From owner-freebsd-net@FreeBSD.ORG Sun Sep 1 01:38:20 2013 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id BA512D6D for ; Sun, 1 Sep 2013 01:38:20 +0000 (UTC) (envelope-from btv1==95617f36530==tgubatayao@barracuda.com) Received: from bsf03.barracuda.com (bsf03.barracuda.com [64.235.145.83]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 804152E64 for ; Sun, 1 Sep 2013 01:38:20 +0000 (UTC) X-ASG-Debug-ID: 1377999493-05b9635b3b681360001-oFaieN Received: from BN-SCL-FE02.Cudanet.local (bn-scl-fe02.cudanet.local [10.8.96.69]) by bsf03.barracuda.com with ESMTP id 8Sv0VtqUTg8hSNp3 (version=TLSv1 cipher=AES128-SHA bits=128 verify=NO); Sat, 31 Aug 2013 18:38:13 -0700 (PDT) X-Barracuda-Envelope-From: tgubatayao@barracuda.com Received: from bn-scl-be02.Cudanet.local (10.8.1.52) by BN-SCL-FE02.Cudanet.local (10.8.96.69) with Microsoft SMTP Server (TLS) id 8.3.298.1; Sat, 31 Aug 2013 18:38:13 -0700 Received: from BN-SCL-MBX03.Cudanet.local ([fe80::e5b6:9fef:a4d2:a5ba]) by bn-scl-be02.Cudanet.local ([::1]) with mapi; Sat, 31 Aug 2013 18:38:13 -0700 From: "T.C. Gubatayao" To: Barney Cordoba , Luigi Rizzo , Alan Somers Date: Sat, 31 Aug 2013 18:38:12 -0700 Subject: RE: Flow ID, LACP, and igb Thread-Topic: Flow ID, LACP, and igb X-ASG-Orig-Subj: RE: Flow ID, LACP, and igb Thread-Index: Ac6mR3pN1wEcLwZWQ96rJZRpG6eRNwAaarS4 Message-ID: References: <521BBD21.4070304@freebsd.org> <521EE8DA.3060107@freebsd.org> , <1377952913.44129.YahooMailNeo@web121605.mail.ne1.yahoo.com> In-Reply-To: <1377952913.44129.YahooMailNeo@web121605.mail.ne1.yahoo.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Barracuda-Connect: bn-scl-fe02.cudanet.local[10.8.96.69] X-Barracuda-Start-Time: 1377999493 X-Barracuda-Encrypted: AES128-SHA X-Barracuda-URL: http://10.8.98.66:8000/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at barracuda.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.62 X-Barracuda-Spam-Status: No, SCORE=0.62 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=9.0 tests=COMMA_SUBJECT, THREAD_INDEX, THREAD_TOPIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.139982 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)... 0.60 COMMA_SUBJECT Subject is like 'Re: FDSDS, this is a subject' Cc: Jack F Vogel , "Justin T. Gibbs" , Andre Oppermann , "net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Sep 2013 01:38:20 -0000 On Sat, Aug 31, 2013 at 8:41 AM, Barney Cordoba = wrote: > Also, the *most* important thing is distribution with realistic data. The= goal > should be to use the most trivial function that gives the most balanced > distribution with real numbers. Faster is not better if the result is an > unbalanced distribution. Agreed, with a caveat. It's critical that this distribution be by "flow", = so that out of order packet delivery is minimized. > Many of your ports will be 80 and 53, and if you're going through a route= r > your ethernets may not be very unique, so why even bother to include them= ? > Does getting a good distribution require that you hash every element > individually, or can you get the same distribution with a faster, simpler= way > of creating the seed? > > There's also the other consideration of packet size. Packets on port 53 a= re > likely to be smaller than packets on port 80. What you want is equal > distribution PER PORT on the ports that will carry that vast majority of = your > traffic. Unfortunately, trying to evenly distribute traffic per port based on packet size will likely result in the reordering of packets, and bandwidth wasted = on TCP retransmissions. > Or better yet, use the same number of queues on igb as you have LAGG port= s, > and use the queue id (or RSS) as the hash, so that your traffic is sync'd > between the ethernet adapter queues and the LAGG ports. The card has alre= ady > done the work for you. Isn't this hash for selecting an outbound link? The ingress adapter hash (= RSS) won't help for packets originating from the host, or for packets that may h= ave been translated or otherwise modified while traversing the stack. T.C.= From owner-freebsd-net@FreeBSD.ORG Sun Sep 1 02:18:29 2013 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id A3941EB for ; Sun, 1 Sep 2013 02:18:29 +0000 (UTC) (envelope-from barney_cordoba@yahoo.com) Received: from nm18-vm4.bullet.mail.ne1.yahoo.com (nm18-vm4.bullet.mail.ne1.yahoo.com [98.138.91.178]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 345702FB4 for ; Sun, 1 Sep 2013 02:18:28 +0000 (UTC) Received: from [98.138.90.56] by nm18.bullet.mail.ne1.yahoo.com with NNFMP; 01 Sep 2013 02:15:33 -0000 Received: from [98.138.88.232] by tm9.bullet.mail.ne1.yahoo.com with NNFMP; 01 Sep 2013 02:15:33 -0000 Received: from [127.0.0.1] by omp1032.mail.ne1.yahoo.com with NNFMP; 01 Sep 2013 02:15:33 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 731413.57168.bm@omp1032.mail.ne1.yahoo.com Received: (qmail 59793 invoked by uid 60001); 1 Sep 2013 02:15:33 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1378001733; bh=z49vOQO4Dfyl4k79zwsmy1cBF4QGkM84J0GvGYV3bNk=; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:References:Message-ID:Date:From:Reply-To:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=INnh61X/+YVyVzysXowwHPUUphlLaZOvdf3lfK3TFEOimvLdbDshHRB4hos9Y4t0bGmFPWkNwz0DbaVqw932QdL3SIyOstXRE8n5Z1XX3ZWehcUiAyDt/R2nnNEJcta/v2/gdqOJX8mVfsRXP7YP9m6KB5fMaJ5A/OgpPAvT5uw= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:References:Message-ID:Date:From:Reply-To:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=kHcvjUBOKTpmZlP+vbvhB5tKQ6gMJ1heq501oZuqwZtueDw1ADfEAknXIO+oq6nFph4MzaMI23JAB7NMMie20lehs2ZimgJqdjJGBQPp+9rsyAnHTdCYhvHQzm7b5XTdtim423fRii5gsmCyTzH/xG8xEGoyij1/rToffUi7RDs=; X-YMail-OSG: kWC2Na4VM1k.J.4rwkrzPOQ.y1Tl1NOLnzdYn80XOhyRJAM TuBf_lFE3.WDn93U.HoZvVkmElu7MqYgF1DazgiafsbwMdi3gIvTUKgDaLw_ anmhwTijDPSy1N13khA5kIpQeeSJeUBj4qkWIxyjd8S_.MH54t_5Repf50or 0l3ctkS1wBBgESXHbKmnOn4I0o5tJGLSCJZaQwMmT7I8h8afcsmWHwye1Dva cxRCed9dy9bIwUatbsZMtBD.poFcFOBiC24_teUjGkfXZzl8wMggg.0LcOjF V6y_3sLExuJHTQiI5lvfiobksbYNkDWXoBICQyskxSzmq4jNB06dC3mrNTGg R_CEydwuL3LzBxlzqGR.hawIo1e7CEchsD2yTgnF3_3zlE0PrsQ_Axtqa535 cnRPL0EvVIo3nr6iOpC96LZs3f3_iKgkmA1KSZXD7X772gqFhsOhpwzXdaj6 tkMLDawV0_mCAGYbFjqmcMAF7eoNFb9XVgEKMhaMYqWfzjyJfRvLshJfd309 2WNmUATEpwd4LLIdEvnMboIJPkMIMJoqqYSViDEmew4ZyPV0K.ElxRnmmmIT bdPRhZ_7PmzLCr1VNjOb_BQ-- Received: from [98.203.118.124] by web121606.mail.ne1.yahoo.com via HTTP; Sat, 31 Aug 2013 19:15:33 PDT X-Rocket-MIMEInfo: 002.001, Tm8sIG5vLiBUaGUgZW50aXJlIHBvaW50IG9mIHRoZSBoYXNoIGlzIHRvIHNlcGFyYXRlIHRoZSAiY29ubmVjdGlvbnMiLiBCdXQgd2hlbiB0ZXN0aW5nIHlvdSBzaG91bGQKdXNlIHJlYWxpc3RpYyBhc3N1bXB0aW9ucy4gWW91J3JlIG5vdCBzcGxpdHRpbmcgcGFja2V0cywgc28gdGhlIGJpZyBwYWNrZXRzIHdpbGwgbWVzcyB1cCB5b3VyIGRpc3RyaWJ1dGlvbgppZiB5b3UgZG9uJ3QgZ2V0IGl0IHJpZ2h0LsKgCgpPZiBjb3Vyc2UgdGhlcmUncyBub3RoaW5nIHJlYWxseSB3cm9uZyB3aXRoIE9PTyBwYWNrZXQBMAEBAQE- X-Mailer: YahooMailWebService/0.8.156.576 References: <521BBD21.4070304@freebsd.org> <521EE8DA.3060107@freebsd.org> , <1377952913.44129.YahooMailNeo@web121605.mail.ne1.yahoo.com> Message-ID: <1378001733.36695.YahooMailNeo@web121606.mail.ne1.yahoo.com> Date: Sat, 31 Aug 2013 19:15:33 -0700 (PDT) From: Barney Cordoba Subject: Re: Flow ID, LACP, and igb To: "T.C. Gubatayao" , Luigi Rizzo , Alan Somers In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: Jack F Vogel , "Justin T. Gibbs" , Andre Oppermann , "net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: Barney Cordoba List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Sep 2013 02:18:29 -0000 No, no. The entire point of the hash is to separate the "connections". But = when testing you should=0Ause realistic assumptions. You're not splitting p= ackets, so the big packets will mess up your distribution=0Aif you don't ge= t it right.=A0=0A=0AOf course there's nothing really wrong with OOO packets= . We had this discussion before; lots of people=0Ahave round robin dual hom= ing without any ill effects. It's just not an issue.=0A=0A=0ABC=0A=0A______= __________________________=0A From: T.C. Gubatayao =0ATo: Barney Cordoba ; Luigi Rizzo ; Alan Somers =0ACc: Jack F Vogel ; Justin T. Gibbs ; Andre Oppermann ; "net@freebsd.org" =0ASent: Saturday, August 31, 2= 013 9:38 PM=0ASubject: RE: Flow ID, LACP, and igb=0A =0A=0AOn Sat, Aug 31, = 2013 at 8:41 AM, Barney Cordoba wrote:=0A=0A> Al= so, the *most* important thing is distribution with realistic data. The goa= l=0A> should be to use the most trivial function that gives the most balanc= ed=0A> distribution with real numbers. Faster is not better if the result i= s an=0A> unbalanced distribution.=0A=0AAgreed, with a caveat.=A0 It's criti= cal that this distribution be by "flow", so=0Athat out of order packet deli= very is minimized.=0A=0A> Many of your ports will be 80 and 53, and if you'= re going through a router=0A> your ethernets may not be very unique, so why= even bother to include them?=0A> Does getting a good distribution require = that you hash every element=0A> individually, or can you get the same distr= ibution with a faster, simpler way=0A> of creating the seed?=0A>=0A> There'= s also the other consideration of packet size. Packets on port 53 are=0A> l= ikely to be smaller than packets on port 80. What you want is equal=0A> dis= tribution PER PORT on the ports that will carry that vast majority of your= =0A> traffic.=0A=0AUnfortunately, trying to evenly distribute traffic per p= ort based on packet=0Asize will likely result in the reordering of packets,= and bandwidth wasted on=0ATCP retransmissions.=0A=0A> Or better yet, use t= he same number of queues on igb as you have LAGG ports,=0A> and use the que= ue id (or RSS) as the hash, so that your traffic is sync'd=0A> between the = ethernet adapter queues and the LAGG ports. The card has already=0A> done t= he work for you.=0A=0AIsn't this hash for selecting an outbound link?=A0 Th= e ingress adapter hash (RSS)=0Awon't help for packets originating from the = host, or for packets that may have=0Abeen translated or otherwise modified = while traversing the stack.=0A=0AT.C.=0A___________________________________= ____________=0Afreebsd-net@freebsd.org mailing list=0Ahttp://lists.freebsd.= org/mailman/listinfo/freebsd-net=0ATo unsubscribe, send any mail to "freebs= d-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Sun Sep 1 02:27:30 2013 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 26ECF1A9; Sun, 1 Sep 2013 02:27:30 +0000 (UTC) (envelope-from rizzo.unipi@gmail.com) Received: from mail-lb0-x22d.google.com (mail-lb0-x22d.google.com [IPv6:2a00:1450:4010:c04::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id BDF532FFD; Sun, 1 Sep 2013 02:27:28 +0000 (UTC) Received: by mail-lb0-f173.google.com with SMTP id o14so2863597lbi.32 for ; Sat, 31 Aug 2013 19:27:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=Ylc3iq3b1QlBvxdTgYhUgUqLOYU0kK7Dv2264U9ZG1k=; b=aHY+jNZ7CRomOgDJ0DjWOp9DZwfGq0D2aEaI1E7mBMy2aD0MFFw2z7LDjetbuFI/XL zPiq3hDfcDlZLV6jf041gpCCUHWXMWQJfRxc9+f95a63+xRvrWCfd5coqV1a4XJYT9eq +AMuYXwoaY8XUgaVb0STpTXeXu9kYDP1W1eMRAuRa6O0G5+OvRK3H4VgNagW6CmmZDx+ yJ9Yf/h/FWML3NNiO/sYC7CJyLi1Xi323yKSWMkP2dzXqKrDBVWVjbT3xYP1iuRvFg7g STqhbJxOcjFor3tl8NnRLVrJ6nXpU/d3hFl/hpkf11n6bbklV2xSv3vu1sGFr/15Topc YCEg== MIME-Version: 1.0 X-Received: by 10.152.3.42 with SMTP id 10mr15000594laz.22.1378002445948; Sat, 31 Aug 2013 19:27:25 -0700 (PDT) Sender: rizzo.unipi@gmail.com Received: by 10.114.200.165 with HTTP; Sat, 31 Aug 2013 19:27:25 -0700 (PDT) In-Reply-To: <1378001733.36695.YahooMailNeo@web121606.mail.ne1.yahoo.com> References: <521BBD21.4070304@freebsd.org> <521EE8DA.3060107@freebsd.org> <1377952913.44129.YahooMailNeo@web121605.mail.ne1.yahoo.com> <1378001733.36695.YahooMailNeo@web121606.mail.ne1.yahoo.com> Date: Sun, 1 Sep 2013 04:27:25 +0200 X-Google-Sender-Auth: uHUCtpiAk6eN9tIh6b_OBc3skCM Message-ID: Subject: Re: Flow ID, LACP, and igb From: Luigi Rizzo To: Barney Cordoba Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: Andre Oppermann , Alan Somers , "net@freebsd.org" , Jack F Vogel , "Justin T. Gibbs" , "T.C. Gubatayao" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Sep 2013 02:27:30 -0000 On Sun, Sep 1, 2013 at 4:15 AM, Barney Cordoba wrote: > ... > [your point on testing with realistic assumptions is surely a valid one] > > Of course there's nothing really wrong with OOO packets. We had this > discussion before; lots of people > have round robin dual homing without any ill effects. It's just not an > issue. > It depends on where you are. It may not be an issue if the reordering is not large enough to trigger retransmissions, but even then it is annoying as it causes more work in the endpoint -- it prevents LRO from working, and even on the host stack it takes more work to sort where an out of order segment goes than appending an in-order one to the socket buffer. cheers luigi From owner-freebsd-net@FreeBSD.ORG Sun Sep 1 04:34:45 2013 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 2AE152CE; Sun, 1 Sep 2013 04:34:45 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 007E9253D; Sun, 1 Sep 2013 04:34:45 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r814Yibk021094; Sun, 1 Sep 2013 04:34:44 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id r814Yirx021093; Sun, 1 Sep 2013 04:34:44 GMT (envelope-from linimon) Date: Sun, 1 Sep 2013 04:34:44 GMT Message-Id: <201309010434.r814Yirx021093@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/181703: [re] [patch] Fix Realtek 8111G Ethernet controller not being detected X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Sep 2013 04:34:45 -0000 Old Synopsis: Realtek 8111G Ethernet controller not detected New Synopsis: [re] [patch] Fix Realtek 8111G Ethernet controller not being detected Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Sun Sep 1 04:33:24 UTC 2013 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=181703 From owner-freebsd-net@FreeBSD.ORG Sun Sep 1 04:35:30 2013 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 5D58F3B8; Sun, 1 Sep 2013 04:35:30 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 330DC2553; Sun, 1 Sep 2013 04:35:30 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r814ZUZc021179; Sun, 1 Sep 2013 04:35:30 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id r814ZUMV021178; Sun, 1 Sep 2013 04:35:30 GMT (envelope-from linimon) Date: Sun, 1 Sep 2013 04:35:30 GMT Message-Id: <201309010435.r814ZUMV021178@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/181699: [ipsec] [patch] IPsec does scale to large SPD / SADB X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Sep 2013 04:35:30 -0000 Old Synopsis: IPsec does scale to large SPD / SADB New Synopsis: [ipsec] [patch] IPsec does scale to large SPD / SADB Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Sun Sep 1 04:35:02 UTC 2013 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=181699 From owner-freebsd-net@FreeBSD.ORG Sun Sep 1 08:11:40 2013 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 34986827; Sun, 1 Sep 2013 08:11:40 +0000 (UTC) (envelope-from melifaro@ipfw.ru) Received: from mail.ipfw.ru (mail.ipfw.ru [IPv6:2a01:4f8:120:6141::2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id ECF3F2D0F; Sun, 1 Sep 2013 08:11:39 +0000 (UTC) Received: from secured.by.ipfw.ru ([95.143.220.47] helo=ws.su29.net) by mail.ipfw.ru with esmtpsa (TLSv1:CAMELLIA256-SHA:256) (Exim 4.76 (FreeBSD)) (envelope-from ) id 1VFz0D-000EXb-52; Sun, 01 Sep 2013 08:11:05 +0400 Message-ID: <5222F661.1090601@ipfw.ru> Date: Sun, 01 Sep 2013 12:10:09 +0400 From: "Alexander V. Chernikov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130728 Thunderbird/17.0.7 MIME-Version: 1.0 To: jfv@FreeBSD.org Subject: Re: Flow ID, LACP, and igb References: In-Reply-To: X-Enigmail-Version: 1.5.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: "Justin T. Gibbs" , Alan Somers , net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Sep 2013 08:11:40 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 26.08.2013 21:18, Justin T. Gibbs wrote: > Hi Net, > > I'm an infrequent traveler through the networking code and would > appreciate some feedback on some proposed solutions to issues > Spectra has seen with outbound LACP traffic. > > lacp_select_tx_port() uses the flow ID if it is available in the > outbound mbuf to select the outbound port. The igb driver uses the > msix queue of the inbound packet to set a packet's flow ID. This > doesn't provide enough Not sure about igb, but ixgbe (according to advanced RX descriptor format, 7.1.6.2 @ 82599 datasheet) can provide 'real' RSS value which can be used in m_flowid instead of NIC queue id. (And, by the way, another RSS-related problem: there are cases when setting flowid does more harm, for example - PPPoE frames always being received at Q0. Partially this can be solved by analyzing RSS type from the same RX descriptor format (e.g. don't set flowid for RSS type 0x0), but there are other cases like GRE tunneling (where you probably want to perform deeper inspection in SW). So, can we have some kind of per-NIC sysctl disabling setting flowid on given port? (Yes, this should be in some kind of `ethtool` binary but we still don't have it..) ) Jack, what do you think? > bits of information to yield a high quality flow ID. If, for > example, the switch controlling inbound packet distribution does a > poor job, the outbound packet distribution will also be poorly > distributed. > > The majority of the adapters supported by this driver will compute > the Toeplitz RSS hash. Using this data seems to work quite well in > our tests (3 member LAGG group). Is there any reason we shouldn't > use the RSS hash for flow ID? > > We also tried disabling the use of flow ID and doing the hash > directly in the driver. Unfortunately, the current hash is pretty > weak. It multiplies by 33, which yield very poor distributions if > you need to mod the result by 3 (e.g. LAGG group with 3 members). > Alan modified the driver to use the FNV hash, which is already in > the kernel, and this yielded much better results. He is still > benchmarking the impact of this change. Assuming we can get decent > flow ID data, this should only impact outbound UDP, since the stack > doesn't provide a flow ID in this case. > > Are there other checksums we should be looking at in addition to > FNV? > > Thanks, Justin > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net To > unsubscribe, send any mail to > "freebsd-net-unsubscribe@freebsd.org" > -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.20 (FreeBSD) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlIi9mEACgkQwcJ4iSZ1q2meFQCfQ9QO+y/9ArTXQBAB9RDCGVY2 SpEAnAgZ0vRYuJ0HMamCnpd8q8/yxsLE =RlDE -----END PGP SIGNATURE----- From owner-freebsd-net@FreeBSD.ORG Sun Sep 1 08:55:06 2013 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id A2962C0C for ; Sun, 1 Sep 2013 08:55:06 +0000 (UTC) (envelope-from yuri@rawbw.com) Received: from shell0.rawbw.com (shell0.rawbw.com [198.144.192.45]) by mx1.freebsd.org (Postfix) with ESMTP id 3F4CF2E9B for ; Sun, 1 Sep 2013 08:55:05 +0000 (UTC) Received: from eagle.yuri.org (stunnel@localhost [127.0.0.1]) (authenticated bits=0) by shell0.rawbw.com (8.14.4/8.14.4) with ESMTP id r818sx9V066801 for ; Sun, 1 Sep 2013 01:54:59 -0700 (PDT) (envelope-from yuri@rawbw.com) Message-ID: <522300E3.6050303@rawbw.com> Date: Sun, 01 Sep 2013 01:54:59 -0700 From: Yuri User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130822 Thunderbird/17.0.8 MIME-Version: 1.0 To: net@FreeBSD.org Subject: Packet loss when 'control' messages are present with large data (sendmsg(2)) Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Sep 2013 08:55:06 -0000 I found the case when sendmsg(2) silently loses packets for AF_LOCAL domain when large packets with control part in them are sent. Here is how: There is the watermark limit on sockbuf determined by net.local.stream.sendspace, default is 8192 bytes (field sockbuf.sb_hiwat). When sendmsg(2) sends large enough data (8K+ that hits this 8192 limit) with control message, sosend_generic will be cutting the message data into separate mbufs based on 'sbspace' (derived from the above-mentioned sb_hiwat limit) with adjustment for control message size as it sees it. This way it tries to make sure this sb_hiwat limit is enforced. However, down on uipc level control message is being further modified in two ways: unp_internalize modifies it into some 'internal' form, also unp_addsockcred function adds another control message when LOCAL_CREDS are requested by client. Both functions only increase control message size beyond its original size (seen by sosend_generic). So that the first final mbuf sent (concatenation of control and data) will always be larger than 'sbspace' limit that sosend_generic was cutting data for. There is also the function sbappendcontrol_locked. It checks the 'sbspace' limit again, and discards the packet when sbspace llimit is exceeded. Its result code is essentially ignored in uipc_send. I believe, sbappendcontrol_locked shouldn't be checking space at all, since packets are expected to be properly sized to begin with. But this won't be the right fix, since sizes would be exceeding the sbspace limit anyway. sosend_default is one level up over uipc level, and it doesn't know what uipc will do with control message. Therefore it can't know what the real adjustment for control message is needed (to properly cut data). It wrongly takes the original control size and this makes the first packet too large and discarded by sbappendcontrol_locked. To solve the problem, I propose to add another function into struct pr_usrreqs: int (*pru_finalizecontrol)(struct socket *so, int flags, struct mbuf **pcontrol); This function will be called from sosend_default and sosend_dgram. uipc_finalizecontrol will do the same that unp_internalize and unp_addsockcred do on uipc level, and it will allow sosend_default to see the final version of the control message, and properly split data into pieces when data is large enough to hit the limit. I felt I better discuss such addition to struct pr_usrreqs, because it might seem like an overkill to add this function just to solve this one local issue. But it seems there is no other solution (other than just ignoring the occasionally larger mbuf size). I can easily make a patch fixing this issue with this new function. Yuri From owner-freebsd-net@FreeBSD.ORG Sun Sep 1 15:45:26 2013 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 558B1640 for ; Sun, 1 Sep 2013 15:45:26 +0000 (UTC) (envelope-from barney_cordoba@yahoo.com) Received: from nm19-vm5.bullet.mail.ne1.yahoo.com (nm19-vm5.bullet.mail.ne1.yahoo.com [98.138.91.241]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 06CA123F0 for ; Sun, 1 Sep 2013 15:45:25 +0000 (UTC) Received: from [98.138.101.128] by nm19.bullet.mail.ne1.yahoo.com with NNFMP; 01 Sep 2013 15:45:19 -0000 Received: from [98.138.89.194] by tm16.bullet.mail.ne1.yahoo.com with NNFMP; 01 Sep 2013 15:45:19 -0000 Received: from [127.0.0.1] by omp1052.mail.ne1.yahoo.com with NNFMP; 01 Sep 2013 15:45:19 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 466928.55584.bm@omp1052.mail.ne1.yahoo.com Received: (qmail 68749 invoked by uid 60001); 1 Sep 2013 15:45:19 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1378050319; bh=Z6bh2tuvZfEjVaxdRzEKK1q0dLpvj6f5ryFybnb3Jq4=; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:References:Message-ID:Date:From:Reply-To:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=uNeqxboGkpp6nQC2mUyIVq9XJ+Sl+B66NpKjGINTIdLfJrKs8bakZA5fyd0gyBb+qQ5lciIAApNNjS1p530Kww1LIozc79HN8xdjB3eYggrdhoXs2mMmRer6wpHdqDoBVHQzCFKou6m+TBbjh3dUDPB8ZvFEg9jneX8tnqw7tYc= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:References:Message-ID:Date:From:Reply-To:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=CyeYMuUJxohp7Sm/fXFitUD+965nZ+m2RjEVkYNznI5DHt/k30EgvfUan4h7PMQW7KQsQD32yUnYpQjZSYPlXv+9G8ezSuEPjzX8Grm81OIHd2Ti0EnR3IwjlFProAFt0ZsEao/N54HGdg3BIL3vghxmn5+VWvb/s4GJT8ibM3w=; X-YMail-OSG: zdhPu_UVM1kB7XCaHj6Ivz7sd7r23IFaBU.sCGQDR7XsyCb U7Ld8mgVYaShS2FkOs0clg5sGwlU1.FFqvk4qxlfUZERdF6fgZcoOPUnuyCh HPUNxtw7xWGEhsq8pxCmvTpSSFvGxtdeT5thZv82.DzJIjLUBMZ6P1BEWO7W kAVm0B6QsOWhWyEuINtl8zYqXD5Ag0nU1nql9AK7NTVU_x1kouqTTTjrUCPv DGJzk2boFkJ_qj5eZJJUq04YVq8TQuuh2kXT_YiDbTZSECM1d2Yi0DnEviJF dZ_0pwlzYfb21Xsi78N1.boDv1lIFnCBa6TGaO4ORnEkBfWOkdJTzCSzcaLd YJumUk2cGWfs.hwIEKv4pKWkmmjc0qNt2qUuKv5yy640zDIBB02xd8dIei2C 8lW8JW5QC5C04uYqkEI7EfPTapl15gb7UamOI.KzsIpBcfTOcJq9tAZO82AC wtIJXgna0XGWcn_l7uFGH_nnUmfrwIPDmAeekPxOebamgsWb9EC0D2OdSO_k 1Pwm3WpMhC4su3nF9IRylK19gM5ad13IfpVTZXUYmuVm7qxLeKdACP7DgtjG GC2u9i6R3vjMmH5s6blwjfQ-- Received: from [98.203.118.124] by web121601.mail.ne1.yahoo.com via HTTP; Sun, 01 Sep 2013 08:45:19 PDT X-Rocket-MIMEInfo: 002.001, CgpDb21jYXN0IHNlbmRzIHBhY2tldHMgT09PLiBXaXRoIGFueSBkZWNlbnQgbnVtYmVyIG9mIGludGVybmV0IGhvcHMgeW91J3JlIGxpa2VseSB0byBlbmNvdW50ZXIgYSBsb2FkCmJhbGFuY2VyIG9yIHBhY2tldCBzaGFwZXIgdGhhdCBzZW5kcyBwYWNrZXRzIE9PTywgc28geW91IGp1c3QgY2FuJ3QgYmUgd29ycmllZCBhYm91dCBpdC4gSW4gZmFjdCwgeW91cgpkZXNpZ25zIE1VU1Qgd29yayB3aXRoIE9PTyBwYWNrZXRzLsKgCgpHZXR0aW5nIGJhbGFuY2Ugb24geW91ciBsb2FkIGJhbGFuY2VkIGxpbmVzIGkBMAEBAQE- X-Mailer: YahooMailWebService/0.8.156.576 References: <521BBD21.4070304@freebsd.org> <521EE8DA.3060107@freebsd.org> <1377952913.44129.YahooMailNeo@web121605.mail.ne1.yahoo.com> <1378001733.36695.YahooMailNeo@web121606.mail.ne1.yahoo.com> Message-ID: <1378050319.62710.YahooMailNeo@web121601.mail.ne1.yahoo.com> Date: Sun, 1 Sep 2013 08:45:19 -0700 (PDT) From: Barney Cordoba Subject: Re: Flow ID, LACP, and igb To: Luigi Rizzo In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: Andre Oppermann , Alan Somers , "net@freebsd.org" , Jack F Vogel , "Justin T. Gibbs" , "T.C. Gubatayao" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: Barney Cordoba List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Sep 2013 15:45:26 -0000 =0A=0AComcast sends packets OOO. With any decent number of internet hops yo= u're likely to encounter a load=0Abalancer or packet shaper that sends pack= ets OOO, so you just can't be worried about it. In fact, your=0Adesigns MUS= T work with OOO packets.=A0=0A=0AGetting balance on your load balanced line= s is certainly a bigger upside than the additional CPU used.=0AYou can buy = a faster processor for your "stack" for a lot less than you can buy bandwid= th.=A0=0A=0AFrankly my opinion of LRO is that it's a science project suitab= le for labs only. It's a trick to get more bandwidth=0Athan your bus capaci= ty; the answer is to not run PCIe2 if you need pcie3. You can use it intern= ally if you have=0Acontrol of all of the machines. When I modify a driver t= he first thing that I do is rip it out.=0A=0ABC=0A=0A=0A___________________= _____________=0A From: Luigi Rizzo =0ATo: Barney Cordob= a =0ACc: Andre Oppermann ; Al= an Somers ; "net@freebsd.org" ; Jack = F Vogel ; Justin T. Gibbs ; T.C. Gubata= yao =0ASent: Saturday, August 31, 2013 10:27 PM= =0ASubject: Re: Flow ID, LACP, and igb=0A =0A=0AOn Sun, Sep 1, 2013 at 4:15= AM, Barney Cordoba wrote:=0A=0A> ...=0A>=0A=0A[y= our point on testing with realistic assumptions is surely a valid one]=0A= =0A=0A>=0A> Of course there's nothing really wrong with OOO packets. We had= this=0A> discussion before; lots of people=0A> have round robin dual homin= g without any ill effects. It's just not an=0A> issue.=0A>=0A=0AIt depends = on where you are.=0AIt may not be an issue if the reordering is not large e= nough to=0Atrigger retransmissions, but even then it is annoying as it caus= es=0Amore work in the endpoint -- it prevents LRO from working, and even=0A= on the host stack it takes more work to sort where an out of order=0Asegmen= t goes than appending an in-order one to the socket buffer.=0A=0Acheers=0Al= uigi=0A_______________________________________________=0Afreebsd-net@freebs= d.org mailing list=0Ahttp://lists.freebsd.org/mailman/listinfo/freebsd-net= =0ATo unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Sun Sep 1 18:00:15 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 9B48C1A1; Sun, 1 Sep 2013 18:00:15 +0000 (UTC) (envelope-from kubito@gmail.com) Received: from mail-lb0-x22d.google.com (mail-lb0-x22d.google.com [IPv6:2a00:1450:4010:c04::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B70DC2C21; Sun, 1 Sep 2013 18:00:14 +0000 (UTC) Received: by mail-lb0-f173.google.com with SMTP id o14so3188539lbi.32 for ; Sun, 01 Sep 2013 11:00:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:sender:to:subject:from:reply-to:cc; bh=+vScTl6jLxxdO0zrKnhx6+3hp3il4IhUZeB8DTKTJEE=; b=MgX5pePLKJI3FZRq+CgwbVi0MF283niaPV2Pb68tyKa/YJdOkkCP4la5+q5BPUPDTZ LB8/7NUKok35zMx/ivFJmGvNL/nm6bjZXwRJoT+NvfuZP/6OQ3kRXXMzwfN9yeJg01to aDb3K7j7gmPsr6XMII5Zbg9NXYb2iOGmpHGGaus9c1rAzFiDEV8DwSnNIDyT6sbWcddg jqMI0ylA2J0JKQyrahdW6mp3qgixYntdv/p/CTP2A5GUk5Yk0GFL5118QQPlA7FETfc9 vGVMNFDTfktaSiPJ+B8+JC1BBhP82o5HRDazxaCcs/mdyVwqOWndxoUySynB3WUtupcn lIcw== X-Received: by 10.152.26.72 with SMTP id j8mr17837819lag.19.1378058412473; Sun, 01 Sep 2013 11:00:12 -0700 (PDT) Received: from localhost (a91-154-115-217.elisa-laajakaista.fi. [91.154.115.217]) by mx.google.com with ESMTPSA id b1sm4378725lah.6.1969.12.31.16.00.00 (version=TLSv1.2 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sun, 01 Sep 2013 11:00:11 -0700 (PDT) Message-ID: <522380ab.811c980a.63e7.ffff829a@mx.google.com> Date: Sun, 01 Sep 2013 11:00:11 -0700 (PDT) Sender: Raphael Kubo da Costa To: FreeBSD-gnats-submit@freebsd.org Subject: [PATCH] Add product ID for Asus USB-BT400 Bluetooth adaptor From: Raphael Kubo da Costa X-send-pr-version: 3.114 X-GNATS-Notify: Cc: freebsd-net@FreeBSD.org, hselasky@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: Raphael Kubo da Costa List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Sep 2013 18:00:15 -0000 >Submitter-Id: current-users >Originator: Raphael Kubo da Costa >Organization: FreeBSD Project >Confidential: no >Synopsis: [PATCH] Add product ID for Asus USB-BT400 Bluetooth adaptor >Severity: non-critical >Priority: low >Category: kern >Class: change-request >Release: FreeBSD 10.0-CURRENT amd64 >Environment: System: FreeBSD orwell 10.0-CURRENT FreeBSD 10.0-CURRENT #2 r255096: Sat Aug 31 19:35:58 EEST 2013 root@orwell:/usr/obj/usr/src/sys/ORWELL amd64 >Description: The attached patch adds another product ID to the list of devices using the BCM20702A0 chipset, ASUS USB-BT400. usbconfig dump_device_desc output: ugen2.3: at usbus2, cfg=0 md=HOST spd=FULL (12Mbps) pwr=ON (100mA) bLength = 0x0012 bDescriptorType = 0x0001 bcdUSB = 0x0200 bDeviceClass = 0x00ff bDeviceSubClass = 0x0001 bDeviceProtocol = 0x0001 bMaxPacketSize0 = 0x0040 idVendor = 0x0b05 idProduct = 0x17cb bcdDevice = 0x0112 iManufacturer = 0x0001 iProduct = 0x0002 iSerialNumber = 0x0003 <000272C64400> bNumConfigurations = 0x0001 >How-To-Repeat: >Fix: --- ng_ubt-asus-17cb.diff begins here --- Index: sys/netgraph/bluetooth/drivers/ubt/ng_ubt.c =================================================================== --- sys/netgraph/bluetooth/drivers/ubt/ng_ubt.c (revision 255096) +++ sys/netgraph/bluetooth/drivers/ubt/ng_ubt.c (working copy) @@ -494,6 +494,7 @@ /* Broadcom BCM20702A0 */ { USB_VPI(USB_VENDOR_ASUS, 0x17b5, 0) }, + { USB_VPI(USB_VENDOR_ASUS, 0x17cb, 0) }, { USB_VPI(USB_VENDOR_LITEON, 0x2003, 0) }, { USB_VPI(USB_VENDOR_FOXCONN, 0xe042, 0) }, { USB_VPI(USB_VENDOR_DELL, 0x8197, 0) }, --- ng_ubt-asus-17cb.diff ends here --- From owner-freebsd-net@FreeBSD.ORG Sun Sep 1 18:39:50 2013 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 143B61B0; Sun, 1 Sep 2013 18:39:50 +0000 (UTC) (envelope-from rakuco@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id DD15D2E69; Sun, 1 Sep 2013 18:39:49 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r81Idnv0099307; Sun, 1 Sep 2013 18:39:49 GMT (envelope-from rakuco@freefall.freebsd.org) Received: (from rakuco@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id r81Idncm099306; Sun, 1 Sep 2013 18:39:49 GMT (envelope-from rakuco) Date: Sun, 1 Sep 2013 18:39:49 GMT Message-Id: <201309011839.r81Idncm099306@freefall.freebsd.org> To: rakuco@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: rakuco@FreeBSD.org Subject: Re: kern/181728: [PATCH] Add product ID for Asus USB-BT400 Bluetooth adaptor X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Sep 2013 18:39:50 -0000 Synopsis: [PATCH] Add product ID for Asus USB-BT400 Bluetooth adaptor Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: rakuco Responsible-Changed-When: Sun Sep 1 18:39:45 UTC 2013 Responsible-Changed-Why: On to freebsd-net. http://www.freebsd.org/cgi/query-pr.cgi?pr=181728 From owner-freebsd-net@FreeBSD.ORG Sun Sep 1 18:40:40 2013 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id F0C322AA; Sun, 1 Sep 2013 18:40:40 +0000 (UTC) (envelope-from eadler@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id C702C2EAC; Sun, 1 Sep 2013 18:40:40 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r81IeeJw099499; Sun, 1 Sep 2013 18:40:40 GMT (envelope-from eadler@freefall.freebsd.org) Received: (from eadler@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id r81IeeNW099498; Sun, 1 Sep 2013 18:40:40 GMT (envelope-from eadler) Date: Sun, 1 Sep 2013 18:40:40 GMT Message-Id: <201309011840.r81IeeNW099498@freefall.freebsd.org> To: eadler@FreeBSD.org, freebsd-net@FreeBSD.org, eadler@FreeBSD.org From: eadler@FreeBSD.org Subject: Re: kern/181728: [PATCH] Add product ID for Asus USB-BT400 Bluetooth adaptor X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Sep 2013 18:40:41 -0000 Synopsis: [PATCH] Add product ID for Asus USB-BT400 Bluetooth adaptor Responsible-Changed-From-To: freebsd-net->eadler Responsible-Changed-By: eadler Responsible-Changed-When: Sun Sep 1 18:40:40 UTC 2013 Responsible-Changed-Why: I'll take it. http://www.freebsd.org/cgi/query-pr.cgi?pr=181728 From owner-freebsd-net@FreeBSD.ORG Sun Sep 1 20:25:27 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 4D25364B; Sun, 1 Sep 2013 20:25:27 +0000 (UTC) (envelope-from universite@ukr.net) Received: from otrada.od.ua (universite-1-pt.tunnel.tserv24.sto1.ipv6.he.net [IPv6:2001:470:27:140::2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9625123E2; Sun, 1 Sep 2013 20:25:26 +0000 (UTC) Received: from [10.0.0.10] ([10.0.0.10]) (authenticated bits=0) by otrada.od.ua (8.14.4/8.14.5) with ESMTP id r81KPJjU051505; Sun, 1 Sep 2013 23:25:19 +0300 (EEST) (envelope-from universite@ukr.net) Message-ID: <5223A2A9.8030602@ukr.net> Date: Sun, 01 Sep 2013 23:25:13 +0300 From: "Vladislav V. Prodan" User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130801 Thunderbird/17.0.8 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: Quagga not support password for neighbor References: <66067.1363878392.12938546996697300992@ffe17.ukr.net> In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (otrada.od.ua [89.209.81.54]); Sun, 01 Sep 2013 23:25:19 +0300 (EEST) X-Spam-Status: No, score=-101.0 required=5.0 tests=ALL_TRUSTED, FREEMAIL_FROM, T_TO_NO_BRKTS_FREEMAIL, USER_IN_WHITELIST autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mary-teresa.otrada.od.ua Cc: =?UTF-8?B?RXJtYWwgTHXDp2k=?= X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Sep 2013 20:25:27 -0000 Up. You can set examples? I added these options in my kernel and rebuild: options TCP_SIGNATURE options IPSEC device crypto device cryptodev ... I added these lines to /etc/rc.conf: ... ipsec_enable="YES" ipsec_file="/etc/ipsec.conf" ... and /etc/ipsec.conf output: flush; add ZZZ.245.YYY.67 ZZZ.245.YYY.1 tcp 0x1000 -A tcp-md5 "XXXXXXXX"; add ZZZ.245.YYY.67 ZZZ.245.YYY.2 tcp 0x1000 -A tcp-md5 "XXXXXXXX"; add ZZZ.107.YYY.12 ZZZ.107.YYY.1 tcp 0x1000 -A tcp-md5 "XXXXXXXX"; add ZZZ.107.YYY.12 ZZZ.107.YYY.199 tcp 0x1000 -A tcp-md5 "XXXXXXXX"; 21.03.2013 17:52, Ermal Luçi wrote: > You need a kernel with TCP_SIGNATURE option and insert policy routes with > setkey. > > > On Thu, Mar 21, 2013 at 4:06 PM, Vladislav Prodan wrote: > >> >> FreeBSD 8.2-STABLE >> quagga-0.99.21 Free RIPv1, RIPv2, OSPFv2, BGP4, IS-IS route software >> >> BGP.as11111(config-router)# neighbor XXX.XXX.YYY.YYY password testtest >> % Error while applying TCP-Sig to session(s) >> >> No one to share the patch with the Linux version of quagga, so get to work >> option password? >> >> Thanks! >> -- Vladislav V. Prodan System & Network Administrator http://support.od.ua +380 67 4584408, +380 99 4060508 VVP88-RIPE From owner-freebsd-net@FreeBSD.ORG Sun Sep 1 20:45:58 2013 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 930BD9CA; Sun, 1 Sep 2013 20:45:58 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-we0-x229.google.com (mail-we0-x229.google.com [IPv6:2a00:1450:400c:c03::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id A40F624CB; Sun, 1 Sep 2013 20:45:57 +0000 (UTC) Received: by mail-we0-f169.google.com with SMTP id p60so824734wes.28 for ; Sun, 01 Sep 2013 13:45:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=ttI8uk4Qrp0LVP9AwHqevDQngsRuFRoNKH5U5ZHlUCs=; b=tgJksMKrZfnylKBn2LldHUZefduOZqTbvNqGsM4beYtbb1nMJf7yD/4eQR77LMxZSY AnNILaT3AWyINUucY0Dsxcc9PM92LCdxxswmOxxZp8VNCawrL1gSsFY7XiL9ofDUSgW5 2bEUSkp25ld3sqHmpj6/bdckoK7DL4OpnMNLpFMjpxk17C4JlFtDpW2LRu4FaME+yoiI gJ5avWMupLUNb+lg84hj+Wg5dc7AlGA715VYhf9qGGRTMRlI/0yYZQ944QvCBZLrKMLl pK7uadWZWzrIiid2fQEKyocddMLfM2v2Xa/PcI8d110Krm1tvxjewWYurKIMUtEYNkYz dQ/w== MIME-Version: 1.0 X-Received: by 10.194.21.131 with SMTP id v3mr2364060wje.44.1378068355923; Sun, 01 Sep 2013 13:45:55 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.216.146.2 with HTTP; Sun, 1 Sep 2013 13:45:55 -0700 (PDT) In-Reply-To: <5222F661.1090601@ipfw.ru> References: <5222F661.1090601@ipfw.ru> Date: Sun, 1 Sep 2013 13:45:55 -0700 X-Google-Sender-Auth: jXdfGbnCMHrX24KDTuAK3w4ja1M Message-ID: Subject: Re: Flow ID, LACP, and igb From: Adrian Chadd To: "Alexander V. Chernikov" Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: Jack F Vogel , "Justin T. Gibbs" , Alan Somers , "net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Sep 2013 20:45:58 -0000 > Not sure about igb, but ixgbe (according to advanced RX descriptor > format, 7.1.6.2 @ 82599 datasheet) can provide 'real' RSS value which > can be used in m_flowid instead of NIC queue id. > > (And, by the way, another RSS-related problem: > there are cases when setting flowid does more harm, for example - > PPPoE frames always being received at Q0. > Partially this can be solved by analyzing RSS type from the same RX > descriptor format (e.g. don't set flowid for RSS type 0x0), but there > are other cases like GRE tunneling (where you probably want to perform > deeper inspection in SW). > > So, can we have some kind of per-NIC sysctl disabling setting flowid > on given port? > > (Yes, this should be in some kind of `ethtool` binary but we still > don't have it..) > ) > What specifically are you asking for? Disabling the flowid tagging of mbufs? Or changing the LACP hashing? You can configure LACP to not use the flowid from the NIC and do the hashing yourself. -adrian From owner-freebsd-net@FreeBSD.ORG Sun Sep 1 20:51:20 2013 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id C7AD0B7D; Sun, 1 Sep 2013 20:51:20 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-wi0-x22b.google.com (mail-wi0-x22b.google.com [IPv6:2a00:1450:400c:c05::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id A24532515; Sun, 1 Sep 2013 20:51:19 +0000 (UTC) Received: by mail-wi0-f171.google.com with SMTP id hq12so1179379wib.10 for ; Sun, 01 Sep 2013 13:51:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=qwKV+wetXtX2L8fzprJcef2IwG9zoUZn6GZq4SNa1tk=; b=Fcxk7pwlrG4uXUWM1ZImr69Za9unx+vvRarWaGCABKOp11TvATZPIKU/a2bW3mu2cs Q+tFdYMjjt+ojSYFDmXieA2e0vzamEGGSX90xdHLeR09RzsbEh/Aw1NOqKdaQ+gakAVz RyHlkyavj46FhC8xPcXWVNty0qsolnqnqrj0SRrKo9kxNNb5pc3rz1atKI7+biazVrbe MiB1BE3v98z/cIh2i9QgnKAKs9ZY7T2FrYA8xywKPnQznTAwZ+BDM+MWDalDKOfQdNz/ Hj/JW3Yy+0fRzPjLC5SECkjjCseR6ptHXrBSXdzFWvnXxvCsu/b8Odx2ln7xD5S+wPqZ Q1qg== MIME-Version: 1.0 X-Received: by 10.180.37.164 with SMTP id z4mr10937379wij.30.1378068677832; Sun, 01 Sep 2013 13:51:17 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.216.146.2 with HTTP; Sun, 1 Sep 2013 13:51:17 -0700 (PDT) In-Reply-To: <1378050319.62710.YahooMailNeo@web121601.mail.ne1.yahoo.com> References: <521BBD21.4070304@freebsd.org> <521EE8DA.3060107@freebsd.org> <1377952913.44129.YahooMailNeo@web121605.mail.ne1.yahoo.com> <1378001733.36695.YahooMailNeo@web121606.mail.ne1.yahoo.com> <1378050319.62710.YahooMailNeo@web121601.mail.ne1.yahoo.com> Date: Sun, 1 Sep 2013 13:51:17 -0700 X-Google-Sender-Auth: Dsg0NSrMPg0YIKpmRWgs3S4mwxM Message-ID: Subject: Re: Flow ID, LACP, and igb From: Adrian Chadd To: Barney Cordoba Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: Andre Oppermann , Alan Somers , "net@freebsd.org" , Jack F Vogel , "Justin T. Gibbs" , Luigi Rizzo , "T.C. Gubatayao" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Sep 2013 20:51:20 -0000 Yo, LRO is an interesting hack that seems to do a good trick of hiding the ridiculous locking and unfriendly cache behaviour that we do per-packet. It helps with LAN test traffic where things are going out in batches from the TCP layer so the RX layer "sees" these frames in-order and can do LRO. When you disable it, I don't easily get 10GE LAN TCP performance. That has to be fixed. Given how fast the CPU cores, bus interconnect and memory interconnects are, I don't think there should be any reason why we can't hit 10GE traffic on a LAN with LRO disabled (in both software and hardware.) Now that I have the PMC sandy bridge stuff working right (but no PEBS, I have to talk to Intel about that in a bit more detail before I think about hacking that in) we can get actual live information about this stuff. But the last time I looked, there's just too much per-packet latency going on. The root cause looks like it's a toss up between scheduling, locking and just lots of code running to completion per-frame. As I said, that all has to die somehow. 2c, -adrian On 1 September 2013 08:45, Barney Cordoba wrote: > > > Comcast sends packets OOO. With any decent number of internet hops you're > likely to encounter a load > balancer or packet shaper that sends packets OOO, so you just can't be > worried about it. In fact, your > designs MUST work with OOO packets. > > Getting balance on your load balanced lines is certainly a bigger upside > than the additional CPU used. > You can buy a faster processor for your "stack" for a lot less than you > can buy bandwidth. > > Frankly my opinion of LRO is that it's a science project suitable for labs > only. It's a trick to get more bandwidth > than your bus capacity; the answer is to not run PCIe2 if you need pcie3. > You can use it internally if you have > control of all of the machines. When I modify a driver the first thing > that I do is rip it out. > > BC > > > ________________________________ > From: Luigi Rizzo > To: Barney Cordoba > Cc: Andre Oppermann ; Alan Somers ; > "net@freebsd.org" ; Jack F Vogel ; > Justin T. Gibbs ; T.C. Gubatayao < > tgubatayao@barracuda.com> > Sent: Saturday, August 31, 2013 10:27 PM > Subject: Re: Flow ID, LACP, and igb > > > On Sun, Sep 1, 2013 at 4:15 AM, Barney Cordoba >wrote: > > > ... > > > > [your point on testing with realistic assumptions is surely a valid one] > > > > > > Of course there's nothing really wrong with OOO packets. We had this > > discussion before; lots of people > > have round robin dual homing without any ill effects. It's just not an > > issue. > > > > It depends on where you are. > It may not be an issue if the reordering is not large enough to > trigger retransmissions, but even then it is annoying as it causes > more work in the endpoint -- it prevents LRO from working, and even > on the host stack it takes more work to sort where an out of order > segment goes than appending an in-order one to the socket buffer. > > cheers > luigi > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Sun Sep 1 21:04:04 2013 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id D5531DB1; Sun, 1 Sep 2013 21:04:04 +0000 (UTC) (envelope-from melifaro@ipfw.ru) Received: from mail.ipfw.ru (mail.ipfw.ru [IPv6:2a01:4f8:120:6141::2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 96350257A; Sun, 1 Sep 2013 21:04:04 +0000 (UTC) Received: from secured.by.ipfw.ru ([95.143.220.47] helo=ws.su29.net) by mail.ipfw.ru with esmtpsa (TLSv1:CAMELLIA256-SHA:256) (Exim 4.76 (FreeBSD)) (envelope-from ) id 1VGB3f-000L1c-63; Sun, 01 Sep 2013 21:03:27 +0400 Message-ID: <5223AB67.60207@ipfw.ru> Date: Mon, 02 Sep 2013 01:02:31 +0400 From: "Alexander V. Chernikov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130728 Thunderbird/17.0.7 MIME-Version: 1.0 To: Adrian Chadd Subject: Re: Flow ID, LACP, and igb References: <5222F661.1090601@ipfw.ru> In-Reply-To: X-Enigmail-Version: 1.5.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Jack F Vogel , "Justin T. Gibbs" , Alan Somers , "net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Sep 2013 21:04:04 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 02.09.2013 00:45, Adrian Chadd wrote: > > > Not sure about igb, but ixgbe (according to advanced RX descriptor > format, 7.1.6.2 @ 82599 datasheet) can provide 'real' RSS value > which can be used in m_flowid instead of NIC queue id. > > (And, by the way, another RSS-related problem: there are cases when > setting flowid does more harm, for example - PPPoE frames always > being received at Q0. Partially this can be solved by analyzing RSS > type from the same RX descriptor format (e.g. don't set flowid for > RSS type 0x0), but there are other cases like GRE tunneling (where > you probably want to perform deeper inspection in SW). > > So, can we have some kind of per-NIC sysctl disabling setting > flowid on given port? > > (Yes, this should be in some kind of `ethtool` binary but we still > don't have it..) ) > > > What specifically are you asking for? Disabling the flowid tagging > of I'm talking about some small ixgbe (and maybe igb?) changes related to the generation of mbuf flowid. More specifically: 1) As far as I understand, ixgbe generates u16 hash which is then used to compute receive queue number. It seems that this value can be set by NIC in per-packet advanced RX descriptor, so it can be used as better flowid value (which should be optional). 2) There are cases when we shouldn't simply mark all packets as received by q0 (since NIC hash doesn't known how to hash them), so disabling setting m_flowid for given port can help a lot. > mbufs? Or changing the LACP hashing? It is not related to lagg directly. Actually, it is, but for the very special case like 'routing on the stick' when we're forwarding packets back to the same lagg interface. In this case, we can set (pre-computed) static RX queue flowids to force forwarder packet fall to the same NIC and the same TX queue id. This approach minimizes egress mutex contention (I forgot to mention patch implementing this stuff in my 'Network stack changes' letter). > > You can configure LACP to not use the flowid from the NIC and do > the hashing yourself. Yup, I'm aware of that :) > > > -adrian > -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.20 (FreeBSD) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlIjq2cACgkQwcJ4iSZ1q2ncAACfVmiVTtyno7hcxG59HZs8cSyq umwAnjQ6r4V3UCO8T0uE3gMZmeMveUMB =thS0 -----END PGP SIGNATURE----- From owner-freebsd-net@FreeBSD.ORG Mon Sep 2 05:15:22 2013 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 8B3E9C56; Mon, 2 Sep 2013 05:15:22 +0000 (UTC) (envelope-from yuri@rawbw.com) Received: from shell0.rawbw.com (shell0.rawbw.com [198.144.192.45]) by mx1.freebsd.org (Postfix) with ESMTP id 5AD7E2E3A; Mon, 2 Sep 2013 05:15:22 +0000 (UTC) Received: from eagle.yuri.org (stunnel@localhost [127.0.0.1]) (authenticated bits=0) by shell0.rawbw.com (8.14.4/8.14.4) with ESMTP id r824sDsk002565; Sun, 1 Sep 2013 21:54:13 -0700 (PDT) (envelope-from yuri@rawbw.com) Message-ID: <522419F4.5010605@rawbw.com> Date: Sun, 01 Sep 2013 21:54:12 -0700 From: Yuri User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130822 Thunderbird/17.0.8 MIME-Version: 1.0 To: current@freebsd.org, net@freebsd.org Subject: [PATCH] Packet loss when 'control' messages are present with large data (sendmsg(2)) References: <522300E3.6050303@rawbw.com> In-Reply-To: <522300E3.6050303@rawbw.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Sep 2013 05:15:22 -0000 Please check in this patch: http://www.freebsd.org/cgi/query-pr.cgi?pr=181741 Please MFC into 9.X Description of the problem is within PR. Thanks, Yuri From owner-freebsd-net@FreeBSD.ORG Mon Sep 2 06:42:09 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 04589425; Mon, 2 Sep 2013 06:42:09 +0000 (UTC) (envelope-from mailinglists@martinlaabs.de) Received: from relay04.alfahosting-server.de (relay04.alfahosting-server.de [109.237.142.240]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B43EE21B6; Mon, 2 Sep 2013 06:42:08 +0000 (UTC) Received: by relay04.alfahosting-server.de (Postfix, from userid 1001) id 8384932D176F; Mon, 2 Sep 2013 08:42:06 +0200 (CEST) X-Spam-DCC: : X-Spam-Level: X-Spam-Status: No, score=0.0 required=7.0 tests=BAYES_50 autolearn=disabled version=3.2.5 Received: from alfa3018.alfahosting-server.de (alfa3018.alfahosting-server.de [109.237.140.30]) by relay04.alfahosting-server.de (Postfix) with ESMTPS id 9CB4E32D0CC9; Mon, 2 Sep 2013 08:42:04 +0200 (CEST) Received: from laabs.hf.ifn.et.tu-dresden.de (hfsync.ifn.et.tu-dresden.de [141.30.128.60]) by alfa3018.alfahosting-server.de (Postfix) with ESMTPSA id 84615515DE98; Mon, 2 Sep 2013 08:42:04 +0200 (CEST) Message-ID: <5224333C.8070305@martinlaabs.de> Date: Mon, 02 Sep 2013 08:42:04 +0200 From: Martin Laabs User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130809 Thunderbird/17.0.8 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: No UDP/TCP IPv6 connectivity (only) to router using gif interface - maybe ARM related Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Virus-Status: No X-Virus-Checker-Version: clamassassin 1.2.4 with ClamAV 0.97.3/17790/Mon Sep 2 06:48:29 2013 Cc: freebsd-arm , freebsd-current@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Sep 2013 06:42:09 -0000 Hi, I tried to set up my raspberry PI as an ipv6 router. As a tunnel broker I use sixxs. Now I observed an interesting behavior: Every host from my network can reach the ipv6 world. The ipv6 world can also reach every host in my network. However - the router itself is unable to make udp or tcp connection to the "world" and is also unable to accept connections form the "world" ICMP however works properly. I had a look to the tcpdump and when trying to connect i.e. to www.kame.net the rasperry router sends a syn packet and get a syn/ack packet back. The rest of the handshake is missing. I tried also some udp with netcat (nc -6 -u -l 5555 on the server and nc -6 -u 5555 on the client) This works great for internal (ethernet) traffic but when the data should go through the tunnel if fails. The last test is maybe the most significant to describe the bug: Start netcat to listen for UDP packages on an external host: nc -6 -u -l 5555 Connect from the RPI-Router to that host nc -6 -u 2001:4dd0:xxxx:xxxx::2 5555 Now it is possible to send data from the RPI router to the external host but the opposite direction does not work. Tcpdump however shows that the udp package arrives but it is not "forwarded" to the application. So for me it seems to be a problem with the handling of the receiving data in the gif interface. This behavior is independent from net.inet6.ip6.forwarding or net.inet6.ip6.redirect status. The router system: FreeBSD raspberry-pi.xxx 10.0-CURRENT FreeBSD 10.0-CURRENT #2 r254984 Best regards, Martin Laabs From owner-freebsd-net@FreeBSD.ORG Mon Sep 2 11:06:49 2013 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 0021D142 for ; Mon, 2 Sep 2013 11:06:48 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E144623AD for ; Mon, 2 Sep 2013 11:06:48 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r82B6mtr016102 for ; Mon, 2 Sep 2013 11:06:48 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id r82B6mW2016100 for freebsd-net@FreeBSD.org; Mon, 2 Sep 2013 11:06:48 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 2 Sep 2013 11:06:48 GMT Message-Id: <201309021106.r82B6mW2016100@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-net@FreeBSD.org Subject: Current problem reports assigned to freebsd-net@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Sep 2013 11:06:49 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/181703 net [re] [patch] Fix Realtek 8111G Ethernet controller not o kern/181699 net [ipsec] [patch] IPsec does scale to large SPD / SADB o kern/181657 net [bpf] [patch] BPF_COP/BPF_COPX instruction reservation o kern/181257 net [bge] bge link status change o kern/181236 net [igb] igb driver unstable work o kern/181225 net [infiniband] [patch] unloading ipoib crashes the kerne o kern/181135 net [netmap] [patch] sys/dev/netmap patch for Linux compat o kern/181131 net [netmap] [patch] sys/dev/netmap memory allocation impr o kern/181006 net [run] [patch] mbuf leak in run(4) driver o kern/180893 net [if_ethersubr] [patch] Packets received with own LLADD o kern/180844 net [panic] [re] Intermittent panic (re driver?) o kern/180775 net [bxe] if_bxe driver broken with Broadcom BCM57711 card o kern/180722 net [bluetooth] bluetooth takes 30-50 attempts to pair to s kern/180468 net [request] LOCAL_PEERCRED support for PF_INET o kern/180065 net [netinet6] [patch] Multicast loopback to own host brok o kern/179926 net [lacp] [patch] active aggregator selection bug o kern/179824 net [ixgbe] System (9.1-p4) hangs on heavy ixgbe network t o kern/179733 net [lagg] [patch] interface loses capabilities when proto o kern/179429 net [tap] STP enabled tap bridge o kern/179299 net [igb] Intel X540-T2 - unstable driver a kern/179264 net [vimage] [pf] Core dump with Packet filter and VIMAGE o kern/178947 net [arp] arp rejecting not working o kern/178782 net [ixgbe] 82599EB SFP does not work with passthrough und o kern/178612 net [run] kernel panic due the problems with run driver o kern/178472 net [ip6] [patch] make return code consistent with IPv4 co o kern/178079 net [tcp] Switching TCP CC algorithm panics on sparc64 wit s kern/178071 net FreeBSD unable to recongize Kontron (Industrial Comput o kern/177905 net [xl] [panic] ifmedia_set when pluging CardBus LAN card o kern/177618 net [bridge] Problem with bridge firewall with trunk ports o kern/177417 net [ip6] Invalid protocol value in ipsec6_common_input_cb o kern/177402 net [igb] [pf] problem with ethernet driver igb + pf / alt o kern/177400 net [jme] JMC25x 1000baseT establishment issues o kern/177366 net [ieee80211] negative malloc(9) statistics for 80211nod f kern/177362 net [netinet] [patch] Wrong control used to return TOS o kern/177194 net [netgraph] Unnamed netgraph nodes for vlan interfaces o kern/177139 net [igb] igb drops ethernet ports 2 and 3 o kern/176884 net [re] re0 flapping up/down o kern/176671 net [epair] MAC address for epair device not unique o kern/176484 net [ipsec] [enc] [patch] panic: IPsec + enc(4); device na o kern/176446 net [netinet] [patch] Concurrency in ixgbe driving out-of- o kern/176420 net [kernel] [patch] incorrect errno for LOCAL_PEERCRED o kern/176419 net [kernel] [patch] socketpair support for LOCAL_PEERCRED o kern/176401 net [netgraph] page fault in netgraph o kern/176167 net [ipsec][lagg] using lagg and ipsec causes immediate pa o kern/176097 net [lagg] [patch] lagg/lacp broken when aggregated interf o kern/176027 net [em] [patch] flow control systcl consistency for em dr o kern/176026 net [tcp] [patch] TCP wrappers caused quite a lot of warni o bin/175974 net ppp(8): logic issue o kern/175864 net [re] Intel MB D510MO, onboard ethernet not working aft o kern/175852 net [amd64] [patch] in_cksum_hdr() behaves differently on o kern/175734 net no ethernet detected on system with EG20T PCH chipset o kern/175267 net [pf] [tap] pf + tap keep state problem o kern/175236 net [epair] [gif] epair and gif Devices On Bridge o kern/175182 net [panic] kernel panic on RADIX_MPATH when deleting rout o kern/175153 net [tcp] will there miss a FIN when do TSO? o kern/174959 net [net] [patch] rnh_walktree_from visits spurious nodes o kern/174958 net [net] [patch] rnh_walktree_from makes unreasonable ass o kern/174897 net [route] Interface routes are broken o kern/174851 net [bxe] [patch] UDP checksum offload is wrong in bxe dri o kern/174850 net [bxe] [patch] bxe driver does not receive multicasts o kern/174849 net [bxe] [patch] bxe driver can hang kernel when reset o kern/174822 net [tcp] Page fault in tcp_discardcb under high traffic o kern/174602 net [gif] [ipsec] traceroute issue on gif tunnel with ipse o kern/174535 net [tcp] TCP fast retransmit feature works strange o kern/173871 net [gif] process of 'ifconfig gif0 create hangs' when if_ o kern/173475 net [tun] tun(4) stays opened by PID after process is term o kern/173201 net [ixgbe] [patch] Missing / broken ixgbe sysctl's and tu o kern/173137 net [em] em(4) unable to run at gigabit with 9.1-RC2 o kern/173002 net [patch] data type size problem in if_spppsubr.c o kern/172895 net [ixgb] [ixgbe] do not properly determine link-state o kern/172683 net [ip6] Duplicate IPv6 Link Local Addresses o kern/172675 net [netinet] [patch] sysctl_tcp_hc_list (net.inet.tcp.hos o kern/172113 net [panic] [e1000] [patch] 9.1-RC1/amd64 panices in igb(4 o kern/171840 net [ip6] IPv6 packets transmitting only on queue 0 o kern/171739 net [bce] [panic] bce related kernel panic o kern/171711 net [dummynet] [panic] Kernel panic in dummynet o kern/171532 net [ndis] ndis(4) driver includes 'pccard'-specific code, o kern/171531 net [ndis] undocumented dependency for ndis(4) o kern/171524 net [ipmi] ipmi driver crashes kernel by reboot or shutdow s kern/171508 net [epair] [request] Add the ability to name epair device o kern/171228 net [re] [patch] if_re - eeprom write issues o kern/170701 net [ppp] killl ppp or reboot with active ppp connection c o kern/170267 net [ixgbe] IXGBE_LE32_TO_CPUS is probably an unintentiona o kern/170081 net [fxp] pf/nat/jails not working if checksum offloading o kern/169898 net ifconfig(8) fails to set MTU on multiple interfaces. o kern/169676 net [bge] [hang] system hangs, fully or partially after re o kern/169664 net [bgp] Wrongful replacement of interface connected net o kern/169620 net [ng] [pf] ng_l2tp incoming packet bypass pf firewall o kern/169459 net [ppp] umodem/ppp/3g stopped working after update from o kern/169438 net [ipsec] ipv4-in-ipv6 tunnel mode IPsec does not work p kern/168294 net [ixgbe] [patch] ixgbe driver compiled in kernel has no o kern/168246 net [em] Multiple em(4) not working with qemu o kern/168245 net [arp] [regression] Permanent ARP entry not deleted on o kern/168244 net [arp] [regression] Unable to manually remove permanent o kern/168183 net [bce] bce driver hang system o kern/167947 net [setfib] [patch] arpresolve checks only the default FI o kern/167603 net [ip] IP fragment reassembly's broken: file transfer ov o kern/167500 net [em] [panic] Kernel panics in em driver o kern/167325 net [netinet] [patch] sosend sometimes return EINVAL with o kern/167202 net [igmp]: Sending multiple IGMP packets crashes kernel o kern/166462 net [gre] gre(4) when using a tunnel source address from c o kern/166285 net [arp] FreeBSD v8.1 REL p8 arp: unknown hardware addres o kern/166255 net [net] [patch] It should be possible to disable "promis p kern/165903 net mbuf leak o kern/165622 net [ndis][panic][patch] Unregistered use of FPU in kernel s kern/165562 net [request] add support for Intel i350 in FreeBSD 7.4 o kern/165526 net [bxe] UDP packets checksum calculation whithin if_bxe o kern/165488 net [ppp] [panic] Fatal trap 12 jails and ppp , kernel wit o kern/165305 net [ip6] [request] Feature parity between IP_TOS and IPV6 o kern/165296 net [vlan] [patch] Fix EVL_APPLY_VLID, update EVL_APPLY_PR o kern/165181 net [igb] igb freezes after about 2 weeks of uptime o kern/165174 net [patch] [tap] allow tap(4) to keep its address on clos o kern/165152 net [ip6] Does not work through the issue of ipv6 addresse o kern/164495 net [igb] connect double head igb to switch cause system t o kern/164490 net [pfil] Incorrect IP checksum on pfil pass from ip_outp o kern/164475 net [gre] gre misses RUNNING flag after a reboot o kern/164265 net [netinet] [patch] tcp_lro_rx computes wrong checksum i o kern/163903 net [igb] "igb0:tx(0)","bpf interface lock" v2.2.5 9-STABL o kern/163481 net freebsd do not add itself to ping route packet o kern/162927 net [tun] Modem-PPP error ppp[1538]: tun0: Phase: Clearing o kern/162558 net [dummynet] [panic] seldom dummynet panics o kern/162153 net [em] intel em driver 7.2.4 don't compile o kern/162110 net [igb] [panic] RELENG_9 panics on boot in IGB driver - o kern/162028 net [ixgbe] [patch] misplaced #endif in ixgbe.c o kern/161277 net [em] [patch] BMC cannot receive IPMI traffic after loa o kern/160873 net [igb] igb(4) from HEAD fails to build on 7-STABLE o kern/160750 net Intel PRO/1000 connection breaks under load until rebo o kern/160693 net [gif] [em] Multicast packet are not passed from GIF0 t o kern/160293 net [ieee80211] ppanic] kernel panic during network setup o kern/160206 net [gif] gifX stops working after a while (IPv6 tunnel) o kern/159817 net [udp] write UDPv4: No buffer space available (code=55) o kern/159629 net [ipsec] [panic] kernel panic with IPsec in transport m o kern/159621 net [tcp] [panic] panic: soabort: so_count o kern/159603 net [netinet] [patch] in_ifscrubprefix() - network route c o kern/159601 net [netinet] [patch] in_scrubprefix() - loopback route re o kern/159294 net [em] em watchdog timeouts o kern/159203 net [wpi] Intel 3945ABG Wireless LAN not support IBSS o kern/158930 net [bpf] BPF element leak in ifp->bpf_if->bif_dlist o kern/158726 net [ip6] [patch] ICMPv6 Router Announcement flooding limi o kern/158694 net [ix] [lagg] ix0 is not working within lagg(4) o kern/158665 net [ip6] [panic] kernel pagefault in in6_setscope() o kern/158635 net [em] TSO breaks BPF packet captures with em driver f kern/157802 net [dummynet] [panic] kernel panic in dummynet o kern/157785 net amd64 + jail + ipfw + natd = very slow outbound traffi o kern/157418 net [em] em driver lockup during boot on Supermicro X9SCM- o kern/157410 net [ip6] IPv6 Router Advertisements Cause Excessive CPU U o kern/157287 net [re] [panic] INVARIANTS panic (Memory modified after f o kern/157200 net [network.subr] [patch] stf(4) can not communicate betw o kern/157182 net [lagg] lagg interface not working together with epair o kern/156877 net [dummynet] [panic] dummynet move_pkt() null ptr derefe o kern/156667 net [em] em0 fails to init on CURRENT after March 17 o kern/156408 net [vlan] Routing failure when using VLANs vs. Physical e o kern/156328 net [icmp]: host can ping other subnet but no have IP from o kern/156317 net [ip6] Wrong order of IPv6 NS DAD/MLD Report o kern/156283 net [ip6] [patch] nd6_ns_input - rtalloc_mpath does not re o kern/156279 net [if_bridge][divert][ipfw] unable to correctly re-injec o kern/156226 net [lagg]: failover does not announce the failover to swi o kern/156030 net [ip6] [panic] Crash in nd6_dad_start() due to null ptr o kern/155680 net [multicast] problems with multicast s kern/155642 net [new driver] [request] Add driver for Realtek RTL8191S o kern/155597 net [panic] Kernel panics with "sbdrop" message o kern/155420 net [vlan] adding vlan break existent vlan o kern/155177 net [route] [panic] Panic when inject routes in kernel o kern/155010 net [msk] ntfs-3g via iscsi using msk driver cause kernel o kern/154943 net [gif] ifconfig gifX create on existing gifX clears IP s kern/154851 net [new driver] [request]: Port brcm80211 driver from Lin o kern/154850 net [netgraph] [patch] ng_ether fails to name nodes when t o kern/154679 net [em] Fatal trap 12: "em1 taskq" only at startup (8.1-R o kern/154600 net [tcp] [panic] Random kernel panics on tcp_output o kern/154557 net [tcp] Freeze tcp-session of the clients, if in the gat o kern/154443 net [if_bridge] Kernel module bridgestp.ko missing after u o kern/154286 net [netgraph] [panic] 8.2-PRERELEASE panic in netgraph o kern/154255 net [nfs] NFS not responding o kern/154214 net [stf] [panic] Panic when creating stf interface o kern/154185 net race condition in mb_dupcl p kern/154169 net [multicast] [ip6] Node Information Query multicast add o kern/154134 net [ip6] stuck kernel state in LISTEN on ipv6 daemon whic o kern/154091 net [netgraph] [panic] netgraph, unaligned mbuf? o conf/154062 net [vlan] [patch] change to way of auto-generatation of v o kern/153937 net [ral] ralink panics the system (amd64 freeBSDD 8.X) wh o kern/153936 net [ixgbe] [patch] MPRC workaround incorrectly applied to o kern/153816 net [ixgbe] ixgbe doesn't work properly with the Intel 10g o kern/153772 net [ixgbe] [patch] sysctls reference wrong XON/XOFF varia o kern/153497 net [netgraph] netgraph panic due to race conditions o kern/153454 net [patch] [wlan] [urtw] Support ad-hoc and hostap modes o kern/153308 net [em] em interface use 100% cpu o kern/153244 net [em] em(4) fails to send UDP to port 0xffff o kern/152893 net [netgraph] [panic] 8.2-PRERELEASE panic in netgraph o kern/152853 net [em] tftpd (and likely other udp traffic) fails over e o kern/152828 net [em] poor performance on 8.1, 8.2-PRE o kern/152569 net [net]: Multiple ppp connections and routing table prob o kern/152235 net [arp] Permanent local ARP entries are not properly upd o kern/152141 net [vlan] [patch] encapsulate vlan in ng_ether before out o kern/152036 net [libc] getifaddrs(3) returns truncated sockaddrs for n o kern/151690 net [ep] network connectivity won't work until dhclient is o kern/151681 net [nfs] NFS mount via IPv6 leads to hang on client with o kern/151593 net [igb] [panic] Kernel panic when bringing up igb networ o kern/150920 net [ixgbe][igb] Panic when packets are dropped with heade o kern/150557 net [igb] igb0: Watchdog timeout -- resetting o kern/150251 net [patch] [ixgbe] Late cable insertion broken o kern/150249 net [ixgbe] Media type detection broken o bin/150224 net ppp(8) does not reassign static IP after kill -KILL co f kern/149969 net [wlan] [ral] ralink rt2661 fails to maintain connectio o kern/149643 net [rum] device not sending proper beacon frames in ap mo o kern/149609 net [panic] reboot after adding second default route o kern/149117 net [inet] [patch] in_pcbbind: redundant test o kern/149086 net [multicast] Generic multicast join failure in 8.1 o kern/148018 net [flowtable] flowtable crashes on ia64 o kern/147912 net [boot] FreeBSD 8 Beta won't boot on Thinkpad i1300 11 o kern/147894 net [ipsec] IPv6-in-IPv4 does not work inside an ESP-only o kern/147155 net [ip6] setfb not work with ipv6 o kern/146845 net [libc] close(2) returns error 54 (connection reset by f kern/146792 net [flowtable] flowcleaner 100% cpu's core load o kern/146719 net [pf] [panic] PF or dumynet kernel panic o kern/146534 net [icmp6] wrong source address in echo reply o kern/146427 net [mwl] Additional virtual access points don't work on m f kern/146394 net [vlan] IP source address for outgoing connections o bin/146377 net [ppp] [tun] Interface doesn't clear addresses when PPP o kern/146358 net [vlan] wrong destination MAC address o kern/146165 net [wlan] [panic] Setting bssid in adhoc mode causes pani o kern/146082 net [ng_l2tp] a false invaliant check was performed in ng_ o kern/146037 net [panic] mpd + CoA = kernel panic o kern/145825 net [panic] panic: soabort: so_count o kern/145728 net [lagg] Stops working lagg between two servers. p kern/145600 net TCP/ECN behaves different to CE/CWR than ns2 reference f kern/144917 net [flowtable] [panic] flowtable crashes system [regressi o kern/144882 net MacBookPro =>4.1 does not connect to BSD in hostap wit o kern/144874 net [if_bridge] [patch] if_bridge frees mbuf after pfil ho o conf/144700 net [rc.d] async dhclient breaks stuff for too many people o kern/144616 net [nat] [panic] ip_nat panic FreeBSD 7.2 f kern/144315 net [ipfw] [panic] freebsd 8-stable reboot after add ipfw o kern/144231 net bind/connect/sendto too strict about sockaddr length o kern/143846 net [gif] bringing gif3 tunnel down causes gif0 tunnel to s kern/143673 net [stf] [request] there should be a way to support multi s kern/143666 net [ip6] [request] PMTU black hole detection not implemen o kern/143622 net [pfil] [patch] unlock pfil lock while calling firewall o kern/143593 net [ipsec] When using IPSec, tcpdump doesn't show outgoin o kern/143591 net [ral] RT2561C-based DLink card (DWL-510) fails to work o kern/143208 net [ipsec] [gif] IPSec over gif interface not working o kern/143034 net [panic] system reboots itself in tcp code [regression] o kern/142877 net [hang] network-related repeatable 8.0-STABLE hard hang o kern/142774 net Problem with outgoing connections on interface with mu o kern/142772 net [libc] lla_lookup: new lle malloc failed f kern/142518 net [em] [lagg] Problem on 8.0-STABLE with em and lagg o kern/142018 net [iwi] [patch] Possibly wrong interpretation of beacon- o kern/141861 net [wi] data garbled with WEP and wi(4) with Prism 2.5 f kern/141741 net Etherlink III NIC won't work after upgrade to FBSD 8, o kern/140742 net rum(4) Two asus-WL167G adapters cannot talk to each ot o kern/140682 net [netgraph] [panic] random panic in netgraph f kern/140634 net [vlan] destroying if_lagg interface with if_vlan membe o kern/140619 net [ifnet] [patch] refine obsolete if_var.h comments desc o kern/140346 net [wlan] High bandwidth use causes loss of wlan connecti o kern/140142 net [ip6] [panic] FreeBSD 7.2-amd64 panic w/IPv6 o kern/140066 net [bwi] install report for 8.0 RC 2 (multiple problems) o kern/139387 net [ipsec] Wrong lenth of PF_KEY messages in promiscuous o bin/139346 net [patch] arp(8) add option to remove static entries lis o kern/139268 net [if_bridge] [patch] allow if_bridge to forward just VL p kern/139204 net [arp] DHCP server replies rejected, ARP entry lost bef o kern/139117 net [lagg] + wlan boot timing (EBUSY) o kern/138850 net [dummynet] dummynet doesn't work correctly on a bridge o kern/138782 net [panic] sbflush_internal: cc 0 || mb 0xffffff004127b00 o kern/138688 net [rum] possibly broken on 8 Beta 4 amd64: able to wpa a o kern/138678 net [lo] FreeBSD does not assign linklocal address to loop o kern/138407 net [gre] gre(4) interface does not come up after reboot o kern/138332 net [tun] [lor] ifconfig tun0 destroy causes LOR if_adata/ o kern/138266 net [panic] kernel panic when udp benchmark test used as r f kern/138029 net [bpf] [panic] periodically kernel panic and reboot o kern/137881 net [netgraph] [panic] ng_pppoe fatal trap 12 p bin/137841 net [patch] wpa_supplicant(8) cannot verify SHA256 signed p kern/137776 net [rum] panic in rum(4) driver on 8.0-BETA2 o bin/137641 net ifconfig(8): various problems with "vlan_device.vlan_i o kern/137392 net [ip] [panic] crash in ip_nat.c line 2577 o kern/137372 net [ral] FreeBSD doesn't support wireless interface from o kern/137089 net [lagg] lagg falsely triggers IPv6 duplicate address de o kern/136911 net [netgraph] [panic] system panic on kldload ng_bpf.ko t o kern/136618 net [pf][stf] panic on cloning interface without unit numb o kern/135502 net [periodic] Warning message raised by rtfree function i o kern/134583 net [hang] Machine with jail freezes after random amount o o kern/134531 net [route] [panic] kernel crash related to routes/zebra o kern/134157 net [dummynet] dummynet loads cpu for 100% and make a syst o kern/133969 net [dummynet] [panic] Fatal trap 12: page fault while in o kern/133968 net [dummynet] [panic] dummynet kernel panic o kern/133736 net [udp] ip_id not protected ... o kern/133595 net [panic] Kernel Panic at pcpu.h:195 o kern/133572 net [ppp] [hang] incoming PPTP connection hangs the system o kern/133490 net [bpf] [panic] 'kmem_map too small' panic on Dell r900 o kern/133235 net [netinet] [patch] Process SIOCDLIFADDR command incorre f kern/133213 net arp and sshd errors on 7.1-PRERELEASE o kern/133060 net [ipsec] [pfsync] [panic] Kernel panic with ipsec + pfs o kern/132889 net [ndis] [panic] NDIS kernel crash on load BCM4321 AGN d o conf/132851 net [patch] rc.conf(5): allow to setfib(1) for service run o kern/132734 net [ifmib] [panic] panic in net/if_mib.c o kern/132705 net [libwrap] [patch] libwrap - infinite loop if hosts.all o kern/132672 net [ndis] [panic] ndis with rt2860.sys causes kernel pani o kern/132354 net [nat] Getting some packages to ipnat(8) causes crash o kern/132277 net [crypto] [ipsec] poor performance using cryptodevice f o kern/131781 net [ndis] ndis keeps dropping the link o kern/131776 net [wi] driver fails to init o kern/131753 net [altq] [panic] kernel panic in hfsc_dequeue o bin/131365 net route(8): route add changes interpretation of network f kern/130820 net [ndis] wpa_supplicant(8) returns 'no space on device' o kern/130628 net [nfs] NFS / rpc.lockd deadlock on 7.1-R o kern/130525 net [ndis] [panic] 64 bit ar5008 ndisgen-erated driver cau o kern/130311 net [wlan_xauth] [panic] hostapd restart causing kernel pa o kern/130109 net [ipfw] Can not set fib for packets originated from loc f kern/130059 net [panic] Leaking 50k mbufs/hour f kern/129719 net [nfs] [panic] Panic during shutdown, tcp_ctloutput: in o kern/129517 net [ipsec] [panic] double fault / stack overflow f kern/129508 net [carp] [panic] Kernel panic with EtherIP (may be relat o kern/129219 net [ppp] Kernel panic when using kernel mode ppp o kern/129197 net [panic] 7.0 IP stack related panic o bin/128954 net ifconfig(8) deletes valid routes o bin/128602 net [an] wpa_supplicant(8) crashes with an(4) o kern/128448 net [nfs] 6.4-RC1 Boot Fails if NFS Hostname cannot be res o bin/128295 net [patch] ifconfig(8) does not print TOE4 or TOE6 capabi o bin/128001 net wpa_supplicant(8), wlan(4), and wi(4) issues o kern/127826 net [iwi] iwi0 driver has reduced performance and connecti o kern/127815 net [gif] [patch] if_gif does not set vlan attributes from o kern/127724 net [rtalloc] rtfree: 0xc5a8f870 has 1 refs f bin/127719 net [arp] arp: Segmentation fault (core dumped) f kern/127528 net [icmp]: icmp socket receives icmp replies not owned by p kern/127360 net [socket] TOE socket options missing from sosetopt() o bin/127192 net routed(8) removes the secondary alias IP of interface f kern/127145 net [wi]: prism (wi) driver crash at bigger traffic o kern/126895 net [patch] [ral] Add antenna selection (marked as TBD) o kern/126874 net [vlan]: Zebra problem if ifconfig vlanX destroy o kern/126695 net rtfree messages and network disruption upon use of if_ o kern/126339 net [ipw] ipw driver drops the connection o kern/126075 net [inet] [patch] internet control accesses beyond end of o bin/125922 net [patch] Deadlock in arp(8) o kern/125920 net [arp] Kernel Routing Table loses Ethernet Link status o kern/125845 net [netinet] [patch] tcp_lro_rx() should make use of hard o kern/125258 net [socket] socket's SO_REUSEADDR option does not work o kern/125239 net [gre] kernel crash when using gre o kern/124341 net [ral] promiscuous mode for wireless device ral0 looses o kern/124225 net [ndis] [patch] ndis network driver sometimes loses net o kern/124160 net [libc] connect(2) function loops indefinitely o kern/124021 net [ip6] [panic] page fault in nd6_output() o kern/123968 net [rum] [panic] rum driver causes kernel panic with WPA. o kern/123892 net [tap] [patch] No buffer space available o kern/123890 net [ppp] [panic] crash & reboot on work with PPP low-spee o kern/123858 net [stf] [patch] stf not usable behind a NAT o kern/123758 net [panic] panic while restarting net/freenet6 o bin/123633 net ifconfig(8) doesn't set inet and ether address in one o kern/123559 net [iwi] iwi periodically disassociates/associates [regre o bin/123465 net [ip6] route(8): route add -inet6 -interfac o kern/123463 net [ipsec] [panic] repeatable crash related to ipsec-tool o conf/123330 net [nsswitch.conf] Enabling samba wins in nsswitch.conf c o kern/123160 net [ip] Panic and reboot at sysctl kern.polling.enable=0 o kern/122989 net [swi] [panic] 6.3 kernel panic in swi1: net o kern/122954 net [lagg] IPv6 EUI64 incorrectly chosen for lagg devices f kern/122780 net [lagg] tcpdump on lagg interface during high pps wedge o kern/122685 net It is not visible passing packets in tcpdump(1) o kern/122319 net [wi] imposible to enable ad-hoc demo mode with Orinoco o kern/122290 net [netgraph] [panic] Netgraph related "kmem_map too smal o kern/122252 net [ipmi] [bge] IPMI problem with BCM5704 (does not work o kern/122033 net [ral] [lor] Lock order reversal in ral0 at bootup ieee o bin/121895 net [patch] rtsol(8)/rtsold(8) doesn't handle managed netw s kern/121774 net [swi] [panic] 6.3 kernel panic in swi1: net o kern/121555 net [panic] Fatal trap 12: current process = 12 (swi1: net o kern/121534 net [ipl] [nat] FreeBSD Release 6.3 Kernel Trap 12: o kern/121443 net [gif] [lor] icmp6_input/nd6_lookup o kern/121437 net [vlan] Routing to layer-2 address does not work on VLA o bin/121359 net [patch] [security] ppp(8): fix local stack overflow in o kern/121257 net [tcp] TSO + natd -> slow outgoing tcp traffic o kern/121181 net [panic] Fatal trap 3: breakpoint instruction fault whi o kern/120966 net [rum] kernel panic with if_rum and WPA encryption o kern/120566 net [request]: ifconfig(8) make order of arguments more fr o kern/120304 net [netgraph] [patch] netgraph source assumes 32-bit time o kern/120266 net [udp] [panic] gnugk causes kernel panic when closing U o bin/120060 net routed(8) deletes link-level routes in the presence of o kern/119945 net [rum] [panic] rum device in hostap mode, cause kernel o kern/119791 net [nfs] UDP NFS mount of aliased IP addresses from a Sol o kern/119617 net [nfs] nfs error on wpa network when reseting/shutdown f kern/119516 net [ip6] [panic] _mtx_lock_sleep: recursed on non-recursi o kern/119432 net [arp] route add -host -iface causes arp e o kern/119225 net [wi] 7.0-RC1 no carrier with Prism 2.5 wifi card [regr o kern/118727 net [netgraph] [patch] [request] add new ng_pf module o kern/117423 net [vlan] Duplicate IP on different interfaces o bin/117339 net [patch] route(8): loading routing management commands o bin/116643 net [patch] [request] fstat(1): add INET/INET6 socket deta o kern/116185 net [iwi] if_iwi driver leads system to reboot o kern/115239 net [ipnat] panic with 'kmem_map too small' using ipnat o kern/115019 net [netgraph] ng_ether upper hook packet flow stops on ad o kern/115002 net [wi] if_wi timeout. failed allocation (busy bit). ifco o kern/114915 net [patch] [pcn] pcn (sys/pci/if_pcn.c) ethernet driver f o kern/113432 net [ucom] WARNING: attempt to net_add_domain(netgraph) af o kern/112722 net [ipsec] [udp] IP v4 udp fragmented packet reject o kern/112686 net [patm] patm driver freezes System (FreeBSD 6.2-p4) i38 o bin/112557 net [patch] ppp(8) lock file should not use symlink name o kern/112528 net [nfs] NFS over TCP under load hangs with "impossible p o kern/111537 net [inet6] [patch] ip6_input() treats mbuf cluster wrong o kern/111457 net [ral] ral(4) freeze o kern/110284 net [if_ethersubr] Invalid Assumption in SIOCSIFADDR in et o kern/110249 net [kernel] [regression] [patch] setsockopt() error regre o kern/109470 net [wi] Orinoco Classic Gold PC Card Can't Channel Hop o bin/108895 net pppd(8): PPPoE dead connections on 6.2 [regression] f kern/108197 net [panic] [gif] [ip6] if_delmulti reference counting pan o kern/107944 net [wi] [patch] Forget to unlock mutex-locks o conf/107035 net [patch] bridge(8): bridge interface given in rc.conf n o kern/106444 net [netgraph] [panic] Kernel Panic on Binding to an ip to o kern/106316 net [dummynet] dummynet with multipass ipfw drops packets o kern/105945 net Address can disappear from network interface s kern/105943 net Network stack may modify read-only mbuf chain copies o bin/105925 net problems with ifconfig(8) and vlan(4) [regression] o kern/104851 net [inet6] [patch] On link routes not configured when usi o kern/104751 net [netgraph] kernel panic, when getting info about my tr o kern/104738 net [inet] [patch] Reentrant problem with inet_ntoa in the o kern/103191 net Unpredictable reboot o kern/103135 net [ipsec] ipsec with ipfw divert (not NAT) encodes a pac o kern/102540 net [netgraph] [patch] supporting vlan(4) by ng_fec(4) o conf/102502 net [netgraph] [patch] ifconfig name does't rename netgrap o kern/102035 net [plip] plip networking disables parallel port printing o kern/100709 net [libc] getaddrinfo(3) should return TTL info o kern/100519 net [netisr] suggestion to fix suboptimal network polling o kern/98597 net [inet6] Bug in FreeBSD 6.1 IPv6 link-local DAD procedu o bin/98218 net wpa_supplicant(8) blacklist not working o kern/97306 net [netgraph] NG_L2TP locks after connection with failed o conf/97014 net [gif] gifconfig_gif? in rc.conf does not recognize IPv f kern/96268 net [socket] TCP socket performance drops by 3000% if pack o kern/95519 net [ral] ral0 could not map mbuf o kern/95288 net [pppd] [tty] [panic] if_ppp panic in sys/kern/tty_subr o kern/95277 net [netinet] [patch] IP Encapsulation mask_match() return o kern/95267 net packet drops periodically appear f kern/93378 net [tcp] Slow data transfer in Postfix and Cyrus IMAP (wo o kern/93019 net [ppp] ppp and tunX problems: no traffic after restarti o kern/92880 net [libc] [patch] almost rewritten inet_network(3) functi s kern/92279 net [dc] Core faults everytime I reboot, possible NIC issu o kern/91859 net [ndis] if_ndis does not work with Asus WL-138 o kern/91364 net [ral] [wep] WF-511 RT2500 Card PCI and WEP o kern/91311 net [aue] aue interface hanging o kern/87421 net [netgraph] [panic]: ng_ether + ng_eiface + if_bridge o kern/86871 net [tcp] [patch] allocation logic for PCBs in TIME_WAIT s o kern/86427 net [lor] Deadlock with FASTIPSEC and nat o kern/85780 net 'panic: bogus refcnt 0' in routing/ipv6 o bin/85445 net ifconfig(8): deprecated keyword to ifconfig inoperativ o bin/82975 net route change does not parse classfull network as given o kern/82881 net [netgraph] [panic] ng_fec(4) causes kernel panic after o kern/82468 net Using 64MB tcp send/recv buffers, trafficflow stops, i o bin/82185 net [patch] ndp(8) can delete the incorrect entry o kern/81095 net IPsec connection stops working if associated network i o kern/78968 net FreeBSD freezes on mbufs exhaustion (network interface o kern/78090 net [ipf] ipf filtering on bridged packets doesn't work if o kern/77341 net [ip6] problems with IPV6 implementation o kern/75873 net Usability problem with non-RFC-compliant IP spoof prot s kern/75407 net [an] an(4): no carrier after short time a kern/71474 net [route] route lookup does not skip interfaces marked d o kern/71469 net default route to internet magically disappears with mu o kern/68889 net [panic] m_copym, length > size of mbuf chain o kern/66225 net [netgraph] [patch] extend ng_eiface(4) control message o kern/65616 net IPSEC can't detunnel GRE packets after real ESP encryp s kern/60293 net [patch] FreeBSD arp poison patch a kern/56233 net IPsec tunnel (ESP) over IPv6: MTU computation is wrong s bin/41647 net ifconfig(8) doesn't accept lladdr along with inet addr o kern/39937 net ipstealth issue a kern/38554 net [patch] changing interface ipaddress doesn't seem to w o kern/31940 net ip queue length too short for >500kpps o kern/31647 net [libc] socket calls can return undocumented EINVAL o kern/30186 net [libc] getaddrinfo(3) does not handle incorrect servna f kern/24959 net [patch] proper TCP_NOPUSH/TCP_CORK compatibility o conf/23063 net [arp] [patch] for static ARP tables in rc.network o kern/21998 net [socket] [patch] ident only for outgoing connections o kern/5877 net [socket] sb_cc counts control data as well as data dat 463 problems total. From owner-freebsd-net@FreeBSD.ORG Mon Sep 2 12:47:25 2013 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 09BD0A1C for ; Mon, 2 Sep 2013 12:47:25 +0000 (UTC) (envelope-from barney_cordoba@yahoo.com) Received: from nm5-vm5.bullet.mail.ne1.yahoo.com (nm5-vm5.bullet.mail.ne1.yahoo.com [98.138.91.227]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B6BA72C93 for ; Mon, 2 Sep 2013 12:47:24 +0000 (UTC) Received: from [98.138.90.57] by nm5.bullet.mail.ne1.yahoo.com with NNFMP; 02 Sep 2013 12:47:18 -0000 Received: from [98.138.89.170] by tm10.bullet.mail.ne1.yahoo.com with NNFMP; 02 Sep 2013 12:47:18 -0000 Received: from [127.0.0.1] by omp1026.mail.ne1.yahoo.com with NNFMP; 02 Sep 2013 12:47:18 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 69213.72948.bm@omp1026.mail.ne1.yahoo.com Received: (qmail 56592 invoked by uid 60001); 2 Sep 2013 12:47:18 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1378126037; bh=NlR6CvQ7sW4XKZvv5yuY6sYfGhYkwR8F50gQvexzxEs=; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:References:Message-ID:Date:From:Reply-To:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=wbUdaq/b2NfORJmBZa4Edz4kyfA75CK0oqIu7lgtSt3avuXgntAZELMHf6XzTxRAdRVseUc/lG07E62V7sKvXYUAvdXI5qFHmtTUfcL7hZANEz41wDVWD5g/nx3Khe6i7wpad6rupoM1d+3ZFPKl9Wn8XM3BadKeNpYyGLLyUcA= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:References:Message-ID:Date:From:Reply-To:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=QIYPxo2bnectl5mblGEq7u0J/+0xxJsO6uHnshu6KGqY7Q9nHuGgHw9pgD7lzXfA1W3ntvrejmlglIjwRNfSuPVEDWY46L3IYP+uWNqHWBEDawpQkl7+rabR6R5hPOernb6XltxrCultkZ+qGVsmYFGyl6spSvdvJGtJR+HKAQA=; X-YMail-OSG: S1POVpkVM1lokt2FACqMN1phGlmIIfQmROHz0GUnPc36Uch hxCmxQYzLIK4_OpPTPnpzUsJGK0owRZUVWt4FupX63MFWFnoBtH6CbvUZdHQ SIZ1KD4JSTyGkJ8SSmf4Jpxc7.90vHQANsYGYzntwKfO310PDLEqkEJ_L53J PHZ74mR17feOcZK89HNeSPDog3hj3_sVdri3ew2iYm42OOzM6vfp0788SYzm ZtNXAWUqX9UqFCKXp_jLPfFIldGwuzWY8MqMLkVIcWdg3NySX8lNimJ2aX1Y _8rOtLEPyGxKHwfI5Ta2TJ2atlbIuBm5Hi3mM0lcJdcQbXJsyIhKyUejtKHk lQi.F.gMWXp1ILe4TYKGF5aQrwv1K_1U32yod4VPNcQoKH46DN9DTj5aCESy aOR2kgAgd7lsUNFcFALKRA4pB1.HutvOtwcA8pOpidKi.ynW462OIEWAY6jM ekgn24mjkKbYmTjxn4UuQm4RerU6Kw0oLRtbqOQ4f7FM6A8OdyobCmSAkASR _xY7ZCYGi2L2MNi3FC7W7aiF9o2n6mq2YA.Tv0KxvccQnXlGKW.XfsbsTiai y81SrnIkOt7DpQWox Received: from [98.203.118.124] by web121603.mail.ne1.yahoo.com via HTTP; Mon, 02 Sep 2013 05:47:17 PDT X-Rocket-MIMEInfo: 002.001, QXJlIHlvdSB1c2luZyBhIHBjaWUzIGJ1cz8gT2YgY291cnNlIHRoaXMgaXMgb25seSBhbiBpc3N1ZSBmb3IgMTBnOyB3aGF0IHBjdCBvZgpGcmVlQlNEIHVzZXJzIGhhdmUgYSBsb2FkIG92ZXIgOS41R2Ivcz8gSXQncyBjb21wbGV0ZWx5IHVubmVjZXNzYXJ5IGZvciBpZ2IKb3IgZW0gZHJpdmVyLCBzbyB3aHkgaXMgaXQgdXNlZD8gYmVjYXVzZSBpdCdzIHRoZXJlLgoKSGVyZSdzIG15IGFyZ3VtZW50IGFnYWluc3QgaXQuIFRoZSBoYW5kZnVsIG9mIGJyYWlucyBjYXBhYmxlIG9mIGRvaW5nIGRyaXZlciBkZXYBMAEBAQE- X-Mailer: YahooMailWebService/0.8.156.576 References: <521BBD21.4070304@freebsd.org> <521EE8DA.3060107@freebsd.org> <1377952913.44129.YahooMailNeo@web121605.mail.ne1.yahoo.com> <1378001733.36695.YahooMailNeo@web121606.mail.ne1.yahoo.com> <1378050319.62710.YahooMailNeo@web121601.mail.ne1.yahoo.com> Message-ID: <1378126037.56348.YahooMailNeo@web121603.mail.ne1.yahoo.com> Date: Mon, 2 Sep 2013 05:47:17 -0700 (PDT) From: Barney Cordoba Subject: Re: Flow ID, LACP, and igb To: Adrian Chadd In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: Andre Oppermann , Alan Somers , "net@freebsd.org" , Jack F Vogel , "Justin T. Gibbs" , Luigi Rizzo , "T.C. Gubatayao" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: Barney Cordoba List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Sep 2013 12:47:25 -0000 Are you using a pcie3 bus? Of course this is only an issue for 10g; what pc= t of=0AFreeBSD users have a load over 9.5Gb/s? It's completely unnecessary = for igb=0Aor em driver, so why is it used? because it's there.=0A=0AHere's = my argument against it. The handful of brains capable of doing driver devel= opment=0Abecome consumed with BS like LRO and the things that need to be fi= xed, like=0Abuffer management and basic driver design flaws, never get fixe= d. The offload=0Acode makes the driver code a virtual mess that can only be= maintained by Jack and=0A1 other guy in the entire world. And it takes 10 = times longer to make a simple change or=0Ato add support for a new NIC.=A0= =0A=0AIn a week I ripped out the offload crap and the 9000 sysctls, elimina= ted the=A0=0A"consumer buffer" problem, reduced locking by 40% and now the = igb driver=0Auses 20% less cpu with a full gig load.=0A=0AAnd the code is c= leaner and more easily maintained.=0A=0ABC=0A=0A=0A________________________= ________=0A From: Adrian Chadd =0ATo: Barney Cordoba =0ACc: Andre Oppermann ; Alan S= omers ; "net@freebsd.org" ; Jack F Vo= gel ; Justin T. Gibbs ; Luigi Rizzo ; T.C. Gubatayao =0ASent: Sunda= y, September 1, 2013 4:51 PM=0ASubject: Re: Flow ID, LACP, and igb=0A =0A= =0AYo,=0A=0ALRO is an interesting hack that seems to do a good trick of hid= ing the=0Aridiculous locking and unfriendly cache behaviour that we do per-= packet.=0A=0AIt helps with LAN test traffic where things are going out in b= atches from=0Athe TCP layer so the RX layer "sees" these frames in-order an= d can do LRO.=0AWhen you disable it, I don't easily get 10GE LAN TCP perfor= mance. That has=0Ato be fixed. Given how fast the CPU cores, bus interconne= ct and memory=0Ainterconnects are, I don't think there should be any reason= why we can't=0Ahit 10GE traffic on a LAN with LRO disabled (in both softwa= re and hardware.)=0A=0ANow that I have the PMC sandy bridge stuff working r= ight (but no PEBS, I=0Ahave to talk to Intel about that in a bit more detai= l before I think about=0Ahacking that in) we can get actual live informatio= n about this stuff. But=0Athe last time I looked, there's just too much per= -packet latency going on.=0AThe root cause looks like it's a toss up betwee= n scheduling, locking and=0Ajust lots of code running to completion per-fra= me. As I said, that all has=0Ato die somehow.=0A=0A2c,=0A=0A=0A=0A-adrian= =0A=0A=0A=0AOn 1 September 2013 08:45, Barney Cordoba wrote:=0A=0A>=0A>=0A> Comcast sends packets OOO. With any decent numb= er of internet hops you're=0A> likely to encounter a load=0A> balancer or p= acket shaper that sends packets OOO, so you just can't be=0A> worried about= it. In fact, your=0A> designs MUST work with OOO packets.=0A>=0A> Getting = balance on your load balanced lines is certainly a bigger upside=0A> than t= he additional CPU used.=0A> You can buy a faster processor for your "stack"= for a lot less than you=0A> can buy bandwidth.=0A>=0A> Frankly my opinion = of LRO is that it's a science project suitable for labs=0A> only. It's a tr= ick to get more bandwidth=0A> than your bus capacity; the answer is to not = run PCIe2 if you need pcie3.=0A> You can use it internally if you have=0A> = control of all of the machines. When I modify a driver the first thing=0A> = that I do is rip it out.=0A>=0A> BC=0A>=0A>=0A> ___________________________= _____=0A>=A0 From: Luigi Rizzo =0A> To: Barney Cordoba = =0A> Cc: Andre Oppermann ; Ala= n Somers ;=0A> "net@freebsd.org" ; Ja= ck F Vogel ;=0A> Justin T. Gibbs ; T.C.= Gubatayao <=0A> tgubatayao@barracuda.com>=0A> Sent: Saturday, August 31, 2= 013 10:27 PM=0A> Subject: Re: Flow ID, LACP, and igb=0A>=0A>=0A> On Sun, Se= p 1, 2013 at 4:15 AM, Barney Cordoba >wrote:= =0A>=0A> > ...=0A> >=0A>=0A> [your point on testing with realistic assumpti= ons is surely a valid one]=0A>=0A>=0A> >=0A> > Of course there's nothing re= ally wrong with OOO packets. We had this=0A> > discussion before; lots of p= eople=0A> > have round robin dual homing without any ill effects. It's just= not an=0A> > issue.=0A> >=0A>=0A> It depends on where you are.=0A> It may = not be an issue if the reordering is not large enough to=0A> trigger retran= smissions, but even then it is annoying as it causes=0A> more work in the e= ndpoint -- it prevents LRO from working, and even=0A> on the host stack it = takes more work to sort where an out of order=0A> segment goes than appendi= ng an in-order one to the socket buffer.=0A>=0A> cheers=0A> luigi=0A> _____= __________________________________________=0A> freebsd-net@freebsd.org mail= ing list=0A> http://lists.freebsd.org/mailman/listinfo/freebsd-net=0A> To u= nsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"=0A> ____= ___________________________________________=0A> freebsd-net@freebsd.org mai= ling list=0A> http://lists.freebsd.org/mailman/listinfo/freebsd-net=0A> To = unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"=0A>=0A_= ______________________________________________=0Afreebsd-net@freebsd.org ma= iling list=0Ahttp://lists.freebsd.org/mailman/listinfo/freebsd-net=0ATo uns= ubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Mon Sep 2 13:01:42 2013 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 76939F19 for ; Mon, 2 Sep 2013 13:01:42 +0000 (UTC) (envelope-from cochard@gmail.com) Received: from mail-vc0-x22a.google.com (mail-vc0-x22a.google.com [IPv6:2607:f8b0:400c:c03::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 310C32DA4 for ; Mon, 2 Sep 2013 13:01:42 +0000 (UTC) Received: by mail-vc0-f170.google.com with SMTP id kw10so3130690vcb.1 for ; Mon, 02 Sep 2013 06:01:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc:content-type; bh=P9soiYOWUfiJA7hVZIASjyIoEFWYMYH84pdkSMMvMwE=; b=o8mp20tS0DUMKB3HxoBi96y26u+Kz7wORmWXcy8mdEM/ODtAXAPQ3C1hNps+YA59p8 PJaLXRqIbNbnxT99u9k/iuT7iLSA2tyecc29GpOWIl9ZGdaki38Yu5P0yPVfvIQ7MbrJ uPDHfHhEbpmkjJodl3dmXr6eM14HGkNRXL7bYAvAT0f4CVSeO6RtocarOUNe50j0MKTY Wg9X4usknZG+3HmVKC+TDHeFlD6FLxwjCQnF/FETa4mXm50DXp+2t2YJfMBzyHGQBgeK yVTxewFNmy3MRd1cRROSL677oVv3zrDSbV/ah6mjtz17S/Qa17QyQSH1FpOIC4o0NKLk kDWA== X-Received: by 10.52.52.231 with SMTP id w7mr10224301vdo.12.1378126901227; Mon, 02 Sep 2013 06:01:41 -0700 (PDT) MIME-Version: 1.0 Sender: cochard@gmail.com Received: by 10.58.221.9 with HTTP; Mon, 2 Sep 2013 06:01:21 -0700 (PDT) In-Reply-To: <1378126037.56348.YahooMailNeo@web121603.mail.ne1.yahoo.com> References: <521BBD21.4070304@freebsd.org> <521EE8DA.3060107@freebsd.org> <1377952913.44129.YahooMailNeo@web121605.mail.ne1.yahoo.com> <1378001733.36695.YahooMailNeo@web121606.mail.ne1.yahoo.com> <1378050319.62710.YahooMailNeo@web121601.mail.ne1.yahoo.com> <1378126037.56348.YahooMailNeo@web121603.mail.ne1.yahoo.com> From: =?ISO-8859-1?Q?Olivier_Cochard=2DLabb=E9?= Date: Mon, 2 Sep 2013 15:01:21 +0200 X-Google-Sender-Auth: X9i8ieHUeOuPZJPbqCmbry5xRZc Message-ID: Subject: Re: Flow ID, LACP, and igb To: Barney Cordoba Content-Type: text/plain; charset=ISO-8859-1 Cc: "net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Sep 2013 13:01:42 -0000 On Mon, Sep 2, 2013 at 2:47 PM, Barney Cordoba wrote: > > In a week I ripped out the offload crap and the 9000 sysctls, eliminated the > "consumer buffer" problem, reduced locking by 40% and now the igb driver > uses 20% less cpu with a full gig load. > Wow! where is the patch ? I would like to test it too. Thanks, Olivier From owner-freebsd-net@FreeBSD.ORG Mon Sep 2 19:55:07 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 89F27C21 for ; Mon, 2 Sep 2013 19:55:07 +0000 (UTC) (envelope-from rmcintosh@nitemare.net) Received: from mail-qe0-f53.google.com (mail-qe0-f53.google.com [209.85.128.53]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4C9192E8F for ; Mon, 2 Sep 2013 19:55:06 +0000 (UTC) Received: by mail-qe0-f53.google.com with SMTP id 1so2353680qee.40 for ; Mon, 02 Sep 2013 12:55:00 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:date:message-id:subject:from:to :content-type; bh=siFuYm6qg4BDDymYO52WHbaZTLNCNBtZSunA8qP9+sw=; b=cAQgNdnt7vdnakYKwnUm5aSRXAEy/Lpv7m1qqwe2Ua6DJSCZzLHC3BYqfRi2GIoa1j IASZv2MzZSspg5SvG/PZ/Oa8XaNA0Zw1w2T7QKRGtJDtENvMMjC20JQ9y52anYHHOU8C Q+WEFymapjutduO7mA7mLTW+UwC7GFuoAw2dGhvHSs8wwfWCSlLgieBbXyrY+i1kL4xL CIqrNezGJdXcGD1cZaDI8Syp0XoRXqHrjQu/espNQZg9d2aS/QXm4Ec9PWcd1fbJMsAe xbH1CLQfQXsj+hw0v4uwPQDEaTagyuin8Ujr8fBOwaCJ2HxMBDa4w72WywHS1en8PMhN 3/hA== X-Gm-Message-State: ALoCoQnUCvSxhe56WBPfwIY74uR9F+D3P32GUnSqCNGhPzfCm+hhCqp5dUc+PSPkBQnCL8qSvrSr MIME-Version: 1.0 X-Received: by 10.49.116.178 with SMTP id jx18mr5061768qeb.81.1378151700307; Mon, 02 Sep 2013 12:55:00 -0700 (PDT) Received: by 10.49.60.10 with HTTP; Mon, 2 Sep 2013 12:55:00 -0700 (PDT) X-Originating-IP: [2001:470:8ca5:2001:7529:198d:3be5:bdd2] Date: Mon, 2 Sep 2013 15:55:00 -0400 Message-ID: Subject: QLE3142-CU-CK driver? From: Ryan McIntosh To: freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Sep 2013 19:55:07 -0000 NetXen Part #: NX3-20GCU pciconf -lv shows: none2@pci0:1:0:0: class=0x020000 card=0x01264040 chip=0x01004040 rev=0x42 hdr=0x00 vendor = 'NetXen Incorporated' device = 'NX3031 Multifunction 1/10-Gigabit Server Adapter' class = network subclass = ethernet none3@pci0:1:0:1: class=0x020000 card=0x01264040 chip=0x01004040 rev=0x42 hdr=0x00 vendor = 'NetXen Incorporated' device = 'NX3031 Multifunction 1/10-Gigabit Server Adapter' class = network subclass = ethernet I found opensolaris has a CDDL driver; however I have no idea how involved that would be to convert without looking indepth or if its even legally possible. http://fxr.watson.org/fxr/source/common/io/ntxn/?v=OPENSOLARIS I've done considerable amount of searching to this particular card and turned up that there's no driver/associations for it yet for FreeBSD. Is there anyway I can assist in getting this built? I reached out to Qlogic with the response of "FreeBSD is an unsupported Operating System" (sounds like a generic response as, from what it looks, like _they_ built the qlxgb driver). Let me know anything I can do to help or if this is a lost cause for any particular reason. Thanks in advance. Ryan From owner-freebsd-net@FreeBSD.ORG Mon Sep 2 20:06:56 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 16EBD1B9; Mon, 2 Sep 2013 20:06:56 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from onlyone.friendlyhosting.spb.ru (onlyone.friendlyhosting.spb.ru [IPv6:2a01:4f8:131:60a2::2]) by mx1.freebsd.org (Postfix) with ESMTP id CD5172F45; Mon, 2 Sep 2013 20:06:55 +0000 (UTC) Received: from lion.home.serebryakov.spb.ru (unknown [IPv6:2001:470:923f:1:61fd:95c3:8111:539a]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.friendlyhosting.spb.ru (Postfix) with ESMTPSA id 174184AC2D; Tue, 3 Sep 2013 00:06:48 +0400 (MSK) Date: Tue, 3 Sep 2013 00:06:46 +0400 From: Lev Serebryakov Organization: FreeBSD X-Priority: 3 (Normal) Message-ID: <9510628781.20130903000646@serebryakov.spb.ru> To: Xin Li Subject: Re: Why default route is not installed last? In-Reply-To: <521670FF.6080407@delphij.net> References: <521670FF.6080407@delphij.net> MIME-Version: 1.0 Content-Type: text/plain; charset=windows-1251 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-net@freebsd.org" , freebsd-rc@freebsd.org, d@delphij.net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: lev@FreeBSD.org List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Sep 2013 20:06:56 -0000 Hello, Xin. You wrote 23 =E0=E2=E3=F3=F1=F2=E0 2013 =E3., 0:13:51: XL> I've noticed that we do not install default route last (after other XL> static routes). I think we should probably install it last, since the XL> administrator may legitimately configure a static route (e.g. this XL> IPv6 address goes to this interface) that is required by the default XL> route. It is why I need patch network.subr after each upgrade on all my Hetzner servers: they use IPv6 default route pointed to static route (not link-local one)... --=20 // Black Lion AKA Lev Serebryakov From owner-freebsd-net@FreeBSD.ORG Mon Sep 2 20:43:00 2013 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 64EAB3D4; Mon, 2 Sep 2013 20:43:00 +0000 (UTC) (envelope-from hrs@FreeBSD.org) Received: from mail.allbsd.org (gatekeeper.allbsd.org [IPv6:2001:2f0:104:e001::32]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id D03C22229; Mon, 2 Sep 2013 20:42:59 +0000 (UTC) Received: from alph.d.allbsd.org (p2049-ipbf1102funabasi.chiba.ocn.ne.jp [122.26.101.49]) (authenticated bits=128) by mail.allbsd.org (8.14.5/8.14.5) with ESMTP id r82KgUUv036733 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 3 Sep 2013 05:42:40 +0900 (JST) (envelope-from hrs@FreeBSD.org) Received: from localhost (localhost [IPv6:::1]) (authenticated bits=0) by alph.d.allbsd.org (8.14.5/8.14.5) with ESMTP id r82KgSBS085354; Tue, 3 Sep 2013 05:42:29 +0900 (JST) (envelope-from hrs@FreeBSD.org) Date: Tue, 03 Sep 2013 05:41:45 +0900 (JST) Message-Id: <20130903.054145.729685298814952552.hrs@allbsd.org> To: d@delphij.net, delphij@delphij.net Subject: Re: Why default route is not installed last? From: Hiroki Sato In-Reply-To: <521BA31C.5000807@delphij.net> References: <521B9A1B.7080908@freebsd.org> <521BA31C.5000807@delphij.net> X-PGPkey-fingerprint: BDB3 443F A5DD B3D0 A530 FFD7 4F2C D3D8 2793 CF2D X-Mailer: Mew version 6.5 on Emacs 24.3 / Mule 6.0 (HANACHIRUSATO) Mime-Version: 1.0 Content-Type: Multipart/Signed; protocol="application/pgp-signature"; micalg=pgp-sha1; boundary="--Security_Multipart(Tue_Sep__3_05_41_45_2013_061)--" Content-Transfer-Encoding: 7bit X-Virus-Scanned: clamav-milter 0.97.4 at gatekeeper.allbsd.org X-Virus-Status: Clean X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (mail.allbsd.org [133.31.130.32]); Tue, 03 Sep 2013 05:42:40 +0900 (JST) X-Spam-Status: No, score=-90.5 required=13.0 tests=CONTENT_TYPE_PRESENT, DIRECTOCNDYN,DYN_PBL,RCVD_IN_PBL,SPF_SOFTFAIL,USER_IN_WHITELIST, X_CHINESE_RELAY autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on gatekeeper.allbsd.org Cc: freebsd-rc@FreeBSD.org, kpaasial@gmail.com, freebsd-net@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Sep 2013 20:43:00 -0000 ----Security_Multipart(Tue_Sep__3_05_41_45_2013_061)-- Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Xin Li wrote in <521BA31C.5000807@delphij.net>: de> > That has always been specifically not supported. default route de> > needs to be directly attached. in fact the routing tables only ever de> > deliver the 'next hop' de> de> Well, depends on whether the 'next hop' is an IP or an interface. For de> instance one can have a valid configuration that they have a static de> route of: de> de> 2607:5300:XXXX:XXXX:ff:ff:ff:ff -prefixlen 128 -interface em0 de> de> Then have 2607:5300:XXXX:XXXX:ff:ff:ff:ff as default router. de> de> This configuration is not possible with the current rc.d startup order. Ah, I see. I personally do not like this kind of configurations but it should be supported as a dirty workaround. The patch is correct, so please go ahead with it. -- Hiroki ----Security_Multipart(Tue_Sep__3_05_41_45_2013_061)-- Content-Type: application/pgp-signature Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.13 (FreeBSD) iEYEABECAAYFAlIk+AkACgkQTyzT2CeTzy27hACdGQ87Ml7IBvsa2h+rBaf3F8Mt VSYAoI7s+sRHhqgKKBANWX0npJLDoNc2 =IHF1 -----END PGP SIGNATURE----- ----Security_Multipart(Tue_Sep__3_05_41_45_2013_061)---- From owner-freebsd-net@FreeBSD.ORG Tue Sep 3 00:05:12 2013 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 854D3D21; Tue, 3 Sep 2013 00:05:12 +0000 (UTC) (envelope-from lists@rewt.org.uk) Received: from hosted.mx.as41113.net (abby.lhr1.as41113.net [91.208.177.20]) by mx1.freebsd.org (Postfix) with ESMTP id EA32424FA; Tue, 3 Sep 2013 00:05:10 +0000 (UTC) Received: from jwhlaptop (unknown [91.208.177.70]) (using TLSv1.2 with cipher AES128-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: lists@rewt.org.uk) by hosted.mx.as41113.net (Postfix) with ESMTPSA id 3cTT1m17NYz1Mh; Tue, 3 Sep 2013 01:05:00 +0100 (BST) From: "Joe Holden" To: "'Barney Cordoba'" , "'Adrian Chadd'" References: <521BBD21.4070304@freebsd.org> <521EE8DA.3060107@freebsd.org> <1377952913.44129.YahooMailNeo@web121605.mail.ne1.yahoo.com> <1378001733.36695.YahooMailNeo@web121606.mail.ne1.yahoo.com> <1378050319.62710.YahooMailNeo@web121601.mail.ne1.yahoo.com> <1378126037.56348.YahooMailNeo@web121603.mail.ne1.yahoo.com> In-Reply-To: <1378126037.56348.YahooMailNeo@web121603.mail.ne1.yahoo.com> Subject: RE: Flow ID, LACP, and igb Date: Tue, 3 Sep 2013 01:04:53 +0100 Message-ID: <25fd01cea839$39cbc1a0$ad6344e0$@rewt.org.uk> X-Mailer: Microsoft Outlook 15.0 Thread-Index: AQFi1TQRBLb279T2Rv2tkFTwf9fLJgJpf8UqAiRrO1IBHW2yLwKNzOQLANQ6q8kCYv/xbwH3hdr4AlvyGOUBIldnpwJhCGKaAkN+YYQC66diKQEG3zmDmb7bh6A= Content-Language: en-gb Cc: 'Andre Oppermann' , 'Alan Somers' , net@freebsd.org, 'Jack F Vogel' , "'Justin T. Gibbs'" , 'Luigi Rizzo' , "'T.C. Gubatayao'" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Sep 2013 00:05:12 -0000 Your argument is horseshit on the basis that many x86 and non-x86 (especially mips) usable NICs will happily do linerate (I see you don't understand how network interfaces actually work... that is pps and frame sizes are relevant not throughput) on stock FreeBSD without any tuning whatsoever. Also: a modern Realtek will do higher pps before becoming useless than a 2 or 3 generation old 1000 G/CT. This is *with* PCI-X at 133mhz and 64bit as well as PCIe gen2. You should also consider the people buying interfaces from people like Chelsio (who support FreeBSD rather well considering their customer base includes basically 0 FreeBSD users) who sell 20/80G PCIe interface cards. In reality CPU load is entirely irrelevant since 10G won't bother a decent CPU even with the glaring inefficiencies of the FreeBSD stack - as long as it isn't live locked who cares? Ultimately there are very few driver problems and some quite serious stack design problems which driver behaviour exacerbates. > -----Original Message----- > From: owner-freebsd-net@freebsd.org [mailto:owner-freebsd- > net@freebsd.org] On Behalf Of Barney Cordoba > Sent: 02 September 2013 13:47 > To: Adrian Chadd > Cc: Andre Oppermann; Alan Somers; net@freebsd.org; Jack F Vogel; Justin T. > Gibbs; Luigi Rizzo; T.C. Gubatayao > Subject: Re: Flow ID, LACP, and igb > > Are you using a pcie3 bus? Of course this is only an issue for 10g; what pct of > FreeBSD users have a load over 9.5Gb/s? It's completely unnecessary for igb > or em driver, so why is it used? because it's there. > > Here's my argument against it. The handful of brains capable of doing driver > development become consumed with BS like LRO and the things that need > to be fixed, like buffer management and basic driver design flaws, never get > fixed. The offload code makes the driver code a virtual mess that can only be > maintained by Jack and > 1 other guy in the entire world. And it takes 10 times longer to make a simple > change or to add support for a new NIC. > > In a week I ripped out the offload crap and the 9000 sysctls, eliminated the > "consumer buffer" problem, reduced locking by 40% and now the igb driver > uses 20% less cpu with a full gig load. > > And the code is cleaner and more easily maintained. > > BC > > > ________________________________ > From: Adrian Chadd > To: Barney Cordoba > Cc: Andre Oppermann ; Alan Somers > ; "net@freebsd.org" ; Jack F > Vogel ; Justin T. Gibbs ; Luigi Rizzo > ; T.C. Gubatayao > Sent: Sunday, September 1, 2013 4:51 PM > Subject: Re: Flow ID, LACP, and igb > > > Yo, > > LRO is an interesting hack that seems to do a good trick of hiding the > ridiculous locking and unfriendly cache behaviour that we do per-packet. > > It helps with LAN test traffic where things are going out in batches from the > TCP layer so the RX layer "sees" these frames in-order and can do LRO. > When you disable it, I don't easily get 10GE LAN TCP performance. That has > to be fixed. Given how fast the CPU cores, bus interconnect and memory > interconnects are, I don't think there should be any reason why we can't hit > 10GE traffic on a LAN with LRO disabled (in both software and hardware.) > > Now that I have the PMC sandy bridge stuff working right (but no PEBS, I > have to talk to Intel about that in a bit more detail before I think about > hacking that in) we can get actual live information about this stuff. But the > last time I looked, there's just too much per-packet latency going on. > The root cause looks like it's a toss up between scheduling, locking and just > lots of code running to completion per-frame. As I said, that all has to die > somehow. > > 2c, > > > > -adrian > > > > On 1 September 2013 08:45, Barney Cordoba > wrote: > > > > > > > Comcast sends packets OOO. With any decent number of internet hops > > you're likely to encounter a load balancer or packet shaper that sends > > packets OOO, so you just can't be worried about it. In fact, your > > designs MUST work with OOO packets. > > > > Getting balance on your load balanced lines is certainly a bigger > > upside than the additional CPU used. > > You can buy a faster processor for your "stack" for a lot less than > > you can buy bandwidth. > > > > Frankly my opinion of LRO is that it's a science project suitable for > > labs only. It's a trick to get more bandwidth than your bus capacity; > > the answer is to not run PCIe2 if you need pcie3. > > You can use it internally if you have > > control of all of the machines. When I modify a driver the first thing > > that I do is rip it out. > > > > BC > > > > > > ________________________________ > >  From: Luigi Rizzo > > To: Barney Cordoba > > Cc: Andre Oppermann ; Alan Somers > >; "net@freebsd.org" ; Jack F > >Vogel ; Justin T. Gibbs ; T.C. > >Gubatayao < tgubatayao@barracuda.com> > > Sent: Saturday, August 31, 2013 10:27 PM > > Subject: Re: Flow ID, LACP, and igb > > > > > > On Sun, Sep 1, 2013 at 4:15 AM, Barney Cordoba > > > >wrote: > > > > > ... > > > > > > > [your point on testing with realistic assumptions is surely a valid > > one] > > > > > > > > > > Of course there's nothing really wrong with OOO packets. We had this > > > discussion before; lots of people have round robin dual homing > > > without any ill effects. It's just not an issue. > > > > > > > It depends on where you are. > > It may not be an issue if the reordering is not large enough to > > trigger retransmissions, but even then it is annoying as it causes > > more work in the endpoint -- it prevents LRO from working, and even > > on the host stack it takes more work to sort where an out of order > > segment goes than appending an in-order one to the socket buffer. > > > > cheers > > luigi > > _______________________________________________ > > freebsd-net@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-net > > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > _______________________________________________ > > freebsd-net@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-net > > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Tue Sep 3 14:17:52 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 6C8E194B for ; Tue, 3 Sep 2013 14:17:52 +0000 (UTC) (envelope-from tom@claimlynx.com) Received: from mail-bk0-f44.google.com (mail-bk0-f44.google.com [209.85.214.44]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 005F92FE7 for ; Tue, 3 Sep 2013 14:17:51 +0000 (UTC) Received: by mail-bk0-f44.google.com with SMTP id mz10so2190455bkb.3 for ; Tue, 03 Sep 2013 07:17:49 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:date:message-id:subject:from:to :content-type; bh=GmhL4bQqO9eyojwuwBAPSTV4wUbLazthUqm8eCyXNPw=; b=jDGTd7eCmjy0z1TmfRdEd3jXQhEHB98N6dZl2XoxxQgxsZYwVcF7SyhHN+1Yw9v3hi HhDmfQEMkuIA+UE+ZGlx+HN7UWwZM5p8ysIVnPpIbtmVF4+QySh+HBduYLw2MmiCo1Ja GqOu971sl79VdQIgpozkFuoQWjHRNuuLoB+ma5RS5KmDbpnjaRPE/eCoGnRiR36Vg8GW aB25C+7xZQXb/K/P/dy/u08+tsL0Pe4dXi1EJIfC1giR+FXw7lHDBn0muHSDD6/SDG+a 0fx/d9pTl0DZVbIYgP4pvt+zyKv71m+0i4umP17lo6NEHGnpNrkw+tPDRA+3OMuwYnYZ QAQg== X-Gm-Message-State: ALoCoQlD75K2faLQjN4lU+v0e0GV6HoNijywcsiHUc32htMjGZg4JANgBa6OwC9PehNNnTJhglkHIhRnOaWiyBIIqxcxrVqqOh7zsF29OU7qQp/hA0CgONU= MIME-Version: 1.0 X-Received: by 10.204.224.142 with SMTP id io14mr2273254bkb.27.1378217869618; Tue, 03 Sep 2013 07:17:49 -0700 (PDT) Received: by 10.204.101.68 with HTTP; Tue, 3 Sep 2013 07:17:49 -0700 (PDT) Date: Tue, 3 Sep 2013 09:17:49 -0500 Message-ID: Subject: Asymmetric routing vs. pf From: Thomas Johnson To: freebsd-net@freebsd.org Content-Type: text/plain; charset=US-ASCII X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Sep 2013 14:17:52 -0000 Hello, I am in the process of trying to build up a new firewall cluster using FreeBSD 9.2 (-PRERELEASE, r254572) and pf. I am running into some issues with asymmetric routing, and wondering if there is some piece of configuration I'm missing/misusing, or if my configuration just isn't workable (I suspect this). The firewall cluster consists of two hosts, with identical hardware configurations. The only difference is that the WAN interface of each firewall is connected to a different uplink (using BIRD for BGP). The hosts are connected to each other, running an iBGP session and PFSync. On the LAN side, I am using CARP to provide internal hosts a gateway. When I add pf into the mix everything works wonderfully, until I start testing asymmetric routing situations. I can fabricate a situation where the inbound leg of the connection comes over one uplink, with the response going across the other. When I create such a connection (SSH in this case), I get the following state entries on the two hosts in the cluster. On the host receiving the inbound traffic (from the WAN): root@edge1b:~-> pfctl -vvs states | grep -A 3 :22 all tcp 162.219.166.197:22 <- 192.168.8.1:31467 CLOSED:SYN_SENT [0 + 1040] [1785120388 + 4294964376] age 00:00:21, expires in 00:00:12, 17:0 pkts, 3813:0 bytes, rule 1 id: 5221f22a0000017c creatorid: 754319ef all tcp 192.168.8.1:31467 -> 162.219.166.197:22 SYN_SENT:CLOSED [1785120388 + 4294964376] [0 + 1040] age 00:00:21, expires in 00:00:12, 17:0 pkts, 3813:0 bytes, rule 20 id: 5221f22a0000017d creatorid: 754319ef And on the host with the outbound shortest path (is also the LAN CARP master): root@edge1a:~-> pfctl -vvs states | grep -A 3 :22 all tcp 162.219.166.197:22 <- 192.168.8.1:31467 SYN_SENT:ESTABLISHED [983303631 + 61855] wscale 6 [1785117466 + 69482] wscale 6 age 382765:14:04, expires in 00:00:10, 0:20 pkts, 0:4729 bytes id: 5221f22a0000017c creatorid: 754319ef all tcp 192.168.8.1:31467 -> 162.219.166.197:22 ESTABLISHED:SYN_SENT [1785117466 + 69482] wscale 6 [983303631 + 61855] wscale 6 age 382765:14:04, expires in 00:00:10, 0:20 pkts, 0:4729 bytes id: 5221f22a0000017d creatorid: 754319ef As expected, the connection works until the timers expire on the half-open states. I was hoping that PF/PFSync would be smart enough to match up the connection and "do the right thing," but that doesn't seem to be the case. I've tried using synproxy state and sloppy state, but neither seem to work (more likely I'm not using them right), and the reading I've done suggests that both have serious drawbacks. My pf configuration is as follows (the relevant bits): wan_if="em0" lan_if="em1" xo_if="em2" vpn1="162.219.166.197" table {self} # Section 2: Options # # RST blocked connections set block-policy drop # We don't care about OS fingerprinting set fingerprints "/dev/null" # Increase state table sizes. Defaults are too small. set limit { states 200000, frags 20000, src-nodes 20000 } # Skip pf processing on lo0, just make sure that the default policy # for inbound to is block set skip on {$xo_if lo0} # Section 3: Traffic Normalization # scrub in all # Section 6: Policy # ## Default policy block log all ## "Ingress" traffic handling. pass in all ## Outbound connection handling pass out on {$wan_if} modulate state # Allow SSH traffic to vpn1 # Could maybe be made to work, though OpenBSD doesn't recommend for routine use. #pass out log on $lan_if proto tcp from any to $vpn1 port 22 synproxy state # Does not fix state issue. #pass out log on $lan_if proto tcp from any to $vpn1 port 22 keep state (sloppy) pass out log on $lan_if proto tcp from any to $vpn1 port 22 #### END OF pf.conf #### Is the *best* solution here to separate the roles of firewalling and routing? I have an extra set of hosts that could become firewalls, making these hosts strictly routers. Would OpenBSD fare any better in this scenario (thinking specifically of the pfsync "defer" functionality)? Thanks! -- Thomas Johnson ClaimLynx, Inc. -- This e-mail and any files transmitted with it are confidential and are intended solely for the use of the individual or entity to whom they are addressed. If you are not the intended recipient or the individual responsible for delivering the e-mail to the intended recipient, please be advised that you have received this e-mail in error and that any use, dissemination, forwarding, printing, or copying of this e-mail is strictly prohibited. If you have received this communication in error, please return it to the sender immediately and delete the original message and any copy of it from your computer system. If you have any questions concerning this message, please contact the sender or call ClaimLynx at (952) 593-5969. From owner-freebsd-net@FreeBSD.ORG Tue Sep 3 14:19:09 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 35E46A22; Tue, 3 Sep 2013 14:19:09 +0000 (UTC) (envelope-from zbeeble@gmail.com) Received: from mail-vb0-x234.google.com (mail-vb0-x234.google.com [IPv6:2607:f8b0:400c:c02::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id C1C5B201C; Tue, 3 Sep 2013 14:19:08 +0000 (UTC) Received: by mail-vb0-f52.google.com with SMTP id f12so3840164vbg.39 for ; Tue, 03 Sep 2013 07:19:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=8gPdn8i94IJ+H9vwn/kY6bNenX/Rf0MoGQyMWuRmaew=; b=Jd7qMSwwpI+Ri462oPC8X8arKMFMtsvXvz/7M9YhcjMFYyuVCYBbmqRyheDUUKPm3x Z1oN5nStJqyFlv0AkKyKQkM+rAARVAjiiFU5oYzz6J85dyhjgVDB0rQNSyccin3r1Kd3 XEIitrPabe//5lsK04L/OfhlazXDB80ydEz4G52EWyUwqPCYdtxgAnttbqdHD6eTCN7N Zm1VXIGjnmNozNiF0IENo18oN3aWwkXOYst1MTgF7MBSATF1HyDR6pQiHwVoIbwaxJi0 G+TVt7vGXxqUkIH1pJUL+asV5Kw0ojCEhcUxHh1v5VdqdP6Jpnj4gQNE350f2icmjNq3 aEfQ== MIME-Version: 1.0 X-Received: by 10.58.235.69 with SMTP id uk5mr28776563vec.17.1378217947776; Tue, 03 Sep 2013 07:19:07 -0700 (PDT) Received: by 10.220.122.1 with HTTP; Tue, 3 Sep 2013 07:19:07 -0700 (PDT) In-Reply-To: <5224333C.8070305@martinlaabs.de> References: <5224333C.8070305@martinlaabs.de> Date: Tue, 3 Sep 2013 10:19:07 -0400 Message-ID: Subject: Re: No UDP/TCP IPv6 connectivity (only) to router using gif interface - maybe ARM related From: Zaphod Beeblebrox To: Martin Laabs Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: FreeBSD Net , freebsd-arm , freebsd-current X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Sep 2013 14:19:09 -0000 Whether you feel it right, or not, net.inet.ip.forwarding must be 1 for gif to work (even for IPv6). On Mon, Sep 2, 2013 at 2:42 AM, Martin Laabs wrote: > Hi, > > I tried to set up my raspberry PI as an ipv6 router. As a tunnel broker I > use sixxs. Now I observed an interesting behavior: > > Every host from my network can reach the ipv6 world. The ipv6 world can > also reach every host in my network. However - the router itself is unable > to make udp or tcp connection to the "world" and is also unable to accept > connections form the "world" > ICMP however works properly. > I had a look to the tcpdump and when trying to connect i.e. to > www.kame.net > the rasperry router sends a syn packet and get a syn/ack packet back. The > rest of the handshake is missing. > I tried also some udp with netcat (nc -6 -u -l 5555 on the server and nc -6 > -u 5555 on the client) > This works great for internal (ethernet) traffic but when the data should > go through the tunnel if fails. > > The last test is maybe the most significant to describe the bug: > > Start netcat to listen for UDP packages on an external host: > > nc -6 -u -l 5555 > > Connect from the RPI-Router to that host > > nc -6 -u 2001:4dd0:xxxx:xxxx::2 5555 > > Now it is possible to send data from the RPI router to the external host > but the opposite direction does not work. Tcpdump however shows that the > udp package arrives but it is not "forwarded" to the application. > So for me it seems to be a problem with the handling of the receiving data > in the gif interface. > > This behavior is independent from net.inet6.ip6.forwarding or > net.inet6.ip6.redirect status. > > The router system: FreeBSD raspberry-pi.xxx 10.0-CURRENT FreeBSD > 10.0-CURRENT #2 r254984 > > Best regards, > Martin Laabs > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Tue Sep 3 18:07:56 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id B7369B4B; Tue, 3 Sep 2013 18:07:56 +0000 (UTC) (envelope-from mailinglists@martinlaabs.de) Received: from relay03.alfahosting-server.de (relay03.alfahosting-server.de [109.237.142.239]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 712952734; Tue, 3 Sep 2013 18:07:56 +0000 (UTC) Received: by relay03.alfahosting-server.de (Postfix, from userid 1001) id 4D4B932C0B96; Tue, 3 Sep 2013 20:07:48 +0200 (CEST) X-Spam-DCC: : X-Spam-Level: X-Spam-Status: No, score=0.0 required=7.0 tests=BAYES_50 autolearn=disabled version=3.2.5 Received: from alfa3018.alfahosting-server.de (alfa3018.alfahosting-server.de [109.237.140.30]) by relay03.alfahosting-server.de (Postfix) with ESMTPS id 141A232C0BB1; Tue, 3 Sep 2013 20:07:33 +0200 (CEST) Received: from desktop-01.martinlaabs.de (p54B30B56.dip0.t-ipconnect.de [84.179.11.86]) by alfa3018.alfahosting-server.de (Postfix) with ESMTPSA id AF108515C050; Tue, 3 Sep 2013 20:07:32 +0200 (CEST) Message-ID: <52262563.6020407@martinlaabs.de> Date: Tue, 03 Sep 2013 20:07:31 +0200 From: Martin Laabs User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130809 Thunderbird/17.0.8 MIME-Version: 1.0 To: FreeBSD Net Subject: Re: No UDP/TCP IPv6 connectivity (only) to router using gif interface - maybe ARM related References: <5224333C.8070305@martinlaabs.de> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Virus-Status: No X-Virus-Checker-Version: clamassassin 1.2.4 with ClamAV 0.97.3/17800/Tue Sep 3 18:37:24 2013 Cc: freebsd-arm , freebsd-current X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Sep 2013 18:07:56 -0000 Hi , > Whether you feel it right, or not, net.inet.ip.forwarding must be 1 for gif > to work (even for IPv6). This is a unintuitive behavior. But fortunately this solve this problem. I suggest adding setting this sysctl if ipv6_gateway_enable is enabled in the rc.conf. Best regards, Martin Laabs From owner-freebsd-net@FreeBSD.ORG Tue Sep 3 19:27:36 2013 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 7D34E317; Tue, 3 Sep 2013 19:27:36 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from albert.catwhisker.org (mx.catwhisker.org [198.144.209.73]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4975721E3; Tue, 3 Sep 2013 19:27:35 +0000 (UTC) Received: from albert.catwhisker.org (localhost [127.0.0.1]) by albert.catwhisker.org (8.14.7/8.14.7) with ESMTP id r83JRYxu019517; Tue, 3 Sep 2013 12:27:34 -0700 (PDT) (envelope-from david@albert.catwhisker.org) Received: (from david@localhost) by albert.catwhisker.org (8.14.7/8.14.7/Submit) id r83JRYoV019516; Tue, 3 Sep 2013 12:27:34 -0700 (PDT) (envelope-from david) Date: Tue, 3 Sep 2013 12:27:34 -0700 From: David Wolfskill To: FreeBSD Net Subject: Re: TSO and FreeBSD vs Linux Message-ID: <20130903192734.GA19406@albert.catwhisker.org> References: <520A6D07.5080106@freebsd.org> <5214F506.3070706@freebsd.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="3V7upXqbjpZ4EhLz" Content-Disposition: inline In-Reply-To: <5214F506.3070706@freebsd.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Sep 2013 19:27:36 -0000 --3V7upXqbjpZ4EhLz Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Aug 21, 2013 at 07:12:38PM +0200, Andre Oppermann wrote: > On 13.08.2013 19:29, Julian Elischer wrote: > > I have been tracking down a performance embarrassment on AMAZON EC2 and= have found it I think. > > Our OS cousins over at Linux land have implemented some interesting beh= aviour when TSO is in use. >=20 > There used to be a different problem with EC2 and FreeBSD TSO. The Xen h= ypervisor > doesn't like large 64K TSO bursts we generate, the drivers drops the whol= e TSO chain, > TCP gets upset and turns off TSO alltogether leaving the connection going= at one > packet a time as in the old days. > ... My apologies for jumping in so late -- I'm not subscribed to -net@. At work, I received a new desktop machine a few months ago; here's a recent history of what it has been running: FreeBSD 9.2-PRERELEASE #4 r254801M/254827:902501: Sun Aug 25 05:15:29 PDT = 2013 root@dwolf-fbsd:/usr/obj/usr/src/sys/DWOLF amd64 FreeBSD 9.2-PRERELEASE #5 r255066M/255091:902503: Sat Aug 31 11:58:53 PDT = 2013 root@dwolf-fbsd:/usr/obj/usr/src/sys/DWOLF amd64 FreeBSD 9.2-PRERELEASE #5 r255104M/255115:902503: Sun Sep 1 05:02:12 PDT = 2013 root@dwolf-fbsd:/usr/obj/usr/src/sys/DWOLF amd64 Now, I like to have a "private playground" for doing things with machines, so I make use of both em(4) NICs on the machine: em0 connects to the rest of the work network; em1 is connected to a switch I brought in from home, and to which I connect "other things" (such as my laptop). And because I'm fairly comfortable with them, I use IPFW & natd. For some folks here, none of that should come as a surprise. :-}) For reference, the em(4) devices in question are: em0@pci0:0:25:0: class=3D0x020000 card=3D0x060d15d9 chip=3D0x10ef808= 6 rev=3D0x06 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D '82578DM Gigabit Network Connection' and em1@pci0:3:0:0: class=3D0x020000 card=3D0x060d15d9 chip=3D0x10d38086 rev=3D= 0x00 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D '82574L Gigabit Network Connection' I noticed that when I tried to write files to NFS, I could write small files OK, but larger ones seemed to ... hang. Note: We don't use jumbo frames. (Work IT is convinced that they don't help. I'm trying to better-understand their reasoning.) Further poking around showed that (under the above conditions): * natd CPU% was climbing as more of the file was copied, up to 2^21 bytes. (At that point, nothing further was saved on NFS.) * dhcpd CPU% was also climbing. I tried killing that, but doing so didn't affect the other results. (Killing natd made connectivity cease, given the IPFW rules in effect.) * Performing a tcpdump while trying to copy a file of length 117709618 showed lots of TCP retransmissions. In fact, I'd hazard that every TCP packet was getting retransmitted. * "ifconfig -v em0" showed flags TSO4 & VLAN_HWTSO turned on. * "sysctl net.inet.tcp.tso" showed "1" -- enabled. As soon as I issued "sudo net.inet.tcp.tso=3D0" ... the copy worked without a hitch or a whine. And I was able to copy all 117709618 bytes, not just 2097152 (2^21). Is the above expected? It came rather as a surprise to me. Peace, david --=20 David H. Wolfskill david@catwhisker.org Taliban: Evil cowards with guns afraid of truth from a 14-year old girl. See http://www.catwhisker.org/~david/publickey.gpg for my public key. --3V7upXqbjpZ4EhLz Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.21 (FreeBSD) iEYEARECAAYFAlImOCUACgkQmprOCmdXAD1PlwCfRI35ikbwABwsF9L8S91GH+LM 0K4AnRYK8kglQA2RzY5R39gSnP/NmYKI =/Nw8 -----END PGP SIGNATURE----- --3V7upXqbjpZ4EhLz-- From owner-freebsd-net@FreeBSD.ORG Tue Sep 3 20:23:32 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 42DFB89B for ; Tue, 3 Sep 2013 20:23:32 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 1188A26A5 for ; Tue, 3 Sep 2013 20:23:32 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id B7FA6B965; Tue, 3 Sep 2013 16:23:30 -0400 (EDT) From: John Baldwin To: freebsd-net@freebsd.org Subject: Re: Does pthread_set_name_np() work? Date: Tue, 3 Sep 2013 16:06:55 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p28; KDE/4.5.5; amd64; ; ) References: <1377102969.15111.YahooMailBasic@web125803.mail.ne1.yahoo.com> In-Reply-To: <1377102969.15111.YahooMailBasic@web125803.mail.ne1.yahoo.com> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201309031606.55154.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Tue, 03 Sep 2013 16:23:30 -0400 (EDT) Cc: Laurie Jennings X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Sep 2013 20:23:32 -0000 On Wednesday, August 21, 2013 12:36:09 pm Laurie Jennings wrote: > Im trying to set the names of threads so I can distinguish them in top -H, but it doesn't seem to > take the thread id as valid. > > err=pthread_set_name_np(pthread_self(),"FOO"); This function returns void, not an error, so you can't trust the return value. -- John Baldwin From owner-freebsd-net@FreeBSD.ORG Tue Sep 3 20:56:11 2013 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 23ECD56F for ; Tue, 3 Sep 2013 20:56:11 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-we0-x22b.google.com (mail-we0-x22b.google.com [IPv6:2a00:1450:400c:c03::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9A169290F for ; Tue, 3 Sep 2013 20:56:10 +0000 (UTC) Received: by mail-we0-f171.google.com with SMTP id p57so5326277wes.30 for ; Tue, 03 Sep 2013 13:56:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=84nGUnMQZ1iSQTq05XyEsoDDFhJ+SmNH2I66UTQk+68=; b=R1woGGrSG2nSxP7TsMhM7W+fUgIrYVEOnPfk2CsdCMdYXFBPsFw3D3FaMkg8/dEoP1 30lp2L35KnJDaCX9eWesQjx+tC4e9SdVjshLYDwPR6St2t4DAq2VCrSLjXJtLq07+6na hP54Gv7pb1gz+utUlT6AYaKpWMSClJo5vFU577ywvAt4Rrqwn/xxfyzbslEuwAA1H8P4 KmEdW6u0u/t+Vun/X0o+9fV8Lh4m3aZuf+RZfG2b0V4KGAu2WKhFudlBH/u773BVxEth NASkEGw67fTZ1TlCM0FhdeHNeA0PeoFFfNmmrobhYh7OiK/C6XwS9PN8IPYXAW48QjAR NIfQ== MIME-Version: 1.0 X-Received: by 10.194.109.35 with SMTP id hp3mr3575125wjb.55.1378241767379; Tue, 03 Sep 2013 13:56:07 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.216.146.2 with HTTP; Tue, 3 Sep 2013 13:56:07 -0700 (PDT) In-Reply-To: <20130903192734.GA19406@albert.catwhisker.org> References: <520A6D07.5080106@freebsd.org> <5214F506.3070706@freebsd.org> <20130903192734.GA19406@albert.catwhisker.org> Date: Tue, 3 Sep 2013 13:56:07 -0700 X-Google-Sender-Auth: KMmlj-DFyiZuMiqatTSZwSzOEOc Message-ID: Subject: Re: TSO and FreeBSD vs Linux From: Adrian Chadd To: David Wolfskill Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Sep 2013 20:56:11 -0000 ... this is bad behaviour. So yes, it needs to be chased up and repaired. Thanks for finding it out! On 3 September 2013 12:27, David Wolfskill wrote: > On Wed, Aug 21, 2013 at 07:12:38PM +0200, Andre Oppermann wrote: > > On 13.08.2013 19:29, Julian Elischer wrote: > > > I have been tracking down a performance embarrassment on AMAZON EC2 > and have found it I think. > > > Our OS cousins over at Linux land have implemented some interesting > behaviour when TSO is in use. > > > > There used to be a different problem with EC2 and FreeBSD TSO. The Xen > hypervisor > > doesn't like large 64K TSO bursts we generate, the drivers drops the > whole TSO chain, > > TCP gets upset and turns off TSO alltogether leaving the connection > going at one > > packet a time as in the old days. > > ... > > My apologies for jumping in so late -- I'm not subscribed to -net@. > > At work, I received a new desktop machine a few months ago; here's a > recent history of what it has been running: > > FreeBSD 9.2-PRERELEASE #4 r254801M/254827:902501: Sun Aug 25 05:15:29 PDT > 2013 root@dwolf-fbsd:/usr/obj/usr/src/sys/DWOLF amd64 > FreeBSD 9.2-PRERELEASE #5 r255066M/255091:902503: Sat Aug 31 11:58:53 PDT > 2013 root@dwolf-fbsd:/usr/obj/usr/src/sys/DWOLF amd64 > FreeBSD 9.2-PRERELEASE #5 r255104M/255115:902503: Sun Sep 1 05:02:12 PDT > 2013 root@dwolf-fbsd:/usr/obj/usr/src/sys/DWOLF amd64 > > Now, I like to have a "private playground" for doing things with > machines, so I make use of both em(4) NICs on the machine: em0 connects > to the rest of the work network; em1 is connected to a switch I brought > in from home, and to which I connect "other things" (such as my laptop). > And because I'm fairly comfortable with them, I use IPFW & natd. For > some folks here, none of that should come as a surprise. :-}) > > For reference, the em(4) devices in question are: > > em0@pci0:0:25:0: class=0x020000 card=0x060d15d9 chip=0x10ef8086 > rev=0x06 hdr=0x00 > vendor = 'Intel Corporation' > device = '82578DM Gigabit Network Connection' > > and > > em1@pci0:3:0:0: class=0x020000 card=0x060d15d9 chip=0x10d38086 rev=0x00 > hdr=0x00 > vendor = 'Intel Corporation' > device = '82574L Gigabit Network Connection' > > > > I noticed that when I tried to write files to NFS, I could write small > files OK, but larger ones seemed to ... hang. > > Note: We don't use jumbo frames. (Work IT is convinced that they > don't help. I'm trying to better-understand their reasoning.) > > Further poking around showed that (under the above conditions): > * natd CPU% was climbing as more of the file was copied, up to 2^21 > bytes. (At that point, nothing further was saved on NFS.) > * dhcpd CPU% was also climbing. I tried killing that, but doing so > didn't affect the other results. (Killing natd made connectivity > cease, given the IPFW rules in effect.) > * Performing a tcpdump while trying to copy a file of length 117709618 > showed lots of TCP retransmissions. In fact, I'd hazard that every TCP > packet was getting retransmitted. > * "ifconfig -v em0" showed flags TSO4 & VLAN_HWTSO turned on. > * "sysctl net.inet.tcp.tso" showed "1" -- enabled. > > As soon as I issued "sudo net.inet.tcp.tso=0" ... the copy worked without > a hitch or a whine. And I was able to copy all 117709618 bytes, not just > 2097152 (2^21). > > Is the above expected? It came rather as a surprise to me. > > Peace, > david > -- > David H. Wolfskill david@catwhisker.org > Taliban: Evil cowards with guns afraid of truth from a 14-year old girl. > > See http://www.catwhisker.org/~david/publickey.gpg for my public key. > From owner-freebsd-net@FreeBSD.ORG Tue Sep 3 21:39:04 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 0D4289AF for ; Tue, 3 Sep 2013 21:39:04 +0000 (UTC) (envelope-from juris.kaminskis@gmail.com) Received: from mail-wi0-x229.google.com (mail-wi0-x229.google.com [IPv6:2a00:1450:400c:c05::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9980D2C34 for ; Tue, 3 Sep 2013 21:39:03 +0000 (UTC) Received: by mail-wi0-f169.google.com with SMTP id hj3so2431247wib.2 for ; Tue, 03 Sep 2013 14:39:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=H5WA7qvpgLVyyHNoSX1HeFf1Ex1xKMOqVOyX/aXe6fE=; b=Yi1h8cPhIFMY2L1St2NuIDVd7Zut3r4pnQ8zczo6UeH8RAteXS3qDYO0B+VEBqCg7a xws/4PkLHUn9z7v7wkS20AgRDFuoFl/205WLJ03iMtIiHGxSeOc7rn8N4e+EgNPFvCvs DgW735AOAOJmw5H9jO0CDX7uFKTMCI3iDGZBrRcXZ2z0UEb9LrLTfcacnbptcOWwrZa+ WEovinoVqE8iXy7YuLP9EmTxP00pTYP/IINRp1KzKxf6E6DUG00aJA2CxW1fBfSJaWR+ i02gZRrUTuuQoIZyipElHrh/oGefDcMPYe7AcDCd1IrNznZeHHoW2zbXMH2Nl6bTcnRk DeRQ== MIME-Version: 1.0 X-Received: by 10.180.12.243 with SMTP id b19mr19798882wic.18.1378244341861; Tue, 03 Sep 2013 14:39:01 -0700 (PDT) Received: by 10.194.83.225 with HTTP; Tue, 3 Sep 2013 14:39:01 -0700 (PDT) Date: Wed, 4 Sep 2013 00:39:01 +0300 Message-ID: Subject: Restart of wireless service creates crash in system From: Juris Kaminskis To: freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Sep 2013 21:39:04 -0000 Hello, I was trying to set up new wireless network. I had sucessful setup in my home over wireless but when I moved to hotel and wanted to hook up to wireless network i got some problems. first I changed /etc/wpa_supplicant.conf to read new network ssid and psk. then I did: ifconfig wlan0 destroy and then after: service netif start After I got crash of my freebsd system and reboot. Contents of my /etc/rc.conf: wlans_iwi0="wlan0" ifconfig_wlan0="WPA DHCP" and details of /var/crash/core.txt.8 (not all): Unread portion of the kernel message buffer: wlan0: ieee80211_new_state_locked: pending SCAN -> AUTH transition lost Fatal trap 12: page fault while in kernel mode cpuid = 0; apic id = 00 fault virtual address = 0xc66561d4 fault code = supervisor read, page not present instruction pointer = 0x20:0xc0751582 stack pointer = 0x28:0xc3bdab58 frame pointer = 0x28:0xc3bdac08 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, def32 1, gran 1 Even if I did something wrong probably system should not just crash and reboot. But I also do not know what I did wrong. What is interesting that after crash and reboot of system my /etc/wpa_supplicant.conf become empty. And to actually fix my wireless network I changed once again /etc/wpa_supplicant.conf and just rebooted computer which did the trick. regards Juris From owner-freebsd-net@FreeBSD.ORG Tue Sep 3 22:37:10 2013 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 39AF1354 for ; Tue, 3 Sep 2013 22:37:10 +0000 (UTC) (envelope-from kudzu@tenebras.com) Received: from mail-oa0-f51.google.com (mail-oa0-f51.google.com [209.85.219.51]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 02A2720D4 for ; Tue, 3 Sep 2013 22:37:09 +0000 (UTC) Received: by mail-oa0-f51.google.com with SMTP id h1so7555974oag.10 for ; Tue, 03 Sep 2013 15:37:03 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:content-type; bh=+/hlADK1MrP4qI/OjlQOKXqJSBXZQ774kBH0X+rmNqM=; b=cgiIxT6VW7UqE9tZfQ29dOAlENMQqWl/79JJAvJbOcMieOiaDNSe45URX7cGYHdcvG MRzDP+z7zZrJWTAUxRxTGFi5Mf+hNwQDfwP0SbfDNjmSJss9tMIiYEMfKo8eIhGX+Rth T8WFj2ECGrBpZC+jnUXaJu+qyFJveSfZZhs5EPLSlmxot2XZwAyPn4i6jHOOq3m2cNwh zGAEfnW9Sqr9+ZseRZ7vFTcSDNwSk3VX/YMu68gIiliIvojy1ijfPbTo3KJ895s3QPcK SFLkspsre3ACVvbpLfAwOZPDHL6kvTGmrpT1BAsUfOqsEGddN8z8IiWiULX03AOoVHLE i2ow== X-Gm-Message-State: ALoCoQlzMrNSxKalDHZ2I0G7SKzCjv7H1FJJ8xeG6AKzpnM5iOx2oPSGITJYzf9irA9XRtp9O/0+ MIME-Version: 1.0 X-Received: by 10.60.133.71 with SMTP id pa7mr10218093oeb.44.1378247823291; Tue, 03 Sep 2013 15:37:03 -0700 (PDT) Received: by 10.60.21.69 with HTTP; Tue, 3 Sep 2013 15:37:03 -0700 (PDT) In-Reply-To: References: <520A6D07.5080106@freebsd.org> <5214F506.3070706@freebsd.org> <20130903192734.GA19406@albert.catwhisker.org> Date: Tue, 3 Sep 2013 18:37:03 -0400 Message-ID: Subject: Re: TSO and FreeBSD vs Linux From: Michael Sierchio To: FreeBSD Net Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Sep 2013 22:37:10 -0000 I must have discovered this and forgotten - all my AMIs have net.inet.tcp.tso=0 dev.xn.0.enable_lro=0 or ifconfig xn0 -tso -lro - M On Tue, Sep 3, 2013 at 4:56 PM, Adrian Chadd wrote: > ... this is bad behaviour. So yes, it needs to be chased up and repaired. > > Thanks for finding it out! > > > On 3 September 2013 12:27, David Wolfskill wrote: > > > On Wed, Aug 21, 2013 at 07:12:38PM +0200, Andre Oppermann wrote: > > > On 13.08.2013 19:29, Julian Elischer wrote: > > > > I have been tracking down a performance embarrassment on AMAZON EC2 > > and have found it I think. > > > > Our OS cousins over at Linux land have implemented some interesting > > behaviour when TSO is in use. > > > > > > There used to be a different problem with EC2 and FreeBSD TSO. The Xen > > hypervisor > > > doesn't like large 64K TSO bursts we generate, the drivers drops the > > whole TSO chain, > > > TCP gets upset and turns off TSO alltogether leaving the connection > > going at one > > > packet a time as in the old days. > > > ... > > > > My apologies for jumping in so late -- I'm not subscribed to -net@. > > > > At work, I received a new desktop machine a few months ago; here's a > > recent history of what it has been running: > > > > FreeBSD 9.2-PRERELEASE #4 r254801M/254827:902501: Sun Aug 25 05:15:29 > PDT > > 2013 root@dwolf-fbsd:/usr/obj/usr/src/sys/DWOLF amd64 > > FreeBSD 9.2-PRERELEASE #5 r255066M/255091:902503: Sat Aug 31 11:58:53 > PDT > > 2013 root@dwolf-fbsd:/usr/obj/usr/src/sys/DWOLF amd64 > > FreeBSD 9.2-PRERELEASE #5 r255104M/255115:902503: Sun Sep 1 05:02:12 > PDT > > 2013 root@dwolf-fbsd:/usr/obj/usr/src/sys/DWOLF amd64 > > > > Now, I like to have a "private playground" for doing things with > > machines, so I make use of both em(4) NICs on the machine: em0 connects > > to the rest of the work network; em1 is connected to a switch I brought > > in from home, and to which I connect "other things" (such as my laptop). > > And because I'm fairly comfortable with them, I use IPFW & natd. For > > some folks here, none of that should come as a surprise. :-}) > > > > For reference, the em(4) devices in question are: > > > > em0@pci0:0:25:0: class=0x020000 card=0x060d15d9 chip=0x10ef8086 > > rev=0x06 hdr=0x00 > > vendor = 'Intel Corporation' > > device = '82578DM Gigabit Network Connection' > > > > and > > > > em1@pci0:3:0:0: class=0x020000 card=0x060d15d9 chip=0x10d38086 rev=0x00 > > hdr=0x00 > > vendor = 'Intel Corporation' > > device = '82574L Gigabit Network Connection' > > > > > > > > I noticed that when I tried to write files to NFS, I could write small > > files OK, but larger ones seemed to ... hang. > > > > Note: We don't use jumbo frames. (Work IT is convinced that they > > don't help. I'm trying to better-understand their reasoning.) > > > > Further poking around showed that (under the above conditions): > > * natd CPU% was climbing as more of the file was copied, up to 2^21 > > bytes. (At that point, nothing further was saved on NFS.) > > * dhcpd CPU% was also climbing. I tried killing that, but doing so > > didn't affect the other results. (Killing natd made connectivity > > cease, given the IPFW rules in effect.) > > * Performing a tcpdump while trying to copy a file of length 117709618 > > showed lots of TCP retransmissions. In fact, I'd hazard that every TCP > > packet was getting retransmitted. > > * "ifconfig -v em0" showed flags TSO4 & VLAN_HWTSO turned on. > > * "sysctl net.inet.tcp.tso" showed "1" -- enabled. > > > > As soon as I issued "sudo net.inet.tcp.tso=0" ... the copy worked without > > a hitch or a whine. And I was able to copy all 117709618 bytes, not just > > 2097152 (2^21). > > > > Is the above expected? It came rather as a surprise to me. > > > > Peace, > > david > > -- > > David H. Wolfskill david@catwhisker.org > > Taliban: Evil cowards with guns afraid of truth from a 14-year old girl. > > > > See http://www.catwhisker.org/~david/publickey.gpg for my public key. > > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Tue Sep 3 22:49:36 2013 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 3E0D68B5 for ; Tue, 3 Sep 2013 22:49:36 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from albert.catwhisker.org (mx.catwhisker.org [198.144.209.73]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E045021CE for ; Tue, 3 Sep 2013 22:49:35 +0000 (UTC) Received: from albert.catwhisker.org (localhost [127.0.0.1]) by albert.catwhisker.org (8.14.7/8.14.7) with ESMTP id r83MnTt4020959; Tue, 3 Sep 2013 15:49:29 -0700 (PDT) (envelope-from david@albert.catwhisker.org) Received: (from david@localhost) by albert.catwhisker.org (8.14.7/8.14.7/Submit) id r83MnScQ020958; Tue, 3 Sep 2013 15:49:28 -0700 (PDT) (envelope-from david) Date: Tue, 3 Sep 2013 15:49:28 -0700 From: David Wolfskill To: FreeBSD Net Subject: Re: TSO and FreeBSD vs Linux Message-ID: <20130903224928.GQ1577@albert.catwhisker.org> References: <520A6D07.5080106@freebsd.org> <5214F506.3070706@freebsd.org> <20130903192734.GA19406@albert.catwhisker.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="TSQPSNmi3T91JED+" Content-Disposition: inline In-Reply-To: <20130903192734.GA19406@albert.catwhisker.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Sep 2013 22:49:36 -0000 --TSQPSNmi3T91JED+ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Sep 03, 2013 at 12:27:34PM -0700, David Wolfskill wrote: > ... > As soon as I issued "sudo net.inet.tcp.tso=3D0" ... the copy worked witho= ut > a hitch or a whine. And I was able to copy all 117709618 bytes, not just > 2097152 (2^21). The above command should (of course) have read sudo sysctl net.inet.tcp.tso=3D0 Also: I normally had the em0 NIC on the machine in question connected to a Netgear GS105 (5-port Gigabit switch). In the process of trouble-shooting the problem with NFS writes, I bypassed that switch and connected the em0 NIC directly to the jack in my cube. In that configuration, the em0 NIC showed "media: Ethernet 1000baseT (autoselect)", while connected to the GS105, it showed "media: Ethernet 100baseTX (autoselect)". While the NFS write worked whether or not I had the GS105 in the path, it seemed ... suboptimal ... to have a NIC capable of 1000baseT connected to a Gigabit switch, but negotiating at 100baseTX. So I tried setting the media via "ifconfig em0 media 1000baseT"; after a few seconds, it finally woke back up, and now reports "media: Ethernet 1000baseT (1000baseT )". So it appears that the em(4) driver and Intel 82578DM NIC fail to negotiate 1000baseT with the Netgear GS105. Peace, david --=20 David H. Wolfskill david@catwhisker.org Taliban: Evil cowards with guns afraid of truth from a 14-year old girl. See http://www.catwhisker.org/~david/publickey.gpg for my public key. --TSQPSNmi3T91JED+ Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.21 (FreeBSD) iEYEARECAAYFAlImZ3gACgkQmprOCmdXAD1VggCfQSAR6YpT4aJmNih4+Y3IW21G xZoAn0oA8FpQm0iGPgrwHpxinj6xIIcX =DUco -----END PGP SIGNATURE----- --TSQPSNmi3T91JED+-- From owner-freebsd-net@FreeBSD.ORG Tue Sep 3 23:11:16 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 5F18519E for ; Tue, 3 Sep 2013 23:11:16 +0000 (UTC) (envelope-from kurt.buff@gmail.com) Received: from mail-ea0-x22a.google.com (mail-ea0-x22a.google.com [IPv6:2a00:1450:4013:c01::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E7DA623C8 for ; Tue, 3 Sep 2013 23:11:15 +0000 (UTC) Received: by mail-ea0-f170.google.com with SMTP id h14so3313342eak.15 for ; Tue, 03 Sep 2013 16:11:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=j/mxm7jOFeNNVhS9FnQWZP4Z11rtE651XlZowpdVlRM=; b=iWDLaA0vsU1FY6XQTApOco8rOiauQ7Db7n9aAyZLseUteNuoOO9KdkV21gYgFZ1Mic AlYLvoGzv1G5fzQ5ts/8tScI3efWUd3vT9ZGeTTd51dwXIf7mEGsmdikl5VsSSplIFnd RBv6pF4gr9aHka6l3Ep9nHHpMsElkUFhnQ1ee2KInX2CehBSSrwqtCi8EFO/sJK26+HZ Y2vrvVenTdOOlmxUK+tx4ngGZNV9Bn9+MwaUzT3IJh8qmdQdPtoVgYXK8OLwTAW0eTgQ XIL6JzgvnNkJGK7N1mk6630mqXbD7pxDg+U+EwojOeWsLd9miFjJufzW9+DaxH04AsrL 6o3A== MIME-Version: 1.0 X-Received: by 10.14.4.1 with SMTP id 1mr50103764eei.21.1378249874102; Tue, 03 Sep 2013 16:11:14 -0700 (PDT) Received: by 10.14.142.209 with HTTP; Tue, 3 Sep 2013 16:11:14 -0700 (PDT) Date: Tue, 3 Sep 2013 16:11:14 -0700 Message-ID: Subject: Question regarding security run output From: Kurt Buff To: freebsd-net@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Sep 2013 23:11:16 -0000 Over the three-day US weekend, I was working on some stuff, and found an interesting set of entries in the daily security run emails all three days. The output looks as follows: ntop.example.com kernel log messages: +++ /tmp/security.IUGsscCR 2013-08-26 03:02:24.000000000 -0700 +arp: unknown hardware address format (0x4500) (from 00:05:b7:de:cd:79 to 72:6e:61:6c:2c:70) +arp: unknown hardware address format (0x0100) (from 00:05:b7:de:cd:79 to 6c:3d:31:37:2c:6e) +arp: unknown hardware address format (0x4500) (from 00:05:b7:de:cd:a3 to 77:72:69:74:74:65) +arp: unknown hardware address format (0x0000) (from 00:05:b7:de:cd:71 to 2d:0d:0a:62:6f:64) This box is monitoring a mirror port on a procurve switch, using an unnumbered interface. My investigation led me to the engineering lab, and I'm querying them regarding the equipment, but I don't know what the above entries signal. Does anyone have a clue they can throw me on this? I also find it interesting that the MAC addresses are either unknown, or belong to Arbor Networks. We don't have any Arbor Networks equipment, though I suppose they could vend them to an OEM. I'm going to see if I can trace them down and get some idea of what's running around in that lab. Thanks, Kurt From owner-freebsd-net@FreeBSD.ORG Wed Sep 4 00:36:51 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id C4C65F84 for ; Wed, 4 Sep 2013 00:36:51 +0000 (UTC) (envelope-from ddesimone@verio.net) Received: from relay2-bcrtfl2.verio.net (relay2-bcrtfl2.verio.net [131.103.218.177]) by mx1.freebsd.org (Postfix) with ESMTP id 76ECC29B3 for ; Wed, 4 Sep 2013 00:36:51 +0000 (UTC) Received: from iad-wprd-xchw01.corp.verio.net (iad-wprd-xchw01.corp.verio.net [198.87.7.164]) by relay2-bcrtfl2.verio.net (Postfix) with ESMTP id C93711FF0018 for ; Tue, 3 Sep 2013 20:10:02 -0400 (EDT) thread-index: Ac6pAxnmNjkgTtsGQhWc8vj3Y3Vwxg== Received: from hometx-733b1p1.corp.verio.net ([10.144.2.53]) by iad-wprd-xchw01.corp.verio.net over TLS secured channel with Microsoft SMTPSVC(6.0.3790.4675); Tue, 3 Sep 2013 20:10:01 -0400 Received: by hometx-733b1p1.corp.verio.net (sSMTP sendmail emulation); Tue, 03 Sep 2013 19:09:59 -0500 Date: Tue, 3 Sep 2013 19:09:59 -0500 Content-Transfer-Encoding: 7bit From: "David DeSimone" To: Content-class: urn:content-classes:message X-MimeOLE: Produced By Microsoft MimeOLE V6.00.3790.4913 Subject: Re: Question regarding security run output Importance: normal Priority: normal Message-ID: <20130904000959.GG19904@verio.net> Mail-Followup-To: freebsd-net@freebsd.org References: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: Precedence: bulk User-Agent: Mutt/1.5.20 (2009-12-10) X-OriginalArrivalTime: 04 Sep 2013 00:10:01.0423 (UTC) FILETIME=[19445DF0:01CEA903] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Sep 2013 00:36:51 -0000 Kurt Buff wrote: > > Over the three-day US weekend, I was working on some stuff, and found an > interesting set of entries in the daily security run emails all three days. > > The output looks as follows: > > ntop.example.com kernel log messages: > > +++ /tmp/security.IUGsscCR 2013-08-26 03:02:24.000000000 -0700 > > +arp: unknown hardware address format (0x4500) (from 00:05:b7:de:cd:79 to > 72:6e:61:6c:2c:70) > > +arp: unknown hardware address format (0x0100) (from 00:05:b7:de:cd:79 to > 6c:3d:31:37:2c:6e) > > +arp: unknown hardware address format (0x4500) (from 00:05:b7:de:cd:a3 to > 77:72:69:74:74:65) > > +arp: unknown hardware address format (0x0000) (from 00:05:b7:de:cd:71 to > 2d:0d:0a:62:6f:64) These are all interesting because the destination MAC address is composed entirely of valid ASCII characters. 72:6e:61:6c:2c:70 = "rnal,p" 6c:3d:31:37:2c:6e = "l=17,n" 77:72:69:74:74:65 = "writte" 2d:0d:0a:62:6f:64 = "-\r\nbod" > This box is monitoring a mirror port on a procurve switch, using an > unnumbered interface. > > My investigation led me to the engineering lab, and I'm querying them > regarding the equipment, but I don't know what the above entries signal. > Does anyone have a clue they can throw me on this? > > I also find it interesting that the MAC addresses are either unknown, or > belong to Arbor Networks. We don't have any Arbor Networks equipment, > though I suppose they could vend them to an OEM. I'm going to see if I can > trace them down and get some idea of what's running around in that lab. Is there some hardware NIC fault causing DMA from random places in memory on these devices, or some other data leak propogating through the stack on them? It is probably worth capturing the odd packets and analyzing them further to see why they look the way they do. -- David DeSimone == Network Admin == fox@verio.net "I don't like spinach, and I'm glad I don't, because if I liked it I'd eat it, and I just hate it." -- Clarence Darrow This email message is intended for the use of the person to whom it has been sent, and may contain information that is confidential or legally protected. If you are not the intended recipient or have received this message in error, you are not authorized to copy, distribute, or otherwise use this message or its attachments. Please notify the sender immediately by return e-mail and permanently delete this message and any attachments. Verio Inc. makes no warranty that this email is error or virus free. Thank you. From owner-freebsd-net@FreeBSD.ORG Wed Sep 4 01:58:02 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id AC549520 for ; Wed, 4 Sep 2013 01:58:02 +0000 (UTC) (envelope-from kurt.buff@gmail.com) Received: from mail-ee0-x236.google.com (mail-ee0-x236.google.com [IPv6:2a00:1450:4013:c00::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 448CA2EB0 for ; Wed, 4 Sep 2013 01:58:02 +0000 (UTC) Received: by mail-ee0-f54.google.com with SMTP id e53so3381023eek.13 for ; Tue, 03 Sep 2013 18:58:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=xOIUDM2W4O+LSy4G/Co7bX8CD0jL50bFaU3jUUOl2j4=; b=FvNN2ByelfWCc6acFKelafuzZU8rRoqmSkRQXEE0MVppZv9Ur8JN0ahfhNiNkM3sei D7PowEoghKhkMS4CEGIbd7L4TH2HRVgRE01MRvinVCfRV6+FuGmIsBdxRIhWuRUtH2Cp 1kpTl4emHPdiWNOKO+3M2EFeNDWoS7pVmgbVspaYxMPudHmpHI1Tnyiu0phIqL3vXsgT AK6eGFkLM6EqB4PkqO4x3xt6t+2j+ENfwNymt7iBFqQ9Xw9MMLP7oc1EtBCnbEsoBo12 qENdkHneZ94gv621n3/heH0JwlWjgQlomtTKg9ZjMlqRU2PHejKIrTBeIvdKIGTMSKAF rabA== MIME-Version: 1.0 X-Received: by 10.14.210.8 with SMTP id t8mr479366eeo.39.1378259880614; Tue, 03 Sep 2013 18:58:00 -0700 (PDT) Received: by 10.14.142.209 with HTTP; Tue, 3 Sep 2013 18:58:00 -0700 (PDT) In-Reply-To: <20130904000959.GG19904@verio.net> References: <20130904000959.GG19904@verio.net> Date: Tue, 3 Sep 2013 18:58:00 -0700 Message-ID: Subject: Re: Question regarding security run output From: Kurt Buff To: freebsd-net@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Sep 2013 01:58:02 -0000 On Tue, Sep 3, 2013 at 5:09 PM, David DeSimone wrote: > Kurt Buff wrote: >> >> Over the three-day US weekend, I was working on some stuff, and found an >> interesting set of entries in the daily security run emails all three days. >> >> The output looks as follows: >> >> ntop.example.com kernel log messages: >> >> +++ /tmp/security.IUGsscCR 2013-08-26 03:02:24.000000000 -0700 >> >> +arp: unknown hardware address format (0x4500) (from 00:05:b7:de:cd:79 to >> 72:6e:61:6c:2c:70) >> >> +arp: unknown hardware address format (0x0100) (from 00:05:b7:de:cd:79 to >> 6c:3d:31:37:2c:6e) >> >> +arp: unknown hardware address format (0x4500) (from 00:05:b7:de:cd:a3 to >> 77:72:69:74:74:65) >> >> +arp: unknown hardware address format (0x0000) (from 00:05:b7:de:cd:71 to >> 2d:0d:0a:62:6f:64) > > These are all interesting because the destination MAC address is > composed entirely of valid ASCII characters. > > 72:6e:61:6c:2c:70 = "rnal,p" > > 6c:3d:31:37:2c:6e = "l=17,n" > > 77:72:69:74:74:65 = "writte" > > 2d:0d:0a:62:6f:64 = "-\r\nbod" That is indeed interesting. >> This box is monitoring a mirror port on a procurve switch, using an >> unnumbered interface. >> >> My investigation led me to the engineering lab, and I'm querying them >> regarding the equipment, but I don't know what the above entries signal. >> Does anyone have a clue they can throw me on this? >> >> I also find it interesting that the MAC addresses are either unknown, or >> belong to Arbor Networks. We don't have any Arbor Networks equipment, >> though I suppose they could vend them to an OEM. I'm going to see if I can >> trace them down and get some idea of what's running around in that lab. > > > Is there some hardware NIC fault causing DMA from random places in > memory on these devices, or some other data leak propogating through the > stack on them? It is probably worth capturing the odd packets and > analyzing them further to see why they look the way they do. Unknown, but the units in that lab are a system/product under development, and after talking with them this afternoon, they did mention they were having some problems. I shall pass your tidbits on to them, and see if it rings a bell for them. Many thanks for the help. Kurt From owner-freebsd-net@FreeBSD.ORG Wed Sep 4 04:54:18 2013 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 8D6D0EA for ; Wed, 4 Sep 2013 04:54:18 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 60F462BA4 for ; Wed, 4 Sep 2013 04:54:17 +0000 (UTC) Received: from Julian-MBP3.local (ppp121-45-245-177.lns20.per2.internode.on.net [121.45.245.177]) (authenticated bits=0) by vps1.elischer.org (8.14.6/8.14.6) with ESMTP id r844rwXb051219 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Tue, 3 Sep 2013 21:54:05 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <5226BCE1.40107@freebsd.org> Date: Wed, 04 Sep 2013 12:53:53 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:17.0) Gecko/20130801 Thunderbird/17.0.8 MIME-Version: 1.0 To: David Wolfskill Subject: Re: TSO and FreeBSD vs Linux References: <520A6D07.5080106@freebsd.org> <5214F506.3070706@freebsd.org> <20130903192734.GA19406@albert.catwhisker.org> <20130903224928.GQ1577@albert.catwhisker.org> In-Reply-To: <20130903224928.GQ1577@albert.catwhisker.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Sep 2013 04:54:18 -0000 On 9/4/13 6:49 AM, David Wolfskill wrote: > On Tue, Sep 03, 2013 at 12:27:34PM -0700, David Wolfskill wrote: >> ... >> As soon as I issued "sudo net.inet.tcp.tso=0" ... the copy worked without >> a hitch or a whine. And I was able to copy all 117709618 bytes, not just >> 2097152 (2^21). > The above command should (of course) have read > > sudo sysctl net.inet.tcp.tso=0 > > Also: I normally had the em0 NIC on the machine in question connected to > a Netgear GS105 (5-port Gigabit switch). In the process of > trouble-shooting the problem with NFS writes, I bypassed that switch and > connected the em0 NIC directly to the jack in my cube. > > In that configuration, the em0 NIC showed "media: Ethernet 1000baseT > (autoselect)", while connected to the GS105, it showed "media: Ethernet > 100baseTX (autoselect)". > > While the NFS write worked whether or not I had the GS105 in the path, > it seemed ... suboptimal ... to have a NIC capable of 1000baseT > connected to a Gigabit switch, but negotiating at 100baseTX. > > So I tried setting the media via "ifconfig em0 media 1000baseT"; after a > few seconds, it finally woke back up, and now reports "media: Ethernet > 1000baseT (1000baseT )". > > So it appears that the em(4) driver and Intel 82578DM NIC fail to > negotiate 1000baseT with the Netgear GS105. yeah auto-negotiation seems a bit fragile.. not just for us either.. I often end up hardwiring it in rc.conf. > Peace, > david From owner-freebsd-net@FreeBSD.ORG Wed Sep 4 05:19:32 2013 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 83738BE6; Wed, 4 Sep 2013 05:19:32 +0000 (UTC) (envelope-from hoomanfazaeli@gmail.com) Received: from mail-we0-x230.google.com (mail-we0-x230.google.com [IPv6:2a00:1450:400c:c03::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E6E072DC8; Wed, 4 Sep 2013 05:19:31 +0000 (UTC) Received: by mail-we0-f176.google.com with SMTP id u56so3288196wes.35 for ; Tue, 03 Sep 2013 22:19:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=kpfU4GTMTbiD6W07pD1hXOpPX4G6TVomo8bz66NtsiA=; b=S2iHGQsWvoJiC5fyF3CeJrbXt7Ka8c2WGEAH7/9ZSsTG3lnKFNfE7TOoLR+fpCozKn si771iqU5/Te8JRCwEV6TwAU8BJUqnIQVTHfHZMn7NoUi+2Idh72Mvoqnuz69zE7FcW4 xf6e3UZRhTbX2BXbk3eFskXO+rxRMcX8DJvE8bOzMnQbW+423odudNrY0aO0qGG38Fs6 Td2WX6qamPboL2ckCJjqcPzEiBrDibT3r5CIZZElR7PLP1rnkLp8qqN+dz8O1RpRJI7L i9wJTqvJsK2xMhfQkglZjESKiX+2sevEDSmwBk7vq5LAUxDqMuJSbc1ZpeWl0+qiXzlu u1mg== X-Received: by 10.180.9.69 with SMTP id x5mr510004wia.41.1378271970222; Tue, 03 Sep 2013 22:19:30 -0700 (PDT) Received: from [192.168.2.30] ([2.176.240.65]) by mx.google.com with ESMTPSA id ey2sm1057695wib.5.1969.12.31.16.00.00 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 03 Sep 2013 22:19:29 -0700 (PDT) Message-ID: <5226C2E0.8020800@gmail.com> Date: Wed, 04 Sep 2013 09:49:28 +0430 From: Hooman Fazaeli User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130215 Thunderbird/17.0.3 MIME-Version: 1.0 To: Julian Elischer Subject: Re: TSO and FreeBSD vs Linux References: <520A6D07.5080106@freebsd.org> <5214F506.3070706@freebsd.org> <20130903192734.GA19406@albert.catwhisker.org> <20130903224928.GQ1577@albert.catwhisker.org> <5226BCE1.40107@freebsd.org> In-Reply-To: <5226BCE1.40107@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD Net , David Wolfskill X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Sep 2013 05:19:32 -0000 On 9/4/2013 9:23 AM, Julian Elischer wrote: > On 9/4/13 6:49 AM, David Wolfskill wrote: >> On Tue, Sep 03, 2013 at 12:27:34PM -0700, David Wolfskill wrote: >>> ... >>> As soon as I issued "sudo net.inet.tcp.tso=0" ... the copy worked without >>> a hitch or a whine. And I was able to copy all 117709618 bytes, not just >>> 2097152 (2^21). >> The above command should (of course) have read >> >> sudo sysctl net.inet.tcp.tso=0 >> >> Also: I normally had the em0 NIC on the machine in question connected to >> a Netgear GS105 (5-port Gigabit switch). In the process of >> trouble-shooting the problem with NFS writes, I bypassed that switch and >> connected the em0 NIC directly to the jack in my cube. >> >> In that configuration, the em0 NIC showed "media: Ethernet 1000baseT >> (autoselect)", while connected to the GS105, it showed "media: Ethernet >> 100baseTX (autoselect)". >> >> While the NFS write worked whether or not I had the GS105 in the path, >> it seemed ... suboptimal ... to have a NIC capable of 1000baseT >> connected to a Gigabit switch, but negotiating at 100baseTX. >> >> So I tried setting the media via "ifconfig em0 media 1000baseT"; after a >> few seconds, it finally woke back up, and now reports "media: Ethernet >> 1000baseT (1000baseT )". >> >> So it appears that the em(4) driver and Intel 82578DM NIC fail to >> negotiate 1000baseT with the Netgear GS105. > > yeah auto-negotiation seems a bit fragile.. not just for us either.. > I often end up hardwiring it in rc.conf. > I had also experienced similar problems (one case was 82574 with cisco 3550). I also remember cases when auto-select worked but fixed media did not (link was settled down to 100 or half-duplex) I am curious as what is the exact technical reason(s) for such media problems? Are they more hardware or driver related? -- Best regards. Hooman Fazaeli From owner-freebsd-net@FreeBSD.ORG Wed Sep 4 08:05:04 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 0897B69C for ; Wed, 4 Sep 2013 08:05:04 +0000 (UTC) (envelope-from syuu@dokukino.com) Received: from mail-pb0-x22f.google.com (mail-pb0-x22f.google.com [IPv6:2607:f8b0:400e:c01::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id CC2102C69 for ; Wed, 4 Sep 2013 08:05:03 +0000 (UTC) Received: by mail-pb0-f47.google.com with SMTP id rr4so7134596pbb.34 for ; Wed, 04 Sep 2013 01:05:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dokukino.com; s=google; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=mjtroT/ZlAGF34h6Z5vKMNqM4xiEX4sqPbU7Jw/f8hk=; b=OAm21qmAVcodOHU2QebxGOYZ9rwXFPdc4QRk8gj5jyFTB8fy7CZ/YOGAka++o2899V Ha5Kv6KsJ3LisIg2PsIp5RiH/gFjflOk46XGlCUyLHF6S8/Ic2Gb0vnp6/sQq3O7l0rx r9v8jMMRgDzVTVcqJ9/gFNGFsP9Ma9HKG4/Bw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type; bh=mjtroT/ZlAGF34h6Z5vKMNqM4xiEX4sqPbU7Jw/f8hk=; b=M+9pm4n9eVtOCFEE+wH+yHzqXApLNl45pHQIHMOBkDGlHptAqp9z7I5y+nqsDBqf2y OrIP+eF+TBW4rFTuhFfcIuT+vm6orcIWzZf4vWyrjL50KpsKer9SmMOuvpdXR08usXYE qyCm6bgjr8ulI2XK12kZ03TV6RXHgtruT7Ru/+iP2KK/mGYdzsuxKjGK4BcxV7CvcjIE bNG8kRyhyxXMbIEQBnjcgMZolNU/RNWPXD2nNnjGoAOMiq2JSsxAwHcRufvWQ7mwXJ8E KBRoUk1e8VVy70tzRXsn8r5wfLBvWJs0/k/xW8EoIOlGkhSEpc0O7/24jfeb3VGVY8Bm RjxA== X-Gm-Message-State: ALoCoQnkCz8HqXld2D4TJQ3UwQxufP8YVQgiJWLWWtlDRNJC9FMj6WVnSOWGBK0BlAbBmPR0/hqO X-Received: by 10.68.113.99 with SMTP id ix3mr1726875pbb.180.1378281903436; Wed, 04 Sep 2013 01:05:03 -0700 (PDT) MIME-Version: 1.0 Received: by 10.68.54.137 with HTTP; Wed, 4 Sep 2013 01:04:23 -0700 (PDT) In-Reply-To: References: From: Takuya ASADA Date: Wed, 4 Sep 2013 17:04:23 +0900 Message-ID: Subject: Re: Multiqueue support for bpf To: Luigi Rizzo Content-Type: multipart/mixed; boundary=047d7b6d82dc41494f04e58a43ab X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Sep 2013 08:05:04 -0000 --047d7b6d82dc41494f04e58a43ab Content-Type: text/plain; charset=UTF-8 Hi, This is 2nd version of multiqueue bpf patch, I think I fixed things what you commented on previous mail. Here's a change list of the patch: - Drop added functions on struct ifnet(if_get_[rt]xqueue_len/if_get_[rt]xqueue_affinity). HW queue number and queue affinity informations are maybe useful for some applications, but it's not really directly related to multiqueue bpf. I think we should discuss them separately. - Use BITSET for queue mask. It seems better to use BITSET for queue mask structure, instead of boolean array. - Drop tcpdump/libpcap changes. It also should discuss separately. - M_QUEUEID/IFCAP_QUEUEID M_QUEUEID is the flag for mbuf which contains hw queue id. IFCAP_QUEUEID is the flag which means the driver has ability to set queue id on mbuf. 2013/7/3 Luigi Rizzo > > > > On Tue, Jul 2, 2013 at 5:56 PM, Takuya ASADA wrote: > >> Hi, >> >> Do you have an updated URL for the diffs ? The link below from your >>> original message >>> seems not working now (NXDOMAIN) >>> >>> http://www.dokukino.com/mq_bpf_20110813.diff >>> >> >> Changes with recent head is on my repository: >> http://svnweb.freebsd.org/base/user/syuu/mq_bpf/ >> And I attached a diff file on this mail. >> >> > thanks for the diffs (the URL to the repo is useful too, > but a URL to generate diffs is more convenient for reviewing changes). > > I believe it still needs a bit of work before being merged. > > My comments (in order of the patch): > > === ifnet.9 (and related code in if.c, sockio.h) === > - if_get_rxqueue_len()/if_get_rxqueue_len() is not a good name, > as to me at least it suggests that it returns the size of the > individual queue, rather than the number of queues. > > - cpu affinity (in userspace) is a bitmask, whereas in the BSD kernel > we almost never use the term "affinity", and favour "couid" or "oncpu" > (i.e. an individual CPU id). > I think you should either rename if_get_txqueue_affinity(), or make > the return type a cpuset (which seems more sensible as the return > value is passed to userspace) > > === bpf.4 (and related code) === > > - the ioctl() to attach/detach/report queues attached to a specific > bpf descriptor talk about "mask bit" but neither the internal nor > the external implementation deal with bits. > I'd rather document those ioctl as "attaching queue to file descriptor". > > - the BPF ioctl names are generally inconsistent (using either S or SET > and G or GET for the setter and getter methods). > But you should pick one of the patterns and stick with it, > not introduce a third variant (GT/ST). > Given we are in 2013 we might go for the long form GET and SET > so i suggest the following (spaces for clarity) > > +#define BIOC ENA QMASK _IO('B', 133) > +#define BIOC DIS QMASK _IO('B', 134) > +#define BIOC SET RXQMASK _IOWR('B', 135, uint32_t) > +#define BIOC CLR RXQMASK _IOWR('B', 136, uint32_t) > +#define BIOC GET RXQMASK _IOR('B', 137, uint32_t) > +#define BIOC SET TXQMASK _IOWR('B', 138, uint32_t) > +#define BIOC CLR TXQMASK _IOWR('B', 139, uint32_t) > +#define BIOC GET TXQMASK _IOR('B', 140, uint32_t) > +#define BIOC SET OTHERMASK _IO('B', 141) > +#define BIOC CLR OTHERMASK _IO('B', 142) > +#define BIOC GET OTHERMASK _IOR('B', 143, uint32_t) > > Also related: the existing ioctls() use u_int as argumnts, rather > than uint32_t. I personally prefer the uint32_t form, but you > should at least add a comment to indicate that the choice is > deliberate. > > === if.c === > > > - you have a KASSERT to report if ifp->if_get_*xqueue_affinity() is not > set, but i'd rather run the function only if is set, so you can > have a multiqueue interface which does not bind queues to specific cores > (which i am not sure is always a great idea; too many processes > statically bound to the same queue mean you lose opportunity to > parallelize work.) > > === mbuf.h === > > as mentioned earlier, the modification to struct mbuf should > be avoided if possible at all. It seems that you need just one > direction bit (which maybe is available already from the context) > and one queue identifier, which in the rx path, at least in your > implementation is always a replica of the 'flowid' field. > Can you see if perhaps the flowid field can be (ab)used on the > tx path as well ? > > > === if.h === > > - in if.h, can you use individual variables instead of arrays > for ifr_queue_affinity_index and friends ? > The macros to map the fields of ifr_ifru one > level up are a necessary evil, > but there is no point in using the arrays. > > - SIOCGIFTXQAFFINITY seems to use the receive function (copy&paste typo) > talks about > Also, this function is probably something that should be coordinated > with work on generic multiqueue support > > > === bpf.c === > > - in linux (and hopefully in FreeBSD at some point) the number of queues > can be changed at runtime. > So i suggest that you cache the current number of queues when > you allocate the arrays (qm_*xq_qmask[] ) rather than invoking > ifp->if_get_*xqueue_len() everytime you need to do a boundary check. > This will save us from all sort of problems later. > > - in terms of code, the six BIOC*XQMASK are very similar, you are probably > better off having one single case in the switch > > - can you some comments in the code for the chunk at @@ -2117,6 +2391,42 @@ > I do not completely understand why you are returning if the *queue tag > in the mbuf is out of range (my impression is that you should > just continue, or if you think the packet is incorrect it should > be filtered out before entering the LIST_FOREACH() ). > Secondly, you should use the cached value of *queue_len > > > > cheers > luigi > > > -- > -----------------------------------------+------------------------------- > Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione > http://www.iet.unipi.it/~luigi/ . Universita` di Pisa > TEL +39-050-2211611 . via Diotisalvi 2 > Mobile +39-338-6809875 . 56122 PISA (Italy) > > -----------------------------------------+------------------------------- > > --047d7b6d82dc41494f04e58a43ab Content-Type: application/octet-stream; name="mq_bpf_201309041611.diff" Content-Disposition: attachment; filename="mq_bpf_201309041611.diff" Content-Transfer-Encoding: base64 X-Attachment-Id: f_hl691xcn0 SW5kZXg6IHNoYXJlL21hbi9tYW40L2JwZi40Cj09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0KLS0tIHNoYXJlL21hbi9tYW40 L2JwZi40CSguLi4vaGVhZCkJKHJldmlzaW9uIDI1NTE4MCkKKysrIHNoYXJlL21hbi9tYW40L2Jw Zi40CSguLi4vdXNlci9zeXV1L21xX2JwZikJKHJldmlzaW9uIDI1NTIwMCkKQEAgLTYzMSw2ICs2 MzEsMzYgQEAKIC5WdCBiemhfa2VybmVsX2dlbgogYWdhaW5zdAogLlZ0IGJ6aF91c2VyX2dlbiAu CisuSXQgRHYgQklPQ1FNQVNLRU5BQkxFCitFbmFibGVzIG11bHRpcXVldWUgZmlsdGVyIG9uIHRo ZSBkZXNjcmlwdG9yLgorCisuSXQgRHYgQklPQ1FNQVNLRElTQUJMRQorRGlzYWJsZXMgbXVsdGlx dWV1ZSBmaWx0ZXIgb24gdGhlIGRlc2NyaXB0b3IuCisKKy5JdCBEdiBCSU9DR1JYUU1BU0sKKy5Q cSBMaSBzdHJ1Y3QgYnBmX3FtYXNrX2JpdHMKK1NldCBSWCBxdWV1ZSBtYXNrIGJpdHMuCisKKy5J dCBEdiBCSU9DU1JYUU1BU0sKKy5QcSBMaSBzdHJ1Y3QgYnBmX3FtYXNrX2JpdHMKK0dldCBSWCBx dWV1ZSBtYXNrIGJpdHMuCisKKy5JdCBEdiBCSU9DR1RYUU1BU0sKKy5QcSBMaSBzdHJ1Y3QgYnBm X3FtYXNrX2JpdHMKK1NldCBUWCBxdWV1ZSBtYXNrIGJpdHMuCisKKy5JdCBEdiBCSU9DU1RYUU1B U0sKKy5QcSBMaSBzdHJ1Y3QgYnBmX3FtYXNrX2JpdHMKK0dldCBUWCBxdWV1ZSBtYXNrIGJpdHMu CisKKy5JdCBEdiBCSU9DR05PUU1BU0sKKy5QcSBMaSBpbnQKK1NldCBtYXNrIGJpdCBmb3IgdGhl IHBhY2tldHMgd2hpY2ggbm90IHRpZWQgd2l0aCBhbnkgcXVldWVzLgorCisuSXQgRHYgQklPQ1NO T1FNQVNLCisuUHEgTGkgaW50CitHZXQgbWFzayBiaXQgZm9yIHRoZSBwYWNrZXRzIHdoaWNoIG5v dCB0aWVkIHdpdGggYW55IHF1ZXVlcy4KKwogLkVsCiAuU2ggQlBGIEhFQURFUgogT25lIG9mIHRo ZSBmb2xsb3dpbmcgc3RydWN0dXJlcyBpcyBwcmVwZW5kZWQgdG8gZWFjaCBwYWNrZXQgcmV0dXJu ZWQgYnkKQEAgLTEwMzcsNiArMTA2NywyMyBAQAogCUJQRl9TVE1UKEJQRl9SRVQrQlBGX0ssIDAp LAogfTsKIC5FZAorLlNoIE1VTFRJUVVFVUUgU1VQUE9SVAorTXVsdGlxdWV1ZSBuZXR3b3JrIGlu dGVyZmFjZSBzdXBwb3J0IGZ1bmN0aW9uIHByb3ZpZGVzIGludGVyZmFjZXMgZm9yIAorbXVsdGl0 aHJlYWRlZCBwYWNrZXQgcHJvY2Vzc2luZyB1c2luZyBicGYuCisKK05vcm1hbCBicGYgY2FuIHJl Y2VpdmUgcGFja2V0cyBmcm9tIHNwZWNpZmllZCBpbnRlcmZhY2UsIG11bHRpcXVldWUgc3VwcG9y dCAKK2Z1bmN0aW9uIGNhbiByZWNlaXZlIHBhY2tldHMgZnJvbSBzcGVjaWZpZWQgaGFyZHdhcmUg cXVldWUuCisKK1RoaXMgZGlzdHJpYnV0ZXMgYnBmIHdvcmtsb2FkIG9uIG11bHRpcGxlIHRocmVh ZHMsIGFsc28gcmVkdWNlcyBsb2NrIAorY29udGVudGlvbiBvbiBicGYuCisKK1RvIG1ha2UgeW91 ciBwcm9ncmFtIG11bHRpdGhyZWFkZWQsIHlvdSdsbCBuZWVkIHRvIG9wZW4gYnBmIGRlc2NyaXB0 b3Igb24gZWFjaCAKK3RocmVhZCwgZW5hYmxlIG11bHRpcXVldWUgc3VwcG9ydCBieSBCSU9DUU1B U0tFTkFCTEUgaW9jdGwsIGFuZCBzZXQgcXVldWUgbWFzayBieSBCSU9DU1JYUU1BU0sgLyBCSU9D U1RYUU1BU0sgLyBCSU9DU05PUU1BU0sgaW9jdGxzLgorCitRdWV1ZSBsZW5ndGggYW5kIHF1ZXVl IGFmZmluaXR5IGluZm9ybWF0aW9uIG1heSB1c2VmdWwgdG8gb3B0aW1pemUgc2V0dGluZyAKK3F1 ZXVlIG1hc2sgb24gYnBmIGRlc2NyaXB0b3IsIHNlZQorLlhyIG5ldGludHJvIDQgLgorCiAuU2gg U0VFIEFMU08KIC5YciB0Y3BkdW1wIDEgLAogLlhyIGlvY3RsIDIgLApJbmRleDogc3lzL2Rldi9p eGdiZS9peGdiZS5jCj09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT0KLS0tIHN5cy9kZXYvaXhnYmUvaXhnYmUuYwkoLi4uL2hl YWQpCShyZXZpc2lvbiAyNTUxODApCisrKyBzeXMvZGV2L2l4Z2JlL2l4Z2JlLmMJKC4uLi91c2Vy L3N5dXUvbXFfYnBmKQkocmV2aXNpb24gMjU1MjAwKQpAQCAtNzUxLDYgKzc1MSwxMSBAQAogCQkJ CUlGUV9EUlZfUFJFUEVORCgmaWZwLT5pZl9zbmQsIG1faGVhZCk7CiAJCQlicmVhazsKIAkJfQor CisJCW1faGVhZC0+bV9mbGFncyB8PSBNX1FVRVVFSUQ7CisJCW1faGVhZC0+bV9wa3RoZHIucXVl dWVpZCA9IHR4ci0+bWU7CisJCW1faGVhZC0+bV9wa3RoZHIucXVldWV0eXBlID0gUVVFVUVUWVBF X1RYOworCiAJCS8qIFNlbmQgYSBjb3B5IG9mIHRoZSBmcmFtZSB0byB0aGUgQlBGIGxpc3RlbmVy ICovCiAJCUVUSEVSX0JQRl9NVEFQKGlmcCwgbV9oZWFkKTsKIApAQCAtODQ5LDYgKzg1NCwxMSBA QAogCQlkcmJyX2FkdmFuY2UoaWZwLCB0eHItPmJyKTsKICNlbmRpZgogCQllbnF1ZXVlZCsrOwor IAorCQluZXh0LT5tX2ZsYWdzIHw9IE1fUVVFVUVJRDsKKyAJCW5leHQtPm1fcGt0aGRyLnF1ZXVl aWQgPSB0eHItPm1lOworCQluZXh0LT5tX3BrdGhkci5xdWV1ZXR5cGUgPSBRVUVVRVRZUEVfVFg7 CisKIAkJLyogU2VuZCBhIGNvcHkgb2YgdGhlIGZyYW1lIHRvIHRoZSBCUEYgbGlzdGVuZXIgKi8K IAkJRVRIRVJfQlBGX01UQVAoaWZwLCBuZXh0KTsKIAkJaWYgKChpZnAtPmlmX2Rydl9mbGFncyAm IElGRl9EUlZfUlVOTklORykgPT0gMCkKQEAgLTI2MzcsNiArMjY0Nyw3IEBACiAJaWZwLT5pZl9j YXBhYmlsaXRpZXMgfD0gSUZDQVBfVkxBTl9IV1RBR0dJTkcKIAkJCSAgICAgfCAgSUZDQVBfVkxB Tl9IV1RTTwogCQkJICAgICB8ICBJRkNBUF9WTEFOX01UVTsKKwlpZnAtPmlmX2NhcGFiaWxpdGll cyB8PSBJRkNBUF9RVUVVRUlEOwogCWlmcC0+aWZfY2FwZW5hYmxlID0gaWZwLT5pZl9jYXBhYmls aXRpZXM7CiAKIAkvKgpAQCAtNDU0Niw4ICs0NTU3LDEwIEBACiAJCQkJaXhnYmVfcnhfY2hlY2tz dW0oc3RhdGVyciwgc2VuZG1wLCBwdHlwZSk7CiAjaWYgX19GcmVlQlNEX3ZlcnNpb24gPj0gODAw MDAwCiAJCQlzZW5kbXAtPm1fcGt0aGRyLmZsb3dpZCA9IHF1ZS0+bXNpeDsKLQkJCXNlbmRtcC0+ bV9mbGFncyB8PSBNX0ZMT1dJRDsKKwkJCXNlbmRtcC0+bV9mbGFncyB8PSAoTV9GTE9XSUQgfCBN X1FVRVVFSUQpOwogI2VuZGlmCisJCQlzZW5kbXAtPm1fcGt0aGRyLnF1ZXVlaWQgPSBxdWUtPm1z aXg7CisJCQlzZW5kbXAtPm1fcGt0aGRyLnF1ZXVldHlwZSA9IFFVRVVFVFlQRV9SWDsKIAkJfQog bmV4dF9kZXNjOgogCQlidXNfZG1hbWFwX3N5bmMocnhyLT5yeGRtYS5kbWFfdGFnLCByeHItPnJ4 ZG1hLmRtYV9tYXAsCkluZGV4OiBzeXMvZGV2L2UxMDAwL2lmX2lnYi5jCj09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0KLS0t IHN5cy9kZXYvZTEwMDAvaWZfaWdiLmMJKC4uLi9oZWFkKQkocmV2aXNpb24gMjU1MTgwKQorKysg c3lzL2Rldi9lMTAwMC9pZl9pZ2IuYwkoLi4uL3VzZXIvc3l1dS9tcV9icGYpCShyZXZpc2lvbiAy NTUyMDApCkBAIC05MjAsNiArOTIwLDEwIEBACiAJCQlicmVhazsKIAkJfQogCisJCW1faGVhZC0+ bV9mbGFncyB8PSBNX1FVRVVFSUQ7CisJCW1faGVhZC0+bV9wa3RoZHIucXVldWVpZCA9IHR4ci0+ bWU7CisJCW1faGVhZC0+bV9wa3RoZHIucXVldWV0eXBlID0gUVVFVUVUWVBFX1RYOworCiAJCS8q IFNlbmQgYSBjb3B5IG9mIHRoZSBmcmFtZSB0byB0aGUgQlBGIGxpc3RlbmVyICovCiAJCUVUSEVS X0JQRl9NVEFQKGlmcCwgbV9oZWFkKTsKIApAQCAtMTAxOSw2ICsxMDIzLDkgQEAKIAkJaWZwLT5p Zl9vYnl0ZXMgKz0gbmV4dC0+bV9wa3RoZHIubGVuOwogCQlpZiAobmV4dC0+bV9mbGFncyAmIE1f TUNBU1QpCiAJCQlpZnAtPmlmX29tY2FzdHMrKzsKKwkJbmV4dC0+bV9mbGFncyB8PSBNX1FVRVVF SUQ7CisJCW5leHQtPm1fcGt0aGRyLnF1ZXVlaWQgPSB0eHItPm1lOworCQluZXh0LT5tX3BrdGhk ci5xdWV1ZXR5cGUgPSBRVUVVRVRZUEVfVFg7CiAJCUVUSEVSX0JQRl9NVEFQKGlmcCwgbmV4dCk7 CiAJCWlmICgoaWZwLT5pZl9kcnZfZmxhZ3MgJiBJRkZfRFJWX1JVTk5JTkcpID09IDApCiAJCQli cmVhazsKQEAgLTMxNjksNiArMzE3Niw3IEBACiAJKiogZW5hYmxlIHRoaXMgYW5kIGdldCBmdWxs IGhhcmR3YXJlIHRhZyBmaWx0ZXJpbmcuCiAJKi8KIAlpZnAtPmlmX2NhcGFiaWxpdGllcyB8PSBJ RkNBUF9WTEFOX0hXRklMVEVSOworCWlmcC0+aWZfY2FwYWJpbGl0aWVzIHw9IElGQ0FQX1FVRVVF SUQ7CiAKIAkvKgogCSAqIFNwZWNpZnkgdGhlIG1lZGlhIHR5cGVzIHN1cHBvcnRlZCBieSB0aGlz IGFkYXB0ZXIgYW5kIHJlZ2lzdGVyCkBAIC00ODkxLDggKzQ4OTksMTEgQEAKIAkJCX0KICNpZm5k ZWYgSUdCX0xFR0FDWV9UWAogCQkJcnhyLT5mbXAtPm1fcGt0aGRyLmZsb3dpZCA9IHF1ZS0+bXNp eDsKLQkJCXJ4ci0+Zm1wLT5tX2ZsYWdzIHw9IE1fRkxPV0lEOworCQkJcnhyLT5mbXAtPm1fZmxh Z3MgfD0gKE1fRkxPV0lEIHwgTV9RVUVVRUlEKTsKICNlbmRpZgorCQkJcnhyLT5mbXAtPm1fcGt0 aGRyLnF1ZXVlaWQgPSBxdWUtPm1zaXg7CisJCQlyeHItPmZtcC0+bV9wa3RoZHIucXVldWV0eXBl ID0gUVVFVUVUWVBFX1RYOworCiAJCQlzZW5kbXAgPSByeHItPmZtcDsKIAkJCS8qIE1ha2Ugc3Vy ZSB0byBzZXQgTV9QS1RIRFIuICovCiAJCQlzZW5kbXAtPm1fZmxhZ3MgfD0gTV9QS1RIRFI7Cklu ZGV4OiBzeXMvZGV2L214Z2UvaWZfbXhnZS5jCj09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0KLS0tIHN5cy9kZXYvbXhnZS9p Zl9teGdlLmMJKC4uLi9oZWFkKQkocmV2aXNpb24gMjU1MTgwKQorKysgc3lzL2Rldi9teGdlL2lm X214Z2UuYwkoLi4uL3VzZXIvc3l1dS9tcV9icGYpCShyZXZpc2lvbiAyNTUyMDApCkBAIC0yMjcy LDYgKzIyNzIsMTAgQEAKIAkJaWYgKG0gPT0gTlVMTCkgewogCQkJcmV0dXJuOwogCQl9CisJCW0t Pm1fZmxhZ3MgfD0gTV9RVUVVRUlEOworCQltLT5tX3BrdGhkci5xdWV1ZWlkID0gKHNzIC0gc2Mt PnNzKTsKKwkJbS0+bV9wa3RoZHIucXVldWV0eXBlID0gUVVFVUVUWVBFX1RYOworCiAJCS8qIGxl dCBCUEYgc2VlIGl0ICovCiAJCUJQRl9NVEFQKGlmcCwgbSk7CiAKQEAgLTIzMDYsNiArMjMxMCwx MCBAQAogCiAJaWYgKCFkcmJyX25lZWRzX2VucXVldWUoaWZwLCB0eC0+YnIpICYmCiAJICAgICgo dHgtPm1hc2sgLSAodHgtPnJlcSAtIHR4LT5kb25lKSkgPiB0eC0+bWF4X2Rlc2MpKSB7CisJCW0t Pm1fZmxhZ3MgfD0gTV9RVUVVRUlEOworCQltLT5tX3BrdGhkci5xdWV1ZWlkID0gKHNzIC0gc2Mt PnNzKTsKKwkJbS0+bV9wa3RoZHIucXVldWV0eXBlID0gUVVFVUVUWVBFX1RYOworCiAJCS8qIGxl dCBCUEYgc2VlIGl0ICovCiAJCUJQRl9NVEFQKGlmcCwgbSk7CiAJCS8qIGdpdmUgaXQgdG8gdGhl IG5pYyAqLwpAQCAtMjcxOCw3ICsyNzI2LDkgQEAKIAkvKiBmbG93aWQgb25seSB2YWxpZCBpZiBS U1MgaGFzaGluZyBpcyBlbmFibGVkICovCiAJaWYgKHNjLT5udW1fc2xpY2VzID4gMSkgewogCQlt LT5tX3BrdGhkci5mbG93aWQgPSAoc3MgLSBzYy0+c3MpOwotCQltLT5tX2ZsYWdzIHw9IE1fRkxP V0lEOworCQltLT5tX2ZsYWdzIHw9IChNX0ZMT1dJRCB8IE1fUVVFVUVJRCk7CisJCW0tPm1fcGt0 aGRyLnF1ZXVlaWQgPSAoc3MgLSBzYy0+c3MpOworCQltLT5tX3BrdGhkci5xdWV1ZXR5cGUgPSBR VUVVRVRZUEVfUlg7CiAJfQogCS8qIHBhc3MgdGhlIGZyYW1lIHVwIHRoZSBzdGFjayAqLwogCSgq aWZwLT5pZl9pbnB1dCkoaWZwLCBtKTsKQEAgLTQ4OTMsNiArNDkwMyw3IEBACiAjaWYgZGVmaW5l ZChJTkVUKSB8fCBkZWZpbmVkKElORVQ2KQogCWlmcC0+aWZfY2FwYWJpbGl0aWVzIHw9IElGQ0FQ X0xSTzsKICNlbmRpZgorCWlmcC0+aWZfY2FwYWJpbGl0aWVzIHw9IElGQ0FQX1FVRVVFSUQ7CiAK ICNpZmRlZiBNWEdFX05FV19WTEFOX0FQSQogCWlmcC0+aWZfY2FwYWJpbGl0aWVzIHw9IElGQ0FQ X1ZMQU5fSFdUQUdHSU5HIHwgSUZDQVBfVkxBTl9IV0NTVU07CkluZGV4OiBzeXMvbmV0L2JwZnEu aAo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09Ci0tLSBzeXMvbmV0L2JwZnEuaAkoLi4uL2hlYWQpCShyZXZpc2lvbiAwKQor Kysgc3lzL25ldC9icGZxLmgJKC4uLi91c2VyL3N5dXUvbXFfYnBmKQkocmV2aXNpb24gMjU1MjAw KQpAQCAtMCwwICsxLDMyIEBACisjaWZuZGVmIF9ORVRfQlBGUV9IXworI2RlZmluZSBfTkVUX0JQ RlFfSF8KKworI2luY2x1ZGUgPHN5cy9wYXJhbS5oPgorI2luY2x1ZGUgPHN5cy9iaXRzZXQuaD4K KyNpbmNsdWRlIDxzeXMvX2JpdHNldC5oPgorCisjZGVmaW5lIEJQRlFfQklUUwkJCTI1NgorQklU U0VUX0RFRklORShicGZfcW1hc2tfYml0cywgQlBGUV9CSVRTKTsKKworI2RlZmluZQlCUEZRX0NM UihuLCBwKQkJCUJJVF9DTFIoQlBGUV9CSVRTLCBuLCBwKQorI2RlZmluZQlCUEZRX0NPUFkoZiwg dCkJCQlCSVRfQ09QWShCUEZRX0JJVFMsIGYsIHQpCisjZGVmaW5lCUJQRlFfSVNTRVQobiwgcCkJ CUJJVF9JU1NFVChCUEZRX0JJVFMsIG4sIHApCisjZGVmaW5lCUJQRlFfU0VUKG4sIHApCQkJQklU X1NFVChCUEZRX0JJVFMsIG4sIHApCisjZGVmaW5lCUJQRlFfWkVSTyhwKSAJCQlCSVRfWkVSTyhC UEZRX0JJVFMsIHApCisjZGVmaW5lCUJQRlFfRklMTChwKSAJCQlCSVRfRklMTChCUEZRX0JJVFMs IHApCisjZGVmaW5lCUJQRlFfU0VUT0YobiwgcCkJCUJJVF9TRVRPRihCUEZRX0JJVFMsIG4sIHAp CisjZGVmaW5lCUJQRlFfRU1QVFkocCkJCQlCSVRfRU1QVFkoQlBGUV9CSVRTLCBwKQorI2RlZmlu ZQlCUEZRX0lTRlVMTFNFVChwKQkJQklUX0lTRlVMTFNFVChCUEZRX0JJVFMsIHApCisjZGVmaW5l CUJQRlFfU1VCU0VUKHAsIGMpCQlCSVRfU1VCU0VUKEJQRlFfQklUUywgcCwgYykKKyNkZWZpbmUJ QlBGUV9PVkVSTEFQKHAsIGMpCQlCSVRfT1ZFUkxBUChCUEZRX0JJVFMsIHAsIGMpCisjZGVmaW5l CUJQRlFfQ01QKHAsIGMpCQkJQklUX0NNUChCUEZRX0JJVFMsIHAsIGMpCisjZGVmaW5lCUJQRlFf T1IoZCwgcykJCQlCSVRfT1IoQlBGUV9CSVRTLCBkLCBzKQorI2RlZmluZQlCUEZRX0FORChkLCBz KQkJCUJJVF9BTkQoQlBGUV9CSVRTLCBkLCBzKQorI2RlZmluZQlCUEZRX05BTkQoZCwgcykJCQlC SVRfTkFORChCUEZRX0JJVFMsIGQsIHMpCisjZGVmaW5lCUJQRlFfQ0xSX0FUT01JQyhuLCBwKQkJ QklUX0NMUl9BVE9NSUMoQlBGUV9CSVRTLCBuLCBwKQorI2RlZmluZQlCUEZRX1NFVF9BVE9NSUMo biwgcCkJCUJJVF9TRVRfQVRPTUlDKEJQRlFfQklUUywgbiwgcCkKKyNkZWZpbmUJQlBGUV9PUl9B VE9NSUMoZCwgcykJCUJJVF9PUl9BVE9NSUMoQlBGUV9CSVRTLCBkLCBzKQorI2RlZmluZQlCUEZR X0NPUFlfU1RPUkVfUkVMKGYsIHQpCUJJVF9DT1BZX1NUT1JFX1JFTChCUEZRX0JJVFMsIGYsIHQp CisjZGVmaW5lCUJQRlFfRkZTKHApCQkJQklUX0ZGUyhCUEZRX0JJVFMsIHApCisKKyNlbmRpZgpJ bmRleDogc3lzL25ldC9pZi5oCj09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0KLS0tIHN5cy9uZXQvaWYuaAkoLi4uL2hlYWQp CShyZXZpc2lvbiAyNTUxODApCisrKyBzeXMvbmV0L2lmLmgJKC4uLi91c2VyL3N5dXUvbXFfYnBm KQkocmV2aXNpb24gMjU1MjAwKQpAQCAtMjMxLDYgKzIzMSw3IEBACiAjZGVmaW5lCUlGQ0FQX05F VE1BUAkJMHgxMDAwMDAgLyogbmV0bWFwIG1vZGUgc3VwcG9ydGVkL2VuYWJsZWQgKi8KICNkZWZp bmUJSUZDQVBfUlhDU1VNX0lQVjYJMHgyMDAwMDAgIC8qIGNhbiBvZmZsb2FkIGNoZWNrc3VtIG9u IElQdjYgUlggKi8KICNkZWZpbmUJSUZDQVBfVFhDU1VNX0lQVjYJMHg0MDAwMDAgIC8qIGNhbiBv ZmZsb2FkIGNoZWNrc3VtIG9uIElQdjYgVFggKi8KKyNkZWZpbmUJSUZDQVBfUVVFVUVJRAkJMHg4 MDAwMDAgIC8qIGRyaXZlciBzdXBwb3J0cyBxdWV1ZWlkIG5vdGlmeSAqLwogCiAjZGVmaW5lIElG Q0FQX0hXQ1NVTV9JUFY2CShJRkNBUF9SWENTVU1fSVBWNiB8IElGQ0FQX1RYQ1NVTV9JUFY2KQog CkluZGV4OiBzeXMvbmV0L2JwZi5jCj09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0KLS0tIHN5cy9uZXQvYnBmLmMJKC4uLi9o ZWFkKQkocmV2aXNpb24gMjU1MTgwKQorKysgc3lzL25ldC9icGYuYwkoLi4uL3VzZXIvc3l1dS9t cV9icGYpCShyZXZpc2lvbiAyNTUyMDApCkBAIC04MTksNiArODE5LDkgQEAKIAlzaXplID0gZC0+ YmRfYnVmc2l6ZTsKIAlicGZfYnVmZmVyX2lvY3RsX3NibGVuKGQsICZzaXplKTsKIAorIAlkLT5i ZF9xbWFzay5xbV9lbmFibGVkID0gRkFMU0U7CisJQlBGUV9MT0NLX0lOSVQoJmQtPmJkX3FtYXNr KTsKKwogCXJldHVybiAoMCk7CiB9CiAKQEAgLTE2OTcsNyArMTcwMCwxOTEgQEAKIAljYXNlIEJJ T0NST1RaQlVGOgogCQllcnJvciA9IGJwZl9pb2N0bF9yb3R6YnVmKHRkLCBkLCAoc3RydWN0IGJw Zl96YnVmICopYWRkcik7CiAJCWJyZWFrOworCisJY2FzZSBCSU9DUU1BU0tFTkFCTEU6CisJCXsK KwkJCXN0cnVjdCBpZm5ldCAqaWZwOworCisJCQlpZiAoZC0+YmRfYmlmID09IE5VTEwpIHsKKwkJ CQkvKgorCQkJCSAqIE5vIGludGVyZmFjZSBhdHRhY2hlZCB5ZXQuCisJCQkJICovCisJCQkJZXJy b3IgPSBFSU5WQUw7CisJCQkJYnJlYWs7CisJCQl9CisJCQlCUEZRX1dMT0NLKCZkLT5iZF9xbWFz ayk7CisJCQlpZiAoZC0+YmRfcW1hc2sucW1fZW5hYmxlZCkgeworCQkJCUJQRlFfV1VOTE9DSygm ZC0+YmRfcW1hc2spOworCQkJCWVycm9yID0gRUlOVkFMOworCQkJCWJyZWFrOworCQkJfQorCQkJ aWZwID0gZC0+YmRfYmlmLT5iaWZfaWZwOworCQkJaWYgKCEoaWZwLT5pZl9jYXBhYmlsaXRpZXMg JiBJRkNBUF9RVUVVRUlEKSkgeworCQkJCUJQRlFfV1VOTE9DSygmZC0+YmRfcW1hc2spOworCQkJ CWVycm9yID0gRUlOVkFMOworCQkJCWJyZWFrOworCQkJfQorCQkJQlBGUV9aRVJPKCZkLT5iZF9x bWFzay5xbV9yeHFtYXNrKTsKKwkJCUJQRlFfWkVSTygmZC0+YmRfcW1hc2sucW1fdHhxbWFzayk7 CisJCQlkLT5iZF9xbWFzay5xbV9ub3FtYXNrID0gRkFMU0U7CisJCQlkLT5iZF9xbWFzay5xbV9l bmFibGVkID0gVFJVRTsKKwkJCUJQRlFfV1VOTE9DSygmZC0+YmRfcW1hc2spOworCQkJYnJlYWs7 CisJCX0KKworCWNhc2UgQklPQ1FNQVNLRElTQUJMRToKKwkJeworCQkJaWYgKGQtPmJkX2JpZiA9 PSBOVUxMKSB7CisJCQkJLyoKKwkJCQkgKiBObyBpbnRlcmZhY2UgYXR0YWNoZWQgeWV0LgorCQkJ CSAqLworCQkJCWVycm9yID0gRUlOVkFMOworCQkJCWJyZWFrOworCQkJfQorCQkJQlBGUV9XTE9D SygmZC0+YmRfcW1hc2spOworCQkJaWYgKCFkLT5iZF9xbWFzay5xbV9lbmFibGVkKSB7CisJCQkJ QlBGUV9XVU5MT0NLKCZkLT5iZF9xbWFzayk7CisJCQkJZXJyb3IgPSBFSU5WQUw7CisJCQkJYnJl YWs7CisJCQl9CisJCQlkLT5iZF9xbWFzay5xbV9lbmFibGVkID0gRkFMU0U7CisJCQlCUEZRX1dV TkxPQ0soJmQtPmJkX3FtYXNrKTsKKwkJCWJyZWFrOworCQl9CisKKwljYXNlIEJJT0NHUlhRTUFT SzoKKwkJeworCQkJc3RydWN0IGJwZl9xbWFza19iaXRzICpxbWFzayA9IChzdHJ1Y3QgYnBmX3Ft YXNrX2JpdHMgKilhZGRyOworCisJCQlpZiAoZC0+YmRfYmlmID09IE5VTEwpIHsKKwkJCQkvKgor CQkJCSAqIE5vIGludGVyZmFjZSBhdHRhY2hlZCB5ZXQuCisJCQkJICovCisJCQkJZXJyb3IgPSBF SU5WQUw7CisJCQkJYnJlYWs7CisJCQl9CisJCQlCUEZRX1dMT0NLKCZkLT5iZF9xbWFzayk7CisJ CQlpZiAoIWQtPmJkX3FtYXNrLnFtX2VuYWJsZWQpIHsKKwkJCQlCUEZRX1dVTkxPQ0soJmQtPmJk X3FtYXNrKTsKKwkJCQllcnJvciA9IEVJTlZBTDsKKwkJCQlicmVhazsKKwkJCX0KKwkJCUJQRlFf Q09QWSgmZC0+YmRfcW1hc2sucW1fcnhxbWFzaywgcW1hc2spOworCQkJQlBGUV9XVU5MT0NLKCZk LT5iZF9xbWFzayk7CisJCQlicmVhazsKKworCQl9CisKKwljYXNlIEJJT0NTUlhRTUFTSzoKKwkJ eworCQkJc3RydWN0IGJwZl9xbWFza19iaXRzICpxbWFzayA9IChzdHJ1Y3QgYnBmX3FtYXNrX2Jp dHMgKilhZGRyOworCisJCQlpZiAoZC0+YmRfYmlmID09IE5VTEwpIHsKKwkJCQkvKgorCQkJCSAq IE5vIGludGVyZmFjZSBhdHRhY2hlZCB5ZXQuCisJCQkJICovCisJCQkJZXJyb3IgPSBFSU5WQUw7 CQorCQkJCWJyZWFrOworCQkJfQorCQkJQlBGUV9XTE9DSygmZC0+YmRfcW1hc2spOworCQkJaWYg KCFkLT5iZF9xbWFzay5xbV9lbmFibGVkKSB7CisJCQkJQlBGUV9XVU5MT0NLKCZkLT5iZF9xbWFz ayk7CisJCQkJZXJyb3IgPSBFSU5WQUw7CisJCQkJYnJlYWs7CisJCQl9CisJCQlCUEZRX0NPUFko cW1hc2ssICZkLT5iZF9xbWFzay5xbV9yeHFtYXNrKTsKKwkJCUJQRlFfV1VOTE9DSygmZC0+YmRf cW1hc2spOworCQkJYnJlYWs7CisJCX0KKworCWNhc2UgQklPQ0dUWFFNQVNLOgorCQl7CisJCQlz dHJ1Y3QgYnBmX3FtYXNrX2JpdHMgKnFtYXNrID0gKHN0cnVjdCBicGZfcW1hc2tfYml0cyAqKWFk ZHI7CisKKwkJCWlmIChkLT5iZF9iaWYgPT0gTlVMTCkgeworCQkJCS8qCisJCQkJICogTm8gaW50 ZXJmYWNlIGF0dGFjaGVkIHlldC4KKwkJCQkgKi8KKwkJCQllcnJvciA9IEVJTlZBTDsKKwkJCQli cmVhazsKKwkJCX0KKwkJCUJQRlFfV0xPQ0soJmQtPmJkX3FtYXNrKTsKKwkJCWlmICghZC0+YmRf cW1hc2sucW1fZW5hYmxlZCkgeworCQkJCUJQRlFfV1VOTE9DSygmZC0+YmRfcW1hc2spOworCQkJ CWVycm9yID0gRUlOVkFMOworCQkJCWJyZWFrOworCQkJfQorCQkJQlBGUV9DT1BZKCZkLT5iZF9x bWFzay5xbV90eHFtYXNrLCBxbWFzayk7CisJCQlCUEZRX1dVTkxPQ0soJmQtPmJkX3FtYXNrKTsK KwkJCWJyZWFrOworCQl9CisKKwljYXNlIEJJT0NTVFhRTUFTSzoKKwkJeworCQkJc3RydWN0IGJw Zl9xbWFza19iaXRzICpxbWFzayA9IChzdHJ1Y3QgYnBmX3FtYXNrX2JpdHMgKilhZGRyOworCisJ CQlpZiAoZC0+YmRfYmlmID09IE5VTEwpIHsKKwkJCQkvKgorCQkJCSAqIE5vIGludGVyZmFjZSBh dHRhY2hlZCB5ZXQuCisJCQkJICovCisJCQkJZXJyb3IgPSBFSU5WQUw7CQorCQkJCWJyZWFrOwor CQkJfQorCQkJQlBGUV9XTE9DSygmZC0+YmRfcW1hc2spOworCQkJaWYgKCFkLT5iZF9xbWFzay5x bV9lbmFibGVkKSB7CisJCQkJQlBGUV9XVU5MT0NLKCZkLT5iZF9xbWFzayk7CisJCQkJZXJyb3Ig PSBFSU5WQUw7CisJCQkJYnJlYWs7CisJCQl9CisJCQlCUEZRX0NPUFkocW1hc2ssICZkLT5iZF9x bWFzay5xbV90eHFtYXNrKTsKKwkJCUJQRlFfV1VOTE9DSygmZC0+YmRfcW1hc2spOworCQkJYnJl YWs7CisJCX0KKworCWNhc2UgQklPQ0dOT1FNQVNLOgorCQl7CisJCQlib29sZWFuX3QgKm5vcW1h c2sgPSAoYm9vbGVhbl90ICopYWRkcjsKKwkJCWlmIChkLT5iZF9iaWYgPT0gTlVMTCkgeworCQkJ CS8qCisJCQkJICogTm8gaW50ZXJmYWNlIGF0dGFjaGVkIHlldC4KKwkJCQkgKi8KKwkJCQllcnJv ciA9IEVJTlZBTDsKKwkJCQlicmVhazsKKwkJCX0KKwkJCUJQRlFfV0xPQ0soJmQtPmJkX3FtYXNr KTsKKwkJCWlmICghZC0+YmRfcW1hc2sucW1fZW5hYmxlZCkgeworCQkJCUJQRlFfV1VOTE9DSygm ZC0+YmRfcW1hc2spOworCQkJCWVycm9yID0gRUlOVkFMOworCQkJCWJyZWFrOworCQkJfQorCQkJ Km5vcW1hc2sgPSBkLT5iZF9xbWFzay5xbV9ub3FtYXNrOworCQkJQlBGUV9XVU5MT0NLKCZkLT5i ZF9xbWFzayk7CisJCQlicmVhazsKKwkJfQorCisJY2FzZSBCSU9DU05PUU1BU0s6CisJCXsKKwkJ CWJvb2xlYW5fdCAqbm9xbWFzayA9IChib29sZWFuX3QgKilhZGRyOworCisJCQlpZiAoZC0+YmRf YmlmID09IE5VTEwpIHsKKwkJCQkvKgorCQkJCSAqIE5vIGludGVyZmFjZSBhdHRhY2hlZCB5ZXQu CisJCQkJICovCisJCQkJZXJyb3IgPSBFSU5WQUw7CQorCQkJCWJyZWFrOworCQkJfQorCQkJQlBG UV9XTE9DSygmZC0+YmRfcW1hc2spOworCQkJaWYgKCFkLT5iZF9xbWFzay5xbV9lbmFibGVkKSB7 CisJCQkJQlBGUV9XVU5MT0NLKCZkLT5iZF9xbWFzayk7CisJCQkJZXJyb3IgPSBFSU5WQUw7CisJ CQkJYnJlYWs7CisJCQl9CisJCQlkLT5iZF9xbWFzay5xbV9ub3FtYXNrID0gKm5vcW1hc2s7CisJ CQlCUEZRX1dVTkxPQ0soJmQtPmJkX3FtYXNrKTsKKwkJCWJyZWFrOworCQl9CiAJfQorCiAJQ1VS Vk5FVF9SRVNUT1JFKCk7CiAJcmV0dXJuIChlcnJvcik7CiB9CkBAIC0yMDQzLDYgKzIyMzAsMTUg QEAKIAlCUEZJRl9STE9DSyhicCk7CiAKIAlMSVNUX0ZPUkVBQ0goZCwgJmJwLT5iaWZfZGxpc3Qs IGJkX25leHQpIHsKKyAJCUJQRlFfUkxPQ0soJmQtPmJkX3FtYXNrKTsKKyAJCWlmIChkLT5iZF9x bWFzay5xbV9lbmFibGVkKSB7CisgCQkJaWYgKCFkLT5iZF9xbWFzay5xbV9ub3FtYXNrKSB7CisJ CQkJQlBGUV9SVU5MT0NLKCZkLT5iZF9xbWFzayk7CisgCQkJCWNvbnRpbnVlOworIAkJCX0KKyAJ CX0KKwkJQlBGUV9SVU5MT0NLKCZkLT5iZF9xbWFzayk7CisKIAkJLyoKIAkJICogV2UgYXJlIG5v dCB1c2luZyBhbnkgbG9ja3MgZm9yIGQgaGVyZSBiZWNhdXNlOgogCQkgKiAxKSBhbnkgZmlsdGVy IGNoYW5nZSBpcyBwcm90ZWN0ZWQgYnkgaW50ZXJmYWNlCkBAIC0yMTE3LDYgKzIzMTMsNDAgQEAK IAlCUEZJRl9STE9DSyhicCk7CiAKIAlMSVNUX0ZPUkVBQ0goZCwgJmJwLT5iaWZfZGxpc3QsIGJk X25leHQpIHsKKyAJCUJQRlFfUkxPQ0soJmQtPmJkX3FtYXNrKTsKKyAJCWlmIChkLT5iZF9xbWFz ay5xbV9lbmFibGVkKSB7CisgCQkJTV9BU1NFUlRQS1RIRFIobSk7CisgCQkJaWYgKG0tPm1fZmxh Z3MgJiBNX1FVRVVFSUQpIHsKKwkJCQlzd2l0Y2ggKG0tPm1fcGt0aGRyLnF1ZXVldHlwZSkgewor CQkJCWNhc2UgUVVFVUVUWVBFX1JYOgorCQkJCQlpZiAoIUJQRlFfSVNTRVQobS0+bV9wa3RoZHIu cXVldWVpZCwKKwkJCQkJCSAmZC0+YmRfcW1hc2sucW1fcnhxbWFzaykpIHsKKyAJCQkJCQlCUEZR X1JVTkxPQ0soJmQtPmJkX3FtYXNrKTsKKyAJCQkJCQljb250aW51ZTsKKyAJCQkJCX0KKwkJCQkJ YnJlYWs7CisJCQkJY2FzZSBRVUVVRVRZUEVfVFg6CisJCQkJCWlmICghQlBGUV9JU1NFVChtLT5t X3BrdGhkci5xdWV1ZWlkLCAKKwkJCQkJCSZkLT5iZF9xbWFzay5xbV9yeHFtYXNrKSkgeworIAkJ CQkJCUJQRlFfUlVOTE9DSygmZC0+YmRfcW1hc2spOworIAkJCQkJCWNvbnRpbnVlOworIAkJCQkJ fQorCQkJCQlicmVhazsKKwkJCQlkZWZhdWx0OgorCQkJCQlpZiAoIWQtPmJkX3FtYXNrLnFtX25v cW1hc2spIHsKKwkJCQkJCUJQRlFfUlVOTE9DSygmZC0+YmRfcW1hc2spOworCQkJCQkJY29udGlu dWU7CisJCQkJCX0KKyAJCQkJfQorIAkJCX1lbHNleworCQkJCWlmICghZC0+YmRfcW1hc2sucW1f bm9xbWFzaykgeworCQkJCQlCUEZRX1JVTkxPQ0soJmQtPmJkX3FtYXNrKTsKKwkJCQkJY29udGlu dWU7CisJCQkJfQorCQkJfQorIAkJfQorIAkJQlBGUV9SVU5MT0NLKCZkLT5iZF9xbWFzayk7Cisg CiAJCWlmIChCUEZfQ0hFQ0tfRElSRUNUSU9OKGQsIG0tPm1fcGt0aGRyLnJjdmlmLCBicC0+Ymlm X2lmcCkpCiAJCQljb250aW51ZTsKIAkJKytkLT5iZF9yY291bnQ7CkBAIC0yMTgwLDYgKzI0MTAs NDAgQEAKIAlCUEZJRl9STE9DSyhicCk7CiAKIAlMSVNUX0ZPUkVBQ0goZCwgJmJwLT5iaWZfZGxp c3QsIGJkX25leHQpIHsKKyAgCQlCUEZRX1JMT0NLKCZkLT5iZF9xbWFzayk7CisgCQlpZiAoZC0+ YmRfcW1hc2sucW1fZW5hYmxlZCkgeworIAkJCU1fQVNTRVJUUEtUSERSKG0pOworIAkJCWlmICht LT5tX2ZsYWdzICYgTV9RVUVVRUlEKSB7CisJCQkJc3dpdGNoIChtLT5tX3BrdGhkci5xdWV1ZXR5 cGUpIHsKKwkJCQljYXNlIFFVRVVFVFlQRV9SWDoKKwkJCQkJaWYgKCFCUEZRX0lTU0VUKG0tPm1f cGt0aGRyLnF1ZXVlaWQsCisJCQkJCQkgJmQtPmJkX3FtYXNrLnFtX3J4cW1hc2spKSB7CisgCQkJ CQkJQlBGUV9SVU5MT0NLKCZkLT5iZF9xbWFzayk7CisgCQkJCQkJY29udGludWU7CisgCQkJCQl9 CisJCQkJCWJyZWFrOworCQkJCWNhc2UgUVVFVUVUWVBFX1RYOgorCQkJCQlpZiAoIUJQRlFfSVNT RVQobS0+bV9wa3RoZHIucXVldWVpZCwgCisJCQkJCQkmZC0+YmRfcW1hc2sucW1fcnhxbWFzaykp IHsKKyAJCQkJCQlCUEZRX1JVTkxPQ0soJmQtPmJkX3FtYXNrKTsKKyAJCQkJCQljb250aW51ZTsK KyAJCQkJCX0KKwkJCQkJYnJlYWs7CisJCQkJZGVmYXVsdDoKKwkJCQkJaWYgKCFkLT5iZF9xbWFz ay5xbV9ub3FtYXNrKSB7CisJCQkJCQlCUEZRX1JVTkxPQ0soJmQtPmJkX3FtYXNrKTsKKwkJCQkJ CWNvbnRpbnVlOworCQkJCQl9CisgCQkJCX0KKyAJCQl9ZWxzZXsKKwkJCQlpZiAoIWQtPmJkX3Ft YXNrLnFtX25vcW1hc2spIHsKKwkJCQkJQlBGUV9SVU5MT0NLKCZkLT5iZF9xbWFzayk7CisJCQkJ CWNvbnRpbnVlOworCQkJCX0KKwkJCX0KKyAJCX0KKyAJCUJQRlFfUlVOTE9DSygmZC0+YmRfcW1h c2spOworCiAJCWlmIChCUEZfQ0hFQ0tfRElSRUNUSU9OKGQsIG0tPm1fcGt0aGRyLnJjdmlmLCBi cC0+YmlmX2lmcCkpCiAJCQljb250aW51ZTsKIAkJKytkLT5iZF9yY291bnQ7CkluZGV4OiBzeXMv bmV0L2JwZmRlc2MuaAo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBzeXMvbmV0L2JwZmRlc2MuaAkoLi4uL2hlYWQp CShyZXZpc2lvbiAyNTUxODApCisrKyBzeXMvbmV0L2JwZmRlc2MuaAkoLi4uL3VzZXIvc3l1dS9t cV9icGYpCShyZXZpc2lvbiAyNTUyMDApCkBAIC00NCw3ICs0NCwyMyBAQAogI2luY2x1ZGUgPHN5 cy9xdWV1ZS5oPgogI2luY2x1ZGUgPHN5cy9jb25mLmg+CiAjaW5jbHVkZSA8bmV0L2lmLmg+Cisj aW5jbHVkZSA8bmV0L2JwZnEuaD4KIAorc3RydWN0IGJwZl9xbWFzayB7CisJaW50CQlxbV9lbmFi bGVkOworCXN0cnVjdCBicGZfcW1hc2tfYml0cyBxbV9yeHFtYXNrOworCXN0cnVjdCBicGZfcW1h c2tfYml0cyBxbV90eHFtYXNrOworCWludAkJcW1fbm9xbWFzazsKKwlzdHJ1Y3Qgcndsb2NrCXFt X2xvY2s7Cit9OworCisjZGVmaW5lIEJQRlFfTE9DS19JTklUKHFtKQlyd19pbml0KCYocW0pLT5x bV9sb2NrLCAicW1hc2sgbG9jayIpCisjZGVmaW5lIEJQRlFfTE9DS19ERVNUUk9ZKHFtKQlyd19k ZXN0cm95KCYocW0pLT5xbV9sb2NrKQorI2RlZmluZSBCUEZRX1JMT0NLKHFtKQkJcndfcmxvY2so JihxbSktPnFtX2xvY2spCisjZGVmaW5lIEJQRlFfUlVOTE9DSyhxbSkJcndfcnVubG9jaygmKHFt KS0+cW1fbG9jaykKKyNkZWZpbmUgQlBGUV9XTE9DSyhxbSkJCXJ3X3dsb2NrKCYocW0pLT5xbV9s b2NrKQorI2RlZmluZSBCUEZRX1dVTkxPQ0socW0pCXJ3X3d1bmxvY2soJihxbSktPnFtX2xvY2sp CisKIC8qCiAgKiBEZXNjcmlwdG9yIGFzc29jaWF0ZWQgd2l0aCBlYWNoIG9wZW4gYnBmIGZpbGUu CiAgKi8KQEAgLTEwMSw2ICsxMTcsNyBAQAogCXVfaW50NjRfdAliZF93ZGNvdW50OwkvKiBudW1i ZXIgb2YgcGFja2V0cyBkcm9wcGVkIGR1cmluZyBhIHdyaXRlICovCiAJdV9pbnQ2NF90CWJkX3pj b3B5OwkvKiBudW1iZXIgb2YgemVybyBjb3B5IG9wZXJhdGlvbnMgKi8KIAl1X2NoYXIJCWJkX2Nv bXBhdDMyOwkvKiAzMi1iaXQgc3RyZWFtIG9uIExQNjQgc3lzdGVtICovCisJc3RydWN0IGJwZl9x bWFzayBiZF9xbWFzazsKIH07CiAKIC8qIFZhbHVlcyBmb3IgYmRfc3RhdGUgKi8KSW5kZXg6IHN5 cy9uZXQvYnBmLmgKPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PQotLS0gc3lzL25ldC9icGYuaAkoLi4uL2hlYWQpCShyZXZp c2lvbiAyNTUxODApCisrKyBzeXMvbmV0L2JwZi5oCSguLi4vdXNlci9zeXV1L21xX2JwZikJKHJl dmlzaW9uIDI1NTIwMCkKQEAgLTQwLDYgKzQwLDggQEAKICNpZm5kZWYgX05FVF9CUEZfSF8KICNk ZWZpbmUgX05FVF9CUEZfSF8KIAorI2luY2x1ZGUgPG5ldC9icGZxLmg+CisKIC8qIEJTRCBzdHls ZSByZWxlYXNlIGRhdGUgKi8KICNkZWZpbmUJQlBGX1JFTEVBU0UgMTk5NjA2CiAKQEAgLTE0Nyw2 ICsxNDksMTQgQEAKICNkZWZpbmUJQklPQ1NFVEZOUglfSU9XKCdCJywgMTMwLCBzdHJ1Y3QgYnBm X3Byb2dyYW0pCiAjZGVmaW5lCUJJT0NHVFNUQU1QCV9JT1IoJ0InLCAxMzEsIHVfaW50KQogI2Rl ZmluZQlCSU9DU1RTVEFNUAlfSU9XKCdCJywgMTMyLCB1X2ludCkKKyNkZWZpbmUJQklPQ1FNQVNL RU5BQkxFCV9JTygnQicsIDEzMykKKyNkZWZpbmUJQklPQ1FNQVNLRElTQUJMRSBfSU8oJ0InLCAx MzQpCisjZGVmaW5lCUJJT0NHUlhRTUFTSwlfSU9SKCdCJywgMTM1LCBzdHJ1Y3QgYnBmX3FtYXNr X2JpdHMpCisjZGVmaW5lCUJJT0NTUlhRTUFTSwlfSU9XKCdCJywgMTM1LCBzdHJ1Y3QgYnBmX3Ft YXNrX2JpdHMpCisjZGVmaW5lCUJJT0NHVFhRTUFTSwlfSU9SKCdCJywgMTM2LCBzdHJ1Y3QgYnBm X3FtYXNrX2JpdHMpCisjZGVmaW5lCUJJT0NTVFhRTUFTSwlfSU9XKCdCJywgMTM3LCBzdHJ1Y3Qg YnBmX3FtYXNrX2JpdHMpCisjZGVmaW5lCUJJT0NHTk9RTUFTSwlfSU9SKCdCJywgMTM4LCBpbnQp CisjZGVmaW5lCUJJT0NTTk9RTUFTSwlfSU9XKCdCJywgMTM5LCBpbnQpCiAKIC8qIE9ic29sZXRl ICovCiAjZGVmaW5lCUJJT0NHU0VFU0VOVAlCSU9DR0RJUkVDVElPTgpJbmRleDogc3lzL3N5cy9t YnVmLmgKPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PQotLS0gc3lzL3N5cy9tYnVmLmgJKC4uLi9oZWFkKQkocmV2aXNpb24g MjU1MTgwKQorKysgc3lzL3N5cy9tYnVmLmgJKC4uLi91c2VyL3N5dXUvbXFfYnBmKQkocmV2aXNp b24gMjU1MjAwKQpAQCAtMTE0LDYgKzExNCwxMSBAQAogCXZvaWQJCQkoKm1fdGFnX2ZyZWUpKHN0 cnVjdCBtX3RhZyAqKTsKIH07CiAKK2VudW0gcXVldWV0eXBlIHsKKwlRVUVVRVRZUEVfUlgsCisJ UVVFVUVUWVBFX1RYCit9OworCiAvKgogICogUmVjb3JkL3BhY2tldCBoZWFkZXIgaW4gZmlyc3Qg bWJ1ZiBvZiBjaGFpbjsgdmFsaWQgb25seSBpZiBNX1BLVEhEUiBpcyBzZXQuCiAgKiBTaXplIElM UDMyOiA0OApAQCAtMTI2LDYgKzEzMSw4IEBACiAKIAkvKiBMYXllciBjcm9zc2luZyBwZXJzaXN0 ZW50IGluZm9ybWF0aW9uLiAqLwogCXVpbnQzMl90CSBmbG93aWQ7CS8qIHBhY2tldCdzIDQtdHVw bGUgc3lzdGVtICovCisJdWludDMyX3QJIHF1ZXVlaWQ7CS8qIGh3IHF1ZXVlIGlkICovCisJdWlu dDMyX3QJIHF1ZXVldHlwZTsJLyogaHcgcXVldWUgdHlwZSAqLwogCXVpbnQ2NF90CSBjc3VtX2Zs YWdzOwkvKiBjaGVja3N1bSBhbmQgb2ZmbG9hZCBmZWF0dXJlcyAqLwogCXVpbnQxNl90CSBmaWJu dW07CS8qIHRoaXMgcGFja2V0IHNob3VsZCB1c2UgdGhpcyBmaWIgKi8KIAl1aW50OF90CQkgY29z cW9zOwkvKiBjbGFzcy9xdWFsaXR5IG9mIHNlcnZpY2UgKi8KQEAgLTIyMyw2ICsyMzAsNyBAQAog I2RlZmluZQlNX1ZMQU5UQUcJMHgwMDAwMDA4MCAvKiBldGhlcl92dGFnIGlzIHZhbGlkICovCiAj ZGVmaW5lCU1fRkxPV0lECTB4MDAwMDAxMDAgLyogZGVwcmVjYXRlZDogZmxvd2lkIGlzIHZhbGlk ICovCiAjZGVmaW5lCU1fTk9GUkVFCTB4MDAwMDAyMDAgLyogZG8gbm90IGZyZWUgbWJ1ZiwgZW1i ZWRkZWQgaW4gY2x1c3RlciAqLworI2RlZmluZQlNX1FVRVVFSUQJMHgwMDAwMDQwMCAvKiBwYWNr ZXQgaGFzIGh3IHF1ZXVlIGlkICovCiAKICNkZWZpbmUJTV9QUk9UTzEJMHgwMDAwMTAwMCAvKiBw cm90b2NvbC1zcGVjaWZpYyAqLwogI2RlZmluZQlNX1BST1RPMgkweDAwMDAyMDAwIC8qIHByb3Rv Y29sLXNwZWNpZmljICovCkluZGV4OiBzYmluL2lmY29uZmlnL2lmY29uZmlnLmMKPT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PQotLS0gc2Jpbi9pZmNvbmZpZy9pZmNvbmZpZy5jCSguLi4vaGVhZCkJKHJldmlzaW9uIDI1NTE4 MCkKKysrIHNiaW4vaWZjb25maWcvaWZjb25maWcuYwkoLi4uL3VzZXIvc3l1dS9tcV9icGYpCShy ZXZpc2lvbiAyNTUyMDApCkBAIC05MTcsNyArOTE3LDcgQEAKICJcMDIwXDFSWENTVU1cMlRYQ1NV TVwzTkVUQ09OU1w0VkxBTl9NVFVcNVZMQU5fSFdUQUdHSU5HXDZKVU1CT19NVFVcN1BPTExJTkci IFwKICJcMTBWTEFOX0hXQ1NVTVwxMVRTTzRcMTJUU082XDEzTFJPXDE0V09MX1VDQVNUXDE1V09M X01DQVNUXDE2V09MX01BR0lDIiBcCiAiXDE3VE9FNFwyMFRPRTZcMjFWTEFOX0hXRklMVEVSXDIz VkxBTl9IV1RTT1wyNExJTktTVEFURVwyNU5FVE1BUCIgXAotIlwyNlJYQ1NVTV9JUFY2XDI3VFhD U1VNX0lQVjYiCisiXDI2UlhDU1VNX0lQVjZcMjdUWENTVU1fSVBWNlwyOFFVRVVFSUQiCiAKIC8q CiAgKiBQcmludCB0aGUgc3RhdHVzIG9mIHRoZSBpbnRlcmZhY2UuICBJZiBhbiBhZGRyZXNzIGZh bWlseSB3YXMK --047d7b6d82dc41494f04e58a43ab-- From owner-freebsd-net@FreeBSD.ORG Wed Sep 4 12:50:12 2013 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id B7C3FA2C for ; Wed, 4 Sep 2013 12:50:12 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 7EE4220BF for ; Wed, 4 Sep 2013 12:50:12 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqIEAAwsJ1KDaFve/2dsb2JhbABaFoMmUYMovguBPHSCJAEBBAEjVgUWGAICDRkCWQYTCYdzBgyoE5FEgSmOGTQHgmmBNAOpW4M8IIFu X-IronPort-AV: E=Sophos;i="4.89,1021,1367985600"; d="scan'208";a="49418467" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 04 Sep 2013 08:50:05 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 9969079283; Wed, 4 Sep 2013 08:50:05 -0400 (EDT) Date: Wed, 4 Sep 2013 08:50:05 -0400 (EDT) From: Rick Macklem To: David Wolfskill Message-ID: <979862494.17918795.1378299005617.JavaMail.root@uoguelph.ca> In-Reply-To: <20130903192734.GA19406@albert.catwhisker.org> Subject: Re: TSO and FreeBSD vs Linux MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.203] X-Mailer: Zimbra 7.2.1_GA_2790 (ZimbraWebClient - FF3.0 (Win)/7.2.1_GA_2790) Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Sep 2013 12:50:12 -0000 David Wolfskill wrote: > On Wed, Aug 21, 2013 at 07:12:38PM +0200, Andre Oppermann wrote: > > On 13.08.2013 19:29, Julian Elischer wrote: > > > I have been tracking down a performance embarrassment on AMAZON > > > EC2 and have found it I think. > > > Our OS cousins over at Linux land have implemented some > > > interesting behaviour when TSO is in use. > > > > There used to be a different problem with EC2 and FreeBSD TSO. The > > Xen hypervisor > > doesn't like large 64K TSO bursts we generate, the drivers drops > > the whole TSO chain, > > TCP gets upset and turns off TSO alltogether leaving the connection > > going at one > > packet a time as in the old days. > > ... > > My apologies for jumping in so late -- I'm not subscribed to -net@. > > At work, I received a new desktop machine a few months ago; here's a > recent history of what it has been running: > > FreeBSD 9.2-PRERELEASE #4 r254801M/254827:902501: Sun Aug 25 > 05:15:29 PDT 2013 root@dwolf-fbsd:/usr/obj/usr/src/sys/DWOLF > amd64 > FreeBSD 9.2-PRERELEASE #5 r255066M/255091:902503: Sat Aug 31 > 11:58:53 PDT 2013 root@dwolf-fbsd:/usr/obj/usr/src/sys/DWOLF > amd64 > FreeBSD 9.2-PRERELEASE #5 r255104M/255115:902503: Sun Sep 1 > 05:02:12 PDT 2013 root@dwolf-fbsd:/usr/obj/usr/src/sys/DWOLF > amd64 > > Now, I like to have a "private playground" for doing things with > machines, so I make use of both em(4) NICs on the machine: em0 > connects > to the rest of the work network; em1 is connected to a switch I > brought > in from home, and to which I connect "other things" (such as my > laptop). > And because I'm fairly comfortable with them, I use IPFW & natd. For > some folks here, none of that should come as a surprise. :-}) > > For reference, the em(4) devices in question are: > > em0@pci0:0:25:0: class=0x020000 card=0x060d15d9 > chip=0x10ef8086 rev=0x06 hdr=0x00 > vendor = 'Intel Corporation' > device = '82578DM Gigabit Network Connection' > > and > > em1@pci0:3:0:0: class=0x020000 card=0x060d15d9 chip=0x10d38086 > rev=0x00 hdr=0x00 > vendor = 'Intel Corporation' > device = '82574L Gigabit Network Connection' > > > > I noticed that when I tried to write files to NFS, I could write > small > files OK, but larger ones seemed to ... hang. > > Note: We don't use jumbo frames. (Work IT is convinced that they > don't help. I'm trying to better-understand their reasoning.) > > Further poking around showed that (under the above conditions): > * natd CPU% was climbing as more of the file was copied, up to 2^21 > bytes. (At that point, nothing further was saved on NFS.) > * dhcpd CPU% was also climbing. I tried killing that, but doing so > didn't affect the other results. (Killing natd made connectivity > cease, given the IPFW rules in effect.) > * Performing a tcpdump while trying to copy a file of length > 117709618 > showed lots of TCP retransmissions. In fact, I'd hazard that every > TCP > packet was getting retransmitted. > * "ifconfig -v em0" showed flags TSO4 & VLAN_HWTSO turned on. > * "sysctl net.inet.tcp.tso" showed "1" -- enabled. > > As soon as I issued "sudo net.inet.tcp.tso=0" ... the copy worked > without > a hitch or a whine. And I was able to copy all 117709618 bytes, not > just > 2097152 (2^21). > > Is the above expected? It came rather as a surprise to me. > Not surprising to me, I'm afraid. When there are serious NFS problems like this, it is often caused by a network fabric issue and broken TSO is at the top of the list w.r.t. cause. rick > Peace, > david > -- > David H. Wolfskill david@catwhisker.org > Taliban: Evil cowards with guns afraid of truth from a 14-year old > girl. > > See http://www.catwhisker.org/~david/publickey.gpg for my public key. > From owner-freebsd-net@FreeBSD.ORG Wed Sep 4 15:29:07 2013 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id D9BCC2B1 for ; Wed, 4 Sep 2013 15:29:07 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost1.sentex.ca (smarthost1-6.sentex.ca [IPv6:2607:f3e0:0:1::12]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 6E3112B9C for ; Wed, 4 Sep 2013 15:29:07 +0000 (UTC) Received: from [192.168.43.26] (pyroxene.sentex.ca [199.212.134.18]) by smarthost1.sentex.ca (8.14.7/8.14.5) with ESMTP id r84FQvNw049424; Wed, 4 Sep 2013 11:26:57 -0400 (EDT) (envelope-from mike@sentex.net) Message-ID: <52275136.1010105@sentex.net> Date: Wed, 04 Sep 2013 11:26:46 -0400 From: Mike Tancsa Organization: Sentex Communications User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:12.0) Gecko/20120428 Thunderbird/12.0.1 MIME-Version: 1.0 To: Rick Macklem Subject: TSO help or hindrance ? (was Re: TSO and FreeBSD vs Linux) References: <979862494.17918795.1378299005617.JavaMail.root@uoguelph.ca> In-Reply-To: <979862494.17918795.1378299005617.JavaMail.root@uoguelph.ca> X-Enigmail-Version: 1.4.2 Content-Type: multipart/mixed; boundary="------------090102030905000709060202" X-Scanned-By: MIMEDefang 2.74 on 64.7.153.18 Cc: FreeBSD Net , David Wolfskill X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Sep 2013 15:29:08 -0000 This is a multi-part message in MIME format. --------------090102030905000709060202 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit On 9/4/2013 8:50 AM, Rick Macklem wrote: > David Wolfskill wrote: >> >> >> I noticed that when I tried to write files to NFS, I could write >> small >> files OK, but larger ones seemed to ... hang. >> * "ifconfig -v em0" showed flags TSO4 & VLAN_HWTSO turned on. >> * "sysctl net.inet.tcp.tso" showed "1" -- enabled. >> >> As soon as I issued "sudo net.inet.tcp.tso=0" ... the copy worked >> without >> a hitch or a whine. And I was able to copy all 117709618 bytes, not >> just >> 2097152 (2^21). >> >> Is the above expected? It came rather as a surprise to me. >> > Not surprising to me, I'm afraid. When there are serious NFS problems > like this, it is often caused by a network fabric issue and broken > TSO is at the top of the list w.r.t. cause. I was just experimenting a bit with iSCSI via FreeNAS and was a little disappointed at the speeds I was getting. So, I tried disabling tso on both boxes and it did seem to speed things up a bit. Data and testing methods attached in a txt file. I did 3 cases. Just boot up FreeNAS and the initiator without tweaks. That had the worst performance. disable tso on the nic as well as via sysctl on both boxes. That had the best performance. re-enable tso on both boxes. That had better performance than the first case, but still not as good as totally disabling it. I am guessing something is not quite being re-enabled properly ? But its different than the other two cases ?!? tgt is FreeNAS-9.1.1-RELEASE-x64 (a752d35) and initiator is r254328 9.2 AMD64 The FreeNAS box has 16G of RAM, so the file is being served out of cache as gstat shows no activity when sending out the file ---Mike -- ------------------- Mike Tancsa, tel +1 519 651 3400 Sentex Communications, mike@sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada http://www.tancsa.com/ --------------090102030905000709060202 Content-Type: text/plain; charset=windows-1252; name="stat.txt" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="stat.txt" 3 data files. notso = tso disabled on tgt and initiator tso-boot = initiator is rebooted and tests are run tso = tso disabled and then re-enabled. For whatever reason its not as bad as post boot, but still worse than no tso 0{mdttestbox}# ministat notso tso-boot x notso + tso-boot +----------------------------------------------------------------------------------------------------------------------+ | + | | +++ x | |+ ++++ xxxx x x x x| | |AM| |___M__A_____| | +----------------------------------------------------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 9 69086843 71749085 69475320 69810666 887628.9 + 9 55846191 56473897 56357956 56302165 208843.3 Difference at 95.0% confidence -1.35085e+07 +/- 644386 -19.3502% +/- 0.923048% (Student's t, pooled s = 644787) 0{mdttestbox}# # ministat tso notso x tso + notso +----------------------------------------------------------------------------------------------------------------------+ | x + | |x xxxxx x + x + + + + + + +| | |____MA___||___________________M___________A_________________________________| | +----------------------------------------------------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 9 68632067 69126677 68779201 68808473 142050.01 + 9 69086843 71749085 69475320 69810666 887628.9 Difference at 95.0% confidence 1.00219e+06 +/- 635239 1.4565% +/- 0.923199% (Student's t, pooled s = 635635) 0{mdttestbox}# cat tso 68779201 68734143 68915705 68752827 68782212 69126677 68828520 68724901 68632067 0{mdttestbox}# cat notso 71749085 69256608 69097532 69086843 70587459 69179672 69475320 69754511 70108963 0{mdttestbox}# cat tso-boot 55846191 56314173 56284204 56095729 56466769 56446535 56357956 56434027 56473897 0{mdttestbox}# 0{mdttestbox}# cat t.sh #!/bin/sh command="dd if=/mnt/test of=/dev/null bs=4096k" for i in `jot 9 1`;do mount /dev/da0a /mnt eval $command 2>&1 | grep bytes | awk -F"[\( ]" '{print $8}' umount /mnt done 0{mdttestbox}# initiator is r254328 9.2 AMD64 The two boxes are linked via private gigE switch em1: flags=8843 metric 0 mtu 9000 options=4209b ether 00:1e:67:49:31:b8 inet 10.254.1.2 netmask 0xffffff00 broadcast 10.254.1.255 inet6 fe80::21e:67ff:fe49:31b8%em1 prefixlen 64 scopeid 0x3 nd6 options=23 media: Ethernet autoselect (1000baseT ) status: active ifconfig igb0 igb0: flags=8843 metric 0 mtu 9000 options=400bb ether 00:15:17:fe:a5:bc inet 10.254.1.1 netmask 0xffffff00 broadcast 10.254.1.255 inet6 fe80::215:17ff:fefe:a5bc%igb0 prefixlen 64 scopeid 0x1 nd6 options=29 media: Ethernet autoselect (1000baseT ) status: active Initiator just does iscontrol -c /etc/iscsi.conf -n freebsd freebsd { authmethod = none initiatorname = mdttestbox.sentex.ca TargetName = iqn.2011-03.org.example.istgt:freebsd TargetAddress = 10.254.1.1:3260,2 } ifconfig igb0 -tso ifconfig em0 -tso and sysctl -w net.inet.tcp.tso=0 --------------090102030905000709060202-- From owner-freebsd-net@FreeBSD.ORG Wed Sep 4 22:03:14 2013 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id BCA9B50E for ; Wed, 4 Sep 2013 22:03:14 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-wg0-x22d.google.com (mail-wg0-x22d.google.com [IPv6:2a00:1450:400c:c00::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 48D652989 for ; Wed, 4 Sep 2013 22:03:14 +0000 (UTC) Received: by mail-wg0-f45.google.com with SMTP id y10so994785wgg.0 for ; Wed, 04 Sep 2013 15:03:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=aCkVyWzpNb580w9ooyeoDrdsINsyg05aSOsAqX8PyfE=; b=rBFPPiwCmS54ImdhPvbkyqbkThbnCFouWkaToytc3BH/dK2CibUBwL+k25YSXNfLNS 19ZE5LNKJTJ1PLQ+rxw3r9t0dJnY/5fZMUG94WZrSmzosZia8Ey3ppIitsSL3NwIx2Lv NW8ro0TrW2Z0rBX9z3DO53aaZ1WW5VXwlNDIBzxHecr5Nzf5dJ3fjB2RrBfZgIe8J/JT vi6Ee6MrOuwdScHvQuN4i0HympQci+O8hWkTVLfMVvvpOb702xqe8qs+gLY0j3wMrKoZ lZtqn8DTl9poRBI4eo7HSzKzHOyvqI6BI1u6PHJ6lYYr9LLhdcsyOSN1LBLN42Akg0Pi 9Kkw== MIME-Version: 1.0 X-Received: by 10.194.175.193 with SMTP id cc1mr57690wjc.54.1378332192706; Wed, 04 Sep 2013 15:03:12 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.216.146.2 with HTTP; Wed, 4 Sep 2013 15:03:12 -0700 (PDT) In-Reply-To: <979862494.17918795.1378299005617.JavaMail.root@uoguelph.ca> References: <20130903192734.GA19406@albert.catwhisker.org> <979862494.17918795.1378299005617.JavaMail.root@uoguelph.ca> Date: Wed, 4 Sep 2013 15:03:12 -0700 X-Google-Sender-Auth: AOTdP_iy6E8KxivJo6Td3gzB_rQ Message-ID: Subject: Re: TSO and FreeBSD vs Linux From: Adrian Chadd To: Rick Macklem Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: FreeBSD Net , David Wolfskill X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Sep 2013 22:03:14 -0000 Hiya, David - can you put together a minimal test case that others can reproduce? I have a bunch of gige intel NICs that I can try this with when I'm back in the office. Thanks, -adrian On 4 September 2013 05:50, Rick Macklem wrote: > David Wolfskill wrote: > > On Wed, Aug 21, 2013 at 07:12:38PM +0200, Andre Oppermann wrote: > > > On 13.08.2013 19:29, Julian Elischer wrote: > > > > I have been tracking down a performance embarrassment on AMAZON > > > > EC2 and have found it I think. > > > > Our OS cousins over at Linux land have implemented some > > > > interesting behaviour when TSO is in use. > > > > > > There used to be a different problem with EC2 and FreeBSD TSO. The > > > Xen hypervisor > > > doesn't like large 64K TSO bursts we generate, the drivers drops > > > the whole TSO chain, > > > TCP gets upset and turns off TSO alltogether leaving the connection > > > going at one > > > packet a time as in the old days. > > > ... > > > > My apologies for jumping in so late -- I'm not subscribed to -net@. > > > > At work, I received a new desktop machine a few months ago; here's a > > recent history of what it has been running: > > > > FreeBSD 9.2-PRERELEASE #4 r254801M/254827:902501: Sun Aug 25 > > 05:15:29 PDT 2013 root@dwolf-fbsd:/usr/obj/usr/src/sys/DWOLF > > amd64 > > FreeBSD 9.2-PRERELEASE #5 r255066M/255091:902503: Sat Aug 31 > > 11:58:53 PDT 2013 root@dwolf-fbsd:/usr/obj/usr/src/sys/DWOLF > > amd64 > > FreeBSD 9.2-PRERELEASE #5 r255104M/255115:902503: Sun Sep 1 > > 05:02:12 PDT 2013 root@dwolf-fbsd:/usr/obj/usr/src/sys/DWOLF > > amd64 > > > > Now, I like to have a "private playground" for doing things with > > machines, so I make use of both em(4) NICs on the machine: em0 > > connects > > to the rest of the work network; em1 is connected to a switch I > > brought > > in from home, and to which I connect "other things" (such as my > > laptop). > > And because I'm fairly comfortable with them, I use IPFW & natd. For > > some folks here, none of that should come as a surprise. :-}) > > > > For reference, the em(4) devices in question are: > > > > em0@pci0:0:25:0: class=0x020000 card=0x060d15d9 > > chip=0x10ef8086 rev=0x06 hdr=0x00 > > vendor = 'Intel Corporation' > > device = '82578DM Gigabit Network Connection' > > > > and > > > > em1@pci0:3:0:0: class=0x020000 card=0x060d15d9 chip=0x10d38086 > > rev=0x00 hdr=0x00 > > vendor = 'Intel Corporation' > > device = '82574L Gigabit Network Connection' > > > > > > > > I noticed that when I tried to write files to NFS, I could write > > small > > files OK, but larger ones seemed to ... hang. > > > > Note: We don't use jumbo frames. (Work IT is convinced that they > > don't help. I'm trying to better-understand their reasoning.) > > > > Further poking around showed that (under the above conditions): > > * natd CPU% was climbing as more of the file was copied, up to 2^21 > > bytes. (At that point, nothing further was saved on NFS.) > > * dhcpd CPU% was also climbing. I tried killing that, but doing so > > didn't affect the other results. (Killing natd made connectivity > > cease, given the IPFW rules in effect.) > > * Performing a tcpdump while trying to copy a file of length > > 117709618 > > showed lots of TCP retransmissions. In fact, I'd hazard that every > > TCP > > packet was getting retransmitted. > > * "ifconfig -v em0" showed flags TSO4 & VLAN_HWTSO turned on. > > * "sysctl net.inet.tcp.tso" showed "1" -- enabled. > > > > As soon as I issued "sudo net.inet.tcp.tso=0" ... the copy worked > > without > > a hitch or a whine. And I was able to copy all 117709618 bytes, not > > just > > 2097152 (2^21). > > > > Is the above expected? It came rather as a surprise to me. > > > Not surprising to me, I'm afraid. When there are serious NFS problems > like this, it is often caused by a network fabric issue and broken > TSO is at the top of the list w.r.t. cause. > > rick > > > Peace, > > david > > -- > > David H. Wolfskill david@catwhisker.org > > Taliban: Evil cowards with guns afraid of truth from a 14-year old > > girl. > > > > See http://www.catwhisker.org/~david/publickey.gpg for my public key. > > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Wed Sep 4 22:20:06 2013 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 59D7BA56; Wed, 4 Sep 2013 22:20:06 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from albert.catwhisker.org (mx.catwhisker.org [198.144.209.73]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 010032A80; Wed, 4 Sep 2013 22:20:05 +0000 (UTC) Received: from albert.catwhisker.org (localhost [127.0.0.1]) by albert.catwhisker.org (8.14.7/8.14.7) with ESMTP id r84MK453029462; Wed, 4 Sep 2013 15:20:04 -0700 (PDT) (envelope-from david@albert.catwhisker.org) Received: (from david@localhost) by albert.catwhisker.org (8.14.7/8.14.7/Submit) id r84MK4KI029461; Wed, 4 Sep 2013 15:20:04 -0700 (PDT) (envelope-from david) Date: Wed, 4 Sep 2013 15:20:04 -0700 From: David Wolfskill To: Adrian Chadd Subject: Re: TSO and FreeBSD vs Linux Message-ID: <20130904222004.GD1577@albert.catwhisker.org> References: <20130903192734.GA19406@albert.catwhisker.org> <979862494.17918795.1378299005617.JavaMail.root@uoguelph.ca> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="9hzSyicXuByfNYJd" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Sep 2013 22:20:06 -0000 --9hzSyicXuByfNYJd Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Sep 04, 2013 at 03:03:12PM -0700, Adrian Chadd wrote: > Hiya, >=20 > David - can you put together a minimal test case that others can reproduc= e? > I have a bunch of gige intel NICs that I can try this with when I'm back = in > the office. > ... On NFS client machine (w/ Intel NICs): * Ensure that net.inet.tcp.tso=3D1. * Create or gain access to a regular file, size > 2^21 bytes. * In my case, "ifconfig em0" showed both TSO4 & VLAN_HWTSO. I have not experimented to determine whether either or both of those is required -- and in my case, the machine is my desktop here at work: I would prefer to avoid breaking it. :-} * Mount a writable file system with adequate free space via NFS. * Write the file (e.g., via cp(1)). If that fails to re-create the problem, perhaps you'll need to have another NIC and run IPFW & natd. Something similar to the stock "simple" configuration (/etc/rc.firewall) should do. During the write, I would run "tcpdump -s 0 -w /tmp/cp.bpf host $NFS_server= ", then look at the captured traffic later. I used wireshark; it pointed out the retransmissions. Peace, david --=20 David H. Wolfskill david@catwhisker.org Taliban: Evil cowards with guns afraid of truth from a 14-year old girl. See http://www.catwhisker.org/~david/publickey.gpg for my public key. --9hzSyicXuByfNYJd Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.21 (FreeBSD) iEYEARECAAYFAlInshMACgkQmprOCmdXAD01JgCeOt5dI1R6uaBgHGXnkkrMzo5A 8d8Amwdd3igbW0d30fhUk6vLda1V+zio =2dk+ -----END PGP SIGNATURE----- --9hzSyicXuByfNYJd-- From owner-freebsd-net@FreeBSD.ORG Wed Sep 4 22:26:02 2013 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 49031C16 for ; Wed, 4 Sep 2013 22:26:02 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-we0-x235.google.com (mail-we0-x235.google.com [IPv6:2a00:1450:400c:c03::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id C51662ADA for ; Wed, 4 Sep 2013 22:26:01 +0000 (UTC) Received: by mail-we0-f181.google.com with SMTP id p60so983005wes.26 for ; Wed, 04 Sep 2013 15:26:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=g6MIP9ijbTlBDo2bxLMiAR82stSeHu9/y2aV4fBnj50=; b=TZJyyLp6FNsJeacb4gE1gnLYMdSmtUrZX5c+eWy3Dy3WgM7uwNtwVWHSzllv5kpSwP 0+zuUsrAOotiqZs4EpBIyaKkARqqu2ZmxLLcQvHVrkfGsNIDEAX++Ib2BdtKLg82cIf6 YcK4mKnlH7GSPBd0uhByWQHu/GVm9y8gkhrjiZPZDctv51ZMYJaKUcBbbzGN7aTXexhj +S4Hur05HKlEElpXHFDhizzRXKAs/ncM1zvCcAR0n/e1FOPbCz7s/D3QPj1PevjmCB8E ehIfhB7vZ3aU4J41Ck+FdOOSOqAP+zhXLJcxRJsMjk3URg/0C1Agg6URrGKJHb3fzgD5 QU6w== MIME-Version: 1.0 X-Received: by 10.180.72.198 with SMTP id f6mr3891858wiv.46.1378333560275; Wed, 04 Sep 2013 15:26:00 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.216.146.2 with HTTP; Wed, 4 Sep 2013 15:25:59 -0700 (PDT) In-Reply-To: <20130904222004.GD1577@albert.catwhisker.org> References: <20130903192734.GA19406@albert.catwhisker.org> <979862494.17918795.1378299005617.JavaMail.root@uoguelph.ca> <20130904222004.GD1577@albert.catwhisker.org> Date: Wed, 4 Sep 2013 15:25:59 -0700 X-Google-Sender-Auth: M-KIqQvA8XeEEflzHwTydJlXl4Q Message-ID: Subject: Re: TSO and FreeBSD vs Linux From: Adrian Chadd To: David Wolfskill Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Sep 2013 22:26:02 -0000 What's netstat -sp tcp show before/after the test? -adrian On 4 September 2013 15:20, David Wolfskill wrote: > On Wed, Sep 04, 2013 at 03:03:12PM -0700, Adrian Chadd wrote: > > Hiya, > > > > David - can you put together a minimal test case that others can > reproduce? > > I have a bunch of gige intel NICs that I can try this with when I'm back > in > > the office. > > ... > > On NFS client machine (w/ Intel NICs): > * Ensure that net.inet.tcp.tso=1. > * Create or gain access to a regular file, size > 2^21 bytes. > * In my case, "ifconfig em0" showed both TSO4 & VLAN_HWTSO. I have not > experimented to determine whether either or both of those is required > -- and in my case, the machine is my desktop here at work: I would > prefer to avoid breaking it. :-} > * Mount a writable file system with adequate free space via NFS. > * Write the file (e.g., via cp(1)). > > If that fails to re-create the problem, perhaps you'll need to have > another NIC and run IPFW & natd. Something similar to the stock > "simple" configuration (/etc/rc.firewall) should do. > > During the write, I would run "tcpdump -s 0 -w /tmp/cp.bpf host > $NFS_server", > then look at the captured traffic later. I used wireshark; it pointed > out the retransmissions. > > Peace, > david > -- > David H. Wolfskill david@catwhisker.org > Taliban: Evil cowards with guns afraid of truth from a 14-year old girl. > > See http://www.catwhisker.org/~david/publickey.gpg for my public key. > From owner-freebsd-net@FreeBSD.ORG Wed Sep 4 22:35:59 2013 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 24A17D72; Wed, 4 Sep 2013 22:35:59 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from albert.catwhisker.org (mx.catwhisker.org [198.144.209.73]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B8B7D2B6E; Wed, 4 Sep 2013 22:35:58 +0000 (UTC) Received: from albert.catwhisker.org (localhost [127.0.0.1]) by albert.catwhisker.org (8.14.7/8.14.7) with ESMTP id r84MZvcx029629; Wed, 4 Sep 2013 15:35:57 -0700 (PDT) (envelope-from david@albert.catwhisker.org) Received: (from david@localhost) by albert.catwhisker.org (8.14.7/8.14.7/Submit) id r84MZus0029628; Wed, 4 Sep 2013 15:35:56 -0700 (PDT) (envelope-from david) Date: Wed, 4 Sep 2013 15:35:56 -0700 From: David Wolfskill To: Adrian Chadd Subject: Re: TSO and FreeBSD vs Linux Message-ID: <20130904223556.GE1577@albert.catwhisker.org> References: <20130903192734.GA19406@albert.catwhisker.org> <979862494.17918795.1378299005617.JavaMail.root@uoguelph.ca> <20130904222004.GD1577@albert.catwhisker.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="NQFQLHPkFLmdEfVA" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Sep 2013 22:35:59 -0000 --NQFQLHPkFLmdEfVA Content-Type: multipart/mixed; boundary="Wo0oCZYLrer5S3Oe" Content-Disposition: inline --Wo0oCZYLrer5S3Oe Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Sep 04, 2013 at 03:25:59PM -0700, Adrian Chadd wrote: > What's netstat -sp tcp show before/after the test? > ... Typescript attached. Peace, david --=20 David H. Wolfskill david@catwhisker.org Taliban: Evil cowards with guns afraid of truth from a 14-year old girl. See http://www.catwhisker.org/~david/publickey.gpg for my public key. --Wo0oCZYLrer5S3Oe Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename=tso Content-Transfer-Encoding: quoted-printable Script started on Wed Sep 4 15:31:50 2013 dwolf-fbsd(9.2-P)[15] sudo sysctl net.inet.tcp.tso=3D1=0D=0D Password:=0D net.inet.tcp.tso: 0 -> 1=0D dwolf-fbsd(9.2-P)[16] netstat -sp tcp=0D=0D tcp:=0D 2555593963 packets sent=0D 4800252 data packets (2675428399 bytes)=0D 2547149487 data packets (7491423231574 bytes) retransmitted=0D 12755 data packets unnecessarily retransmitted=0D 2546909891 resends initiated by MTU discovery=0D 3630427 ack-only packets (270271 delayed)=0D 0 URG only packets=0D 0 window probe packets=0D 2596 window update packets=0D 3453 control packets=0D 10886789 packets received=0D 3928645 acks (for 2807958913 bytes)=0D 26320 duplicate acks=0D 0 acks for unsent data=0D 9070321 packets (9528490768 bytes) received in-sequence=0D 574 completely duplicate packets (128517 bytes)=0D 2 old duplicate packets=0D 104 packets with some dup. data (4084 bytes duped)=0D 90 out-of-order packets (99414 bytes)=0D 833 packets (833 bytes) of data after window=0D 833 window probes=0D 25734 window update packets=0D 6 packets received after close=0D 0 discarded for bad checksums=0D 0 discarded for bad header offset fields=0D 0 discarded because packet too short=0D 0 discarded due to memory problems=0D 1677 connection requests=0D 315 connection accepts=0D 0 bad connection attempts=0D 0 listen queue overflows=0D 6 ignored RSTs in the windows=0D 1933 connections established (including accepts)=0D 2298 connections closed (including 90 drops)=0D 1458 connections updated cached RTT on close=0D 1460 connections updated cached RTT variance on close=0D 900 connections updated cached ssthresh on close=0D 32 embryonic connections dropped=0D 3928644 segments updated rtt (of 2914696 attempts)=0D 128226 retransmit timeouts=0D 2 connections dropped by rexmit timeout=0D 0 persist timeouts=0D 0 connections dropped by persist timeout=0D 0 Connections (fin_wait_2) dropped because of timeout=0D 163 keepalive timeouts=0D 153 keepalive probes sent=0D 10 connections dropped by keepalive=0D 482733 correct ACK header predictions=0D 6883177 correct data packet header predictions=0D 315 syncache entries added=0D 0 retransmitted=0D 0 dupsyn=0D 0 dropped=0D 315 completed=0D 0 bucket overflow=0D 0 cache overflow=0D 0 reset=0D 0 stale=0D 0 aborted=0D 0 badack=0D 0 unreach=0D 0 zone failures=0D 315 cookies sent=0D 0 cookies received=0D 194 hostcache entries added=0D 0 bucket overflow=0D 1802 SACK recovery episodes=0D 3576 segment rexmits in SACK recovery episodes=0D 4987584 byte rexmits in SACK recovery episodes=0D 51765 SACK options (SACK blocks) received=0D 74 SACK options (SACK blocks) sent=0D 0 SACK scoreboard overflow=0D 0 packets with ECN CE bit set=0D 0 packets with ECN ECT(0) bit set=0D 0 packets with ECN ECT(1) bit set=0D 0 successful ECN handshakes=0D 0 times ECN reduced the congestion window=0D dwolf-fbsd(9.2-P)[17] cp ~/Mail/sisyphus ~/tmp=0D=0D ^C^C=0D=0D dwolf-fbsd(9.2-P)[18] ls -lT !$/sisyphus=0D=0D ls -lT ~/tmp/sisyphus=0D ^Cdwolf-fbsd(9.2-P)[19] ls -lT ~/tmp/sisyphus=0D=0D -rw------- 1 dwolf 929 1048576 Sep 4 15:32:31 2013 /homes/dwolf/tmp/sis= yphus=0D dwolf-fbsd(9.2-P)[20] ls -lT ~/tmp/sisyphus=1B[21Dcp ~/Mail/sisyphus ~/tmp= =1B[24Dnetstat -sp tcp=1B[K=0D=0D tcp:=0D 2560478331 packets sent=0D 4800896 data packets (2676331641 bytes)=0D 2552032761 data packets (7507101784670 bytes) retransmitted=0D 12860 data packets unnecessarily retransmitted=0D 2551792768 resends initiated by MTU discovery=0D 3630851 ack-only packets (270311 delayed)=0D 0 URG only packets=0D 0 window probe packets=0D 2596 window update packets=0D 3479 control packets=0D 10888714 packets received=0D 3929168 acks (for 2808876755 bytes)=0D 26579 duplicate acks=0D 0 acks for unsent data=0D 9071336 packets (9529611537 bytes) received in-sequence=0D 582 completely duplicate packets (129829 bytes)=0D 2 old duplicate packets=0D 104 packets with some dup. data (4084 bytes duped)=0D 90 out-of-order packets (99414 bytes)=0D 833 packets (833 bytes) of data after window=0D 833 window probes=0D 25915 window update packets=0D 7 packets received after close=0D 0 discarded for bad checksums=0D 0 discarded for bad header offset fields=0D 0 discarded because packet too short=0D 0 discarded due to memory problems=0D 1703 connection requests=0D 315 connection accepts=0D 0 bad connection attempts=0D 0 listen queue overflows=0D 6 ignored RSTs in the windows=0D 1934 connections established (including accepts)=0D 2325 connections closed (including 90 drops)=0D 1459 connections updated cached RTT on close=0D 1461 connections updated cached RTT variance on close=0D 901 connections updated cached ssthresh on close=0D 32 embryonic connections dropped=0D 3929167 segments updated rtt (of 2915199 attempts)=0D 128468 retransmit timeouts=0D 2 connections dropped by rexmit timeout=0D 0 persist timeouts=0D 0 connections dropped by persist timeout=0D 0 Connections (fin_wait_2) dropped because of timeout=0D 163 keepalive timeouts=0D 153 keepalive probes sent=0D 10 connections dropped by keepalive=0D 482835 correct ACK header predictions=0D 6884084 correct data packet header predictions=0D 315 syncache entries added=0D 0 retransmitted=0D 0 dupsyn=0D 0 dropped=0D 315 completed=0D 0 bucket overflow=0D 0 cache overflow=0D 0 reset=0D 0 stale=0D 0 aborted=0D 0 badack=0D 0 unreach=0D 0 zone failures=0D 315 cookies sent=0D 0 cookies received=0D 195 hostcache entries added=0D 0 bucket overflow=0D 1843 SACK recovery episodes=0D 3648 segment rexmits in SACK recovery episodes=0D 5090892 byte rexmits in SACK recovery episodes=0D 52227 SACK options (SACK blocks) received=0D 74 SACK options (SACK blocks) sent=0D 0 SACK scoreboard overflow=0D 0 packets with ECN CE bit set=0D 0 packets with ECN ECT(0) bit set=0D 0 packets with ECN ECT(1) bit set=0D 0 successful ECN handshakes=0D 0 times ECN reduced the congestion window=0D dwolf-fbsd(9.2-P)[21] netstat -sp tcp=1B[15Dls -lT ~/tmp/sisyphus=1B[21Dcp = ~/Mail/sisyphus ~/tmp=1B[24Dnetstat -sp tcp=1B[K=1B[15Dsudo sysctl net.inet= =2Etcp.tso=3D1=08=1B[K0=0D=0D net.inet.tcp.tso: 1 -> 0=0D dwolf-fbsd(9.2-P)[22] sudo sysctl net.inet.tcp.tso=3D0=1B[30Dnetstat -sp tc= p=1B[K=1B[15Dls -lT ~/tmp/sisyphus=1B[21Dcp ~/Mail/sisyphus ~/tmp=0D=0D dwolf-fbsd(9.2-P)[23] echo $?=0D=0D 0=0D dwolf-fbsd(9.2-P)[24] echo $?=1B[7Dcp ~/Mail/sisyphus ~/tmp=1B[24Dsudo sysc= tl net.inet.tcp.tso=3D0=1B[30Dnetstat -sp tcp=1B[K=1B[15Dls -lT ~/tmp/sisyp= hus=0D=0D -rw------- 1 dwolf 929 117836339 Sep 4 15:33:34 2013 /homes/dwolf/tmp/s= isyphus=0D dwolf-fbsd(9.2-P)[25] ls -lT ~/tmp/sisyphus=1B[21Decho $?=1B[K=1B[7Dcp ~/Ma= il/sisyphus ~/tmp=1B[24Dsudo sysctl net.inet.tcp.tso=3D0=1B[30Dnetstat -sp = tcp=1B[K=0D=0D tcp:=0D 2561662836 packets sent=0D 4888547 data packets (2797024810 bytes)=0D 2553088612 data packets (7510826223526 bytes) retransmitted=0D 12896 data packets unnecessarily retransmitted=0D 2552848084 resends initiated by MTU discovery=0D 3671853 ack-only packets (270403 delayed)=0D 0 URG only packets=0D 0 window probe packets=0D 2596 window update packets=0D 3480 control packets=0D 11026149 packets received=0D 3976120 acks (for 2929572833 bytes)=0D 29410 duplicate acks=0D 0 acks for unsent data=0D 9167030 packets (9648083856 bytes) received in-sequence=0D 597 completely duplicate packets (147049 bytes)=0D 2 old duplicate packets=0D 104 packets with some dup. data (4084 bytes duped)=0D 893 out-of-order packets (1235254 bytes)=0D 833 packets (833 bytes) of data after window=0D 833 window probes=0D 26317 window update packets=0D 7 packets received after close=0D 0 discarded for bad checksums=0D 0 discarded for bad header offset fields=0D 0 discarded because packet too short=0D 0 discarded due to memory problems=0D 1703 connection requests=0D 315 connection accepts=0D 0 bad connection attempts=0D 0 listen queue overflows=0D 6 ignored RSTs in the windows=0D 1934 connections established (including accepts)=0D 2325 connections closed (including 90 drops)=0D 1460 connections updated cached RTT on close=0D 1462 connections updated cached RTT variance on close=0D 901 connections updated cached ssthresh on close=0D 32 embryonic connections dropped=0D 3976119 segments updated rtt (of 2960431 attempts)=0D 128506 retransmit timeouts=0D 2 connections dropped by rexmit timeout=0D 0 persist timeouts=0D 0 connections dropped by persist timeout=0D 0 Connections (fin_wait_2) dropped because of timeout=0D 163 keepalive timeouts=0D 153 keepalive probes sent=0D 10 connections dropped by keepalive=0D 519937 correct ACK header predictions=0D 6970417 correct data packet header predictions=0D 315 syncache entries added=0D 0 retransmitted=0D 0 dupsyn=0D 0 dropped=0D 315 completed=0D 0 bucket overflow=0D 0 cache overflow=0D 0 reset=0D 0 stale=0D 0 aborted=0D 0 badack=0D 0 unreach=0D 0 zone failures=0D 315 cookies sent=0D 0 cookies received=0D 195 hostcache entries added=0D 0 bucket overflow=0D 1945 SACK recovery episodes=0D 4143 segment rexmits in SACK recovery episodes=0D 5780288 byte rexmits in SACK recovery episodes=0D 55903 SACK options (SACK blocks) received=0D 833 SACK options (SACK blocks) sent=0D 0 SACK scoreboard overflow=0D 0 packets with ECN CE bit set=0D 0 packets with ECN ECT(0) bit set=0D 0 packets with ECN ECT(1) bit set=0D 0 successful ECN handshakes=0D 0 times ECN reduced the congestion window=0D dwolf-fbsd(9.2-P)[26] ^D=08=08exit=0D Script done on Wed Sep 4 15:34:54 2013 --Wo0oCZYLrer5S3Oe-- --NQFQLHPkFLmdEfVA Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.21 (FreeBSD) iEYEARECAAYFAlIntcsACgkQmprOCmdXAD1RqwCdF23tLfb18S7keHC9h5DJKujW QqwAn3vUt0yVK5RovXhHtpMgvy/KgS+x =socp -----END PGP SIGNATURE----- --NQFQLHPkFLmdEfVA-- From owner-freebsd-net@FreeBSD.ORG Thu Sep 5 02:09:32 2013 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 9A5D4A22; Thu, 5 Sep 2013 02:09:32 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 6C9882B6C; Thu, 5 Sep 2013 02:09:32 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r8529WeA030817; Thu, 5 Sep 2013 02:09:32 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id r8529WsO030816; Thu, 5 Sep 2013 02:09:32 GMT (envelope-from linimon) Date: Thu, 5 Sep 2013 02:09:32 GMT Message-Id: <201309050209.r8529WsO030816@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/181800: problem with router and vlan interface X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Sep 2013 02:09:32 -0000 Old Synopsis: amd64 New Synopsis: problem with router and vlan interface Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Thu Sep 5 02:08:57 UTC 2013 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=181800 From owner-freebsd-net@FreeBSD.ORG Thu Sep 5 02:10:47 2013 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 9654BB4A; Thu, 5 Sep 2013 02:10:47 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 6AE2D2BB3; Thu, 5 Sep 2013 02:10:47 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r852AlG5032458; Thu, 5 Sep 2013 02:10:47 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id r852Aled032457; Thu, 5 Sep 2013 02:10:47 GMT (envelope-from linimon) Date: Thu, 5 Sep 2013 02:10:47 GMT Message-Id: <201309050210.r852Aled032457@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/181821: [netinet [patch] Use unsigned type when indexing into mfchashtbl X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Sep 2013 02:10:47 -0000 Old Synopsis: Use unsigned type when indexing into mfchashtbl New Synopsis: [netinet [patch] Use unsigned type when indexing into mfchashtbl Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Thu Sep 5 02:10:21 UTC 2013 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=181821 From owner-freebsd-net@FreeBSD.ORG Thu Sep 5 02:11:15 2013 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 348E9C4D; Thu, 5 Sep 2013 02:11:15 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 093562BCF; Thu, 5 Sep 2013 02:11:15 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r852BE66032520; Thu, 5 Sep 2013 02:11:14 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id r852BEC4032519; Thu, 5 Sep 2013 02:11:14 GMT (envelope-from linimon) Date: Thu, 5 Sep 2013 02:11:14 GMT Message-Id: <201309050211.r852BEC4032519@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/181822: [netinet] [patch] drop dead / unused code X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Sep 2013 02:11:15 -0000 Old Synopsis: drop dead / unused code New Synopsis: [netinet] [patch] drop dead / unused code Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Thu Sep 5 02:10:56 UTC 2013 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=181822 From owner-freebsd-net@FreeBSD.ORG Thu Sep 5 02:11:41 2013 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 81B8CD36; Thu, 5 Sep 2013 02:11:41 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 57A7E2BE6; Thu, 5 Sep 2013 02:11:41 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r852BfnK032570; Thu, 5 Sep 2013 02:11:41 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id r852BfYp032569; Thu, 5 Sep 2013 02:11:41 GMT (envelope-from linimon) Date: Thu, 5 Sep 2013 02:11:41 GMT Message-Id: <201309050211.r852BfYp032569@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/181823: [ip6] [patch] make ipv6 mroute return same errror codes as IPv4 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Sep 2013 02:11:41 -0000 Old Synopsis: make ipv6 mroute return same errror codes as IPv4 New Synopsis: [ip6] [patch] make ipv6 mroute return same errror codes as IPv4 Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Thu Sep 5 02:11:23 UTC 2013 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=181823 From owner-freebsd-net@FreeBSD.ORG Thu Sep 5 08:14:02 2013 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id E0C27AC5; Thu, 5 Sep 2013 08:14:02 +0000 (UTC) (envelope-from ae@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B240628C4; Thu, 5 Sep 2013 08:14:02 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r858E2UB017455; Thu, 5 Sep 2013 08:14:02 GMT (envelope-from ae@freefall.freebsd.org) Received: (from ae@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id r858E2fs017454; Thu, 5 Sep 2013 08:14:02 GMT (envelope-from ae) Date: Thu, 5 Sep 2013 08:14:02 GMT Message-Id: <201309050814.r858E2fs017454@freefall.freebsd.org> To: sven@vyatta.com, ae@FreeBSD.org, freebsd-net@FreeBSD.org, ae@FreeBSD.org From: ae@FreeBSD.org Subject: Re: kern/181822: [netinet] [patch] drop dead / unused code X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Sep 2013 08:14:03 -0000 Synopsis: [netinet] [patch] drop dead / unused code State-Changed-From-To: open->patched State-Changed-By: ae State-Changed-When: Thu Sep 5 08:12:49 UTC 2013 State-Changed-Why: Committed. Thanks! Responsible-Changed-From-To: freebsd-net->ae Responsible-Changed-By: ae Responsible-Changed-When: Thu Sep 5 08:12:49 UTC 2013 Responsible-Changed-Why: Take it. http://www.freebsd.org/cgi/query-pr.cgi?pr=181822 From owner-freebsd-net@FreeBSD.ORG Thu Sep 5 08:29:16 2013 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 53404581; Thu, 5 Sep 2013 08:29:16 +0000 (UTC) (envelope-from ae@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 278952AB3; Thu, 5 Sep 2013 08:29:16 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r858TGVn021316; Thu, 5 Sep 2013 08:29:16 GMT (envelope-from ae@freefall.freebsd.org) Received: (from ae@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id r858TFkY021315; Thu, 5 Sep 2013 08:29:15 GMT (envelope-from ae) Date: Thu, 5 Sep 2013 08:29:15 GMT Message-Id: <201309050829.r858TFkY021315@freefall.freebsd.org> To: pawel401@mail.ru, ae@FreeBSD.org, freebsd-net@FreeBSD.org, ae@FreeBSD.org From: ae@FreeBSD.org Subject: Re: kern/181800: problem with router and vlan interface X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Sep 2013 08:29:16 -0000 Synopsis: problem with router and vlan interface State-Changed-From-To: open->closed State-Changed-By: ae State-Changed-When: Thu Sep 5 08:25:17 UTC 2013 State-Changed-Why: Actually the kernel uses right interfaces. Use -W key with netstat(1) and it will not truncate interface names in the output. Responsible-Changed-From-To: freebsd-net->ae Responsible-Changed-By: ae Responsible-Changed-When: Thu Sep 5 08:25:17 UTC 2013 Responsible-Changed-Why: Take it. http://www.freebsd.org/cgi/query-pr.cgi?pr=181800 From owner-freebsd-net@FreeBSD.ORG Thu Sep 5 14:20:03 2013 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 4C6A05EA for ; Thu, 5 Sep 2013 14:20:03 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 206212216 for ; Thu, 5 Sep 2013 14:20:03 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r85EK2qN092270 for ; Thu, 5 Sep 2013 14:20:02 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id r85EK2Cg092269; Thu, 5 Sep 2013 14:20:02 GMT (envelope-from gnats) Date: Thu, 5 Sep 2013 14:20:02 GMT Message-Id: <201309051420.r85EK2Cg092269@freefall.freebsd.org> To: freebsd-net@FreeBSD.org Cc: From: dfilter@FreeBSD.ORG (dfilter service) Subject: Re: kern/181821: commit references a PR X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: dfilter service List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Sep 2013 14:20:03 -0000 The following reply was made to PR kern/181821; it has been noted by GNATS. From: dfilter@FreeBSD.ORG (dfilter service) To: bug-followup@FreeBSD.org Cc: Subject: Re: kern/181821: commit references a PR Date: Thu, 5 Sep 2013 14:16:45 +0000 (UTC) Author: jhb Date: Thu Sep 5 14:16:37 2013 New Revision: 255248 URL: http://svnweb.freebsd.org/changeset/base/255248 Log: Use an unsigned long when indexing into mfchashtbl[] and mf6ctable[]. This matches the types used when computing hash indices and the type of the maximum size of mfchashtbl[]. PR: kern/181821 Submitted by: Sven-Thorsten Dietrich (IPv4) MFC after: 1 week Modified: head/sys/netinet/ip_mroute.c head/sys/netinet6/ip6_mroute.c Modified: head/sys/netinet/ip_mroute.c ============================================================================== --- head/sys/netinet/ip_mroute.c Thu Sep 5 13:53:25 2013 (r255247) +++ head/sys/netinet/ip_mroute.c Thu Sep 5 14:16:37 2013 (r255248) @@ -609,7 +609,7 @@ static void if_detached_event(void *arg __unused, struct ifnet *ifp) { vifi_t vifi; - int i; + u_long i; MROUTER_LOCK(); @@ -705,7 +705,7 @@ static int X_ip_mrouter_done(void) { struct ifnet *ifp; - int i; + u_long i; vifi_t vifi; MROUTER_LOCK(); @@ -797,7 +797,7 @@ set_assert(int i) int set_api_config(uint32_t *apival) { - int i; + u_long i; /* * We can set the API capabilities only if it is the first operation @@ -1433,7 +1433,7 @@ non_fatal: static void expire_upcalls(void *arg) { - int i; + u_long i; CURVNET_SET((struct vnet *) arg); Modified: head/sys/netinet6/ip6_mroute.c ============================================================================== --- head/sys/netinet6/ip6_mroute.c Thu Sep 5 13:53:25 2013 (r255247) +++ head/sys/netinet6/ip6_mroute.c Thu Sep 5 14:16:37 2013 (r255248) @@ -576,7 +576,7 @@ int X_ip6_mrouter_done(void) { mifi_t mifi; - int i; + u_long i; struct mf6c *rt; struct rtdetq *rte; @@ -1341,7 +1341,7 @@ expire_upcalls(void *unused) { struct rtdetq *rte; struct mf6c *mfc, **nptr; - int i; + u_long i; MFC6_LOCK(); for (i = 0; i < MF6CTBLSIZ; i++) { _______________________________________________ svn-src-all@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Thu Sep 5 14:24:34 2013 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 7894E8E9; Thu, 5 Sep 2013 14:24:34 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4E02722E3; Thu, 5 Sep 2013 14:24:34 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r85EOYpV093956; Thu, 5 Sep 2013 14:24:34 GMT (envelope-from jhb@freefall.freebsd.org) Received: (from jhb@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id r85EOX12093955; Thu, 5 Sep 2013 14:24:33 GMT (envelope-from jhb) Date: Thu, 5 Sep 2013 14:24:33 GMT Message-Id: <201309051424.r85EOX12093955@freefall.freebsd.org> To: sven@inter-yacht.com, jhb@FreeBSD.org, freebsd-net@FreeBSD.org, jhb@FreeBSD.org From: jhb@FreeBSD.org Subject: Re: kern/181821: [netinet [patch] Use unsigned type when indexing into mfchashtbl X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Sep 2013 14:24:34 -0000 Synopsis: [netinet [patch] Use unsigned type when indexing into mfchashtbl State-Changed-From-To: open->patched State-Changed-By: jhb State-Changed-When: Thu Sep 5 14:24:14 UTC 2013 State-Changed-Why: Fix committed to HEAD, thanks! Responsible-Changed-From-To: freebsd-net->jhb Responsible-Changed-By: jhb Responsible-Changed-When: Thu Sep 5 14:24:14 UTC 2013 Responsible-Changed-Why: Fix committed to HEAD, thanks! http://www.freebsd.org/cgi/query-pr.cgi?pr=181821 From owner-freebsd-net@FreeBSD.ORG Thu Sep 5 15:30:02 2013 Return-Path: Delivered-To: freebsd-net@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id F1549D63 for ; Thu, 5 Sep 2013 15:30:01 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id D0F4529DC for ; Thu, 5 Sep 2013 15:30:01 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r85FU1rI006946 for ; Thu, 5 Sep 2013 15:30:01 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id r85FU1uo006945; Thu, 5 Sep 2013 15:30:01 GMT (envelope-from gnats) Date: Thu, 5 Sep 2013 15:30:01 GMT Message-Id: <201309051530.r85FU1uo006945@freefall.freebsd.org> To: freebsd-net@FreeBSD.org Cc: From: "Charbon, Julien" Subject: Re: kern/176446: [netinet] [patch] Concurrency in ixgbe driving out-of-order packet process and spurious RST X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: "Charbon, Julien" List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Sep 2013 15:30:02 -0000 The following reply was made to PR kern/176446; it has been noted by GNATS. From: "Charbon, Julien" To: bug-followup@freebsd.org, jcharbon@verisign.com Cc: John Baldwin Subject: Re: kern/176446: [netinet] [patch] Concurrency in ixgbe driving out-of-order packet process and spurious RST Date: Thu, 05 Sep 2013 15:05:15 +0200 This is a multi-part message in MIME format. --------------090007000606090509060801 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Just a PR update: This issue is fixed in releng/9.2 (since 9.2-RC2 and later), especially with these commits: - Fix local timer watchdog using taskque_enqueue(&que->que_task) instead of taskqueue_enqueue(&txr->txq_task)(one line change in ixgbe_local_timer()): http://svnweb.freebsd.org/base?view=revision&revision=251964 - Not calling (and then remove) ixgbe_rearm_queues(): http://svnweb.freebsd.org/base?view=revision&revision=253865 These changes did not reach (yet) stable/8. Joined the current patch for releng/8.4. Thanks to John Balwin for accepting and pushing these changes. -- Julien Charbon --------------090007000606090509060801 Content-Type: text/plain; charset=UTF-8; x-mac-type="0"; x-mac-creator="0"; name="releng-8.4.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="releng-8.4.patch" diff --git a/sys/dev/ixgbe/ixgbe.c b/sys/dev/ixgbe/ixgbe.c index df37621..9a00517 100644 --- a/sys/dev/ixgbe/ixgbe.c +++ b/sys/dev/ixgbe/ixgbe.c @@ -1396,23 +1396,6 @@ ixgbe_disable_queue(struct adapter *adapter, u32 vector) } } -static inline void -ixgbe_rearm_queues(struct adapter *adapter, u64 queues) -{ - u32 mask; - - if (adapter->hw.mac.type == ixgbe_mac_82598EB) { - mask = (IXGBE_EIMS_RTX_QUEUE & queues); - IXGBE_WRITE_REG(&adapter->hw, IXGBE_EICS, mask); - } else { - mask = (queues & 0xFFFFFFFF); - IXGBE_WRITE_REG(&adapter->hw, IXGBE_EICS_EX(0), mask); - mask = (queues >> 32); - IXGBE_WRITE_REG(&adapter->hw, IXGBE_EICS_EX(1), mask); - } -} - - static void ixgbe_handle_que(void *context, int pending) { @@ -2046,14 +2029,13 @@ ixgbe_local_timer(void *arg) (paused == 0)) ++hung; else if (txr->queue_status == IXGBE_QUEUE_WORKING) - taskqueue_enqueue(que->tq, &que->que_task); + taskqueue_enqueue(que->tq, &txr->txq_task); } /* Only truely watchdog if all queues show hung */ if (hung == adapter->num_tx_queues) goto watchdog; out: - ixgbe_rearm_queues(adapter, adapter->que_mask); callout_reset(&adapter->timer, hz, ixgbe_local_timer, adapter); return; @@ -4559,7 +4541,6 @@ next_desc: ** Schedule another interrupt if so. */ if ((staterr & IXGBE_RXD_STAT_DD) != 0) { - ixgbe_rearm_queues(adapter, (u64)(1 << que->msix)); return (TRUE); } --------------090007000606090509060801-- From owner-freebsd-net@FreeBSD.ORG Thu Sep 5 19:20:12 2013 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 330B2CFD; Thu, 5 Sep 2013 19:20:12 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from albert.catwhisker.org (mx.catwhisker.org [198.144.209.73]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id BD87B2A5E; Thu, 5 Sep 2013 19:20:11 +0000 (UTC) Received: from albert.catwhisker.org (localhost [127.0.0.1]) by albert.catwhisker.org (8.14.7/8.14.7) with ESMTP id r85JKAHt036201; Thu, 5 Sep 2013 12:20:10 -0700 (PDT) (envelope-from david@albert.catwhisker.org) Received: (from david@localhost) by albert.catwhisker.org (8.14.7/8.14.7/Submit) id r85JKAAo036200; Thu, 5 Sep 2013 12:20:10 -0700 (PDT) (envelope-from david) Date: Thu, 5 Sep 2013 12:20:10 -0700 From: David Wolfskill To: Adrian Chadd Subject: Re: TSO and FreeBSD vs Linux Message-ID: <20130905192010.GL1577@albert.catwhisker.org> References: <20130903192734.GA19406@albert.catwhisker.org> <979862494.17918795.1378299005617.JavaMail.root@uoguelph.ca> <20130904222004.GD1577@albert.catwhisker.org> <20130904223556.GE1577@albert.catwhisker.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="flDc4cbvG5Y7qIjC" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Sep 2013 19:20:12 -0000 --flDc4cbvG5Y7qIjC Content-Type: multipart/mixed; boundary="TlGginnuNEm5PkhK" Content-Disposition: inline --TlGginnuNEm5PkhK Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Sep 04, 2013 at 03:52:19PM -0700, Adrian Chadd wrote: > .. can you use -z to clear the stats out just before you do a test? >=20 >=20 > -adrian >=20 >=20 > On 4 September 2013 15:35, David Wolfskill wrote: >=20 > > On Wed, Sep 04, 2013 at 03:25:59PM -0700, Adrian Chadd wrote: > > > What's netstat -sp tcp show before/after the test? > > > ... > ... Attached; test was to demonstrate issue w/TSO. Peace, david --=20 David H. Wolfskill david@catwhisker.org Taliban: Evil cowards with guns afraid of truth from a 14-year old girl. See http://www.catwhisker.org/~david/publickey.gpg for my public key. --TlGginnuNEm5PkhK Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename=tso Content-Transfer-Encoding: quoted-printable Script started on Wed Sep 4 16:03:56 2013 dwolf-fbsd(9.2-P)[15] sudo sysctl net.inet.tcp.tso=3D1=0D=0D Password:=0D net.inet.tcp.tso: 0 -> 1=0D dwolf-fbsd(9.2-P)[16] sudo netstat -zsp tcp=0D=0D tcp:=0D 2574837448 packets sent=0D 4990261 data packets (2921163576 bytes)=0D 2566108476 data packets (7552375270794 bytes) retransmitted=0D 13131 data packets unnecessarily retransmitted=0D 2565866462 resends initiated by MTU discovery=0D 3724751 ack-only packets (272449 delayed)=0D 0 URG only packets=0D 0 window probe packets=0D 2596 window update packets=0D 3615 control packets=0D 11197243 packets received=0D 4033650 acks (for 3053804276 bytes)=0D 33051 duplicate acks=0D 0 acks for unsent data=0D 9289364 packets (9789388088 bytes) received in-sequence=0D 617 completely duplicate packets (160537 bytes)=0D 2 old duplicate packets=0D 104 packets with some dup. data (4084 bytes duped)=0D 1633 out-of-order packets (2277306 bytes)=0D 833 packets (833 bytes) of data after window=0D 833 window probes=0D 27123 window update packets=0D 10 packets received after close=0D 0 discarded for bad checksums=0D 0 discarded for bad header offset fields=0D 0 discarded because packet too short=0D 0 discarded due to memory problems=0D 1809 connection requests=0D 319 connection accepts=0D 0 bad connection attempts=0D 0 listen queue overflows=0D 6 ignored RSTs in the windows=0D 1965 connections established (including accepts)=0D 2454 connections closed (including 90 drops)=0D 1483 connections updated cached RTT on close=0D 1485 connections updated cached RTT variance on close=0D 902 connections updated cached ssthresh on close=0D 36 embryonic connections dropped=0D 4033649 segments updated rtt (of 3015788 attempts)=0D 129116 retransmit timeouts=0D 2 connections dropped by rexmit timeout=0D 0 persist timeouts=0D 0 connections dropped by persist timeout=0D 0 Connections (fin_wait_2) dropped because of timeout=0D 167 keepalive timeouts=0D 157 keepalive probes sent=0D 10 connections dropped by keepalive=0D 560619 correct ACK header predictions=0D 7078166 correct data packet header predictions=0D 319 syncache entries added=0D 0 retransmitted=0D 0 dupsyn=0D 0 dropped=0D 319 completed=0D 0 bucket overflow=0D 0 cache overflow=0D 0 reset=0D 0 stale=0D 0 aborted=0D 0 badack=0D 0 unreach=0D 0 zone failures=0D 319 cookies sent=0D 0 cookies received=0D 200 hostcache entries added=0D 0 bucket overflow=0D 2132 SACK recovery episodes=0D 4792 segment rexmits in SACK recovery episodes=0D 6693920 byte rexmits in SACK recovery episodes=0D 60818 SACK options (SACK blocks) received=0D 1532 SACK options (SACK blocks) sent=0D 0 SACK scoreboard overflow=0D 0 packets with ECN CE bit set=0D 0 packets with ECN ECT(0) bit set=0D 0 packets with ECN ECT(1) bit set=0D 0 successful ECN handshakes=0D 0 times ECN reduced the congestion window=0D dwolf-fbsd(9.2-P)[17] ls -lT ~/Mail/sisyphus=0D=0D -rw------- 1 dwolf 929 117842814 Sep 4 15:36:32 2013 /homes/dwolf/Mail/= sisyphus=0D dwolf-fbsd(9.2-P)[18] cp ~/Mail/sisyphus ~/tmp=0D=0D ^C=0D=0D dwolf-fbsd(9.2-P)[19] ls -lT !$/sisyphus=0D=0D ls -lT ~/tmp/sisyphus=0D -rw------- 1 dwolf 929 2097152 Sep 4 16:06:14 2013 /homes/dwolf/tmp/sis= yphus=0D dwolf-fbsd(9.2-P)[20] netstat -sp tcp=0D=0D tcp:=0D 11721951 packets sent=0D 1648 data packets (2153234 bytes)=0D 11719344 data packets (40242113308 bytes) retransmitted=0D 336 data packets unnecessarily retransmitted=0D 11718480 resends initiated by MTU discovery=0D 869 ack-only packets (55 delayed)=0D 0 URG only packets=0D 0 window probe packets=0D 0 window update packets=0D 90 control packets=0D 4143 packets received=0D 1380 acks (for 2224279 bytes)=0D 564 duplicate acks=0D 0 acks for unsent data=0D 2085 packets (2193927 bytes) received in-sequence=0D 2 completely duplicate packets (328 bytes)=0D 0 old duplicate packets=0D 0 packets with some dup. data (0 bytes duped)=0D 0 out-of-order packets (0 bytes)=0D 0 packets (0 bytes) of data after window=0D 0 window probes=0D 227 window update packets=0D 1 packet received after close=0D 0 discarded for bad checksums=0D 0 discarded for bad header offset fields=0D 0 discarded because packet too short=0D 0 discarded due to memory problems=0D 89 connection requests=0D 0 connection accepts=0D 0 bad connection attempts=0D 0 listen queue overflows=0D 0 ignored RSTs in the windows=0D 2 connections established (including accepts)=0D 90 connections closed (including 0 drops)=0D 2 connections updated cached RTT on close=0D 2 connections updated cached RTT variance on close=0D 1 connection updated cached ssthresh on close=0D 0 embryonic connections dropped=0D 1380 segments updated rtt (of 1284 attempts)=0D 452 retransmit timeouts=0D 0 connections dropped by rexmit timeout=0D 0 persist timeouts=0D 0 connections dropped by persist timeout=0D 0 Connections (fin_wait_2) dropped because of timeout=0D 0 keepalive timeouts=0D 0 keepalive probes sent=0D 0 connections dropped by keepalive=0D 350 correct ACK header predictions=0D 1847 correct data packet header predictions=0D 0 syncache entries added=0D 0 retransmitted=0D 0 dupsyn=0D 0 dropped=0D 0 completed=0D 0 bucket overflow=0D 0 cache overflow=0D 0 reset=0D 0 stale=0D 0 aborted=0D 0 badack=0D 0 unreach=0D 0 zone failures=0D 0 cookies sent=0D 0 cookies received=0D 0 hostcache entries added=0D 0 bucket overflow=0D 164 SACK recovery episodes=0D 323 segment rexmits in SACK recovery episodes=0D 463828 byte rexmits in SACK recovery episodes=0D 824 SACK options (SACK blocks) received=0D 0 SACK options (SACK blocks) sent=0D 0 SACK scoreboard overflow=0D 0 packets with ECN CE bit set=0D 0 packets with ECN ECT(0) bit set=0D 0 packets with ECN ECT(1) bit set=0D 0 successful ECN handshakes=0D 0 times ECN reduced the congestion window=0D dwolf-fbsd(9.2-P)[21] sudo sysctl net.inet.tcp.tso=3D0=0D=0D net.inet.tcp.tso: 1 -> 0=0D dwolf-fbsd(9.2-P)[22] sudo netstat -zsp tcp=0D=0D tcp:=0D 11722179 packets sent=0D 1826 data packets (2173970 bytes)=0D 11719344 data packets (40242113308 bytes) retransmitted=0D 336 data packets unnecessarily retransmitted=0D 11718480 resends initiated by MTU discovery=0D 918 ack-only packets (65 delayed)=0D 0 URG only packets=0D 0 window probe packets=0D 0 window update packets=0D 91 control packets=0D 4375 packets received=0D 1500 acks (for 2245016 bytes)=0D 565 duplicate acks=0D 0 acks for unsent data=0D 2219 packets (2208633 bytes) received in-sequence=0D 2 completely duplicate packets (328 bytes)=0D 0 old duplicate packets=0D 0 packets with some dup. data (0 bytes duped)=0D 0 out-of-order packets (0 bytes)=0D 0 packets (0 bytes) of data after window=0D 0 window probes=0D 227 window update packets=0D 1 packet received after close=0D 0 discarded for bad checksums=0D 0 discarded for bad header offset fields=0D 0 discarded because packet too short=0D 0 discarded due to memory problems=0D 89 connection requests=0D 0 connection accepts=0D 0 bad connection attempts=0D 0 listen queue overflows=0D 0 ignored RSTs in the windows=0D 2 connections established (including accepts)=0D 90 connections closed (including 0 drops)=0D 3 connections updated cached RTT on close=0D 3 connections updated cached RTT variance on close=0D 1 connection updated cached ssthresh on close=0D 0 embryonic connections dropped=0D 1500 segments updated rtt (of 1397 attempts)=0D 452 retransmit timeouts=0D 0 connections dropped by rexmit timeout=0D 0 persist timeouts=0D 0 connections dropped by persist timeout=0D 0 Connections (fin_wait_2) dropped because of timeout=0D 0 keepalive timeouts=0D 0 keepalive probes sent=0D 0 connections dropped by keepalive=0D 412 correct ACK header predictions=0D 1945 correct data packet header predictions=0D 0 syncache entries added=0D 0 retransmitted=0D 0 dupsyn=0D 0 dropped=0D 0 completed=0D 0 bucket overflow=0D 0 cache overflow=0D 0 reset=0D 0 stale=0D 0 aborted=0D 0 badack=0D 0 unreach=0D 0 zone failures=0D 0 cookies sent=0D 0 cookies received=0D 0 hostcache entries added=0D 0 bucket overflow=0D 164 SACK recovery episodes=0D 323 segment rexmits in SACK recovery episodes=0D 463828 byte rexmits in SACK recovery episodes=0D 824 SACK options (SACK blocks) received=0D 0 SACK options (SACK blocks) sent=0D 0 SACK scoreboard overflow=0D 0 packets with ECN CE bit set=0D 0 packets with ECN ECT(0) bit set=0D 0 packets with ECN ECT(1) bit set=0D 0 successful ECN handshakes=0D 0 times ECN reduced the congestion window=0D dwolf-fbsd(9.2-P)[23] cp ~/Mail/sisyphus ~/tmp=0D=0D dwolf-fbsd(9.2-P)[24] echo $?=0D=0D 0=0D dwolf-fbsd(9.2-P)[25] netstat -sp tcp=0D=0D tcp:=0D 129165 packets sent=0D 87533 data packets (120483061 bytes)=0D 388 data packets (531932 bytes) retransmitted=0D 1 data packet unnecessarily retransmitted=0D 0 resends initiated by MTU discovery=0D 41244 ack-only packets (74 delayed)=0D 0 URG only packets=0D 0 window probe packets=0D 0 window update packets=0D 0 control packets=0D 137628 packets received=0D 47037 acks (for 120483061 bytes)=0D 2432 duplicate acks=0D 0 acks for unsent data=0D 96306 packets (119505191 bytes) received in-sequence=0D 8 completely duplicate packets (8816 bytes)=0D 0 old duplicate packets=0D 0 packets with some dup. data (0 bytes duped)=0D 834 out-of-order packets (1175280 bytes)=0D 0 packets (0 bytes) of data after window=0D 0 window probes=0D 413 window update packets=0D 0 packets received after close=0D 0 discarded for bad checksums=0D 0 discarded for bad header offset fields=0D 0 discarded because packet too short=0D 0 discarded due to memory problems=0D 0 connection requests=0D 0 connection accepts=0D 0 bad connection attempts=0D 0 listen queue overflows=0D 0 ignored RSTs in the windows=0D 0 connections established (including accepts)=0D 1 connection closed (including 0 drops)=0D 0 connections updated cached RTT on close=0D 0 connections updated cached RTT variance on close=0D 0 connections updated cached ssthresh on close=0D 0 embryonic connections dropped=0D 47037 segments updated rtt (of 45341 attempts)=0D 4 retransmit timeouts=0D 0 connections dropped by rexmit timeout=0D 0 persist timeouts=0D 0 connections dropped by persist timeout=0D 0 Connections (fin_wait_2) dropped because of timeout=0D 0 keepalive timeouts=0D 0 keepalive probes sent=0D 0 connections dropped by keepalive=0D 37227 correct ACK header predictions=0D 86833 correct data packet header predictions=0D 0 syncache entries added=0D 0 retransmitted=0D 0 dupsyn=0D 0 dropped=0D 0 completed=0D 0 bucket overflow=0D 0 cache overflow=0D 0 reset=0D 0 stale=0D 0 aborted=0D 0 badack=0D 0 unreach=0D 0 zone failures=0D 0 cookies sent=0D 0 cookies received=0D 0 hostcache entries added=0D 0 bucket overflow=0D 77 SACK recovery episodes=0D 384 segment rexmits in SACK recovery episodes=0D 526188 byte rexmits in SACK recovery episodes=0D 3190 SACK options (SACK blocks) received=0D 791 SACK options (SACK blocks) sent=0D 0 SACK scoreboard overflow=0D 0 packets with ECN CE bit set=0D 0 packets with ECN ECT(0) bit set=0D 0 packets with ECN ECT(1) bit set=0D 0 successful ECN handshakes=0D 0 times ECN reduced the congestion window=0D dwolf-fbsd(9.2-P)[26] ls -lT ~/tmp/sisyphus=0D=0D -rw------- 1 dwolf 929 117842814 Sep 4 16:07:30 2013 /homes/dwolf/tmp/s= isyphus=0D dwolf-fbsd(9.2-P)[27] rm !$=0D=0D rm ~/tmp/sisyphus=0D dwolf-fbsd(9.2-P)[28] ^D=08=08exit=0D Script done on Wed Sep 4 16:08:58 2013 --TlGginnuNEm5PkhK-- --flDc4cbvG5Y7qIjC Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.21 (FreeBSD) iEYEARECAAYFAlIo2WgACgkQmprOCmdXAD0oSQCfS2+p16EKTrRFBT2gKcDKPPzS QdQAn3f66BFxI2B4gtSPdbKpB3RWhaKG =x2kd -----END PGP SIGNATURE----- --flDc4cbvG5Y7qIjC-- From owner-freebsd-net@FreeBSD.ORG Fri Sep 6 02:30:05 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id A25CF272 for ; Fri, 6 Sep 2013 02:30:05 +0000 (UTC) (envelope-from prvs=19611cea04=agave.spring@cadamericas.com) Received: from cadamericas.com (mail02.amotive.com [173.164.153.20]) (using TLSv1 with cipher RC4-MD5 (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 802B82E60 for ; Fri, 6 Sep 2013 02:30:04 +0000 (UTC) Received: from agave.cadamericas.com ([64.183.139.162]) by amotive.com (mail02.amotive.com) (MDaemon PRO v13.0.2) with ESMTP id md50002407324.msg; Thu, 05 Sep 2013 19:30:06 -0700 X-Spam-Processed: mail02.amotive.com, Thu, 05 Sep 2013 19:30:06 -0700 (not processed: message from trusted or authenticated source) X-MDRemoteIP: 64.183.139.162 X-Return-Path: prvs=19611cea04=agave.spring@cadamericas.com X-Envelope-From: agave.spring@cadamericas.com X-MDaemon-Deliver-To: freebsd-net@freebsd.org Date: Thu, 5 Sep 2013 19:29:01 -0700 To: freebsd-net From: CAD Americas Subject: AUGI & CAD Americas Collaborate for 8-City Tour Message-ID: <2876e90c399c9e2f145319a190955a91@agave.cadamericas.com> X-Priority: 3 X-Mailer: PHPMailer 5.2.1 (http://code.google.com/a/apache-extras.org/p/phpmailer/) X-CampTrackID: 83bb0874-2c31-bed3-9836-52293d7e85d6 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: CAD Americas List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Sep 2013 02:30:05 -0000 TIME IS RUNNING OUT! Register for CAD Americas Training Days by MAY 7 and S= AVE!=0ACAD AMERICAS TRAINING DAYS ARRIVE IN YOUR AREA SOON Join us for this= one-day training event in your area. Whether your focus is Mechanical Desi= gn, Construction, BIM, Electrical Design or Plant Design, there will be ses= sions that will improve your productivity immediately!=0AJune 4June 6June 7= June 12June 13June 26 June 27=0A Cleveland Cincinnati Detroit Atlanta D= allas San Jose San_Bernardino =0ATAKE HOME NEW TOOLS AND TECHNIQUES THAT = WILL IMPROVE YOUR PERFORMANCE IMMEDIATELY=0A=0A=0A=0A=0ALynn AllenTechnical= Evangelist More =0ARobert GreenCAD Mgmt Expert More =0ASteve SchainAutoCAD= Expert More =0ATod StephensRevit Expert More =0AClick here to see current = session descriptions.Please note that sessions will vary by location =0ALea= rn from well-known industry instructors who will share best practices and t= rends, product tips and tricks, new features =E2=80=A6 and more.=0AImprove = your productivity with new techniques that you can put to work right away.= =0AMeet your peers and exchange ideas on how to best use the CAD tools you = have to meet the demands of your job.=0ATake a closer look at services and = technologies offered by resellers in your area.=0AREGISTER BY MAY 7TH AND S= AVERegister for=C2=A0a CAD Americas Training Day by May 7th and save.=0AEar= ly Bird Rate: $150 (Until May 7th)=0AStandard Rate: $195 (AFTER May 7th)=0A= Student/Faculty Rate: $95 (must present current student ID upon check-in at= registration)=0AREGISTER FOR CAD AMERICAS TRAINING TODAY!=0A=0A=0A=0A=0AJo= in us at=0A=0A=0A=0A=0A=0A=0A=0A=0A=0A=0A=0A=C2=A0INTERNATIONAL SPONSORS=0A= =0A=0A=C2=A0=C2=A0 =C2=A0=C2=A0 =0A=0AEDUCATION SPONSOR=0A=0A=0AMEDIA SPONS= ORS=0A=0A=C2=A0=0AThis email was sent to email address: freebsd-net@freebsd= .org Click here to Opt-Out=0A From owner-freebsd-net@FreeBSD.ORG Fri Sep 6 06:15:26 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id D753A7B for ; Fri, 6 Sep 2013 06:15:26 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from mail-pd0-x234.google.com (mail-pd0-x234.google.com [IPv6:2607:f8b0:400e:c02::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id AD20E28D8 for ; Fri, 6 Sep 2013 06:15:26 +0000 (UTC) Received: by mail-pd0-f180.google.com with SMTP id y10so2807637pdj.25 for ; Thu, 05 Sep 2013 23:15:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=SG8ObtXXNYk1qKw39gI1g3IuwXWTx4JAvfuXsaL81jQ=; b=dwR9pvQUdHvyJI81HcsjIYu2e9YGR0S1SfN18YIAfOqEBjwgMCzo0worFR8JEbIWIU 2VsL5lQQvs5hUSdJjUvwYddWrQPWLrmbTxr3X/Vegi2WM+95mY/ZNHRNHCKTG0lBEs/2 pJgj0lfDSw7lGWqnxSTV7lY7jF0rtAHU18rdCzhNDGJtSnjmTZVI0O4lKlAGRHQpEnkr w+PE37HajwuKvKbKz69AiIy9cRQdenaZi7VX78IK3y+QZKFmPdGtbZm5k5ilUMg3Hlmu vkXsRK2zIK0QZWB0z9DDMiCYbEB9IrwHI53kG/7sxxtxjQHkSMKL8wsyUulrClbVc4M+ xN7g== X-Received: by 10.66.142.132 with SMTP id rw4mr2125616pab.6.1378448125455; Thu, 05 Sep 2013 23:15:25 -0700 (PDT) Received: from pyunyh@gmail.com (lpe4.p59-icn.cdngp.net. [114.111.62.249]) by mx.google.com with ESMTPSA id j9sm2335392paj.18.1969.12.31.16.00.00 (version=TLSv1 cipher=RC4-SHA bits=128/128); Thu, 05 Sep 2013 23:15:24 -0700 (PDT) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Fri, 06 Sep 2013 15:15:21 +0900 From: Yonghyeon PYUN Date: Fri, 6 Sep 2013 15:15:21 +0900 To: Guido Falsi Subject: Re: re0 not working at boot on -CURRENT Message-ID: <20130906061521.GB3070@michelle.cdnetworks.com> References: <51DC726D.6040601@madpilot.net> <20130710070431.GE2753@michelle.cdnetworks.com> <51DD9E15.7070609@madpilot.net> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="9amGYk9869ThD9tj" Content-Disposition: inline In-Reply-To: <51DD9E15.7070609@madpilot.net> User-Agent: Mutt/1.4.2.3i Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Sep 2013 06:15:27 -0000 --9amGYk9869ThD9tj Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Wed, Jul 10, 2013 at 07:47:01PM +0200, Guido Falsi wrote: > On 07/10/13 09:04, Yonghyeon PYUN wrote: > >On Tue, Jul 09, 2013 at 10:28:29PM +0200, Guido Falsi wrote: > >>Hi, > >> > >>I have a PC with an integrate re ethernet interface, pciconf identifies > >>it like this: > >> > >>re0@pci0:3:0:0: class=0x020000 card=0x11c01734 chip=0x816810ec rev=0x07 > >>hdr=0x00 > >> > >>I'm running FreeBSD current r252261. > >> > >>As stated in the subject after boot the interface does not work correctly. > >> > >>Using tcpdump on another host I noticed that packets (ICMP echo requests > >>for example) do get sent, and replies generated by the other host, but > >>the kernel does not seem to see them. Except that every now and then > >>some packet does get to the system. > >> > >>I'm seeing packet 7, 27, 47, 66, 86, 106, 125, 144, 164, 183 and so on > >>from a ping which has been running for some time. Just about one every > >>twenty. Some pattern is showing up. > >> > >>this is the output of ifconfig re0 after boot: > >> > >>re0: flags=8843 metric 0 mtu 1500 > >> > >>options=8209b > >> ether 00:19:99:f8:d3:0b > >> inet 172.24.42.13 netmask 0xffffff00 broadcast 172.24.42.255 > >> inet6 fe80::219:99ff:fef8:d30b%re0 prefixlen 64 scopeid 0x2 > >> nd6 options=29 > >> media: Ethernet autoselect (100baseTX ) > >> status: active > >> > >>If I just touch any interface flag with ifconfig, anyone, tso, -txcsum > >>-rxcsum, it starts working flawlessly. It keeps working also if I > >>perform the opposite operation with ifconfig afterwards, so it is not > >>the flag itself fixing it. > >> > >>This is an ifconfig after performing this exercise(it's the same, since > >>I disabled txcsum and reactivated it in this instance): > >> > >>re0: flags=8843 metric 0 mtu 1500 > >> > >>options=8209b > >> ether 00:19:99:f8:d3:0b > >> inet 172.24.42.13 netmask 0xffffff00 broadcast 172.24.42.255 > >> inet6 fe80::219:99ff:fef8:d30b%re0 prefixlen 64 scopeid 0x2 > >> nd6 options=29 > >> media: Ethernet autoselect (100baseTX ) > >> status: active > >> > >>I don't know much about FreeBSD network drivers so i can't make theories > >>about this. I hope someone has an idea what the problem could be. > >> > >>I'm available for any further information needed, test, experiment and > >>so on. > > > >Could you show me dmesg output(re(4) and rgephy(4) only)? > > re0: port > 0xd000-0xd0ff mem 0xf2104000-0xf2104fff,0xf2100000-0xf2103fff irq 17 at > device 0.0 on pci3 > re0: Using 1 MSI-X message > re0: turning off MSI enable bit. > re0: Chip rev. 0x2c800000 > re0: MAC rev. 0x00000000 > re0: Ethernet address: 00:19:99:f8:d3:0b > miibus0: on re0 > rgephy0: PHY 1 on miibus0 > rgephy0: none, 10baseT, 10baseT-FDX, 10baseT-FDX-flow, 100baseTX, > 100baseTX-FDX, 100baseTX-FDX-flow, 1000baseT, 1000baseT-master, > 1000baseT-FDX, 1000baseT-FDX-master, 1000baseT-FDX-flow, > 1000baseT-FDX-flow-master, auto, auto-flow > > Also, I'm loading this as a module, but, for as much as I know, this > should not make any difference. > > > >Did it ever work or you see the issue only on CURRENT? > > Never worked on this machine (I own it since the last days of February). > > I only installed current on it. If needed I can find time to test a > recent 9.x snapshot on it. > > I worked around the problem till now using an USB ethernet adapter, > always wanted to report this problem, but I've been lazy :) > Would you try attached patch and let me know whether it makes any difference? --9amGYk9869ThD9tj Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="re.eri.diff" Index: sys/dev/re/if_re.c =================================================================== --- sys/dev/re/if_re.c (revision 255289) +++ sys/dev/re/if_re.c (working copy) @@ -289,6 +289,9 @@ static int re_miibus_readreg (device_t, int, int); static int re_miibus_writereg (device_t, int, int, int); static void re_miibus_statchg (device_t); +static uint32_t re_eri_read (struct rl_softc *, bus_size_t, int); +static void re_eri_write (struct rl_softc *, bus_size_t, uint32_t, int); + static void re_set_jumbo (struct rl_softc *, int); static void re_set_rxmode (struct rl_softc *); static void re_reset (struct rl_softc *); @@ -602,6 +605,7 @@ re_miibus_statchg(device_t dev) struct rl_softc *sc; struct ifnet *ifp; struct mii_data *mii; + uint32_t exgmac; sc = device_get_softc(dev); mii = device_get_softc(sc->rl_miibus); @@ -627,14 +631,108 @@ re_miibus_statchg(device_t dev) break; } } + + if ((sc->rl_flags & RL_FLAG_LINK) == 0) + return; + /* * RealTek controllers does not provide any interface to * Tx/Rx MACs for resolved speed, duplex and flow-control * parameters. */ + + switch (sc->rl_hwrev->rl_rev) { + case RL_HWREV_8168E_VL: + if (sc->rl_icrev == 0x00100000) { + switch (IFM_SUBTYPE(mii->mii_media_active)) { + case IFM_1000_T: + re_eri_write(sc, 0x1BC, 0x00000011, + RL_ERIAR_EXGMAC); + re_eri_write(sc, 0x1BC, 0x00000005, + RL_ERIAR_EXGMAC); + break; + case IFM_100_TX: + re_eri_write(sc, 0x1BC, 0x0000001F, + RL_ERIAR_EXGMAC); + re_eri_write(sc, 0x1bc, 0x00000005, + RL_ERIAR_EXGMAC); + break; + default: + re_eri_write(sc, 0x1BC, 0x0000001F, + RL_ERIAR_EXGMAC); + re_eri_write(sc, 0x1BC, 0x0000003F, + RL_ERIAR_EXGMAC); + break; + } + } + exgmac = re_eri_read(sc, 0xDC, RL_ERIAR_EXGMAC); + exgmac &= ~0x01; + re_eri_write(sc, 0x1BC, exgmac, RL_ERIAR_EXGMAC); + exgmac |= 0x01; + re_eri_write(sc, 0x1BC, exgmac, RL_ERIAR_EXGMAC); + break; + case RL_HWREV_8168F: + case RL_HWREV_8411: + switch (IFM_SUBTYPE(mii->mii_media_active)) { + case IFM_1000_T: + re_eri_write(sc, 0x1BC, 0x00000011, RL_ERIAR_EXGMAC); + re_eri_write(sc, 0x1BC, 0x00000005, RL_ERIAR_EXGMAC); + break; + default: + re_eri_write(sc, 0x1BC, 0x0000001F, RL_ERIAR_EXGMAC); + re_eri_write(sc, 0x1BC, 0x0000003F, RL_ERIAR_EXGMAC); + break; + } + break; + } } +static uint32_t +re_eri_read(struct rl_softc *sc, bus_size_t addr, int type) +{ + int i; + + CSR_WRITE_4(sc, RL_ERIAR, addr | type | RL_ERIAR_BYTES_MASK | + RL_ERIAR_READ); + for (i = 100; i > 0; i--) { + DELAY(100); + if ((CSR_READ_4(sc, RL_ERIAR) & RL_ERIAR_BUSY) == 0) + break; + } + if (i == 0) { + device_printf(sc->rl_dev, "%s: timed out\n", __func__); + return (0xFFFFFFFF); + } + return (CSR_READ_4(sc, RL_ERIDR)); +} + /* + * ERI is used to access extended GigaMAC register. addr should be + * aligned on 4 bytes boundary. mask(e.g. bit 12~15 of RL_ERIAR) is + * used to determine which bytes in RL_ERIDR should be accessed. + * Because we have to access 32bits quantity regardless of the value + * of mask, make driver access to 4 bytes which in turn means it's + * responsibility of caller to preserve unwanted bytes in write + * access. + */ +static void +re_eri_write(struct rl_softc *sc, bus_size_t addr, uint32_t val, int type) +{ + int i; + + CSR_WRITE_4(sc, RL_ERIDR, val); + CSR_WRITE_4(sc, RL_ERIAR, addr | type | RL_ERIAR_BYTES_MASK | + RL_ERIAR_WRITE); + for (i = 100; i > 0; i--) { + DELAY(100); + if ((CSR_READ_4(sc, RL_ERIAR) & RL_ERIAR_BUSY) == 0) + break; + } + if (i == 0) + device_printf(sc->rl_dev, "%s: timed out\n", __func__); +} + +/* * Set the RX configuration and 64-bit multicast hash filter. */ static void @@ -1357,6 +1455,7 @@ re_attach(device_t dev) device_printf(dev, "no ASPM capability\n"); } + sc->rl_icrev = 0; hw_rev = re_hwrevs; hwrev = CSR_READ_4(sc, RL_TXCFG); switch (hwrev & 0x70000000) { @@ -1367,6 +1466,8 @@ re_attach(device_t dev) break; default: device_printf(dev, "Chip rev. 0x%08x\n", hwrev & 0x7c800000); + device_printf(dev, "IC rev. 0x%08x\n", hwrev & 0x00700000); + sc->rl_icrev = hwrev & 0x00700000; hwrev &= RL_TXCFG_HWREV; break; } @@ -1429,7 +1530,7 @@ re_attach(device_t dev) sc->rl_flags |= RL_FLAG_MACSLEEP; /* FALLTHROUGH */ case RL_HWREV_8168C: - if ((hwrev & 0x00700000) == 0x00200000) + if (sc->rl_icrev == 0x00200000) sc->rl_flags |= RL_FLAG_MACSLEEP; /* FALLTHROUGH */ case RL_HWREV_8168CP: Index: sys/pci/if_rlreg.h =================================================================== --- sys/pci/if_rlreg.h (revision 255289) +++ sys/pci/if_rlreg.h (working copy) @@ -143,6 +143,8 @@ #define RL_MACDBG 0x006D /* 8 bits, 8168C SPIN2 only */ #define RL_GPIO 0x006E /* 8 bits, 8168C SPIN2 only */ #define RL_PMCH 0x006F /* 8 bits */ +#define RL_ERIDR 0x0070 +#define RL_ERIAR 0x0074 #define RL_MAXRXPKTLEN 0x00DA /* 16 bits, chip multiplies by 8 */ #define RL_INTRMOD 0x00E2 /* 16 bits */ @@ -540,6 +542,18 @@ #define RL_GMEDIASTAT_TXFLOW 0x40 /* TX flow control on */ #define RL_GMEDIASTAT_TBI 0x80 /* TBI enabled */ +#define RL_ERIAR_BYTES_1ST 0x00001000 +#define RL_ERIAR_BYTES_2ND 0x00002000 +#define RL_ERIAR_BYTES_3RD 0x00004000 +#define RL_ERIAR_BYTES_4TH 0x00008000 +#define RL_ERIAR_BYTES_MASK 0x0000F000 +#define RL_ERIAR_EXGMAC 0x00000000 +#define RL_ERIAR_MSIX 0x00010000 +#define RL_ERIAR_ASF 0x00020000 +#define RL_ERIAR_READ 0x00000000 +#define RL_ERIAR_WRITE 0x80000000 +#define RL_ERIAR_BUSY 0x80000000 + /* * The RealTek doesn't use a fragment-based descriptor mechanism. * Instead, there are only four register sets, each or which represents @@ -877,6 +891,7 @@ struct rl_softc { bus_dma_tag_t rl_parent_tag; uint8_t rl_type; const struct rl_hwrev *rl_hwrev; + uint32_t rl_icrev; int rl_eecmd_read; int rl_eewidth; int rl_expcap; --9amGYk9869ThD9tj-- From owner-freebsd-net@FreeBSD.ORG Fri Sep 6 19:10:50 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 201A21E8 for ; Fri, 6 Sep 2013 19:10:50 +0000 (UTC) (envelope-from hiren.panchasara@gmail.com) Received: from mail-ea0-x234.google.com (mail-ea0-x234.google.com [IPv6:2a00:1450:4013:c01::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9EAE72803 for ; Fri, 6 Sep 2013 19:10:49 +0000 (UTC) Received: by mail-ea0-f180.google.com with SMTP id h10so1821418eaj.11 for ; Fri, 06 Sep 2013 12:10:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=Jx5Qgf9+3NaknNwd7NX2dM0JHOSB2M1ytc88s9ZIQfs=; b=XHhvrWvpdm61qcLnA+/D9n8T/DwKjRgVaJ348tMWx4faVQ27USGAkuALSV988d2STL JQQSKO7OOFMVtV6IULtMrMnzMr7q5JqvKh4ks2x6QLH/dwh48GTmpLIRNuJQvl4dYPf5 cBG70N0y8RfLKkRun3pHpuJ+EreXfFqH6o5DFHJxTAE3bnJlH16No+H+Wu2tiW27O9pJ 5L9V4ycGDf+OSLCiD96a9qTwZPIETPjG//zHiJpHPJPWdiFjUARPn0L+GGROFaqo5SSn R35Pl1IsN3ZNTY99G+f3bx9/V2smIMOHMh3oIBi4nz8YVzETFJdPAkusoFV8TplVlTBG gwsQ== MIME-Version: 1.0 X-Received: by 10.15.33.132 with SMTP id c4mr6453040eev.2.1378494647916; Fri, 06 Sep 2013 12:10:47 -0700 (PDT) Received: by 10.14.105.137 with HTTP; Fri, 6 Sep 2013 12:10:47 -0700 (PDT) Date: Fri, 6 Sep 2013 12:10:47 -0700 Message-ID: Subject: mbuf autotuning changes From: hiren panchasara To: "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Sep 2013 19:10:50 -0000 tunable_mbinit() in kern_mbuf.c looks like this: 119 /* 120 * The default limit for all mbuf related memory is 1/2 of all 121 * available kernel memory (physical or kmem). 122 * At most it can be 3/4 of available kernel memory. 123 */ 124 realmem = qmin((quad_t)physmem * PAGE_SIZE, 125 vm_map_max(kmem_map) - vm_map_min(kmem_map)); 126 maxmbufmem = realmem / 2; 127 TUNABLE_QUAD_FETCH("kern.ipc.maxmbufmem", &maxmbufmem); 128 if (maxmbufmem > realmem / 4 * 3) 129 maxmbufmem = realmem / 4 * 3; If I am reading the code correctly, we loose the value on line 126 when we do FETCH on line 127. And after line 127, if we havent specified kern.ipc.maxmbufmem (in loader.conf - I guess...), we set that value to 0. And because of that the if condition on line 128 is almost always false? What am I missing here? Thanks, Hiren From owner-freebsd-net@FreeBSD.ORG Fri Sep 6 19:14:22 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 9EB74314 for ; Fri, 6 Sep 2013 19:14:22 +0000 (UTC) (envelope-from nparhar@gmail.com) Received: from mail-pa0-x22d.google.com (mail-pa0-x22d.google.com [IPv6:2607:f8b0:400e:c03::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 775F42848 for ; Fri, 6 Sep 2013 19:14:22 +0000 (UTC) Received: by mail-pa0-f45.google.com with SMTP id bg4so3725317pad.32 for ; Fri, 06 Sep 2013 12:14:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=Uvp2u7LULS0oZyUMjiTMRjyqJrIEXIWEL/wSDO0K3aY=; b=cKECF1SxlmvnyJ55G26L0mPvCj9ixxX6P1q7nRn1ZuG7RPMyyU3lO+RNTZyeRSZxVe fsK9N/MviVG4mipXYdZ8p4ncyrMLVtpsc9+5jqLOrZDz2AbGxFowU8S1BeXkbcPzUw/J zCs9Vz5awAD9pOpAJj489mQc2duRfXgT6KyV02gd5pOaF4ozU0tu2n0kU3VhFacxZ19E /KyIskcVLHj6PBuJQro46qKvRikEVyglWzrLaGNj3LvJvHh/pg+FOCde/LErnwrxR/pI SADu9MuRMXEB89aZWna3+r8BjhjzGuArWUFVKcUp+AHMO2SAJnCfmPwUzzluTQKrztgu ingg== X-Received: by 10.68.96.130 with SMTP id ds2mr4589547pbb.99.1378494862152; Fri, 06 Sep 2013 12:14:22 -0700 (PDT) Received: from [10.192.166.0] (stargate.chelsio.com. [67.207.112.58]) by mx.google.com with ESMTPSA id om2sm5501951pbc.30.1969.12.31.16.00.00 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 06 Sep 2013 12:14:21 -0700 (PDT) Message-ID: <522A298B.30106@gmail.com> Date: Fri, 06 Sep 2013 12:14:19 -0700 From: Navdeep Parhar User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130819 Thunderbird/17.0.8 MIME-Version: 1.0 To: hiren panchasara Subject: Re: mbuf autotuning changes References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Sep 2013 19:14:22 -0000 On 09/06/13 12:10, hiren panchasara wrote: > tunable_mbinit() in kern_mbuf.c looks like this: > > 119 /* > 120 * The default limit for all mbuf related memory is 1/2 of all > 121 * available kernel memory (physical or kmem). > 122 * At most it can be 3/4 of available kernel memory. > 123 */ > 124 realmem = qmin((quad_t)physmem * PAGE_SIZE, > 125 vm_map_max(kmem_map) - vm_map_min(kmem_map)); > 126 maxmbufmem = realmem / 2; > 127 TUNABLE_QUAD_FETCH("kern.ipc.maxmbufmem", &maxmbufmem); > 128 if (maxmbufmem > realmem / 4 * 3) > 129 maxmbufmem = realmem / 4 * 3; > > If I am reading the code correctly, we loose the value on line 126 when we > do FETCH on line 127. > > And after line 127, if we havent specified kern.ipc.maxmbufmem (in > loader.conf - I guess...), we set that value to 0. > > And because of that the if condition on line 128 is almost always false? > > What am I missing here? All the TUNABLE_FOO_FETCH leave the data alone if the tunable isn't actually in the environment. For this specific case, see the the implementation of getenv_quad(). Navdeep From owner-freebsd-net@FreeBSD.ORG Fri Sep 6 19:14:32 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 96D7B3A1 for ; Fri, 6 Sep 2013 19:14:32 +0000 (UTC) (envelope-from bright@mu.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 84783284D for ; Fri, 6 Sep 2013 19:14:32 +0000 (UTC) Received: from Alfreds-MacBook-Pro-9.local (c-67-180-208-218.hsd1.ca.comcast.net [67.180.208.218]) by elvis.mu.org (Postfix) with ESMTPSA id 5AFEC1A3CF7; Fri, 6 Sep 2013 12:14:28 -0700 (PDT) Message-ID: <522A2993.6020206@mu.org> Date: Fri, 06 Sep 2013 12:14:27 -0700 From: Alfred Perlstein User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:17.0) Gecko/20130801 Thunderbird/17.0.8 MIME-Version: 1.0 To: hiren panchasara Subject: Re: mbuf autotuning changes References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Sep 2013 19:14:32 -0000 On 9/6/13 12:10 PM, hiren panchasara wrote: > tunable_mbinit() in kern_mbuf.c looks like this: > > 119 /* > 120 * The default limit for all mbuf related memory is 1/2 of all > 121 * available kernel memory (physical or kmem). > 122 * At most it can be 3/4 of available kernel memory. > 123 */ > 124 realmem = qmin((quad_t)physmem * PAGE_SIZE, > 125 vm_map_max(kmem_map) - vm_map_min(kmem_map)); > 126 maxmbufmem = realmem / 2; > 127 TUNABLE_QUAD_FETCH("kern.ipc.maxmbufmem", &maxmbufmem); > 128 if (maxmbufmem > realmem / 4 * 3) > 129 maxmbufmem = realmem / 4 * 3; > > If I am reading the code correctly, we loose the value on line 126 when we > do FETCH on line 127. > > And after line 127, if we havent specified kern.ipc.maxmbufmem (in > loader.conf - I guess...), we set that value to 0. > > And because of that the if condition on line 128 is almost always false? > > What am I missing here? > > Thanks, > Hiren > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > I think TUNABLE_*_FETCH will only write to the variable if it explicitly set. Meaning, unless the user actually sets a value in loader.conf then 127 is a no-op. -Alfred -- Alfred Perlstein From owner-freebsd-net@FreeBSD.ORG Fri Sep 6 19:36:49 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id DBEBAFB7; Fri, 6 Sep 2013 19:36:49 +0000 (UTC) (envelope-from hiren.panchasara@gmail.com) Received: from mail-ee0-x236.google.com (mail-ee0-x236.google.com [IPv6:2a00:1450:4013:c00::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 3D2F72B59; Fri, 6 Sep 2013 19:36:49 +0000 (UTC) Received: by mail-ee0-f54.google.com with SMTP id e53so1860278eek.27 for ; Fri, 06 Sep 2013 12:36:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Aiqa3XzzMjqU2TGFwKwNON9nZbDdr3l75i0xARuFGM8=; b=R4/7HmZe9Nu0RPQnrPo+b5KDoKQrVo4Dx+f6wydfNfvx0jGKhFcfR60yi0gsKYzcll tmhmt8yI7a/sgR2rhJU91ucyvFlfAzbYV9T+Ycdp5xNXoY9AoXr7Wxw0zZwbCZ7mEZ+c GjhRVFgPvX8Hl8o2qg7C2/8tvKV36ub868pbJcInyDzSAYsynUGsrOEiYQV7l9js+y2D Uzbseo3HCRY5soIee/i36l3/8YwUn3mP0bv1aEz92B65hg3uVDZCbf6p3TApAehyFDF4 Ad/ca4GouKTYiNLuNtaNRWTn/xSmQeaws8iNPhotR3xVUJUwMhp1kDWg+ok04EGHgRQW 7c6g== MIME-Version: 1.0 X-Received: by 10.14.177.199 with SMTP id d47mr6515120eem.14.1378496207559; Fri, 06 Sep 2013 12:36:47 -0700 (PDT) Received: by 10.14.105.137 with HTTP; Fri, 6 Sep 2013 12:36:47 -0700 (PDT) In-Reply-To: <522A2993.6020206@mu.org> References: <522A2993.6020206@mu.org> Date: Fri, 6 Sep 2013 12:36:47 -0700 Message-ID: Subject: Re: mbuf autotuning changes From: hiren panchasara To: Alfred Perlstein , Navdeep Parhar Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Sep 2013 19:36:49 -0000 On Fri, Sep 6, 2013 at 12:14 PM, Alfred Perlstein wrote: > On 9/6/13 12:10 PM, hiren panchasara wrote: > >> tunable_mbinit() in kern_mbuf.c looks like this: >> >> 119 /* >> 120 * The default limit for all mbuf related memory is 1/2 of all >> 121 * available kernel memory (physical or kmem). >> 122 * At most it can be 3/4 of available kernel memory. >> 123 */ >> 124 realmem = qmin((quad_t)physmem * PAGE_SIZE, >> 125 vm_map_max(kmem_map) - vm_map_min(kmem_map)); >> 126 maxmbufmem = realmem / 2; >> 127 TUNABLE_QUAD_FETCH("kern.ipc.**maxmbufmem", &maxmbufmem); >> 128 if (maxmbufmem > realmem / 4 * 3) >> 129 maxmbufmem = realmem / 4 * 3; >> >> If I am reading the code correctly, we loose the value on line 126 when we >> do FETCH on line 127. >> >> And after line 127, if we havent specified kern.ipc.maxmbufmem (in >> loader.conf - I guess...), we set that value to 0. >> >> And because of that the if condition on line 128 is almost always false? >> >> What am I missing here? >> >> Thanks, >> Hiren >> ______________________________**_________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/**mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@**freebsd.org >> " >> >> I think TUNABLE_*_FETCH will only write to the variable if it explicitly > set. > > Meaning, unless the user actually sets a value in loader.conf then 127 is > a no-op. > Thanks Navdeep and Alfred. Thats correct. Its not touching the var if its not set. I guess the other TUNABLE_INT_FETCHs later in the function checking for variable ==0 confused me. i.e. nmbclusters. 131 TUNABLE_INT_FETCH("kern.ipc.nmbclusters", &nmbclusters); 132 if (nmbclusters == 0) 133 nmbclusters = maxmbufmem / MCLBYTES / 4; But those are global variable so here we are just checking if they are explicitly set of not. If not, we will set them. For maxmbufmem, we will set it to 1/2 the realmem. and if user sets it explicitly than we will make sure its not more than 3/4 of the realmem. Thanks again. Hiren From owner-freebsd-net@FreeBSD.ORG Fri Sep 6 19:38:23 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id ECFC3155 for ; Fri, 6 Sep 2013 19:38:23 +0000 (UTC) (envelope-from alfred@freebsd.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id D9F562B97 for ; Fri, 6 Sep 2013 19:38:23 +0000 (UTC) Received: from Alfreds-MacBook-Pro-9.local (c-67-180-208-218.hsd1.ca.comcast.net [67.180.208.218]) by elvis.mu.org (Postfix) with ESMTPSA id 3CAFB1A3E27 for ; Fri, 6 Sep 2013 12:38:23 -0700 (PDT) Message-ID: <522A2F2E.8080808@freebsd.org> Date: Fri, 06 Sep 2013 12:38:22 -0700 From: Alfred Perlstein User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:17.0) Gecko/20130801 Thunderbird/17.0.8 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: mbuf autotuning changes References: <522A2993.6020206@mu.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Sep 2013 19:38:24 -0000 On 9/6/13 12:36 PM, hiren panchasara wrote: > On Fri, Sep 6, 2013 at 12:14 PM, Alfred Perlstein wrote: > >> On 9/6/13 12:10 PM, hiren panchasara wrote: >> >>> tunable_mbinit() in kern_mbuf.c looks like this: >>> >>> 119 /* >>> 120 * The default limit for all mbuf related memory is 1/2 of all >>> 121 * available kernel memory (physical or kmem). >>> 122 * At most it can be 3/4 of available kernel memory. >>> 123 */ >>> 124 realmem = qmin((quad_t)physmem * PAGE_SIZE, >>> 125 vm_map_max(kmem_map) - vm_map_min(kmem_map)); >>> 126 maxmbufmem = realmem / 2; >>> 127 TUNABLE_QUAD_FETCH("kern.ipc.**maxmbufmem", &maxmbufmem); >>> 128 if (maxmbufmem > realmem / 4 * 3) >>> 129 maxmbufmem = realmem / 4 * 3; >>> >>> If I am reading the code correctly, we loose the value on line 126 when we >>> do FETCH on line 127. >>> >>> And after line 127, if we havent specified kern.ipc.maxmbufmem (in >>> loader.conf - I guess...), we set that value to 0. >>> >>> And because of that the if condition on line 128 is almost always false? >>> >>> What am I missing here? >>> >>> Thanks, >>> Hiren >>> ______________________________**_________________ >>> freebsd-net@freebsd.org mailing list >>> http://lists.freebsd.org/**mailman/listinfo/freebsd-net >>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@**freebsd.org >>> " >>> >>> I think TUNABLE_*_FETCH will only write to the variable if it explicitly >> set. >> >> Meaning, unless the user actually sets a value in loader.conf then 127 is >> a no-op. >> > Thanks Navdeep and Alfred. > > Thats correct. Its not touching the var if its not set. > > I guess the other TUNABLE_INT_FETCHs later in the function checking for > variable ==0 confused me. i.e. nmbclusters. > > 131 TUNABLE_INT_FETCH("kern.ipc.nmbclusters", &nmbclusters); > 132 if (nmbclusters == 0) > 133 nmbclusters = maxmbufmem / MCLBYTES / 4; > > But those are global variable so here we are just checking if they are > explicitly set of not. If not, we will set them. > > For maxmbufmem, we will set it to 1/2 the realmem. and if user sets it > explicitly than we will make sure its not more than 3/4 of the realmem. Yes. It's somewhat confusing. I'm all for adding comments to this effect if you have the time and inclination. -Alfred From owner-freebsd-net@FreeBSD.ORG Fri Sep 6 20:09:26 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 1E24A4C2; Fri, 6 Sep 2013 20:09:26 +0000 (UTC) (envelope-from hiren.panchasara@gmail.com) Received: from mail-ee0-x22f.google.com (mail-ee0-x22f.google.com [IPv6:2a00:1450:4013:c00::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 769C520FB; Fri, 6 Sep 2013 20:09:25 +0000 (UTC) Received: by mail-ee0-f47.google.com with SMTP id d49so1849759eek.34 for ; Fri, 06 Sep 2013 13:09:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=crZEcQYlb4awP9TRtF5DMdpwHBh4McNx3osxAjmwwxk=; b=NJwSXUqAYjnZoRucad8yVUYSaiLa4If1QVpd1FJcKX5SmCY6qCi6oacvs7cp1Nav0V XyhV2AIMVDzdT2yIXaH8oQpBLN/W0rvcvu0rFcNxV8TCDvT9jD7dTcEWkGW7ukU3dkaH mfHjEda7g79zOs6zojf3ao8J+iQDAIrQexEdAkNiCWi3ibA3r6PenOLfrkySzR/uckwQ 4WLW3xTBkg2rgVw5iyqyDLdIexfoG3gK81PLSaStI7bfyM+cRlhnflQU+QhE+JGDdfV1 RC75/SFfL7By/hD31yTwWTMW7w6WVxLt0fXR6sdqDP63lEBKradVAQ7akSQrA3rwLRPF vYIg== MIME-Version: 1.0 X-Received: by 10.15.36.9 with SMTP id h9mr6840621eev.30.1378498163867; Fri, 06 Sep 2013 13:09:23 -0700 (PDT) Received: by 10.14.105.137 with HTTP; Fri, 6 Sep 2013 13:09:23 -0700 (PDT) In-Reply-To: <522A2F2E.8080808@freebsd.org> References: <522A2993.6020206@mu.org> <522A2F2E.8080808@freebsd.org> Date: Fri, 6 Sep 2013 13:09:23 -0700 Message-ID: Subject: Re: mbuf autotuning changes From: hiren panchasara To: Alfred Perlstein Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Sep 2013 20:09:26 -0000 On Fri, Sep 6, 2013 at 12:38 PM, Alfred Perlstein wrote: > On 9/6/13 12:36 PM, hiren panchasara wrote: > >> On Fri, Sep 6, 2013 at 12:14 PM, Alfred Perlstein wrote: >> >> On 9/6/13 12:10 PM, hiren panchasara wrote: >>> >>> tunable_mbinit() in kern_mbuf.c looks like this: >>>> >>>> 119 /* >>>> 120 * The default limit for all mbuf related memory is 1/2 of >>>> all >>>> 121 * available kernel memory (physical or kmem). >>>> 122 * At most it can be 3/4 of available kernel memory. >>>> 123 */ >>>> 124 realmem = qmin((quad_t)physmem * PAGE_SIZE, >>>> 125 vm_map_max(kmem_map) - vm_map_min(kmem_map)); >>>> 126 maxmbufmem = realmem / 2; >>>> 127 TUNABLE_QUAD_FETCH("kern.ipc.****maxmbufmem", &maxmbufmem); >>>> >>>> 128 if (maxmbufmem > realmem / 4 * 3) >>>> 129 maxmbufmem = realmem / 4 * 3; >>>> >>>> If I am reading the code correctly, we loose the value on line 126 when >>>> we >>>> do FETCH on line 127. >>>> >>>> And after line 127, if we havent specified kern.ipc.maxmbufmem (in >>>> loader.conf - I guess...), we set that value to 0. >>>> >>>> And because of that the if condition on line 128 is almost always false? >>>> >>>> What am I missing here? >>>> >>>> Thanks, >>>> Hiren >>>> ______________________________****_________________ >>>> freebsd-net@freebsd.org mailing list >>>> http://lists.freebsd.org/****mailman/listinfo/freebsd-net >>>> >>>> > >>>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@**fre** >>>> ebsd.org >>>> > >>>> >>>> " >>>> >>>> I think TUNABLE_*_FETCH will only write to the variable if it >>>> explicitly >>>> >>> set. >>> >>> Meaning, unless the user actually sets a value in loader.conf then 127 is >>> a no-op. >>> >>> Thanks Navdeep and Alfred. >> >> Thats correct. Its not touching the var if its not set. >> >> I guess the other TUNABLE_INT_FETCHs later in the function checking for >> variable ==0 confused me. i.e. nmbclusters. >> >> 131 TUNABLE_INT_FETCH("kern.ipc.**nmbclusters", &nmbclusters); >> 132 if (nmbclusters == 0) >> 133 nmbclusters = maxmbufmem / MCLBYTES / 4; >> >> But those are global variable so here we are just checking if they are >> explicitly set of not. If not, we will set them. >> >> For maxmbufmem, we will set it to 1/2 the realmem. and if user sets it >> explicitly than we will make sure its not more than 3/4 of the realmem. >> > Yes. It's somewhat confusing. > > I'm all for adding comments to this effect if you have the time and > inclination. I guess its verbose enough in kern_mbuf.c I just had to *actually* read getenv_quad() to know that its not setting the variable to 0, it was just the return value. We can probably do: [hirenp@wholecorner ~/commit_head/sys/kern]$ svn diff Index: kern_environment.c =================================================================== --- kern_environment.c (revision 255320) +++ kern_environment.c (working copy) @@ -530,7 +530,8 @@ } /* - * Return a quad_t value from an environment variable. + * Return a quad_t value from an environment variable inside "data". + * If the environment variable is not set, "data" will be unchanged. */ int getenv_quad(const char *name, quad_t *data) cheers, Hiren From owner-freebsd-net@FreeBSD.ORG Fri Sep 6 20:43:01 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 27056D42 for ; Fri, 6 Sep 2013 20:43:01 +0000 (UTC) (envelope-from mad@madpilot.net) Received: from winston.madpilot.net (winston.madpilot.net [78.47.75.155]) by mx1.freebsd.org (Postfix) with ESMTP id 74D6F232A for ; Fri, 6 Sep 2013 20:43:00 +0000 (UTC) Received: from winston.madpilot.net (localhost [127.0.0.1]) by winston.madpilot.net (Postfix) with ESMTP id 3cWrLq4jQbzFTPg; Fri, 6 Sep 2013 22:42:59 +0200 (CEST) X-Virus-Scanned: amavisd-new at madpilot.net Received: from winston.madpilot.net ([127.0.0.1]) by winston.madpilot.net (winston.madpilot.net [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mwyHwNtCOVY1; Fri, 6 Sep 2013 22:42:57 +0200 (CEST) Received: from tommy.madpilot.net (micro.madpilot.net [88.149.173.206]) by winston.madpilot.net (Postfix) with ESMTPSA; Fri, 6 Sep 2013 22:42:57 +0200 (CEST) Message-ID: <522A3E50.8080801@madpilot.net> Date: Fri, 06 Sep 2013 22:42:56 +0200 From: Guido Falsi User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130903 Thunderbird/17.0.8 MIME-Version: 1.0 To: pyunyh@gmail.com Subject: Re: re0 not working at boot on -CURRENT References: <51DC726D.6040601@madpilot.net> <20130710070431.GE2753@michelle.cdnetworks.com> <51DD9E15.7070609@madpilot.net> <20130906061521.GB3070@michelle.cdnetworks.com> In-Reply-To: <20130906061521.GB3070@michelle.cdnetworks.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Sep 2013 20:43:01 -0000 On 09/06/13 08:15, Yonghyeon PYUN wrote: > On Wed, Jul 10, 2013 at 07:47:01PM +0200, Guido Falsi wrote: >> On 07/10/13 09:04, Yonghyeon PYUN wrote: >>> On Tue, Jul 09, 2013 at 10:28:29PM +0200, Guido Falsi wrote: >>>> Hi, >>>> >>>> I have a PC with an integrate re ethernet interface, pciconf identifies >>>> it like this: >>>> >>>> re0@pci0:3:0:0: class=0x020000 card=0x11c01734 chip=0x816810ec rev=0x07 >>>> hdr=0x00 >>>> >>>> I'm running FreeBSD current r252261. >>>> >>>> As stated in the subject after boot the interface does not work correctly. >>>> >>>> Using tcpdump on another host I noticed that packets (ICMP echo requests >>>> for example) do get sent, and replies generated by the other host, but >>>> the kernel does not seem to see them. Except that every now and then >>>> some packet does get to the system. >>>> >>>> I'm seeing packet 7, 27, 47, 66, 86, 106, 125, 144, 164, 183 and so on >>> >from a ping which has been running for some time. Just about one every >>>> twenty. Some pattern is showing up. >>>> >>>> this is the output of ifconfig re0 after boot: >>>> >>>> re0: flags=8843 metric 0 mtu 1500 >>>> >>>> options=8209b >>>> ether 00:19:99:f8:d3:0b >>>> inet 172.24.42.13 netmask 0xffffff00 broadcast 172.24.42.255 >>>> inet6 fe80::219:99ff:fef8:d30b%re0 prefixlen 64 scopeid 0x2 >>>> nd6 options=29 >>>> media: Ethernet autoselect (100baseTX ) >>>> status: active >>>> >>>> If I just touch any interface flag with ifconfig, anyone, tso, -txcsum >>>> -rxcsum, it starts working flawlessly. It keeps working also if I >>>> perform the opposite operation with ifconfig afterwards, so it is not >>>> the flag itself fixing it. >>>> >>>> This is an ifconfig after performing this exercise(it's the same, since >>>> I disabled txcsum and reactivated it in this instance): >>>> >>>> re0: flags=8843 metric 0 mtu 1500 >>>> >>>> options=8209b >>>> ether 00:19:99:f8:d3:0b >>>> inet 172.24.42.13 netmask 0xffffff00 broadcast 172.24.42.255 >>>> inet6 fe80::219:99ff:fef8:d30b%re0 prefixlen 64 scopeid 0x2 >>>> nd6 options=29 >>>> media: Ethernet autoselect (100baseTX ) >>>> status: active >>>> >>>> I don't know much about FreeBSD network drivers so i can't make theories >>>> about this. I hope someone has an idea what the problem could be. >>>> >>>> I'm available for any further information needed, test, experiment and >>>> so on. >>> >>> Could you show me dmesg output(re(4) and rgephy(4) only)? >> >> re0: port >> 0xd000-0xd0ff mem 0xf2104000-0xf2104fff,0xf2100000-0xf2103fff irq 17 at >> device 0.0 on pci3 >> re0: Using 1 MSI-X message >> re0: turning off MSI enable bit. >> re0: Chip rev. 0x2c800000 >> re0: MAC rev. 0x00000000 >> re0: Ethernet address: 00:19:99:f8:d3:0b >> miibus0: on re0 >> rgephy0: PHY 1 on miibus0 >> rgephy0: none, 10baseT, 10baseT-FDX, 10baseT-FDX-flow, 100baseTX, >> 100baseTX-FDX, 100baseTX-FDX-flow, 1000baseT, 1000baseT-master, >> 1000baseT-FDX, 1000baseT-FDX-master, 1000baseT-FDX-flow, >> 1000baseT-FDX-flow-master, auto, auto-flow >> >> Also, I'm loading this as a module, but, for as much as I know, this >> should not make any difference. >> >> >>> Did it ever work or you see the issue only on CURRENT? >> >> Never worked on this machine (I own it since the last days of February). >> >> I only installed current on it. If needed I can find time to test a >> recent 9.x snapshot on it. >> >> I worked around the problem till now using an USB ethernet adapter, >> always wanted to report this problem, but I've been lazy :) >> > > Would you try attached patch and let me know whether it makes any > difference? > Hi! Thanks for looking into this and sorry for the delay in reporting back. Unluckily the patch does not solve nor mitigates the problem. Symptoms are very similar. Here's a small session: root@marvin:~ [1]# ping 172.24.42.1 PING 172.24.42.1 (172.24.42.1): 56 data bytes 64 bytes from 172.24.42.1: icmp_seq=6 ttl=64 time=0.621 ms ^C --- 172.24.42.1 ping statistics --- 33 packets transmitted, 1 packets received, 97.0% packet loss round-trip min/avg/max/stddev = 0.621/0.621/0.621/0.000 ms root@marvin:~ [0]# ifconfig re0 tso root@marvin:~ [0]# ping 172.24.42.1 PING 172.24.42.1 (172.24.42.1): 56 data bytes ping: sendto: No route to host re0: re_eri_read: timed out 64 bytes from 172.24.42.1: icmp_seq=1 ttl=64 time=0.718 ms 64 bytes from 172.24.42.1: icmp_seq=2 ttl=64 time=0.471 ms 64 bytes from 172.24.42.1: icmp_seq=3 ttl=64 time=0.423 ms 64 bytes from 172.24.42.1: icmp_seq=4 ttl=64 time=0.567 ms 64 bytes from 172.24.42.1: icmp_seq=5 ttl=64 time=0.457 ms 64 bytes from 172.24.42.1: icmp_seq=6 ttl=64 time=0.364 ms ^C --- 172.24.42.1 ping statistics --- 7 packets transmitted, 6 packets received, 14.3% packet loss round-trip min/avg/max/stddev = 0.364/0.500/0.718/0.115 ms root@marvin:~ [0]# Only real difference is the re_eri_read timeout. It did not output that error message before. -- Guido Falsi From owner-freebsd-net@FreeBSD.ORG Fri Sep 6 23:36:13 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 5AEE7B74; Fri, 6 Sep 2013 23:36:13 +0000 (UTC) (envelope-from hiren.panchasara@gmail.com) Received: from mail-ee0-x234.google.com (mail-ee0-x234.google.com [IPv6:2a00:1450:4013:c00::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id C620B26FA; Fri, 6 Sep 2013 23:36:12 +0000 (UTC) Received: by mail-ee0-f52.google.com with SMTP id c41so1894401eek.39 for ; Fri, 06 Sep 2013 16:36:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=YJkdJPw9YCklntanhleYd8ZX+TWmZ/fX0KJtM8YbWNo=; b=DNIEsSESXKfyAHTT5kbxOaDYfjlWI+LsQ5UqKDNeWaiTQMjwwnMQtPgxSQVjRpTqVL e7wMi1+FFV7eLBwnozHCNT445WlZQumyI2LPAREevmUbxkveOglLT7DGuAHL/jezll+L fmFqQ199+bWIM11xP5maUItPhYM0mxWMYOa1BbYs7sNllPQ+/3wSW9GpWsB0Tthd+zPP BOtj1zdixlgcVVOAXsmP1kWeEKLaw/ni/Gr7gY6NEF3M8hcl0rkEzv2txO3UVsecnYbl qV9kTS6yteVfxgm0KTSdQXGDyokCHQZXIFiFheaijeV3njMeVcPsuW4363ISihOQAWAe LLoA== MIME-Version: 1.0 X-Received: by 10.14.126.73 with SMTP id a49mr8091176eei.48.1378510569830; Fri, 06 Sep 2013 16:36:09 -0700 (PDT) Received: by 10.14.105.137 with HTTP; Fri, 6 Sep 2013 16:36:09 -0700 (PDT) Date: Fri, 6 Sep 2013 16:36:09 -0700 Message-ID: Subject: mbuf autotuning effect From: hiren panchasara To: "freebsd-net@freebsd.org" , "freebsd-mips@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Sep 2013 23:36:13 -0000 We are seeing an interesting thing on a mips board with 32MB ram. We run out of mbuf very easily and looking at numbers it seems we are only getting 6mb of maxmbufmem. # sysctl -a | grep hw | grep mem hw.physmem: 33554432 hw.usermem: 21774336 hw.realmem: 33554432 # # sysctl -a | grep maxmbuf kern.ipc.maxmbufmem: 6291456 I believe that number is very low for a board with 32mb of ram. Looking at the code: sys/kern/kern_mbuf.c : tunable_mbinit() 124 realmem = qmin((quad_t)physmem * PAGE_SIZE, vm_kmem_size); 125 maxmbufmem = realmem / 2; 126 TUNABLE_QUAD_FETCH("kern.ipc.maxmbufmem", &maxmbufmem); 127 if (maxmbufmem > realmem / 4 * 3) 128 maxmbufmem = realmem / 4 * 3; So, realmem plays important role in determining maxmbufmem. physmem = 32mb PAGE_SIZE = 4096 vm_kmem_size is calculated inside sys/kern/kern_malloc.c : kmeminit() 705 vm_kmem_size = VM_KMEM_SIZE + nmbclusters * PAGE_SIZE; 706 mem_size = cnt.v_page_count; 707 708 #if defined(VM_KMEM_SIZE_SCALE) 709 vm_kmem_size_scale = VM_KMEM_SIZE_SCALE; 710 #endif 711 TUNABLE_INT_FETCH("vm.kmem_size_scale", &vm_kmem_size_scale); 712 if (vm_kmem_size_scale > 0 && 713 (mem_size / vm_kmem_size_scale) > (vm_kmem_size / PAGE_SIZE)) 714 vm_kmem_size = (mem_size / vm_kmem_size_scale) * PAGE_SIZE; here, VM_KMEM_SIZE = 12*1024*1024 nmbclusters = 0 (initially) PAGE_SIZE = 4096 # sysctl -a | grep v_page_count vm.stats.vm.v_page_count: 7035 and VM_KMEM_SIZE_SCALE = 3 for mips. So, vm_kmem_size = 12mb. Going back to tunable_mbinit(), we get realmem = 12mb. and masmbufmem = 6mb. Wanted to see if I am following the code correctly and how autotuning should work here. cheers, Hiren From owner-freebsd-net@FreeBSD.ORG Sat Sep 7 01:12:00 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 67221954; Sat, 7 Sep 2013 01:12:00 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-wg0-x236.google.com (mail-wg0-x236.google.com [IPv6:2a00:1450:400c:c00::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id C36B2201C; Sat, 7 Sep 2013 01:11:59 +0000 (UTC) Received: by mail-wg0-f54.google.com with SMTP id e11so2402009wgh.21 for ; Fri, 06 Sep 2013 18:11:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=8nhYhl82oPmR2/49cgxbZYaWCP3Z8DJpxzxAF259HNU=; b=ills5GmgikcHuQ5xaqfJek4tq73vl5EjO5vNQ0uUx0i6l73/4cGWW9OBtSPn2mbRyp htq1mzI+jW6ofgxBsZzbIjmLteuYFI8WF+uH0WCXBNb5qFcZnUBZ+9IVshQcvSKRQ6ZC z2at3tHjhydWP2RHczVrIdll3ounq+keTVE9vO4EaTCMQYAHfKKmv4xNzlX+sVgfS4Ye 3zaiXtmDJnefj5zQHfTkoX9gIYYwkGCALI+fxXmzt2Qat8P0YLeKXh6R4TbsDa2lYacc gt7pIbr0fDFclRQMSCrD4XGMTF2Mg+vAZD1Eh2YrWSxenyLCVkANLuzXXhHPYf/00KCD 4ugw== MIME-Version: 1.0 X-Received: by 10.180.211.3 with SMTP id my3mr465306wic.46.1378516318289; Fri, 06 Sep 2013 18:11:58 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.216.73.133 with HTTP; Fri, 6 Sep 2013 18:11:58 -0700 (PDT) In-Reply-To: References: Date: Fri, 6 Sep 2013 18:11:58 -0700 X-Google-Sender-Auth: kPN8wb3xpJJNRvh9L4uC7MGiMk0 Message-ID: Subject: Re: mbuf autotuning effect From: Adrian Chadd To: hiren panchasara Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: "freebsd-net@freebsd.org" , "freebsd-mips@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Sep 2013 01:12:00 -0000 Yeah, why is VM_KMEM_SIZE only 12mbyte for MIPS? That's a little low for a platform that has a direct map that's slightly larger than 12mb :) Warner? Juli? -adrian On 6 September 2013 16:36, hiren panchasara wrote: > We are seeing an interesting thing on a mips board with 32MB ram. > > We run out of mbuf very easily and looking at numbers it seems we are only > getting 6mb of maxmbufmem. > > # sysctl -a | grep hw | grep mem > hw.physmem: 33554432 > hw.usermem: 21774336 > hw.realmem: 33554432 > # > # sysctl -a | grep maxmbuf > kern.ipc.maxmbufmem: 6291456 > > I believe that number is very low for a board with 32mb of ram. > > Looking at the code: > > sys/kern/kern_mbuf.c : tunable_mbinit() > > 124 realmem = qmin((quad_t)physmem * PAGE_SIZE, vm_kmem_size); > 125 maxmbufmem = realmem / 2; > 126 TUNABLE_QUAD_FETCH("kern.ipc.maxmbufmem", &maxmbufmem); > 127 if (maxmbufmem > realmem / 4 * 3) > 128 maxmbufmem = realmem / 4 * 3; > > So, realmem plays important role in determining maxmbufmem. > > physmem = 32mb > PAGE_SIZE = 4096 > > vm_kmem_size is calculated inside sys/kern/kern_malloc.c : kmeminit() > > 705 vm_kmem_size = VM_KMEM_SIZE + nmbclusters * PAGE_SIZE; > 706 mem_size = cnt.v_page_count; > 707 > 708 #if defined(VM_KMEM_SIZE_SCALE) > 709 vm_kmem_size_scale = VM_KMEM_SIZE_SCALE; > 710 #endif > 711 TUNABLE_INT_FETCH("vm.kmem_size_scale", &vm_kmem_size_scale); > 712 if (vm_kmem_size_scale > 0 && > 713 (mem_size / vm_kmem_size_scale) > (vm_kmem_size / > PAGE_SIZE)) > 714 vm_kmem_size = (mem_size / vm_kmem_size_scale) * > PAGE_SIZE; > > here, > VM_KMEM_SIZE = 12*1024*1024 > nmbclusters = 0 (initially) > PAGE_SIZE = 4096 > # sysctl -a | grep v_page_count > vm.stats.vm.v_page_count: 7035 > > and VM_KMEM_SIZE_SCALE = 3 for mips. > > So, vm_kmem_size = 12mb. > > Going back to tunable_mbinit(), > we get realmem = 12mb. > and masmbufmem = 6mb. > > > Wanted to see if I am following the code correctly and how autotuning > should work here. > > cheers, > Hiren > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Sat Sep 7 03:26:46 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 71980CDE for ; Sat, 7 Sep 2013 03:26:46 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mail-ie0-f169.google.com (mail-ie0-f169.google.com [209.85.223.169]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 3CBB526AB for ; Sat, 7 Sep 2013 03:26:45 +0000 (UTC) Received: by mail-ie0-f169.google.com with SMTP id tp5so8868972ieb.28 for ; Fri, 06 Sep 2013 20:26:39 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:mime-version:content-type:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=6UMneV7NVC5314i60mzzVrs/Dl+/UKHPXiU07SakrDg=; b=U0/uWdfOqegMRToML7eDC+dRBXxEIYfXu+otCXlsyWbUY0cbeDeB1ZpgeyZdSMjrl1 mISRhQ78x0sLt4Zn3T1/gK23B0kB0saaSxlXGw4/UpBZ5Quq4WoexU3O5nEOBWKQfbYD ISIDPifHJQylim/AexyNPldAY0vF79dxufaU7v+1zkqMDcfjOp9YBCbBwfnt5TJOjitd ieSp9rAWhXKr0ks8H9u4+QGwyp05Zc3BJSaZTumxb9P/iS/v1NFYz47/OcuhDuWUg0zR xEwYQ+d1WNHMTedax8wgeyXXommDUXCFwH9w8qn5eY9XkouPlvUfHemLlN/DTj/usVvT w8wg== X-Gm-Message-State: ALoCoQm+3qH1Z7s/usj/H7NuX9DZA5M4ISlSHBj1uqU389BQsLDhY+6cUdTsYnkGi8kpRZRu0Lzt X-Received: by 10.50.6.71 with SMTP id y7mr832664igy.8.1378524399754; Fri, 06 Sep 2013 20:26:39 -0700 (PDT) Received: from 53.imp.bsdimp.com (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPSA id yt10sm1370121igb.9.1969.12.31.16.00.00 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 06 Sep 2013 20:26:38 -0700 (PDT) Sender: Warner Losh Subject: Re: mbuf autotuning effect Mime-Version: 1.0 (Apple Message framework v1085) Content-Type: text/plain; charset=us-ascii From: Warner Losh In-Reply-To: Date: Fri, 6 Sep 2013 21:26:37 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <9CBFAD35-D651-4E28-BEBB-DC3717F38567@bsdimp.com> References: To: Adrian Chadd X-Mailer: Apple Mail (2.1085) Cc: "freebsd-net@freebsd.org" , hiren panchasara , "freebsd-mips@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Sep 2013 03:26:46 -0000 On Sep 6, 2013, at 7:11 PM, Adrian Chadd wrote: > Yeah, why is VM_KMEM_SIZE only 12mbyte for MIPS? That's a little low = for a > platform that has a direct map that's slightly larger than 12mb :) >=20 > Warner? Juli? All architectures have it at 12MB, except sparc64 where it is 16MB. This = can be changed with the options VM_KMEM_SIZE=3Dxxxxx in the config file. So my guess as to why this is the case: cut and paste worked, so nobody = changed it after that. # Still need to reads hiren's email to comprehend it... Warner >=20 >=20 > -adrian >=20 >=20 >=20 > On 6 September 2013 16:36, hiren panchasara = wrote: >=20 >> We are seeing an interesting thing on a mips board with 32MB ram. >>=20 >> We run out of mbuf very easily and looking at numbers it seems we are = only >> getting 6mb of maxmbufmem. >>=20 >> # sysctl -a | grep hw | grep mem >> hw.physmem: 33554432 >> hw.usermem: 21774336 >> hw.realmem: 33554432 >> # >> # sysctl -a | grep maxmbuf >> kern.ipc.maxmbufmem: 6291456 >>=20 >> I believe that number is very low for a board with 32mb of ram. >>=20 >> Looking at the code: >>=20 >> sys/kern/kern_mbuf.c : tunable_mbinit() >>=20 >> 124 realmem =3D qmin((quad_t)physmem * PAGE_SIZE, = vm_kmem_size); >> 125 maxmbufmem =3D realmem / 2; >> 126 TUNABLE_QUAD_FETCH("kern.ipc.maxmbufmem", &maxmbufmem); >> 127 if (maxmbufmem > realmem / 4 * 3) >> 128 maxmbufmem =3D realmem / 4 * 3; >>=20 >> So, realmem plays important role in determining maxmbufmem. >>=20 >> physmem =3D 32mb >> PAGE_SIZE =3D 4096 >>=20 >> vm_kmem_size is calculated inside sys/kern/kern_malloc.c : kmeminit() >>=20 >> 705 vm_kmem_size =3D VM_KMEM_SIZE + nmbclusters * PAGE_SIZE; >> 706 mem_size =3D cnt.v_page_count; >> 707 >> 708 #if defined(VM_KMEM_SIZE_SCALE) >> 709 vm_kmem_size_scale =3D VM_KMEM_SIZE_SCALE; >> 710 #endif >> 711 TUNABLE_INT_FETCH("vm.kmem_size_scale", = &vm_kmem_size_scale); >> 712 if (vm_kmem_size_scale > 0 && >> 713 (mem_size / vm_kmem_size_scale) > (vm_kmem_size / >> PAGE_SIZE)) >> 714 vm_kmem_size =3D (mem_size / vm_kmem_size_scale) = * >> PAGE_SIZE; >>=20 >> here, >> VM_KMEM_SIZE =3D 12*1024*1024 >> nmbclusters =3D 0 (initially) >> PAGE_SIZE =3D 4096 >> # sysctl -a | grep v_page_count >> vm.stats.vm.v_page_count: 7035 >>=20 >> and VM_KMEM_SIZE_SCALE =3D 3 for mips. >>=20 >> So, vm_kmem_size =3D 12mb. >>=20 >> Going back to tunable_mbinit(), >> we get realmem =3D 12mb. >> and masmbufmem =3D 6mb. >>=20 >>=20 >> Wanted to see if I am following the code correctly and how autotuning >> should work here. >>=20 >> cheers, >> Hiren >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to = "freebsd-net-unsubscribe@freebsd.org" >>=20 > _______________________________________________ > freebsd-mips@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-mips > To unsubscribe, send any mail to = "freebsd-mips-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Sat Sep 7 10:19:21 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 70DCBB19; Sat, 7 Sep 2013 10:19:21 +0000 (UTC) (envelope-from mailinglists@martinlaabs.de) Received: from relay03.alfahosting-server.de (relay03.alfahosting-server.de [109.237.142.239]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 2E2D827D1; Sat, 7 Sep 2013 10:19:20 +0000 (UTC) Received: by relay03.alfahosting-server.de (Postfix, from userid 1001) id 9E28832C144B; Sat, 7 Sep 2013 12:19:13 +0200 (CEST) X-Spam-DCC: : relay01 1356; Body=1 Fuz1=1 Fuz2=1 X-Spam-Level: X-Spam-Status: No, score=0.0 required=7.0 tests=BAYES_50 autolearn=disabled version=3.2.5 Received: from alfa3018.alfahosting-server.de (alfa3018.alfahosting-server.de [109.237.140.30]) by relay03.alfahosting-server.de (Postfix) with ESMTPS id 64F3932C1437; Sat, 7 Sep 2013 12:19:10 +0200 (CEST) Received: from desktop-01.martinlaabs.de (p54B311BD.dip0.t-ipconnect.de [84.179.17.189]) by alfa3018.alfahosting-server.de (Postfix) with ESMTPSA id 23896515DEA1; Sat, 7 Sep 2013 12:19:10 +0200 (CEST) Message-ID: <522AFD9D.9010500@martinlaabs.de> Date: Sat, 07 Sep 2013 12:19:09 +0200 From: Martin Laabs User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130809 Thunderbird/17.0.8 MIME-Version: 1.0 To: "freebsd-net@freebsd.org" Subject: nfsv4 fails with kerberos Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Virus-Status: No X-Virus-Checker-Version: clamassassin 1.2.4 with ClamAV 0.97.3/17824/Sat Sep 7 06:37:51 2013 Cc: freebsd-arm X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Sep 2013 10:19:21 -0000 Hi, I set up a nfsv4 server with kerberos but when starting the nfs server on the arm (RBI-B) board I get the following error message and the first (managing part) of the nfs exits: "nfsd: can't register svc name" This error message is produced by the following code in /usr/src/sys/fs/nfsserver/nfs_nfsdkrpc.c: ==================:<======================= /* An empty string implies AUTH_SYS only. */ if (principal[0] != '\0') { ret2 = rpc_gss_set_svc_name_call(principal, "kerberosv5", GSS_C_INDEFINITE, NFS_PROG, NFS_VER2); ret3 = rpc_gss_set_svc_name_call(principal, "kerberosv5", GSS_C_INDEFINITE, NFS_PROG, NFS_VER3); ret4 = rpc_gss_set_svc_name_call(principal, "kerberosv5", GSS_C_INDEFINITE, NFS_PROG, NFS_VER4); if (!ret2 || !ret3 || !ret4) printf("nfsd: can't register svc name\n"); ==================:<======================= So something went wrong with the principals. Is there a way to get more information or more verbose debugging output from the nfs-server part of the kernel? Thank you, Martin Laabs From owner-freebsd-net@FreeBSD.ORG Sat Sep 7 11:50:08 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 2718FE4E; Sat, 7 Sep 2013 11:50:08 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id CE1782C03; Sat, 7 Sep 2013 11:50:07 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqMEAMURK1KDaFve/2dsb2JhbABbgz9Rgyq+VYE8dIIlAQEBAwEBAQEgKyALBRYOCgICDQUBEwIpAQkmBggHBAEcBIdbBgyxFpEhgSmNHoEFATMHEgGCVoE0A5Usg3iQN4M8IDJ8Bxci X-IronPort-AV: E=Sophos;i="4.90,859,1371096000"; d="scan'208";a="49847019" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-annu.net.uoguelph.ca with ESMTP; 07 Sep 2013 07:50:06 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 2B985B3F4B; Sat, 7 Sep 2013 07:50:06 -0400 (EDT) Date: Sat, 7 Sep 2013 07:50:06 -0400 (EDT) From: Rick Macklem To: Martin Laabs Message-ID: <955745639.19718288.1378554606139.JavaMail.root@uoguelph.ca> In-Reply-To: <522AFD9D.9010500@martinlaabs.de> Subject: Re: nfsv4 fails with kerberos MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.201] X-Mailer: Zimbra 7.2.1_GA_2790 (ZimbraWebClient - FF3.0 (Win)/7.2.1_GA_2790) Cc: freebsd-net@freebsd.org, freebsd-arm X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Sep 2013 11:50:08 -0000 Martin Laabs wrote: > Hi, > > I set up a nfsv4 server with kerberos but when starting the nfs > server on > the arm (RBI-B) board I get the following error message and the first > (managing part) of the nfs exits: > > "nfsd: can't register svc name" > > This error message is produced by the following code in > /usr/src/sys/fs/nfsserver/nfs_nfsdkrpc.c: > > > ==================:<======================= > /* An empty string implies AUTH_SYS only. */ > if (principal[0] != '\0') { > ret2 = rpc_gss_set_svc_name_call(principal, > "kerberosv5", GSS_C_INDEFINITE, NFS_PROG, NFS_VER2); > ret3 = rpc_gss_set_svc_name_call(principal, > "kerberosv5", GSS_C_INDEFINITE, NFS_PROG, NFS_VER3); > ret4 = rpc_gss_set_svc_name_call(principal, > "kerberosv5", GSS_C_INDEFINITE, NFS_PROG, NFS_VER4); > > if (!ret2 || !ret3 || !ret4) > printf("nfsd: can't register svc name\n"); > ==================:<======================= > > So something went wrong with the principals. Is there a way to get > more > information or more verbose debugging output from the nfs-server part > of > the kernel? > The above message normally indicates that the gssd daemon isn't running. Here's a few places you can get info: man nfsv4, gssd http://code.google.com/p/macnfsv4/wiki/FreeBSD8KerberizedNFSSetup - This was done quite a while ago and I should ggo in and update it, but I think it is still mostly correct for server side. (The client in head/10 now does have "host based initiator cred" support.) Feel free to update it. All you should need to do so is a Google login. You need a service principal for "nfs", which means an entry for a principal that looks like: nfs/.@ (Stuff in "<>" needs to be filled in with the answer for your machine.) in /etc/krb5.keytab i the server. rick > Thank you, > Martin Laabs > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to > "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Sat Sep 7 13:24:17 2013 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id AD72884D for ; Sat, 7 Sep 2013 13:24:17 +0000 (UTC) (envelope-from 3_ygrUg0JA5Q216H56zFy56AT4Ay69.0CAB2H3F22zG1.CF4@calendar-server.bounces.google.com) Received: from mail-ee0-x24a.google.com (mail-ee0-x24a.google.com [IPv6:2a00:1450:4013:c00::24a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 139DD20E0 for ; Sat, 7 Sep 2013 13:24:16 +0000 (UTC) Received: by mail-ee0-f74.google.com with SMTP id c1so259725eek.5 for ; Sat, 07 Sep 2013 06:24:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:reply-to:sender:auto-submitted:message-id:date:subject :from:to:content-type; bh=cG5AB+MbmLMCCtjWbLhRpNWzX6WgWZX1kvrzBMI22KU=; b=eQK1D+sbwj2iTJmIGxI79r2kaKZcGRii8CC7UU04MdBMuxF6J2Zao2d2pZKYoYQR2D A4D6MyazIxZiK5xIrIx5aaqtN714ekvZen1uCq/jiWkTzdUTXitdiYJKTy+twiJI7xnU e7dAqSYqDH43TSBB/eRmbaCdx4uh+/allFjHBTo9ZrS8xZ7nGPPZ+caWhnqecbjZOXa0 hkmK2PIin8nkQKrkmQFft1EyXy8Ok2xi4mQYEcYFYd9BfKyMkRfpu9i8QOmQbDefn7ZD BGokeDB07rZRdY5v0FrYFFxuhoZT8vIzyj1qhjYS+Iy7f7jD5+Db17q846lN/6tWS3s5 a7DA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:sender:auto-submitted:message-id:date:subject :from:to:content-type; bh=cG5AB+MbmLMCCtjWbLhRpNWzX6WgWZX1kvrzBMI22KU=; b=iN8mrK+Eopg0j9Ws806zTRxRAU08pnFymo3yi37EPeUeU+ZX6JJDHxq65UortkYUzk QfpYC5Gv0VghG0AxBuO6JCrA5eqyJ8rZVXN2jjXnVNu1CI+jXCvyyvg44PmNr66Y9cVB 1TvXURNt0HEyeaqKyu3TGNVu4nUMG5k0wMXNxrH/oHlry7MK0yVqfc9NIS7C9Nud0d7F EDRMBX/Q295LN6TPUAp+/F3KMHlOUZ62yg8XiCAVx4fW81q8wqfpKfLLSa2j/tKSSVWs o5Qood3auL1gsslFUF9mR0r4Zfu7xXAcZEbDCz1VTFAxguxJw/7TtD2wZcGh5XpUJ4ZM +TlA== MIME-Version: 1.0 X-Received: by 10.14.7.8 with SMTP id 8mr6193237eeo.4.1378560255510; Sat, 07 Sep 2013 06:24:15 -0700 (PDT) Sender: Google Calendar Auto-Submitted: auto-generated Message-ID: <089e0163534454ce7804e5cb124a@google.com> Date: Sat, 07 Sep 2013 13:24:15 +0000 Subject: Invitation: Hello My Dearest, @ Sat Sep 7, 2013 9:30am - 10:30am (edithibrahim5@gmail.com) From: "edithibrahim5@gmail.com" To: "net@freebsd.org" Content-Type: multipart/mixed; boundary=089e0163534454ce6d04e5cb1249 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: "edithibrahim5@gmail.com" List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Sep 2013 13:24:17 -0000 --089e0163534454ce6d04e5cb1249 Content-Type: text/plain; charset=windows-1252; format=flowed; delsp=yes Content-Transfer-Encoding: base64 WW91IGhhdmUgYmVlbiBpbnZpdGVkIHRvIHRoZSBmb2xsb3dpbmcgZXZlbnQuDQoNClRpdGxlOiBI ZWxsbyBNeSBEZWFyZXN0LA0KSGVsbG8gTXkgRGVhcmVzdCwNCg0KDQpQbGVhc2UgcGVybWl0IG1l IHRvIGludHJvZHVjZSBteXNlbGYsIEkgYW0gTWlzcyBFZGl0aCBJYnJhaGltIENvdWxpYmFseSAy MyAgDQp5ZWFycyBvbGQgZmVtYWxlIGZyb20gdGhlIFJlcHVibGljIG9mIEl2b3J5IENvYXN0LCBX ZXN0IEFmcmljYSwgSSdtIHRoZSAgDQpEYXVnaHRlciBvZiBMYXRlIENoaWVmIFNndC4gSWJyYWhp bSBDb3VsaWJhbHkgKEEuSy5BIEdlbmVyYWwgSUIgKS4gTXkgbGF0ZSAgDQpGYXRoZXIgd2FzIGEg d2VsbCBrbm93biBJdm9yeSBDb2FzdCBtaWxpdGFyeSBsZWFkZXIuIEhlIGRpZWQgb24gVGh1cnNk YXkgMjggIA0KQXByaWwgMjAxMSBmb2xsb3dpbmcgYSBmaWdodCB3aXRoIHRoZSBSZXB1YmxpY2Fu IEZvcmNlcyBvZiBJdm9yeSBDb2FzdCAgDQooRlJDSSkuDQoNCg0KWW91IGNhbiByZWFkIG1vcmUg YWJvdXQgbXkgZmF0aGVyIGluIHRoZSBsaW5rIGJlbG93Og0KaHR0cDovL3d3dy5ndWFyZGlhbi5j by51ay93b3JsZC8yMDExL2Fwci8yOC9pdm9yeS1jb2FzdC1yZW5lZ2FkZS13YXJsb3JkLWlicmFo aW0tY291bGliYWx5DQoNCg0KSSBhbSBjb25zdHJhaW5lZCB0byBjb250YWN0IHlvdSBiZWNhdXNl IG9mIHRoZSBtYWx0cmVhdG1lbnQgd2hpY2ggSSBhbSAgDQpyZWNlaXZpbmcgZnJvbSBteSBzdGVw IG1vdGhlci4gU2hlIHBsYW5uZWQgdG8gdGFrZSBhd2F5IGFsbCBteSBsYXRlICANCmZhdGhlcidz IHRyZWFzdXJ5IGFuZCBwcm9wZXJ0aWVzIGZyb20gbWUgc2luY2UgdGhlIHVuZXhwZWN0ZWQgZGVh dGggb2YgbXkgIA0KYmVsb3ZlZCBGYXRoZXIgYmVjYXVzZSBteSBtb3RoZXIgZGllZCBkdXJpbmcg Y2hpbGQgYmlydGggYW5kIEkgd2FzIGxlZnQgIA0KYWxvbmUgd2l0aCBteSBzdGVwIG1vdGhlciB0 byB0YWtlIGNhcmUgb2YgbWUuIE1lYW53aGlsZSBJIHdhbnRlZCB0byB0cmF2ZWwgIA0KdG8gRXVy b3BlLCBidXQgc2hlIGhpZGUgYXdheSBteSB0cmF2ZWxpbmcgZG9jdW1lbnRzLiBMdWNraWx5IHNo ZSBkaWQgbm90ICANCmRpc2NvdmVyIHdoZXJlIEkga2VwdCBteSBmYXRoZXIncyBGaWxlIHdoaWNo IGNvbnRhaW5lZCBpbXBvcnRhbnQgZG9jdW1lbnRzICANCmxpa2Ugd2lsbCBhbmQgZGVwb3NpdCBj ZXJ0aWZpY2F0ZSBvZiBteSBGYXRoZXIncyBmdW5kIHdoaWNoIGJlYXJzIG15IG5hbWUgIA0KYXMg dGhlIG5leHQgb2Yga2luIHRvIGluaGVyaXQgdGhlIG1vbmV5IGluIGhpcyBiYW5rIGFjY291bnQu IE5vdyBJIGFtICANCnByZXNlbnRseSBzdGF5aW5nIGluIHRoZSBSZWZ1Z2VlIE1pc3Npb24gQ2Ft cCBpbiBCdXJraW5hIEZhc28uIEkgYW0gc2Vla2luZyAgDQpmb3IgbG9uZyB0ZXJtIHJlbGF0aW9u c2hpcCBhbmQgaW52ZXN0bWVudCBhc3Npc3RhbmNlLiBNeSBmYXRoZXIgb2YgYmxlc3NlZCAgDQpt ZW1vcnkgZGVwb3NpdGVkIHRoZSBzdW0gb2YgVVMkIDExLjUgTWlsbGlvbiBpbiBCYW5rIE9mIEFm cmljYSBoZXJlIGluICANCkJ1cmtpbmEgRmFzbyB3aXRoIG15IG5hbWUgYXMgdGhlIG5leHQgb2Yg a2luLg0KDQoNCkkgaGF2ZSBjb250YWN0ZWQgdGhlIEJhbmsgdG8gY2xlYXIgdGhlIGRlcG9zaXQg YnV0IHRoZSBCcmFuY2ggTWFuYWdlciB0b2xkICANCm1lIHRoYXQgbXkgbGF0ZSBmYXRoZXIgcGxh Y2UgYW4gaW5zdHJ1Y3Rpb24gb24gdGhlIGRlcG9zaXRlZCBmdW5kIHRoYXQgSSAgDQptdXN0IHBy ZXNlbnQgYSBmb3JlaWduIHRydXN0ZWUgd2hvIHdpbGwgaGVscCBtZSBpbiBpbnZlc3RtZW50IG9m IHRoZSBmdW5kLg0KDQoNCkhvd2V2ZXIsIHRoZSBtYW5hZ2VyIGFkdmlzZWQgbWUgdG8gcHJvdmlk ZSBhIHRydXN0ZWUgd2hvIHdpbGwgc3RhbmQgb24gbXkgIA0KYmVoYWxmIGZvciB0aGUgdHJhbnNm ZXIgb2YgdGhlIGZ1bmQuIEkgd2FudGVkIHRvIGluZm9ybSBteSBzdGVwbW90aGVyIGFib3V0ICAN CnRoaXMgZGVwb3NpdCBidXQgSSBhbSBhZnJhaWQgdGhhdCBzaGUgd2lsbCBub3Qgb2ZmZXIgbWUg YW55dGhpbmcgYWZ0ZXIgdGhlICANCnJlbGVhc2Ugb2YgdGhlIG1vbmV5IGJlY2F1c2Ugc2hlIHRo cmVhdGVuIHRvIGtpbGwgbWUuDQoNCg0KVGhlcmVmb3JlLCBJIGRlY2lkZSB0byBzZWVrIGZvciB5 b3VyIGhlbHAgaW4gdHJhbnNmZXJyaW5nIHRoZSBtb25leSBpbnRvICANCnlvdXIgYmFuayBhY2Nv dW50IHdoaWxlIEkgd2lsbCByZWxvY2F0ZSB0byB5b3VyIGNvdW50cnkgYW5kIHNldHRsZSBkb3du ICANCndpdGggeW91LiBBcyB5b3UgaW5kaWNhdGUgeW91ciBpbnRlcmVzdCB0byBoZWxwIG1lIEkg d2lsbCBnaXZlIHlvdSB0aGUgIA0KYWNjb3VudCBudW1iZXIgYW5kIHRoZSBjb250YWN0IG9mIHRo ZSBiYW5rIHdoZXJlIG15IGxhdGUgYmVsb3ZlZCBmYXRoZXIgIA0KZGVwb3NpdGVkIHRoZSBtb25l eSB3aXRoIG15IG5hbWUgYXMgdGhlIG5leHQgb2Yga2luLiBJdCBpcyBteSBpbnRlbnRpb24gdG8g IA0KY29tcGVuc2F0ZSB5b3Ugd2l0aCAzMCUgb3V0IG9mIHRoZSB0b3RhbCBtb25leSBhZnRlciB0 aGUgdHJhbnNmZXIgZm9yIHlvdXIgIA0KYXNzaXN0YW5jZSBhbmQgdGhlIGJhbGFuY2Ugc2hhbGwg YmUgbXkgaW52ZXN0bWVudCBpbiBhbnkgcHJvZml0YWJsZSB2ZW50dXJlICANCndoaWNoIHlvdSB3 aWxsIHJlY29tbWVuZCB0byBtZSBhcyBJIGhhdmUgbm8gaWRlYSBhYm91dCBmb3JlaWduIGludmVz dG1lbnQuICANClBsZWFzZSBhbGwgY29tbXVuaWNhdGlvbnMgc2hvdWxkIGJlIHRocm91Z2ggdGhp cyBlbWFpbCBhZGRyZXNzIGZvciAgDQpjb25maWRlbnRpYWwgcHVycG9zZXMsIFBsZWFzZSByZXBs eSBtZSBmb3IgbW9yZSBkZXRhaWxzIHZpYSBteSBtYWlsYm94OiAgDQplZGl0aGlicmFoaW0xMUBn bWFpbC5jb20NCg0KDQpUaGFua3MgaW4gYW50aWNpcGF0aW9uIG9mIHlvdXIgcG9zaXRpdmUgcmVz cG9uc2UuDQoNCg0KWW91cnMgc2luY2VyZWx5DQpNaXNzIEVkaXRoIElicmFoaW0gQ291bGliYWx5 DQoNCldoZW46IFNhdCBTZXAgNywgMjAxMyA5OjMwYW0gliAxMDozMGFtIEVhc3Rlcm4gVGltZQ0K Q2FsZW5kYXI6IGVkaXRoaWJyYWhpbTVAZ21haWwuY29tDQpXaG86DQogICAgIChHdWVzdCBsaXN0 IGhhcyBiZWVuIGhpZGRlbiBhdCBvcmdhbml6ZXIncyByZXF1ZXN0KQ0KDQpFdmVudCBkZXRhaWxz OiAgDQpodHRwczovL3d3dy5nb29nbGUuY29tL2NhbGVuZGFyL2V2ZW50P2FjdGlvbj1WSUVXJmVp ZD1hMlYxWTI1dlpHRnlZM05pWW1vek5uUjJjVGh1T1cxaFoyc2dibVYwUUdaeVpXVmljMlF1YjNK biZ0b2s9TWpNalpXUnBkR2hwWW5KaGFHbHROVUJuYldGcGJDNWpiMjA0TnpRek9XTTROelk1Wm1R Mk1ETTBaVE14TURobU56Y3pOalUwWlRGak5ESTFZbVl5WW1FMCZjdHo9QW1lcmljYS9OZXdfWW9y ayZobD1lbg0KDQpJbnZpdGF0aW9uIGZyb20gR29vZ2xlIENhbGVuZGFyOiBodHRwczovL3d3dy5n b29nbGUuY29tL2NhbGVuZGFyLw0KDQpZb3UgYXJlIHJlY2VpdmluZyB0aGlzIGNvdXJ0ZXN5IGVt YWlsIGF0IHRoZSBhY2NvdW50IG5ldEBmcmVlYnNkLm9yZyAgDQpiZWNhdXNlIHlvdSBhcmUgYW4g YXR0ZW5kZWUgb2YgdGhpcyBldmVudC4NCg0KVG8gc3RvcCByZWNlaXZpbmcgZnV0dXJlIG5vdGlm aWNhdGlvbnMgZm9yIHRoaXMgZXZlbnQsIGRlY2xpbmUgdGhpcyBldmVudC4gIA0KQWx0ZXJuYXRp dmVseSB5b3UgY2FuIHNpZ24gdXAgZm9yIGEgR29vZ2xlIGFjY291bnQgYXQgIA0KaHR0cHM6Ly93 d3cuZ29vZ2xlLmNvbS9jYWxlbmRhci8gYW5kIGNvbnRyb2wgeW91ciBub3RpZmljYXRpb24gc2V0 dGluZ3MgZm9yICANCnlvdXIgZW50aXJlIGNhbGVuZGFyLg0K --089e0163534454ce6d04e5cb1249-- From owner-freebsd-net@FreeBSD.ORG Sat Sep 7 19:21:50 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 3773FC92; Sat, 7 Sep 2013 19:21:50 +0000 (UTC) (envelope-from hiren.panchasara@gmail.com) Received: from mail-ee0-x22e.google.com (mail-ee0-x22e.google.com [IPv6:2a00:1450:4013:c00::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 68BD42085; Sat, 7 Sep 2013 19:21:49 +0000 (UTC) Received: by mail-ee0-f46.google.com with SMTP id c13so1861778eek.33 for ; Sat, 07 Sep 2013 12:21:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=MXDiKYPL9UTA75YEcSySs+P7pJZDfsKO9VVvYqTraSU=; b=AanHypU5gnUwh6rmIizsShEUbPN9a/h8/DE1eQtud2FcQ8eQwoJe9pg8VH3lIBsEgO OUJGfnXKN9Xq835iJmwSVWmvgPWWK1v5aePaGME4cMsMffkK1xWz7l0lXqHHE5VgqcCc WiQGOaZWgRMbj5PK4Md6/oElE5gl4jbuS6s+8DWrQutCzREc0WEMUvhk27hZoBPAfvwv ovJcfiW64NoqyBiJXwtVbbt8X+0yMzpPz4EURxfZrta28Cgpqtcu/l+uPudPoltZUEMm 5PVHW+CpJ7w34ehClEgiFSWcf4X1kvqXlYMyCNBDobRIqJTEKbKHAeomsTUhTCCyStHF gBDw== MIME-Version: 1.0 X-Received: by 10.14.103.69 with SMTP id e45mr6825732eeg.51.1378581707816; Sat, 07 Sep 2013 12:21:47 -0700 (PDT) Received: by 10.14.105.137 with HTTP; Sat, 7 Sep 2013 12:21:47 -0700 (PDT) Received: by 10.14.105.137 with HTTP; Sat, 7 Sep 2013 12:21:47 -0700 (PDT) In-Reply-To: <9CBFAD35-D651-4E28-BEBB-DC3717F38567@bsdimp.com> References: <9CBFAD35-D651-4E28-BEBB-DC3717F38567@bsdimp.com> Date: Sat, 7 Sep 2013 12:21:47 -0700 Message-ID: Subject: Re: mbuf autotuning effect From: hiren panchasara To: Warner Losh Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-net@freebsd.org, Adrian Chadd , "freebsd-mips@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Sep 2013 19:21:50 -0000 On Sep 6, 2013 8:26 PM, "Warner Losh" wrote: > > > On Sep 6, 2013, at 7:11 PM, Adrian Chadd wrote: > > > Yeah, why is VM_KMEM_SIZE only 12mbyte for MIPS? That's a little low for a > > platform that has a direct map that's slightly larger than 12mb :) > > > > Warner? Juli? > > All architectures have it at 12MB, except sparc64 where it is 16MB. This can be changed with the options VM_KMEM_SIZE=xxxxx in the config file. Right. Does that mean for any platform, if we do not have nmbclusters pre-set in kmeminit() than we will always have pretty low value of vm_kmem_size. And because of that, if maxmbufmem is not pre-set (via loader.conf) inside tunable_mbinit() , we will have very low value for maxmbufmem too. I hope (partially believe) that my understanding is not entirely correct. Because if its correct, we arw depending on loader.conf instead of actually auto tuning. Thanks, Hiren > > So my guess as to why this is the case: cut and paste worked, so nobody changed it after that. > > # Still need to reads hiren's email to comprehend it... > > Warner > > > > > > > > -adrian > > > > > > > > On 6 September 2013 16:36, hiren panchasara wrote: > > > >> We are seeing an interesting thing on a mips board with 32MB ram. > >> > >> We run out of mbuf very easily and looking at numbers it seems we are only > >> getting 6mb of maxmbufmem. > >> > >> # sysctl -a | grep hw | grep mem > >> hw.physmem: 33554432 > >> hw.usermem: 21774336 > >> hw.realmem: 33554432 > >> # > >> # sysctl -a | grep maxmbuf > >> kern.ipc.maxmbufmem: 6291456 > >> > >> I believe that number is very low for a board with 32mb of ram. > >> > >> Looking at the code: > >> > >> sys/kern/kern_mbuf.c : tunable_mbinit() > >> > >> 124 realmem = qmin((quad_t)physmem * PAGE_SIZE, vm_kmem_size); > >> 125 maxmbufmem = realmem / 2; > >> 126 TUNABLE_QUAD_FETCH("kern.ipc.maxmbufmem", &maxmbufmem); > >> 127 if (maxmbufmem > realmem / 4 * 3) > >> 128 maxmbufmem = realmem / 4 * 3; > >> > >> So, realmem plays important role in determining maxmbufmem. > >> > >> physmem = 32mb > >> PAGE_SIZE = 4096 > >> > >> vm_kmem_size is calculated inside sys/kern/kern_malloc.c : kmeminit() > >> > >> 705 vm_kmem_size = VM_KMEM_SIZE + nmbclusters * PAGE_SIZE; > >> 706 mem_size = cnt.v_page_count; > >> 707 > >> 708 #if defined(VM_KMEM_SIZE_SCALE) > >> 709 vm_kmem_size_scale = VM_KMEM_SIZE_SCALE; > >> 710 #endif > >> 711 TUNABLE_INT_FETCH("vm.kmem_size_scale", &vm_kmem_size_scale); > >> 712 if (vm_kmem_size_scale > 0 && > >> 713 (mem_size / vm_kmem_size_scale) > (vm_kmem_size / > >> PAGE_SIZE)) > >> 714 vm_kmem_size = (mem_size / vm_kmem_size_scale) * > >> PAGE_SIZE; > >> > >> here, > >> VM_KMEM_SIZE = 12*1024*1024 > >> nmbclusters = 0 (initially) > >> PAGE_SIZE = 4096 > >> # sysctl -a | grep v_page_count > >> vm.stats.vm.v_page_count: 7035 > >> > >> and VM_KMEM_SIZE_SCALE = 3 for mips. > >> > >> So, vm_kmem_size = 12mb. > >> > >> Going back to tunable_mbinit(), > >> we get realmem = 12mb. > >> and masmbufmem = 6mb. > >> > >> > >> Wanted to see if I am following the code correctly and how autotuning > >> should work here. > >> > >> cheers, > >> Hiren > >> _______________________________________________ > >> freebsd-net@freebsd.org mailing list > >> http://lists.freebsd.org/mailman/listinfo/freebsd-net > >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > >> > > _______________________________________________ > > freebsd-mips@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-mips > > To unsubscribe, send any mail to "freebsd-mips-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Sat Sep 7 19:30:07 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 2D05E265; Sat, 7 Sep 2013 19:30:07 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-pb0-x230.google.com (mail-pb0-x230.google.com [IPv6:2607:f8b0:400e:c01::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id F3B442113; Sat, 7 Sep 2013 19:30:06 +0000 (UTC) Received: by mail-pb0-f48.google.com with SMTP id ma3so4496498pbc.7 for ; Sat, 07 Sep 2013 12:30:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=8DYCLTHlJzGdrNtOACKu21/6Gi148ctm04t8saE3DYM=; b=qakw+0c+JRQwm6xtA0F31Z0qA6ez2fr1VqLaAdahlQbzX1FCi2i1T+uHwglOTyakVq gOBSVMFlX6/RiThyZgAe3yjqgyZdSDFLC30PMO/XpC2sXu15KUHKLcJ2HN2JeWoqkAxJ Qv6+RjVpt2H1J+k1VI0ipJcLqZaU89I1M+9qivGYSGEVDcaJCennrQfnas0dXPG5ipel gIDfsKBKBDb1qiAyQh68hBfuos5+5ZCBy9JM4RZ5hhq37p7wKmJL3zXW2BdWYn1EMEVQ 5WKt+pwW05Ik6TpxQPNcLl7rRa2UOwQ5MOgveLdigUxfWijlSAn46rNzdiyRBRHKk6zq uaXg== MIME-Version: 1.0 X-Received: by 10.68.26.202 with SMTP id n10mr9817581pbg.97.1378582206659; Sat, 07 Sep 2013 12:30:06 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.70.126.71 with HTTP; Sat, 7 Sep 2013 12:30:06 -0700 (PDT) In-Reply-To: References: <9CBFAD35-D651-4E28-BEBB-DC3717F38567@bsdimp.com> Date: Sat, 7 Sep 2013 12:30:06 -0700 X-Google-Sender-Auth: NcbzLF7OMyP40qAOT8X-d7vbC8Q Message-ID: Subject: Re: mbuf autotuning effect From: Adrian Chadd To: hiren panchasara Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: FreeBSD Net , "freebsd-mips@freebsd.org" , Warner Losh X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Sep 2013 19:30:07 -0000 On 7 September 2013 12:21, hiren panchasara wrote: > > On Sep 6, 2013 8:26 PM, "Warner Losh" wrote: > > > > > > On Sep 6, 2013, at 7:11 PM, Adrian Chadd wrote: > > > > > Yeah, why is VM_KMEM_SIZE only 12mbyte for MIPS? That's a little low > for a > > > platform that has a direct map that's slightly larger than 12mb :) > > > > > > Warner? Juli? > > > > All architectures have it at 12MB, except sparc64 where it is 16MB. This > can be changed with the options VM_KMEM_SIZE=xxxxx in the config file. > > Right. Does that mean for any platform, if we do not have nmbclusters > pre-set in kmeminit() than we will always have pretty low value of > vm_kmem_size. And because of that, if maxmbufmem is not pre-set (via > loader.conf) inside tunable_mbinit() , we will have very low value for > maxmbufmem too. > > I hope (partially believe) that my understanding is not entirely correct. > Because if its correct, we arw depending on loader.conf instead of actually > auto tuning. > > Thanks, > Hiren > > .. so how's this work on i386? ARM? -adrian From owner-freebsd-net@FreeBSD.ORG Sat Sep 7 19:56:06 2013 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 5BC55B0A; Sat, 7 Sep 2013 19:56:06 +0000 (UTC) (envelope-from ian@FreeBSD.org) Received: from mho-02-ewr.mailhop.org (mho-02-ewr.mailhop.org [204.13.248.72]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 3184A2247; Sat, 7 Sep 2013 19:56:06 +0000 (UTC) Received: from c-24-8-230-52.hsd1.co.comcast.net ([24.8.230.52] helo=damnhippie.dyndns.org) by mho-02-ewr.mailhop.org with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.72) (envelope-from ) id 1VIOc0-0008OX-Ro; Sat, 07 Sep 2013 19:56:05 +0000 Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id r87Ju2Up074842; Sat, 7 Sep 2013 13:56:02 -0600 (MDT) (envelope-from ian@FreeBSD.org) X-Mail-Handler: Dyn Standard SMTP by Dyn X-Originating-IP: 24.8.230.52 X-Report-Abuse-To: abuse@dyndns.com (see http://www.dyndns.com/services/sendlabs/outbound_abuse.html for abuse reporting information) X-MHO-User: U2FsdGVkX18bWJ+EOP5Ki+fBG9iHt6q2 Subject: Re: mbuf autotuning effect From: Ian Lepore To: hiren panchasara In-Reply-To: References: <9CBFAD35-D651-4E28-BEBB-DC3717F38567@bsdimp.com> Content-Type: text/plain; charset="us-ascii" Date: Sat, 07 Sep 2013 13:56:02 -0600 Message-ID: <1378583762.1111.512.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: freebsd-net@FreeBSD.org, "freebsd-mips@freebsd.org" , Warner Losh X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Sep 2013 19:56:06 -0000 On Sat, 2013-09-07 at 12:21 -0700, hiren panchasara wrote: > On Sep 6, 2013 8:26 PM, "Warner Losh" wrote: > > > > > > On Sep 6, 2013, at 7:11 PM, Adrian Chadd wrote: > > > > > Yeah, why is VM_KMEM_SIZE only 12mbyte for MIPS? That's a little > low > for a > > > platform that has a direct map that's slightly larger than 12mb :) > > > > > > Warner? Juli? > > > > All architectures have it at 12MB, except sparc64 where it is 16MB. > This > can be changed with the options VM_KMEM_SIZE=xxxxx in the config file. > > Right. Does that mean for any platform, if we do not have nmbclusters > pre-set in kmeminit() than we will always have pretty low value of > vm_kmem_size. And because of that, if maxmbufmem is not pre-set (via > loader.conf) inside tunable_mbinit() , we will have very low value for > maxmbufmem too. > > I hope (partially believe) that my understanding is not entirely > correct. > Because if its correct, we arw depending on loader.conf instead of > actually > auto tuning. > I think the part of this that strikes me as strange is calling 20% of physical memory used for network buffers a "very low value". It seems outrageously high to me. I'd be pissed if that much memory got wasted on network buffers on one of our $work platforms with so little memory. So the fact that you think it's crazy-low and I think it's crazy-high may be a sign that it's auto-tuned to a reasonable compromise, and in both our cases the right fix would be to use the available knobs to tune things for our particular uses. -- Ian From owner-freebsd-net@FreeBSD.ORG Sat Sep 7 20:39:38 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id CFF98BD0; Sat, 7 Sep 2013 20:39:38 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-wi0-x22a.google.com (mail-wi0-x22a.google.com [IPv6:2a00:1450:400c:c05::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 208C72439; Sat, 7 Sep 2013 20:39:37 +0000 (UTC) Received: by mail-wi0-f170.google.com with SMTP id cb5so2073806wib.3 for ; Sat, 07 Sep 2013 13:39:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=Eogs7PtTl8wK6UKeDC50CHhbbeGcRYkfUewmC8zCO0Y=; b=Vsg/5MZiCV1xgikUzKlyAKI0crtBaVDmpF4AVfg0reayRrBYtc2S9W5AYNIK9mXLNz Rxto51P1ffodJHz7aVwDbFcCWgsPw+ebEU9YWQf1yRanhotjJQhSZrfpOg69mdUcO+55 +YdYYPgcjarmPcEe1NygtjH/BtZZXjRZDSJ5eu38nvApolsRivvTKhXu+MPVw6LXzLbz XLQXU2gQbOQMv0F2PR4Jn8nS8oox3v+yoA7vTJ1NRZlEkc372Xvsa2U//DWtgwq+EWE3 9RgAZTOm7jaQi0cdLgnSQ4QcbkcPpBIhXb5khn+MRtz3NIAwAx9AilmxX3R27v9A5qca Zd1w== MIME-Version: 1.0 X-Received: by 10.180.211.206 with SMTP id ne14mr3132494wic.30.1378586376497; Sat, 07 Sep 2013 13:39:36 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.216.73.133 with HTTP; Sat, 7 Sep 2013 13:39:36 -0700 (PDT) In-Reply-To: <1378583762.1111.512.camel@revolution.hippie.lan> References: <9CBFAD35-D651-4E28-BEBB-DC3717F38567@bsdimp.com> <1378583762.1111.512.camel@revolution.hippie.lan> Date: Sat, 7 Sep 2013 13:39:36 -0700 X-Google-Sender-Auth: dO_aVyQDWHdX9IOWuPlMNNMaolo Message-ID: Subject: Re: mbuf autotuning effect From: Adrian Chadd To: Ian Lepore Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: FreeBSD Net , hiren panchasara , "freebsd-mips@freebsd.org" , Warner Losh X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Sep 2013 20:39:39 -0000 On 7 September 2013 12:56, Ian Lepore wrote: > I think the part of this that strikes me as strange is calling 20% of > physical memory used for network buffers a "very low value". It seems > outrageously high to me. I'd be pissed if that much memory got wasted > on network buffers on one of our $work platforms with so little memory. > > So the fact that you think it's crazy-low and I think it's crazy-high > may be a sign that it's auto-tuned to a reasonable compromise, and in > both our cases the right fix would be to use the available knobs to tune > things for our particular uses. > Well, which limit is actually being hit here? 20% of 32mb is still a lot of memory buffers.. Now, for sizing up the needed buffers for wifi: assuming 512 tx, 512 rx buffers for two ath NICs. another 512+512 buffers for each arge NICs. So, 4096 mbufs here, 2k each, so ~ 8mb of RAM. Amusing.. -adrian