From owner-freebsd-net@FreeBSD.ORG Sun Feb 22 01:51:39 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 92F144FA for ; Sun, 22 Feb 2015 01:51:39 +0000 (UTC) Received: from phlegethon.blisses.org (phlegethon.blisses.org [50.56.97.101]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7104A7C4 for ; Sun, 22 Feb 2015 01:51:38 +0000 (UTC) Received: from blisses.org (cocytus.blisses.org [23.25.209.73]) by phlegethon.blisses.org (Postfix) with ESMTPSA id 1BD7A148960; Sat, 21 Feb 2015 20:51:37 -0500 (EST) Date: Sat, 21 Feb 2015 20:51:35 -0500 From: Mason Loring Bliss To: Konstantin Kulikov Subject: Re: NAT question Message-ID: <20150222015135.GD24491@blisses.org> References: <20150221020818.GY24491@blisses.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Feb 2015 01:51:39 -0000 On Sat, Feb 21, 2015 at 10:55:26PM +0400, Konstantin Kulikov wrote: > ipfw nat 1 config ip 1.2.3.4 [...] > Should work (untested though). That looks about right. Thank you. I'll be testing tomorrow I think. Once I got things working in a basic way I want to add in some traffic shaping, but I'll leave that alone until I'm sure my simple initial config doing exactly what I want it to. Again, thank you! -- Mason Loring Bliss mason@blisses.org Ewige Blumenkraft! awake ? sleep : random() & 2 ? dream : sleep; -- Hamlet, Act III, Scene I From owner-freebsd-net@FreeBSD.ORG Sun Feb 22 05:48:03 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A58034DB for ; Sun, 22 Feb 2015 05:48:03 +0000 (UTC) Received: from frv189.fwdcdn.com (frv189.fwdcdn.com [212.42.77.189]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5DE71EA5 for ; Sun, 22 Feb 2015 05:48:02 +0000 (UTC) Received: from [10.10.1.26] (helo=frv197.fwdcdn.com) by frv189.fwdcdn.com with esmtp ID 1YPP3G-0008sM-I6 for freebsd-net@freebsd.org; Sun, 22 Feb 2015 07:25:58 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=ukr.net; s=ffe; h=Content-Transfer-Encoding:Content-Type:MIME-Version:References:In-Reply-To:Message-Id:Cc:To:Subject:From:Date; bh=i1gJoNPFYLfGkvroyIQdg8K0cAAhtJP62wCrzoP8Vtw=; b=qGDhZ5Ww3BqHdaSAFOPl7hi3DJ1N9nzB4wN+LscQICeMSd5lrp1eRC+N9GiPvk23neX11oJQ98/7HAUxyW0C94tghL20F0utsg9oCX69ZloWDAjeqSrP85HN8+FY39gg+ZX/a6+L6ZoSp0gwuz7fme0cTPXo8DH2scDHZIRD3fc=; Received: from [10.10.10.34] (helo=frv34.fwdcdn.com) by frv197.fwdcdn.com with smtp ID 1YPP39-000OOx-11 for freebsd-net@freebsd.org; Sun, 22 Feb 2015 07:25:51 +0200 Date: Sun, 22 Feb 2015 07:25:50 +0200 From: wishmaster Subject: Re[2]: NAT question To: k.kulikov2@gmail.com X-Mailer: mail.ukr.net 5.0 Message-Id: <1424582647.585579533.z0kl61ci@frv34.fwdcdn.com> In-Reply-To: References: <20150221020818.GY24491@blisses.org> MIME-Version: 1.0 Received: from artemrts@ukr.net by frv34.fwdcdn.com; Sun, 22 Feb 2015 07:25:50 +0200 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: binary Content-Disposition: inline X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: freebsd-net@freebsd.org, mason@blisses.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Feb 2015 05:48:03 -0000 --- Original Message --- From: "Konstantin Kulikov" Date: 21 February 2015, 20:55:54 > Hello. > > ipfw nat 1 config ip 1.2.3.4 > ipfw nat 2 config ip 1.2.3.5 > ipfw nat 3 config ip 1.2.3.6 > ipfw add nat 1 ip from 4.5.6.7/32 to any out via $ext > ipfw add nat 2 ip from 4.5.6.0/24 to any out via $ext > ipfw add nat 3 ip from 8.9.0.0/24 to any out via $ext > ipfw add nat 1 ip from any to 1.2.3.4 in via $ext > ipfw add nat 2 ip from any to 1.2.3.5 in via $ext > ipfw add nat 3 ip from any to 1.2.3.6 in via $ext > > Should work (untested though). I think you should use nat global in case of topic starter. > As for your dnat questing I think you want redirect_addr nat option. > > On Sat, Feb 21, 2015 at 5:08 AM, Mason Loring Bliss wrote: > > Hi all. > > > > With iptables, I can say something like: > > > > -t nat -A POSTROUTING -o eth0 -s 4.5.6.7/32 -d 0/0 -j SNAT --to-source 1.2.3.4 > > -t nat -A POSTROUTING -o eth0 -s 4.5.6.0/24 -d 0/0 -j SNAT --to-source 1.2.3.5 > > -t nat -A POSTROUTING -o eth0 -s 8.9.0.0/24 -d 0/0 -j SNAT --to-source 1.2.3.6 > > > > So, traffic going out from 4.5.6.7 goes into the world sourced from 1.2.3.4, > > whereas the rest of 4.5.6/24 goes as 1.2.3.5, and all of 8.9.0/24 comes out > > from 1.2.3.6. > > > > I don't see how to do this with IPFW. I assume there's some way to do it with > > the GENERIC kernel, so I'm assuming natd is deprecated, as it requires a > > custom kernel, as far as I can see. > > > > How do I accomplish this with IPFW? Or do I need to use PF for this? Or are > > those independent of the NAT after all and I want to use something else? If > > that's the case, does it require natd and a custom kernel, or is there > > something that works with a GENERIC kernel? (This will be 10.1, FWIW.) > > > > Thanks. > > > > -- > > Love is a snowmobile racing across the tundra and then suddenly it > > flips over, pinning you underneath. At night, the ice weasels come. > > _______________________________________________ > > freebsd-net@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-net > > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Mon Feb 23 06:54:57 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2C0B6955 for ; Mon, 23 Feb 2015 06:54:57 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [IPv6:2001:4f8:3:ffe0:406a:0:50:2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0CC61384 for ; Mon, 23 Feb 2015 06:54:57 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [127.0.1.5]) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9) with ESMTP id t1N6su4f064163 for ; Mon, 23 Feb 2015 06:54:56 GMT (envelope-from root@phabric-backend.isc.freebsd.org) Received: (from root@localhost) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9/Submit) id t1N6sugG064162; Mon, 23 Feb 2015 06:54:56 GMT (envelope-from root) Date: Mon, 23 Feb 2015 06:54:56 +0000 To: freebsd-net@freebsd.org From: "rodrigc (Craig Rodrigues)" Subject: [Differential] [Changed Subscribers] D1944: PF and VIMAGE fixes Message-ID: <7a22162cd23273c5129eb3d02012bbe7@localhost.localdomain> X-Priority: 3 Thread-Topic: D1944: PF and VIMAGE fixes X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: X-Phabricator-Cc: In-Reply-To: References: Thread-Index: NDc2NzM0MzY4OTdiYThiNTU1MjY2ZDZmMTJiIFTqzsA= X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Feb 2015 06:54:57 -0000 rodrigc added subscribers: freebsd-net, freebsd-pf. REVISION DETAIL https://reviews.freebsd.org/D1944 To: nvass-gmx.com, glebius, rodrigc Cc: freebsd-pf, freebsd-net From owner-freebsd-net@FreeBSD.ORG Mon Feb 23 06:55:31 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 09AC3B08 for ; Mon, 23 Feb 2015 06:55:31 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [IPv6:2001:4f8:3:ffe0:406a:0:50:2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DD7A13C9 for ; Mon, 23 Feb 2015 06:55:30 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [127.0.1.5]) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9) with ESMTP id t1N6tU5Z064772 for ; Mon, 23 Feb 2015 06:55:30 GMT (envelope-from root@phabric-backend.isc.freebsd.org) Received: (from root@localhost) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9/Submit) id t1N6tUoe064771; Mon, 23 Feb 2015 06:55:30 GMT (envelope-from root) Date: Mon, 23 Feb 2015 06:55:30 +0000 To: freebsd-net@freebsd.org From: "rodrigc (Craig Rodrigues)" Subject: [Differential] [Changed Subscribers] D1944: PF and VIMAGE fixes Message-ID: X-Priority: 3 Thread-Topic: D1944: PF and VIMAGE fixes X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: X-Phabricator-Cc: X-Phabricator-Cc: In-Reply-To: References: Thread-Index: NDc2NzM0MzY4OTdiYThiNTU1MjY2ZDZmMTJiIFTqzuI= X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Feb 2015 06:55:31 -0000 rodrigc added a subscriber: freebsd-virtualization. REVISION DETAIL https://reviews.freebsd.org/D1944 To: nvass-gmx.com, glebius, rodrigc Cc: freebsd-virtualization, freebsd-pf, freebsd-net From owner-freebsd-net@FreeBSD.ORG Mon Feb 23 06:55:43 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D6D6BC21 for ; Mon, 23 Feb 2015 06:55:43 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [IPv6:2001:4f8:3:ffe0:406a:0:50:2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B4D453DC for ; Mon, 23 Feb 2015 06:55:43 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [127.0.1.5]) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9) with ESMTP id t1N6thV4064831 for ; Mon, 23 Feb 2015 06:55:43 GMT (envelope-from root@phabric-backend.isc.freebsd.org) Received: (from root@localhost) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9/Submit) id t1N6theU064830; Mon, 23 Feb 2015 06:55:43 GMT (envelope-from root) Date: Mon, 23 Feb 2015 06:55:43 +0000 To: freebsd-net@freebsd.org From: "rodrigc (Craig Rodrigues)" Subject: [Differential] [Updated] D1944: PF and VIMAGE fixes Message-ID: <9a90f4ae2612253a343e96941f8b1bac@localhost.localdomain> X-Priority: 3 Thread-Topic: D1944: PF and VIMAGE fixes X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: X-Phabricator-Cc: X-Phabricator-Cc: In-Reply-To: References: Thread-Index: NDc2NzM0MzY4OTdiYThiNTU1MjY2ZDZmMTJiIFTqzu8= X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Feb 2015 06:55:43 -0000 rodrigc added a reviewer: gnn. REVISION DETAIL https://reviews.freebsd.org/D1944 To: nvass-gmx.com, glebius, rodrigc, gnn Cc: freebsd-virtualization, freebsd-pf, freebsd-net From owner-freebsd-net@FreeBSD.ORG Mon Feb 23 06:56:17 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D4C5DD58 for ; Mon, 23 Feb 2015 06:56:17 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [IPv6:2001:4f8:3:ffe0:406a:0:50:2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B0495405 for ; Mon, 23 Feb 2015 06:56:17 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [127.0.1.5]) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9) with ESMTP id t1N6uHak065091 for ; Mon, 23 Feb 2015 06:56:17 GMT (envelope-from root@phabric-backend.isc.freebsd.org) Received: (from root@localhost) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9/Submit) id t1N6uHOI065090; Mon, 23 Feb 2015 06:56:17 GMT (envelope-from root) Date: Mon, 23 Feb 2015 06:56:17 +0000 To: freebsd-net@freebsd.org From: "rodrigc (Craig Rodrigues)" Subject: [Differential] [Updated] D1944: PF and VIMAGE fixes Message-ID: <9c229b891ad0884148c61e17cf3d7c54@localhost.localdomain> X-Priority: 3 Thread-Topic: D1944: PF and VIMAGE fixes X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: X-Phabricator-Cc: X-Phabricator-Cc: In-Reply-To: References: Thread-Index: NDc2NzM0MzY4OTdiYThiNTU1MjY2ZDZmMTJiIFTqzxE= X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Feb 2015 06:56:17 -0000 rodrigc added reviewers: bz, zec, trociny. REVISION DETAIL https://reviews.freebsd.org/D1944 To: nvass-gmx.com, glebius, rodrigc, gnn, bz, zec, trociny Cc: freebsd-virtualization, freebsd-pf, freebsd-net From owner-freebsd-net@FreeBSD.ORG Mon Feb 23 06:57:03 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 60F77EA9 for ; Mon, 23 Feb 2015 06:57:03 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [IPv6:2001:4f8:3:ffe0:406a:0:50:2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3D3EA5F7 for ; Mon, 23 Feb 2015 06:57:03 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [127.0.1.5]) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9) with ESMTP id t1N6v2xo065919 for ; Mon, 23 Feb 2015 06:57:02 GMT (envelope-from root@phabric-backend.isc.freebsd.org) Received: (from root@localhost) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9/Submit) id t1N6v2Z8065918; Mon, 23 Feb 2015 06:57:02 GMT (envelope-from root) Date: Mon, 23 Feb 2015 06:57:02 +0000 To: freebsd-net@freebsd.org From: "rodrigc (Craig Rodrigues)" Subject: [Differential] [Updated] D1944: PF and VIMAGE fixes Message-ID: <738862ceaaf2aad2fa124a999deb40ee@localhost.localdomain> X-Priority: 3 Thread-Topic: D1944: PF and VIMAGE fixes X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: X-Phabricator-Cc: X-Phabricator-Cc: In-Reply-To: References: Thread-Index: NDc2NzM0MzY4OTdiYThiNTU1MjY2ZDZmMTJiIFTqzz4= X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Feb 2015 06:57:03 -0000 rodrigc added a comment. Nikos has posted these patches to improve VIMAGE support in PF. If some of the folks who are experienced with PF and VIMAGE could take a look, that would be really great. REVISION DETAIL https://reviews.freebsd.org/D1944 To: nvass-gmx.com, glebius, gnn, bz, zec, trociny, rodrigc Cc: freebsd-virtualization, freebsd-pf, freebsd-net From owner-freebsd-net@FreeBSD.ORG Mon Feb 23 08:50:23 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 04DA0103 for ; Mon, 23 Feb 2015 08:50:23 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [IPv6:2001:4f8:3:ffe0:406a:0:50:2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B2F3117E for ; Mon, 23 Feb 2015 08:50:22 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [127.0.1.5]) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9) with ESMTP id t1N8oMJx088949 for ; Mon, 23 Feb 2015 08:50:22 GMT (envelope-from root@phabric-backend.isc.freebsd.org) Received: (from root@localhost) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9/Submit) id t1N8oMKD088947; Mon, 23 Feb 2015 08:50:22 GMT (envelope-from root) Date: Mon, 23 Feb 2015 08:50:22 +0000 To: freebsd-net@freebsd.org From: "hselasky (Hans Petter Selasky)" Subject: [Differential] [Updated, 2, 452 lines] D1438: FreeBSD callout rewrite and cleanup Message-ID: X-Priority: 3 Thread-Topic: D1438: FreeBSD callout rewrite / cleanup X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: X-Phabricator-Cc: In-Reply-To: References: Thread-Index: YzU3ODk0MGM0Y2E4NmE3NjY4YjJlZmFkM2UyIFTq6c4= X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Feb 2015 08:50:23 -0000 hselasky updated this revision to Diff 3922. hselasky added a comment. Replace spinlock_enter/exit with KASSERT's. CHANGES SINCE LAST UPDATE https://reviews.freebsd.org/D1438?vs=3897&id=3922 REVISION DETAIL https://reviews.freebsd.org/D1438 AFFECTED FILES share/man/man9/Makefile share/man/man9/timeout.9 sys/kern/init_main.c sys/kern/kern_clocksource.c sys/kern/kern_condvar.c sys/kern/kern_lock.c sys/kern/kern_switch.c sys/kern/kern_synch.c sys/kern/kern_thread.c sys/kern/kern_timeout.c sys/kern/subr_sleepqueue.c sys/ofed/include/linux/completion.h sys/sys/_callout.h sys/sys/callout.h sys/sys/proc.h To: hselasky, jhb, adrian, markj, emaste, sbruno, imp, lstewart, rwatson, gnn, rrs, kostikbel, delphij, neel, erj, remkolodder, bcr, brueffer, brd, allanjude, wblock Cc: wblock, freebsd-net From owner-freebsd-net@FreeBSD.ORG Mon Feb 23 20:38:56 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F1EAEAA0 for ; Mon, 23 Feb 2015 20:38:56 +0000 (UTC) Received: from mail-ie0-x22b.google.com (mail-ie0-x22b.google.com [IPv6:2607:f8b0:4001:c03::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BA3D3B8D for ; Mon, 23 Feb 2015 20:38:56 +0000 (UTC) Received: by iecrl12 with SMTP id rl12so26708026iec.2 for ; Mon, 23 Feb 2015 12:38:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:content-type; bh=6Ba/JnbNMN9XrsVAEJ84+frQlchHUboJg3o6RIa3g+Q=; b=eb5BqanQzAmp7qx9QYi1Sa+eToKoBiyvWerUrOMtuWUY83SYKERw2FvAwRWZf5EQ/K QTJAiE2aW0e+fqjPPqs6710479KnxccU1J/4cpRo26OrfPO/zdyRtq2EeN3quWsbTCvX wCnOcCIwudRJPrtCJ6G+xTZKT7JtTOIOMdySBr2OwPS0qw3R7I9ClrcrS93uTVNzvrfQ MHJsfXiea2jOAUt3tFZzSDemw/6r68aV+6/G74GP4WwbRA3DFS78aXdXveCKSUMGnlHe sZC3va6ZFX7ou/XF54qEQDlYcytn6mRVEGljDePdeyAuZvyCFQlM6ZBZP+Bkz9nJZEX7 1PZw== MIME-Version: 1.0 X-Received: by 10.50.107.7 with SMTP id gy7mr15351329igb.49.1424723936019; Mon, 23 Feb 2015 12:38:56 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.36.17.66 with HTTP; Mon, 23 Feb 2015 12:38:55 -0800 (PST) Date: Mon, 23 Feb 2015 12:38:55 -0800 X-Google-Sender-Auth: CAMfXzP19_ZD5REfAxiPq1kQCu4 Message-ID: Subject: Looking for help with RSS IPv6 - need software hash code! From: Adrian Chadd To: FreeBSD Net Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Feb 2015 20:38:57 -0000 Hi! The last annoyingish bit for IPv6 RSS support is a full software hash routine to calculate an RSS hash based on the various fields in the IPv6 header. I'm unfortunately out of spare cycles to try and finish it so I'm asking for help. In order to support this in the kernel we need to not only know what to do with the hardware RSS, but also have a fallover path to software hashing if the NIC doesn't hash it for us, or doesn't hash it correctly. For example, if we get IPv4/IPv6 fragments (which yes, are a thing, don't tell me they're not), we need to reassemble the fragments into a single frame, and then re-calculate the RSS hash on the reassembled frame header to figure out where it would've gone. We have to do this because the RSS hash value is also used as an index into the PCBGROUP hash table array - so no matter whether packets in a flow are fragmented or not, they're correctly serialised into the same netisr queue and the PCB information for that flow is in the same PCBGROUP array bucket. For doing "correct" RSS, we need to have support for hashing various fields and this isn't just limited to 2-tuple / 4-tuple hashing. IPv6 has a bunch of mobility header options and RSS has support for these (the TCP_EX and UDP_EX fields.) I've done the software hashing path for IPv4, but I need someone to help me do the IPv6 RSS hash calculation for all the variations - IPv6 2-tuple IPv6 TCP, IPv6 UDP, IPv6 TCP_EX (mobility), UDP_EX (mobility.) The microsoft RSS specification is online and freely available; it has all of these as examples. So, I'm asking for help. If you're able to help, please look at the code in -HEAD in sys/netinet/in_rss.c and sys/netinet6/in6_rss.c. You'll see what's missing. You don't need a NIC that has RSS enabled; if you enable RSS and PCBGROUPS in the kernel (and bump up the number of netisr queues; that still isn't auto-set at boot time) then you'll see that traffic will get distributed by software hashing of the packet headers. Thanks! -adrian From owner-freebsd-net@FreeBSD.ORG Mon Feb 23 23:37:29 2015 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 330D93E8 for ; Mon, 23 Feb 2015 23:37:29 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 18E64214 for ; Mon, 23 Feb 2015 23:37:29 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id t1NNbSs8099100 for ; Mon, 23 Feb 2015 23:37:28 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 197592] can't switch bpf to zero-copy mode Date: Mon, 23 Feb 2015 23:37:29 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.0-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: hiren@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Feb 2015 23:37:29 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=197592 Hiren Panchasara changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-net@FreeBSD.org --- Comment #1 from Hiren Panchasara --- Moving to -net. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Tue Feb 24 06:37:10 2015 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C41575E7; Tue, 24 Feb 2015 06:37:10 +0000 (UTC) Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.59.238]) by mx1.freebsd.org (Postfix) with ESMTP id 889A6381; Tue, 24 Feb 2015 06:37:09 +0000 (UTC) Received: by onelab2.iet.unipi.it (Postfix, from userid 275) id 0C3787300A; Tue, 24 Feb 2015 07:36:00 +0100 (CET) Date: Tue, 24 Feb 2015 07:36:00 +0100 From: Luigi Rizzo To: net@freebsd.org, current@freebsd.org Subject: netmap support for the Intel 40G card in head Message-ID: <20150224063600.GA72100@onelab2.iet.unipi.it> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.20 (2009-06-14) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Feb 2015 06:37:10 -0000 Thanks to Jack Vogel who made available some hardware, I have been able to implement native netmap support for the Intel "ixl" driver, aka xl710, the 40G card. I have committed the code to 'head', and the same works in stable/10 where it will land soon. Testers welcome. I have seen 32 Mpps on tx, 24 Mpps on rx with two ports on the same card connected to each other. This is our second 40G device for which we have native netmap support, which makes FreeBSD quite unique. cheers luigi Date: Tue, 24 Feb 2015 06:20:51 +0000 (UTC) From: Luigi Rizzo Subject: svn commit: r279232 - in head/sys/dev: ixl netmap To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Author: luigi Date: Tue Feb 24 06:20:50 2015 New Revision: 279232 URL: https://svnweb.freebsd.org/changeset/base/279232 Log: Add native netmap support to ixl. Preliminary tests indicate 32 Mpps on tx, 24 Mpps on rx with source and receiver on two different ports of the same 40G card. Optimizations are likely possible. The code follows closely the one for ixgbe so i do not expect stability issues. Hardware kindly supplied by Intel. Reviewed by: Jack Vogel MFC after: 1 week Added: head/sys/dev/netmap/if_ixl_netmap.h (contents, props changed) Modified: head/sys/dev/ixl/if_ixl.c head/sys/dev/ixl/ixl_txrx.c From owner-freebsd-net@FreeBSD.ORG Tue Feb 24 10:54:24 2015 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9682F63E for ; Tue, 24 Feb 2015 10:54:24 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7D23717A for ; Tue, 24 Feb 2015 10:54:24 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id t1OAsODR053926 for ; Tue, 24 Feb 2015 10:54:24 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 182382] [tcp] sysctl to set TCP CC method on BIG ENDIAN systems fails Date: Tue, 24 Feb 2015 10:54:23 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 9.2-PRERELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: jau@iki.fi X-Bugzilla-Status: In Progress X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Feb 2015 10:54:24 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=182382 jau@iki.fi changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jau@iki.fi --- Comment #2 from jau@iki.fi --- (In reply to Mark Linimon from comment #1) This was fixed in 2014 by HPS. The problem occurred due to someone had mistakenly tried to handle sysctl values without copyin/copyout calls. On i386 and amd64 hardware direct references using pointers may work, but on ppc and sparc64 copyin and copyout are needed to move data between the user and kernel address spaces. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Tue Feb 24 11:37:12 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5F659384 for ; Tue, 24 Feb 2015 11:37:12 +0000 (UTC) Received: from mail-wi0-f175.google.com (mail-wi0-f175.google.com [209.85.212.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id ECACF982 for ; Tue, 24 Feb 2015 11:37:10 +0000 (UTC) Received: by mail-wi0-f175.google.com with SMTP id r20so24763514wiv.2 for ; Tue, 24 Feb 2015 03:37:09 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to:cc :content-type; bh=S1C2CTgDdukj20VJDWE1uePHz8BqX9IjEBzkfyQd1vQ=; b=MvX4AiDuWOIQmF5shv+HJ+2/G/nymV3OftdmP2DUSMbujf2bBy/feGUwq+GLMdL1tN JWAs6pj7kCRtxJF2iVBzk9gvCKXIz1Q7xfaQt9tuOftKFiBxxnYkLM7t+MoO54r3l6uu A4jxqSlsx2xiGv95s7d//fE9OLRyd1+2Q6hb9K9NG85iwl59P/OnD+NA7qS+ND+Prwab O8barB+33H9buZS/diW2Fa7ji7IiKyBV90CHPWWXp+s4oJ8moku1aUaeMvX2DWRp2lac Ga/Abr4T5jiJ2FTj39gjZqXZPVOia2I6PYTqg73KJxfvaglvlqJjUTO3398n5S0CHg1s VlCg== X-Gm-Message-State: ALoCoQlDytqeYMFFMMLnKpbtuPRWZQtt1SvC2hUQxxrI69Nu0qNGPY90BOw50221UXIJ2BX9MbH6 X-Received: by 10.181.8.103 with SMTP id dj7mr29545429wid.75.1424777826965; Tue, 24 Feb 2015 03:37:06 -0800 (PST) MIME-Version: 1.0 Received: by 10.27.143.19 with HTTP; Tue, 24 Feb 2015 03:36:46 -0800 (PST) From: Tim Borgeaud Date: Tue, 24 Feb 2015 11:36:46 +0000 Message-ID: Subject: NFS: kernel modules (loading/unloading) and scheduling To: freebsd-net@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: Mark Hills X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Feb 2015 11:37:12 -0000 Hi FreeBSD folks, here at Framestore Mark Hills (cc'd) has been looking at how NFS servers schedule/prioritize incoming requests with a view towards a client/user 'fair share' of a service. We are taking a look at trying out some simple approaches of queuing up and handling requests. There are, obviously, various matters to deal with here, and several complications. However, an initial consideration is how we might best develop and test with the FreeBSD kernel code. Specifically, whether we would be able to unload and reload NFS related kernel modules, or whether there are any other alternatives. It looks like there are several modules involved such as krpc, nfssvc, nfscommon, nfslock, which we can build outside of the kernel itself, but which do not all support unloading. Not being able to reload a module does seem to present a hurdle to development and, therefore, I'd like to know how FreeBSD developers have managed development of the NFS functionality and what approaches may be recommended. It occurs to me that it would be possible for us to consider adding some ability to unload some of these modules, even if it were not suitable for anything other than development. Therefore, an extension of my main, more general, query is to ask how straightforward or fundamentally difficult this may be (for the NFS modules)? Thanks very much -- Tim Borgeaud Systems Developer =E2=80=8BFramestore From owner-freebsd-net@FreeBSD.ORG Tue Feb 24 15:26:34 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 466E1916 for ; Tue, 24 Feb 2015 15:26:34 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [IPv6:2001:4f8:3:ffe0:406a:0:50:2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 008E396E for ; Tue, 24 Feb 2015 15:26:33 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [127.0.1.5]) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9) with ESMTP id t1OFQXad094445 for ; Tue, 24 Feb 2015 15:26:33 GMT (envelope-from root@phabric-backend.isc.freebsd.org) Received: (from root@localhost) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9/Submit) id t1OFQXIu094444; Tue, 24 Feb 2015 15:26:33 GMT (envelope-from root) Date: Tue, 24 Feb 2015 15:26:33 +0000 To: freebsd-net@freebsd.org From: "hselasky (Hans Petter Selasky)" Subject: [Differential] [Updated, 2, 445 lines] D1438: FreeBSD callout rewrite and cleanup Message-ID: X-Priority: 3 Thread-Topic: D1438: FreeBSD callout rewrite / cleanup X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: X-Phabricator-Cc: In-Reply-To: References: Thread-Index: YzU3ODk0MGM0Y2E4NmE3NjY4YjJlZmFkM2UyIFTsmCk= X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Feb 2015 15:26:34 -0000 hselasky updated this revision to Diff 3951. hselasky added a comment. MPSAFE callouts do not support CPU migration with this implementation CHANGES SINCE LAST UPDATE https://reviews.freebsd.org/D1438?vs=3922&id=3951 REVISION DETAIL https://reviews.freebsd.org/D1438 AFFECTED FILES share/man/man9/Makefile share/man/man9/timeout.9 sys/kern/init_main.c sys/kern/kern_clocksource.c sys/kern/kern_condvar.c sys/kern/kern_lock.c sys/kern/kern_switch.c sys/kern/kern_synch.c sys/kern/kern_thread.c sys/kern/kern_timeout.c sys/kern/subr_sleepqueue.c sys/ofed/include/linux/completion.h sys/sys/_callout.h sys/sys/callout.h sys/sys/proc.h To: hselasky, jhb, adrian, markj, emaste, sbruno, imp, lstewart, rwatson, gnn, rrs, kostikbel, delphij, neel, erj, remkolodder, bcr, brueffer, brd, allanjude, wblock Cc: wblock, freebsd-net From owner-freebsd-net@FreeBSD.ORG Tue Feb 24 15:33:39 2015 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 699E0DB1; Tue, 24 Feb 2015 15:33:39 +0000 (UTC) Received: from mail-lb0-x22f.google.com (mail-lb0-x22f.google.com [IPv6:2a00:1450:4010:c04::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E2E1AA7A; Tue, 24 Feb 2015 15:33:38 +0000 (UTC) Received: by lbdu14 with SMTP id u14so25567756lbd.1; Tue, 24 Feb 2015 07:33:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Gfr+RjB7coMGABUqwcAlPOZ/rKUV7JcjbZn/5/sHmME=; b=laVthSMA+o6e/1XefBucKGt8ugLfW0sgU/kyLE69DJ6GaKpv0FskSJjGomCdJCRZhd Wd5k7ogbl2s8tHn+Cugc0F2VgDqNlz+AMeDNmEsgmfe4cFS3lOTmu441fMiP+/aQntX8 7cFYKbyXcJhli4YgiiPpsSoJ6t5EilkeBHBD5ucs8SYi89f3uzcT+NE10gJYOwtIb9r9 LmKTyGgis8xA6nqhMjCjm+ZoEMjBB0JvWgWf1RIpHzDyokhhaKM3Cb/jolkQsfVKSESM 34uejHAwbnrQlIf+iJxavTCk8+E4XidPPnlMBSu0+hneF6Woduk5YetsjO4AlmamN9YS +3qg== MIME-Version: 1.0 X-Received: by 10.152.115.169 with SMTP id jp9mr14884366lab.121.1424792016674; Tue, 24 Feb 2015 07:33:36 -0800 (PST) Received: by 10.114.78.131 with HTTP; Tue, 24 Feb 2015 07:33:36 -0800 (PST) In-Reply-To: <20150224063600.GA72100@onelab2.iet.unipi.it> References: <20150224063600.GA72100@onelab2.iet.unipi.it> Date: Tue, 24 Feb 2015 10:33:36 -0500 Message-ID: Subject: Re: netmap support for the Intel 40G card in head From: Ryan Stone To: Luigi Rizzo Content-Type: text/plain; charset=UTF-8 Cc: FreeBSD Current , FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Feb 2015 15:33:39 -0000 This is great! Thanks to both you and Intel. I'm planning on getting SR-IOV support into head this week, which would allow you to create ixlv instances (on the same hardware). Any chance that you'd have the time to look into supporting SR-IOV for that driver too? From owner-freebsd-net@FreeBSD.ORG Tue Feb 24 18:02:59 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 585E86BC for ; Tue, 24 Feb 2015 18:02:59 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [IPv6:2001:4f8:3:ffe0:406a:0:50:2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 11FDAEB1 for ; Tue, 24 Feb 2015 18:02:59 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [127.0.1.5]) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9) with ESMTP id t1OI2wJi062167 for ; Tue, 24 Feb 2015 18:02:58 GMT (envelope-from root@phabric-backend.isc.freebsd.org) Received: (from root@localhost) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9/Submit) id t1OI2wJ0062166; Tue, 24 Feb 2015 18:02:58 GMT (envelope-from root) Date: Tue, 24 Feb 2015 18:02:58 +0000 To: freebsd-net@freebsd.org From: "hselasky (Hans Petter Selasky)" Subject: [Differential] [Updated, 2, 367 lines] D1438: FreeBSD callout rewrite and cleanup Message-ID: <82e8a53c86b0b7183bbad7833100d477@localhost.localdomain> X-Priority: 3 Thread-Topic: D1438: FreeBSD callout rewrite / cleanup X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: X-Phabricator-Cc: In-Reply-To: References: Thread-Index: YzU3ODk0MGM0Y2E4NmE3NjY4YjJlZmFkM2UyIFTsvNI= X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , , MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Feb 2015 18:02:59 -0000 hselasky updated the summary for this revision. hselasky updated this revision to Diff 3956. hselasky added a comment. Restore how callout_lock() worked, so that a callout can migrate at any time regardless of callout type. Update manual page to reflect this change. Use an unsigned integer for callout buckets. The temporary LIST used by the callout code only needs to be initialized once at startup. CHANGES SINCE LAST UPDATE https://reviews.freebsd.org/D1438?vs=3951&id=3956 REVISION DETAIL https://reviews.freebsd.org/D1438 AFFECTED FILES share/man/man9/Makefile share/man/man9/timeout.9 sys/kern/init_main.c sys/kern/kern_clocksource.c sys/kern/kern_condvar.c sys/kern/kern_lock.c sys/kern/kern_switch.c sys/kern/kern_synch.c sys/kern/kern_thread.c sys/kern/kern_timeout.c sys/kern/subr_sleepqueue.c sys/ofed/include/linux/completion.h sys/sys/_callout.h sys/sys/callout.h sys/sys/proc.h To: hselasky, jhb, adrian, markj, emaste, sbruno, imp, lstewart, rwatson, gnn, rrs, kostikbel, delphij, neel, erj, remkolodder, bcr, brueffer, brd, allanjude, wblock Cc: wblock, freebsd-net From owner-freebsd-net@FreeBSD.ORG Tue Feb 24 19:20:07 2015 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D9923265 for ; Tue, 24 Feb 2015 19:20:07 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BFE719D3 for ; Tue, 24 Feb 2015 19:20:07 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id t1OJK7YP055108 for ; Tue, 24 Feb 2015 19:20:07 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 188899] [cas] cas ethernet driver seems to have issues with some multiport card and mother board combinations Date: Tue, 24 Feb 2015 19:20:08 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.0-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: gavin@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Feb 2015 19:20:07 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=188899 --- Comment #2 from Gavin Atkinson --- Hi, Are you able to provide the Sun part number of the network card, so that I can try and pick one up on eBay? Thanks, Gavin -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Tue Feb 24 21:02:53 2015 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C4ECE11B for ; Tue, 24 Feb 2015 21:02:53 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id AA773801 for ; Tue, 24 Feb 2015 21:02:53 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id t1OL2rbH038259 for ; Tue, 24 Feb 2015 21:02:53 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 188899] [cas] cas ethernet driver seems to have issues with some multiport card and mother board combinations Date: Tue, 24 Feb 2015 21:02:53 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.0-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: clif@eugeneweb.com X-Bugzilla-Status: In Progress X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Feb 2015 21:02:53 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=188899 --- Comment #3 from clif@eugeneweb.com --- Hi Gavin, Yes its Sun P/N: 270-6738-07 and according to this Ebay add: http://www.ebay.com.au/itm/LOT-2-Sun-PCI-X-Quad-Port-GB-Ethernet-Adapter-QGEXPCI-501-6738-10-FREE-US-S-H-/151235315746 It's Sun FRU is: 501-6738-10, which is the number most people use for it but that number does not appear on the card itself. There is a sticker on it that reads 501673800XXXX. Other descriptors are: Qgexpci Quad Port Gigabit PCI / PCI-X. I suppose their is a small possibility that these are actually different cards, but the pictures seem to be identical. There is a great pfsense review about the 501-6738-10 here: http://www.glitchwrks.com/2012/08/03/Quad-Port-PCI-Ethernet-Roundup/ and here are some links on Ebay: http://www.ebay.com/sch/i.html?_from=R40&_sacat=0&_nkw=501-6738-10&_sop=15 http://www.ebay.com/itm/Sun-Microsystems-X4445A-501-6738-10-Quad-GigaSwift-PCI-X-Ethernet-Card-/371261403254?pt=LH_DefaultDomain_0&hash=item5670e77076 http://www.ebay.com/itm/like/331482697268?lpid=82&chn=ps etc... Thanks for looking into it! Clif -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Tue Feb 24 23:00:52 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9D8DCAF4 for ; Tue, 24 Feb 2015 23:00:52 +0000 (UTC) Received: from mail-pa0-x235.google.com (mail-pa0-x235.google.com [IPv6:2607:f8b0:400e:c03::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 694CA89F for ; Tue, 24 Feb 2015 23:00:52 +0000 (UTC) Received: by pabkq14 with SMTP id kq14so224940pab.3 for ; Tue, 24 Feb 2015 15:00:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject :content-type:content-transfer-encoding; bh=+GmgUPHi3rXib2RgCCg3Ay0sbik4lF4ljQ4MYDqNCN8=; b=m9ZqdsC1DxlUEr1cXpA/9UBlZWd50tNaIwq/PK99h0/9pKQOKSfDLw8fEuWZuSjxjN IGglT84V34BSPB592CgEE8spmLOp//+OIUPxiOxwRWz2hlSctB4PE49J1kCHIp9PW3JT F8UgsVtp+hrMuIPyb7avcJmZk1gU+pDVPi1fT0653MkTnVcO7vhSU9cW771XoY+Mv7k7 d6gJHuV0Tc0AKbtn0bYbzzI8WV7J0iFCZP+j2LiNb9iUUxd0dcm17bDdo3TqFWJBi5vc H7N0OJtQOtpCIfm2VihPzE8Mop0tXyW0a4yGz7xCBMrz+C+uhHvk64XxmCnEf0uP7YDg +Mnw== X-Received: by 10.66.235.36 with SMTP id uj4mr104316pac.123.1424818852087; Tue, 24 Feb 2015 15:00:52 -0800 (PST) Received: from [10.192.166.0] (stargate.chelsio.com. [67.207.112.58]) by mx.google.com with ESMTPSA id om9sm39202851pbb.34.2015.02.24.15.00.50 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 24 Feb 2015 15:00:51 -0800 (PST) Message-ID: <54ED02A1.7030202@gmail.com> Date: Tue, 24 Feb 2015 15:00:49 -0800 From: Navdeep Parhar User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 MIME-Version: 1.0 To: "freebsd-net@freebsd.org" Subject: netstat output on a recent head Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Feb 2015 23:00:52 -0000 I see a lot of literal "%s" in netstat's output on head. This on a freshly built system from today.. # netstat -hdw 1 input (Total) output packets errs idrops bytes packets errs bytes colls drops %10s 17 %5s 0 %5s 0 %10s 1.3K %10s 6 %5s 0 %10s 997 %5s 0 %5s 0 %10s 16 %5s 0 %5s 0 %10s 1.1K %10s 5 %5s 0 %10s 848 %5s 0 %5s 0 %10s 14 %5s 0 %5s 0 %10s 902 %10s 4 %5s 0 %10s 629 %5s 0 %5s 0 %10s 18 %5s 0 %5s 0 %10s 1.4K %10s 6 %5s 0 %10s 883 %5s 0 %5s 0 %10s 19 %5s 0 %5s 0 %10s 1.7K %10s 6 %5s 0 %10s 883 %5s 0 %5s 0 %10s 38 %5s 0 %5s 0 %10s 3.2K %10s 11 %5s 0 %10s 1.5K %5s 0 %5s 0 %10s 18 %5s 0 %5s 0 %10s 1.4K %10s 5 %5s 0 %10s 794 %5s 0 %5s 0 %10s 14 %5s 0 %5s 0 %10s 1.2K %10s 5 %5s 0 %10s 794 %5s 0 %5s 0 From owner-freebsd-net@FreeBSD.ORG Tue Feb 24 23:26:05 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7D4BD362; Tue, 24 Feb 2015 23:26:05 +0000 (UTC) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id EDD02B9C; Tue, 24 Feb 2015 23:26:04 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2AeBgAbCO1U/95baINbg1haBIMEwBIKhSdJAoFkAQEBAQEBbw2EDwEBAQMBAQEBIAQnIAsFFhgCAg0ZAikBCSYGCAcEARwEiAYIDbtvmRcBAQEHAQEBAQEBARuBIYlyhB0BARsBMweCLTsSgTEFik6IbYIWgTCDOjmFMIw3IoQMIDEHgQQ5fwEBAQ X-IronPort-AV: E=Sophos;i="5.09,641,1418101200"; d="scan'208";a="192707656" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 24 Feb 2015 18:25:57 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id E36A9B3FD5; Tue, 24 Feb 2015 18:25:57 -0500 (EST) Date: Tue, 24 Feb 2015 18:25:57 -0500 (EST) From: Rick Macklem To: Tim Borgeaud Message-ID: <388835013.10159778.1424820357923.JavaMail.root@uoguelph.ca> In-Reply-To: Subject: Re: NFS: kernel modules (loading/unloading) and scheduling MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [172.17.95.12] X-Mailer: Zimbra 7.2.6_GA_2926 (ZimbraWebClient - FF3.0 (Win)/7.2.6_GA_2926) Cc: freebsd-net@freebsd.org, Alexander Motin , "Kenneth D. Merry" , Mark Hills X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Feb 2015 23:26:05 -0000 Tim Borgeaud wrote: > Hi FreeBSD folks, >=20 > here at Framestore Mark Hills (cc'd) has been looking at how NFS > servers > schedule/prioritize incoming requests with a view towards a > client/user > 'fair share' of a service. >=20 > We are taking a look at trying out some simple approaches of queuing > up and > handling requests. >=20 > There are, obviously, various matters to deal with here, and several > complications. However, an initial consideration is how we might best > develop and test with the FreeBSD kernel code. Specifically, whether > we > would be able to unload and reload NFS related kernel modules, or > whether > there are any other alternatives. >=20 > It looks like there are several modules involved such as krpc, > nfssvc, > nfscommon, nfslock, which we can build outside of the kernel itself, > but > which do not all support unloading. >=20 > Not being able to reload a module does seem to present a hurdle to > development and, therefore, I'd like to know how FreeBSD developers > have > managed development of the NFS functionality and what approaches may > be > recommended. >=20 > It occurs to me that it would be possible for us to consider adding > some > ability to unload some of these modules, even if it were not suitable > for > anything other than development. Therefore, an extension of my main, > more > general, query is to ask how straightforward or fundamentally > difficult > this may be (for the NFS modules)? >=20 Well, the ones that don't allow unloading do so because they can't safely be unloaded. For most (maybe all), the problem is that there is no way for the module to know if there is an RPC in progress, so unloading them when there is traffic arriving from clients could cause a crash. If you can block all incoming NFS RPC requests (maybe just turn off the net interface(s)?), then you could probably unload/reload them without causing any disaster. (Since this hasn't been done, there may be some memory leaks and maybe a variable that needs to be re-initialized in the load.) If you grep for MOD_UNLOAD, you can probably find the code that returns an error and just comment that out. --> Then you can try and if it doesn't crash on the unload... I would have thought a server reboot wouldn't take much longer than the unload/reload and would do the same thing safely. Now, on the "big picture": - I think you'll find that the "interesting case" is when there are several RPC requests to choose from. However, the catch22 here is that, when you get an NFS server to that state, it is heavily overloaded and, as such, isn't performing at an acceptable level. Yes, there probably are short bursts of RPC traffic where the server can choose an RPC ordering, but I suspect these will usually be a burst of I/O on a single file (due to readaheads/write behinds, client side buffer cache flushing etc). --> This brings us to "file handle affinity". Before FreeBSD had shared vnode locks, the vnode lock serialized all operations on a given vnode (ie. file). If these RPCs were handed to different nfsd threads, they were all tied up doing RPCs for one file serially and weren't available for other RPCs. --> This was "solved" by assigning an nfsd thread to do Ops for a given file handle. Then shared vnodes came along and allow many ops to be done on a given file concurrently via different nfsd threads, which I'd argue is a good thing? --> Unfortunately, ken@ found that when read/wrote ops. were done on ZFS "out of sequential order", ZFS's sequential I/O heuristic would fail and decide that the I/O was random. This caused a big performance hit for ZFS. --> As such, he found that file handle affinity does help w.r.t. I/O performance for ZFS because it reduced the "out-of-sequential-order= ness" of the I/O ops. When I discussed this with mav@, he felt that ZFS's sequential I/O heuristic needed to be fixed/improved and that was where the problem should be attacked. So, I think that the benefit of file handle affinity when used with shared vnode locking is still an open question (in general, ignoring the above ZFS case). Since file handle affinity is very hard to do for NFSv4, I like the idea of fixing ZFS. I tend to think that a bias towards doing Getattr/Lookup over Read/Write may help performance (the old "shortest job first" principal), I'm not sure you'll have a big enough queue of outstanding RPCs under normal load for this to make a real difference. I don't think you want to delay doing any RPC "waiting" for a preferred RPC to arrive. Any delay like this will increase RPC response time, which is the main NFS performance issue. rick ps: NFS types are more likely to read freebsd-fs@ than freebsd-net@. pss: I hope ken and mav don't mind me adding them as cc's. > Thanks very much >=20 > -- > Tim Borgeaud > Systems Developer > =E2=80=8BFramestore > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to > "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Wed Feb 25 00:33:32 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 41E7B481; Wed, 25 Feb 2015 00:33:32 +0000 (UTC) Received: from gold.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "gold.funkthat.com", Issuer "gold.funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 1D953270; Wed, 25 Feb 2015 00:33:31 +0000 (UTC) Received: from gold.funkthat.com (localhost [127.0.0.1]) by gold.funkthat.com (8.14.5/8.14.5) with ESMTP id t1P0XV3h007114 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 24 Feb 2015 16:33:31 -0800 (PST) (envelope-from jmg@gold.funkthat.com) Received: (from jmg@localhost) by gold.funkthat.com (8.14.5/8.14.5/Submit) id t1P0XVOi007113; Tue, 24 Feb 2015 16:33:31 -0800 (PST) (envelope-from jmg) Date: Tue, 24 Feb 2015 16:33:31 -0800 From: John-Mark Gurney To: Navdeep Parhar Subject: Re: netstat output on a recent head Message-ID: <20150225003331.GU46794@funkthat.com> References: <54ED02A1.7030202@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <54ED02A1.7030202@gmail.com> X-Operating-System: FreeBSD 9.1-PRERELEASE amd64 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? User-Agent: Mutt/1.5.21 (2010-09-15) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (gold.funkthat.com [127.0.0.1]); Tue, 24 Feb 2015 16:33:31 -0800 (PST) Cc: "freebsd-net@freebsd.org" , marcel@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2015 00:33:32 -0000 Navdeep Parhar wrote this message on Tue, Feb 24, 2015 at 15:00 -0800: > I see a lot of literal "%s" in netstat's output on head. This on a > freshly built system from today.. Might want to revert 279122... Looks like marcel didn't verify his changes complete... Maybe a good time for a unit test? Do people realize that we have gcov in the tree? For changes like this, seems like it'd be nice and easy to verify that all of the changes got coverage... -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-net@FreeBSD.ORG Wed Feb 25 02:45:04 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 94E9B2FE; Wed, 25 Feb 2015 02:45:04 +0000 (UTC) Received: from hergotha.csail.mit.edu (wollman-1-pt.tunnel.tserv4.nyc4.ipv6.he.net [IPv6:2001:470:1f06:ccb::2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3777826E; Wed, 25 Feb 2015 02:45:03 +0000 (UTC) Received: from hergotha.csail.mit.edu (localhost [127.0.0.1]) by hergotha.csail.mit.edu (8.14.9/8.14.9) with ESMTP id t1P2iukq094349; Tue, 24 Feb 2015 21:44:59 -0500 (EST) (envelope-from wollman@hergotha.csail.mit.edu) Received: (from wollman@localhost) by hergotha.csail.mit.edu (8.14.9/8.14.4/Submit) id t1P2iu6N094346; Tue, 24 Feb 2015 21:44:56 -0500 (EST) (envelope-from wollman) Date: Tue, 24 Feb 2015 21:44:56 -0500 (EST) From: Garrett Wollman Message-Id: <201502250244.t1P2iu6N094346@hergotha.csail.mit.edu> To: rmacklem@uoguelph.ca Subject: Re: NFS: kernel modules (loading/unloading) and scheduling References: <388835013.10159778.1424820357923.JavaMail.root@uoguelph.ca> Organization: none X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (hergotha.csail.mit.edu [127.0.0.1]); Tue, 24 Feb 2015 21:44:59 -0500 (EST) X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=disabled version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on hergotha.csail.mit.edu Cc: freebsd-fs@freebsd.org, freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2015 02:45:04 -0000 In article <388835013.10159778.1424820357923.JavaMail.root@uoguelph.ca>, rmacklem@uoguelph.ca writes: >I tend to think that a bias towards doing Getattr/Lookup over Read/Write >may help performance (the old "shortest job first" principal), I'm not >sure you'll have a big enough queue of outstanding RPCs under normal load >for this to make a real difference. I don't think this is a particularly relevant condition here. There are lots of ways RPCs can pile up where you really need to do better work-sharing than the current implementation does. One example is a client that issues lots of concurrent reads (e.g., a compute node running dozens of parallel jobs). Two such systems on gigabit NICs can easily issue large reads fast enough to cause 64 nfsd service threads to blocked while waiting for the socket send buffer to drain. Meanwhile, the file server is completely idle, but unable to respond to incoming requests, and the other users get angry. Rather than assigning new threads to requests from the slow clients, it would be better to let the requests sit until the send buffer drains, and process other clients' requests instead of letting the resources get monopolized by a single user. Lest you think this is purely hypothetical: we actually experienced this problem today, and I verified with "procstat -kk" that all of the nfsd threads were in fact blocked waiting for send buffer space to open up. I was able to restore service immediately by increasing the number of nfsd threads, but I'm unsure to what extent I can do this without breaking other things or hitting other bottlenecks.[1] So I have a user asking me why I haven't enable fair-share scheduling for NFS, and I'm going to have to tell him the answer is "no such thing". -GAWollman [1] What would the right number actually be? We could potentially have many thousands of threads in a compute cluster all operating simultaneously on the same filesystem, well within the I/O capacity of the server, and we'd really like to degrade gracefully rather than falling over when a single slow client soaks up all of the nfsd worker threads. From owner-freebsd-net@FreeBSD.ORG Wed Feb 25 08:01:06 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3BFF050F for ; Wed, 25 Feb 2015 08:01:06 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [IPv6:2001:4f8:3:ffe0:406a:0:50:2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 11D2CBAB for ; Wed, 25 Feb 2015 08:01:06 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [127.0.1.5]) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9) with ESMTP id t1P815qh030021 for ; Wed, 25 Feb 2015 08:01:05 GMT (envelope-from root@phabric-backend.isc.freebsd.org) Received: (from root@localhost) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9/Submit) id t1P815Pl030020; Wed, 25 Feb 2015 08:01:05 GMT (envelope-from root) Date: Wed, 25 Feb 2015 08:01:05 +0000 To: freebsd-net@freebsd.org From: "arybchik (Andrew Rybchenko)" Subject: [Differential] [Commented On] D1697: sfxge: Expect required init_state on data path and in periodic calls Message-ID: X-Priority: 3 Thread-Topic: D1697: sfxge: Expect required init_state on data path and in periodic calls X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: In-Reply-To: References: Thread-Index: MTVmZWRhZmFiZTk2YzBiOTY0NjE3OTVjNDBkIFTtgUE= X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2015 08:01:06 -0000 arybchik added a comment. I did measurements when implemented the patch and just retested once again using pmcstat instruction and BR_MISP_RETIRED.ALL_BRANCHES counters. With the patch applied the number of instruction events is 1% less and number of mispredicted branch events is 5% less. REVISION DETAIL https://reviews.freebsd.org/D1697 To: arybchik, gnn Cc: freebsd-net From owner-freebsd-net@FreeBSD.ORG Wed Feb 25 10:31:02 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 88F8EBAC for ; Wed, 25 Feb 2015 10:31:02 +0000 (UTC) Received: from sola.nimnet.asn.au (paqi.nimnet.asn.au [115.70.110.159]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id F1347F1C for ; Wed, 25 Feb 2015 10:31:01 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by sola.nimnet.asn.au (8.14.2/8.14.2) with ESMTP id t1PAUoTD045271 for ; Wed, 25 Feb 2015 21:30:50 +1100 (EST) (envelope-from smithi@nimnet.asn.au) Date: Wed, 25 Feb 2015 21:30:49 +1100 (EST) From: Ian Smith To: freebsd-net@freebsd.org Subject: What is this? Message-ID: <20150225211159.U38620@sola.nimnet.asn.au> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2015 10:31:02 -0000 This snippet is from an old linux 2.4 router/firewall/proxy box, usually clockwork. Clipped this while monitoring one night, saved it, forgot, but still find it curious and haven't seen anything similar before or since. 31.13.70.1 & 173.252.102.24 are facebook, our guy 192.168.9.21 25/9/2014 what? rpc? no rpc here even internally. .21 is a win7 box. 22:34:15.753436 IP 31.13.70.1.443 > 192.168.9.21.3721: . 21784:23236(1452) ack 15573 win 65340 22:34:15.753560 IP 31.13.70.1.443 > 192.168.9.21.3721: P 23236:23661(425) ack 15573 win 65340 22:34:15.754017 IP 192.168.9.21.3721 > 31.13.70.1.443: . ack 23661 win 65535 22:34:15.828235 IP 173.252.102.24.3660741704 > 192.168.9.21.2049: 735 proc-3090265999 22:34:15.837027 IP 192.168.9.21.2049 > 173.252.102.24.3355443200: reply Unknown rpc response code=239244857 1452 22:34:15.837031 IP 192.168.9.21.2049 > 173.252.102.24.1494367229: reply Unknown rpc response code=3295742795 33 22:34:15.875408 IP 31.13.70.1.443 > 192.168.9.21.3721: . 23661:25113(1452) ack 15573 win 65340 22:34:15.875552 IP 31.13.70.1.443 > 192.168.9.21.3721: P 25113:25677(564) ack 15573 win 65340 22:34:15.875976 IP 192.168.9.21.3721 > 31.13.70.1.443: . ack 25677 win 65535 22:34:16.114979 IP 173.252.102.24.443 > 192.168.9.21.2049: . ack 3841 win 64670 22:34:16.116361 IP 173.252.102.24.443 > 192.168.9.21.2049: . ack 3874 win 64670 22:34:16.117679 IP 173.252.102.24.4046617672 > 192.168.9.21.2049: 758 proc-685943137 22:34:16.124011 IP 192.168.9.21.2049 > 173.252.102.24.2483027968: reply Unknown rpc response code=255805058 1177 22:34:16.400004 IP 173.252.102.24.443 > 192.168.9.21.2049: . ack 5051 win 64670 22:34:20.928488 IP 173.252.102.24.2100460616 > 192.168.9.21.2049: 1410 proc-3156600121 22:34:20.935755 IP 192.168.9.21.2049 > 173.252.102.24.2483027968: reply Unknown rpc response code=269780798 1177 22:34:21.211544 IP 173.252.102.24.443 > 192.168.9.21.2049: . ack 6228 win 64670 Kick me downstairs if it's just some old linux thing, especially the 2-3 giga(what?) port numbers, but otherwise, what is this about? cheers, Ian From owner-freebsd-net@FreeBSD.ORG Wed Feb 25 13:13:23 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9D8B275A for ; Wed, 25 Feb 2015 13:13:23 +0000 (UTC) Received: from mail-wg0-f54.google.com (mail-wg0-f54.google.com [74.125.82.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2FC7C65B for ; Wed, 25 Feb 2015 13:13:21 +0000 (UTC) Received: by wghb13 with SMTP id b13so3582775wgh.0 for ; Wed, 25 Feb 2015 05:13:20 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type; bh=1Xo1tN1ljy6yh1CaW6EUtSDVe8uk1trCfOJYU3DFUtw=; b=SqZ3XJWP8PyYJmbKqC60oDRVm6OpsNE+yeJF5cb0opCh/6uPyc+xVayX3i1To1xvpV O+NNYtyh5leSgweoz+sezcpu91WQbsI/ftNb5f2y8wTRxQLSHdTfyiQ9JeOZ+Lsx49+8 UqrxkNkCenzZ7xnWZEWQ3Qlf6u0wgX5uovUCpptgMQFKeE/V25WXKCIlYGJUTw6V+vHp cqvWuGJwmtzYnBMrq/ihbenufxjs+qetzc0CZ2orObACLIuSq9AW8+AXpHW4eG5InerI MGeB6MwGwGYjwPW2NDRjFJajFXJkyMC3G9gS9r/DrEatrF1cxSmXIhGnyhPx1w7K4LO7 B2PA== X-Gm-Message-State: ALoCoQm75r+AQPDkgi7KwH7gpW0Y+10pkruQ07k/oh0QH1H8QwH+qvSEcXI7I7iS0ICzFFkE5Tkr X-Received: by 10.180.72.98 with SMTP id c2mr40341637wiv.87.1424869999061; Wed, 25 Feb 2015 05:13:19 -0800 (PST) MIME-Version: 1.0 Received: by 10.27.143.19 with HTTP; Wed, 25 Feb 2015 05:12:58 -0800 (PST) In-Reply-To: <201502250244.t1P2iu6N094346@hergotha.csail.mit.edu> References: <388835013.10159778.1424820357923.JavaMail.root@uoguelph.ca> <201502250244.t1P2iu6N094346@hergotha.csail.mit.edu> From: Tim Borgeaud Date: Wed, 25 Feb 2015 13:12:58 +0000 Message-ID: Subject: Re: NFS: kernel modules (loading/unloading) and scheduling To: Garrett Wollman Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: freebsd-fs@freebsd.org, freebsd-net@freebsd.org, rmacklem@uoguelph.ca, Mark Hills X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2015 13:13:23 -0000 Hi Rick, Garret & others, Thanks for the replies and useful info. I may take a look at enabling some kernel module reloading, but, if the usual approach is rebooting, I expect that this won't be an issue. Regarding the NFS functionality itself, I can give a bit more of an overview of our (Framestore) NFS use and what NFS server functionality we are considering. We have NFS storage systems accessed by both an HPC cluster and also multiple desktop users. Like many organizations, we want to have, and get the most from, high performance NFS storage, in terms of both total IO and low latency. One of the demands that we face is to allow users, as nodes of the cluster or more interactive desktops, to be provided with a reasonable level of service from heavily loaded systems. We do want to allow "users" to apply high load, but would like to avoid such requests from starving others. We are considering scheduling of NFS RPCs such that both client transports (xprt) and users themselves are given a fair share, including where a single user is using multiple transports. At the same time, we don't want hurt performance by loss of file handle affinity or similar (though it may be quite nice to remove responsibility from the RPC/NFS layer). We've already prototyped a 'fair' schedule when servicing the RPC requests, which appears to be an improvement. But, as Garrett pointed out, we may have moved onto bottlenecks in sending replies and, possibly, reading requests. It may be that, with such cases as slow clients, overall performance could also be improved. -- Tim Borgeaud Systems Developer On 25 February 2015 at 02:44, Garrett Wollman < wollman@hergotha.csail.mit.edu> wrote: > In article > <388835013.10159778.1424820357923.JavaMail.root@uoguelph.ca>, > rmacklem@uoguelph.ca writes: > > >I tend to think that a bias towards doing Getattr/Lookup over Read/Write > >may help performance (the old "shortest job first" principal), I'm not > >sure you'll have a big enough queue of outstanding RPCs under normal load > >for this to make a real difference. > > I don't think this is a particularly relevant condition here. There > are lots of ways RPCs can pile up where you really need to do better > work-sharing than the current implementation does. One example is a > client that issues lots of concurrent reads (e.g., a compute node > running dozens of parallel jobs). Two such systems on gigabit NICs > can easily issue large reads fast enough to cause 64 nfsd service > threads to blocked while waiting for the socket send buffer to drain. > Meanwhile, the file server is completely idle, but unable to respond > to incoming requests, and the other users get angry. Rather than > assigning new threads to requests from the slow clients, it would be > better to let the requests sit until the send buffer drains, and > process other clients' requests instead of letting the resources get > monopolized by a single user. > > Lest you think this is purely hypothetical: we actually experienced > this problem today, and I verified with "procstat -kk" that all of the > nfsd threads were in fact blocked waiting for send buffer space to > open up. I was able to restore service immediately by increasing the > number of nfsd threads, but I'm unsure to what extent I can do this > without breaking other things or hitting other bottlenecks.[1] So I > have a user asking me why I haven't enable fair-share scheduling for > NFS, and I'm going to have to tell him the answer is "no such thing". > > -GAWollman > > [1] What would the right number actually be? We could potentially > have many thousands of threads in a compute cluster all operating > simultaneously on the same filesystem, well within the I/O capacity of > the server, and we'd really like to degrade gracefully rather than > falling over when a single slow client soaks up all of the nfsd worker > threads. > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Wed Feb 25 14:59:21 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CDCA88BB for ; Wed, 25 Feb 2015 14:59:21 +0000 (UTC) Received: from mail.in-addr.com (mail.in-addr.com [IPv6:2a01:4f8:191:61e8::2525:2525]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 909D5225 for ; Wed, 25 Feb 2015 14:59:21 +0000 (UTC) Received: from gjp by mail.in-addr.com with local (Exim 4.85 (FreeBSD)) (envelope-from ) id 1YQdQk-000JGv-GN; Wed, 25 Feb 2015 14:59:18 +0000 Date: Wed, 25 Feb 2015 14:59:18 +0000 From: Gary Palmer To: Ian Smith Subject: Re: What is this? Message-ID: <20150225145918.GD29176@in-addr.com> References: <20150225211159.U38620@sola.nimnet.asn.au> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20150225211159.U38620@sola.nimnet.asn.au> X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: gpalmer@freebsd.org X-SA-Exim-Scanned: No (on mail.in-addr.com); SAEximRunCond expanded to false Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2015 14:59:21 -0000 On Wed, Feb 25, 2015 at 09:30:49PM +1100, Ian Smith wrote: > This snippet is from an old linux 2.4 router/firewall/proxy box, usually > clockwork. Clipped this while monitoring one night, saved it, forgot, > but still find it curious and haven't seen anything similar before or > since. 31.13.70.1 & 173.252.102.24 are facebook, our guy 192.168.9.21 > > 25/9/2014 what? rpc? no rpc here even internally. .21 is a win7 box. > > 22:34:15.753436 IP 31.13.70.1.443 > 192.168.9.21.3721: . 21784:23236(1452) ack 15573 win 65340 > 22:34:15.753560 IP 31.13.70.1.443 > 192.168.9.21.3721: P 23236:23661(425) ack 15573 win 65340 > 22:34:15.754017 IP 192.168.9.21.3721 > 31.13.70.1.443: . ack 23661 win 65535 > 22:34:15.828235 IP 173.252.102.24.3660741704 > 192.168.9.21.2049: 735 proc-3090265999 > 22:34:15.837027 IP 192.168.9.21.2049 > 173.252.102.24.3355443200: reply Unknown rpc response code=239244857 1452 > 22:34:15.837031 IP 192.168.9.21.2049 > 173.252.102.24.1494367229: reply Unknown rpc response code=3295742795 33 > 22:34:15.875408 IP 31.13.70.1.443 > 192.168.9.21.3721: . 23661:25113(1452) ack 15573 win 65340 > 22:34:15.875552 IP 31.13.70.1.443 > 192.168.9.21.3721: P 25113:25677(564) ack 15573 win 65340 > 22:34:15.875976 IP 192.168.9.21.3721 > 31.13.70.1.443: . ack 25677 win 65535 > 22:34:16.114979 IP 173.252.102.24.443 > 192.168.9.21.2049: . ack 3841 win 64670 > 22:34:16.116361 IP 173.252.102.24.443 > 192.168.9.21.2049: . ack 3874 win 64670 > 22:34:16.117679 IP 173.252.102.24.4046617672 > 192.168.9.21.2049: 758 proc-685943137 > 22:34:16.124011 IP 192.168.9.21.2049 > 173.252.102.24.2483027968: reply Unknown rpc response code=255805058 1177 > 22:34:16.400004 IP 173.252.102.24.443 > 192.168.9.21.2049: . ack 5051 win 64670 > 22:34:20.928488 IP 173.252.102.24.2100460616 > 192.168.9.21.2049: 1410 proc-3156600121 > 22:34:20.935755 IP 192.168.9.21.2049 > 173.252.102.24.2483027968: reply Unknown rpc response code=269780798 1177 > 22:34:21.211544 IP 173.252.102.24.443 > 192.168.9.21.2049: . ack 6228 win 64670 > > Kick me downstairs if it's just some old linux thing, especially the 2-3 > giga(what?) port numbers, but otherwise, what is this about? Supposition: whatever you are using on Linux is seeing the 2049 port number and trying to decode the packet as NFS traffic even though it's not, and the port number isn't a port number at all but a NFS handle or something, but it isn't really, it's just some data from the packet contents that is in the location where the handle would be if the packet were truly NFS. Regards, Gary From owner-freebsd-net@FreeBSD.ORG Wed Feb 25 15:00:13 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 66EA394E; Wed, 25 Feb 2015 15:00:13 +0000 (UTC) Received: from mail-ie0-x235.google.com (mail-ie0-x235.google.com [IPv6:2607:f8b0:4001:c03::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 28A8C231; Wed, 25 Feb 2015 15:00:13 +0000 (UTC) Received: by iecrp18 with SMTP id rp18so5570957iec.1; Wed, 25 Feb 2015 07:00:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=F/sSwcQtUIUg4Fh13WAbgJXyeZphU0papjjQhL2Hrbs=; b=sGRwFnE+NXEbjpg3LdyHMVBl57P0vYoJZzyVS9MVf3avF4ME0SosaBhEbrc20VGn3j RMU1tCDldvsmMsJUoRcRhlKHZL32YfwyPegbYKQd99PmznBihdg/NR7ry7uO6ZHL6e32 2qxgTM38EMV7zGqzf8KxYtj9/7qWczoH7d2mC6grzopb96bwG+pjvv7+m3nHTygiuGtE FkUY/QP7vAiDKKnsSmZ0dwKimtQw+l/YiejPdVYtpI4cc78QwEyc5J4KYCePNtzP6jrX 2wOQE/Z3vK4qWuF9iSCb6a7NF3Xon6Gkz8LP+9C61+B3fhEFFiZcfkEGxYk2VMk+UOiU udxA== X-Received: by 10.107.7.93 with SMTP id 90mr4957372ioh.69.1424876412598; Wed, 25 Feb 2015 07:00:12 -0800 (PST) MIME-Version: 1.0 Received: by 10.64.107.138 with HTTP; Wed, 25 Feb 2015 06:59:52 -0800 (PST) In-Reply-To: <20150217193132.65fe16bb.ohartman@zedat.fu-berlin.de> References: <4E4BF84E-F6FD-4D25-8B2C-2B443894697B@gmail.com> <20150217193132.65fe16bb.ohartman@zedat.fu-berlin.de> From: Luca Pizzamiglio Date: Wed, 25 Feb 2015 15:59:52 +0100 Message-ID: Subject: Re: pcie Realtek 8168G issues (re driver) To: "O. Hartmann" Content-Type: text/plain; charset=UTF-8 Cc: Ben Perrault , freebsd-net@freebsd.org, FreeBSD Current , "freebsd-hardware@freebsd.org" , FreeBSD Hackers X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2015 15:00:13 -0000 Hi, thanks you all for the replies. Unfortunately, the network chip is still not working and I updated the PR (https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=197535) with the last tests. It seems that received packets are not transferred to mbuf or they are transferred, but later, after the mbuf is already freed; moreover, the ring entries are written without looping, overwriting and messing up the whole kernel memory. It looks like a DMA issues, but Apparently it seems a hardware error, but using a Linux distro, it works :( Has someone maybe any other ideas? In the meanwhile I'll get another board with the same chip :O Best regards, Luca On Tue, Feb 17, 2015 at 7:31 PM, O. Hartmann wrote: > Am Tue, 17 Feb 2015 18:32:22 +0100 > Luca Pizzamiglio schrieb: > >> Hi Ben, >> thanks for the tip! tso was already disabled. >> I tried anyway and unfortunately it crashes as before. >> >> I filled a bug report >> (https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=197535) and marius@ >> is giving me a big help on it. >> >> Best regards, >> Luca >> >> On Fri, Feb 13, 2015 at 7:34 PM, Ben Perrault wrote: >> > Luca, >> > >> > I've had the same issue with this interface on both PCIe boards and embedded in a >> > handful of Lenovo products. The one, fairly ugly workaround I've found that makes it >> > work well enough is disable tso ( i.e. ifconfig re0 down && ifconfig re0 -tso && >> > ifconfig re0 up ). This also seems to stop the panics under current. >> > >> > I'm not sure it will work for you - but it has on everyone of those interfaces I've >> > dealt with. >> > >> > Good luck, >> > -bp >> > >> >> On Feb 13, 2015, at 8:06 AM, Luca Pizzamiglio wrote: >> >> >> >> Hi, I'm Luca, >> >> >> >> I've some issues using a PCIe Realtek Ethernet board: >> >> re0@pci0:3:0:0: class=0x020000 card=0x012310ec chip=0x816810ec rev=0x0c hdr=0x00 >> >> vendor = 'Realtek Semiconductor Co., Ltd.' >> >> device = 'RTL8111/8168B PCI Express Gigabit Ethernet controller' >> >> class = network >> >> subclass = ethernet >> >> bar [10] = type I/O Port, range 32, base 0x1000, size 256, enabled >> >> bar [18] = type Memory, range 64, base 0x90500000, size 4096, enabled >> >> bar [20] = type Prefetchable Memory, range 64, base 0x90400000, >> >> size 16384, enabled >> >> cap 01[40] = powerspec 3 supports D0 D1 D2 D3 current D0 >> >> cap 05[50] = MSI supports 1 message, 64 bit >> >> cap 10[70] = PCI-Express 2 endpoint IRQ 1 max data 128(128) link x1(x1) >> >> speed 2.5(2.5) ASPM disabled(L0s/L1) >> >> cap 11[b0] = MSI-X supports 4 messages >> >> Table in map 0x20[0x0], PBA in map 0x20[0x800] >> >> cap 03[d0] = VPD >> >> ecap 0001[100] = AER 1 0 fatal 0 non-fatal 0 corrected >> >> ecap 0002[140] = VC 1 max VC0 >> >> ecap 0003[160] = Serial 1 01000000684ce000 >> >> ecap 0018[170] = LTR 1 >> >> >> >> Rx and Tx don't work. After some minutes the interface is activated I >> >> get kernel panic. >> >> I've already tried to disable MSIx and MSI. >> >> It seems a DMA problem, rx fill the 256 descriptors and the nothing >> >> else until the panic. netstat -s shows now new packets. >> >> >> >> I filled a bug report with more infos: >> >> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=197535 >> >> >> >> could someone kindly pointing some ideas? >> >> >> >> Best regards, >> >> Luca >> >> _______________________________________________ >> >> freebsd-current@freebsd.org mailing list >> >> http://lists.freebsd.org/mailman/listinfo/freebsd-current >> >> To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" >> _______________________________________________ >> freebsd-current@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-current >> To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" > > In September 2014 I filed allready a bug acoording to strange behaviour with a Lenovo > ThinkPad E540 with a Realtek chip: > > > Bug 193743 - RTL8111/8168B PCI Express Gigabit Ethernet > controller: doesn't work properly, problems getting UP automatically From owner-freebsd-net@FreeBSD.ORG Wed Feb 25 16:44:53 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2038C4DC; Wed, 25 Feb 2015 16:44:53 +0000 (UTC) Received: from sola.nimnet.asn.au (paqi.nimnet.asn.au [115.70.110.159]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 77442FF1; Wed, 25 Feb 2015 16:44:52 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by sola.nimnet.asn.au (8.14.2/8.14.2) with ESMTP id t1PGilQv057965; Thu, 26 Feb 2015 03:44:47 +1100 (EST) (envelope-from smithi@nimnet.asn.au) Date: Thu, 26 Feb 2015 03:44:46 +1100 (EST) From: Ian Smith To: Gary Palmer Subject: Re: What is this? In-Reply-To: <20150225145918.GD29176@in-addr.com> Message-ID: <20150226032414.F38620@sola.nimnet.asn.au> References: <20150225211159.U38620@sola.nimnet.asn.au> <20150225145918.GD29176@in-addr.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2015 16:44:53 -0000 On Wed, 25 Feb 2015 14:59:18 +0000, Gary Palmer wrote: > On Wed, Feb 25, 2015 at 09:30:49PM +1100, Ian Smith wrote: > > This snippet is from an old linux 2.4 router/firewall/proxy box, usually > > clockwork. Clipped this while monitoring one night, saved it, forgot, > > but still find it curious and haven't seen anything similar before or > > since. 31.13.70.1 & 173.252.102.24 are facebook, our guy 192.168.9.21 > > > > 25/9/2014 what? rpc? no rpc here even internally. .21 is a win7 box. > > > > 22:34:15.753436 IP 31.13.70.1.443 > 192.168.9.21.3721: . 21784:23236(1452) ack 15573 win 65340 > > 22:34:15.753560 IP 31.13.70.1.443 > 192.168.9.21.3721: P 23236:23661(425) ack 15573 win 65340 > > 22:34:15.754017 IP 192.168.9.21.3721 > 31.13.70.1.443: . ack 23661 win 65535 > > 22:34:15.828235 IP 173.252.102.24.3660741704 > 192.168.9.21.2049: 735 proc-3090265999 > > 22:34:15.837027 IP 192.168.9.21.2049 > 173.252.102.24.3355443200: reply Unknown rpc response code=239244857 1452 > > 22:34:15.837031 IP 192.168.9.21.2049 > 173.252.102.24.1494367229: reply Unknown rpc response code=3295742795 33 > > 22:34:15.875408 IP 31.13.70.1.443 > 192.168.9.21.3721: . 23661:25113(1452) ack 15573 win 65340 > > 22:34:15.875552 IP 31.13.70.1.443 > 192.168.9.21.3721: P 25113:25677(564) ack 15573 win 65340 > > 22:34:15.875976 IP 192.168.9.21.3721 > 31.13.70.1.443: . ack 25677 win 65535 > > 22:34:16.114979 IP 173.252.102.24.443 > 192.168.9.21.2049: . ack 3841 win 64670 > > 22:34:16.116361 IP 173.252.102.24.443 > 192.168.9.21.2049: . ack 3874 win 64670 > > 22:34:16.117679 IP 173.252.102.24.4046617672 > 192.168.9.21.2049: 758 proc-685943137 > > 22:34:16.124011 IP 192.168.9.21.2049 > 173.252.102.24.2483027968: reply Unknown rpc response code=255805058 1177 > > 22:34:16.400004 IP 173.252.102.24.443 > 192.168.9.21.2049: . ack 5051 win 64670 > > 22:34:20.928488 IP 173.252.102.24.2100460616 > 192.168.9.21.2049: 1410 proc-3156600121 > > 22:34:20.935755 IP 192.168.9.21.2049 > 173.252.102.24.2483027968: reply Unknown rpc response code=269780798 1177 > > 22:34:21.211544 IP 173.252.102.24.443 > 192.168.9.21.2049: . ack 6228 win 64670 > > > > Kick me downstairs if it's just some old linux thing, especially the 2-3 > > giga(what?) port numbers, but otherwise, what is this about? > > Supposition: whatever you are using on Linux is seeing the 2049 port > number and trying to decode the packet as NFS traffic even though > it's not, and the port number isn't a port number at all but a NFS handle > or something, but it isn't really, it's just some data from the packet > contents that is in the location where the handle would be if the packet > were truly NFS. Ah, right, of course I should have checked /etc/services first .. nfsd 2049/sctp nfs # NFS server daemon nfsd 2049/tcp nfs # NFS server daemon nfsd 2049/udp nfs # NFS server daemon All that's running on linux is tcpdump -pn on the router - the traffic is with the win7 box - so I guess linux' tcpdump was (mis)interpreting. Seems that traffic was getting through anyway, but I didn't clip much. Thanks, Ian From owner-freebsd-net@FreeBSD.ORG Wed Feb 25 17:08:35 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6CC2F29D for ; Wed, 25 Feb 2015 17:08:35 +0000 (UTC) Received: from mail.xcllnt.net (mail.xcllnt.net [50.0.150.214]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 287BB66A for ; Wed, 25 Feb 2015 17:08:34 +0000 (UTC) Received: from [10.9.1.83] (69-12-170-18.dedicated.static.sonic.net [69.12.170.18]) (authenticated bits=0) by mail.xcllnt.net (8.14.9/8.14.9) with ESMTP id t1PH8WtS030176 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 25 Feb 2015 09:08:34 -0800 (PST) (envelope-from marcel@xcllnt.net) Subject: Re: netstat output on a recent head Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2070.6\)) Content-Type: multipart/signed; boundary="Apple-Mail=_44E2DDC7-EE82-4B7C-B7CD-4C6BB9D5908B"; protocol="application/pgp-signature"; micalg=pgp-sha512 X-Pgp-Agent: GPGMail 2.5b5 From: Marcel Moolenaar In-Reply-To: Date: Wed, 25 Feb 2015 09:08:27 -0800 Message-Id: <11B05307-6402-4515-9466-532550181F7B@xcllnt.net> References: <54ED02A1.7030202@gmail.com> To: Navdeep Parhar X-Mailer: Apple Mail (2.2070.6) Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2015 17:08:35 -0000 --Apple-Mail=_44E2DDC7-EE82-4B7C-B7CD-4C6BB9D5908B Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Feb 25, 2015, at 8:42 AM, Marcel Moolenaar = wrote: >=20 >=20 >> On Feb 24, 2015, at 3:00 PM, Navdeep Parhar = wrote: >>=20 >> I see a lot of literal "%s" in netstat's output on head. This on = a freshly built system from today.. >>=20 >>=20 >> # netstat -hdw 1 >=20 > Oops. >=20 > The bug is with -h. If you remove -h you get the expected output > (except not humanized). I=E2=80=99ll fix it today. Fixed with revision 279288. Sorry for the breakage. -- Marcel Moolenaar marcel@xcllnt.net --Apple-Mail=_44E2DDC7-EE82-4B7C-B7CD-4C6BB9D5908B Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJU7gGLAAoJEIda8t8f0tjjEpcQAKedpaFHcdoonKbg6cKLTGf/ YXgTHx6OOxYOIIe0zSZpUwBAkMvJXNzU+gIG32mAszW4QKiKSv7bLU4AnEjeuCY4 8GKwMxifWLsuOSnzYdVC5vbX4H4TQiH4ofLoSilyelEoIK6AynFQ5fUsgM47lvGj rBRBz1idIZNOUVsfAkgFLecNvTKsIjoGvpEcUd94m/NE0vMF0bp3/13sD2Cc7WMx RgcfLKicqJKro5JbTVqeuj5vuGAA5Te5TyolhUg1V/pdIyphv2rdlQi+C66FFX39 ++nhTTVv1LDvCWulFmAKX5+V0PtPUloC/A5z2lbcPZY6m4ExbvLrVZMHGGML6//4 HHVnPlhx/VEYctKjyQUE/5CB0bV9TyrPrcHGPAPRz4NE30Ms1WPfHK8EeWghBcWN 52WcuWTT0rX2xC/ePna/BvGAytbMDYtcC18hOcOMsZJICHGNiMQCLfgbPZxIwGcB gFH3kPh7feBWZ2f9ApKJFOzRjeYCeLxxVhjx0OXBBvDgYV8kQyfA4unBPLZlkj8i FN+LB69ZSB8OhUSdIGVOvML2UL7d6XgsewdERlxGbcnpvKAuq+9b4htLtp4IAe0M k10DK4xwxiVehY6F/FU3gZjrAC0uu88LvSYBqS7dYkwv4Rp+YMJnndUxdmPmRKKX TD4SZ0Dq3vIxXjCAoe1J =vIm1 -----END PGP SIGNATURE----- --Apple-Mail=_44E2DDC7-EE82-4B7C-B7CD-4C6BB9D5908B-- From owner-freebsd-net@FreeBSD.ORG Wed Feb 25 17:11:36 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2F2A8461 for ; Wed, 25 Feb 2015 17:11:36 +0000 (UTC) Received: from mail.xcllnt.net (mail.xcllnt.net [50.0.150.214]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 06D73792 for ; Wed, 25 Feb 2015 17:11:35 +0000 (UTC) Received: from [10.9.1.83] (69-12-170-18.dedicated.static.sonic.net [69.12.170.18]) (authenticated bits=0) by mail.xcllnt.net (8.14.9/8.14.9) with ESMTP id t1PGgD53030048 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 25 Feb 2015 08:42:16 -0800 (PST) (envelope-from marcel@xcllnt.net) Subject: Re: netstat output on a recent head Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2070.6\)) Content-Type: multipart/signed; boundary="Apple-Mail=_01FE2B0D-D145-486D-BF18-16B2E2AF2D46"; protocol="application/pgp-signature"; micalg=pgp-sha512 X-Pgp-Agent: GPGMail 2.5b5 From: Marcel Moolenaar In-Reply-To: <54ED02A1.7030202@gmail.com> Date: Wed, 25 Feb 2015 08:42:07 -0800 Message-Id: References: <54ED02A1.7030202@gmail.com> To: Navdeep Parhar X-Mailer: Apple Mail (2.2070.6) Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2015 17:11:36 -0000 --Apple-Mail=_01FE2B0D-D145-486D-BF18-16B2E2AF2D46 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Feb 24, 2015, at 3:00 PM, Navdeep Parhar wrote: >=20 > I see a lot of literal "%s" in netstat's output on head. This on a = freshly built system from today.. >=20 >=20 > # netstat -hdw 1 Oops. The bug is with -h. If you remove -h you get the expected output (except not humanized). I=E2=80=99ll fix it today. -- Marcel Moolenaar marcel@xcllnt.net --Apple-Mail=_01FE2B0D-D145-486D-BF18-16B2E2AF2D46 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJU7ftfAAoJEIda8t8f0tjjEN4P/iHliGQ///b+P5EHi2PetSNe /fqRgumi+kRMtmHshwcy6mehJRsuUPNMY4AJVIEjNmgYnZrDIKoXxgEeAKkWFYVn sI83gsn3y+bjUdHp2JKE8nlNDgySFWh/ObDxyK72Zxr4bf+D59lP/VS2wuGXh3YZ NP+tOqjO9tdFh/w2qgTTau04T9r850VwH1BvORYc6403QEA7QeNqG8BVtufAwtAr yvoHPkurnGkW4ZiYqP7R4B6J7/omJ9aLVMod/0GdxHKlihgW0ThyKVrfNo0m2+BR sn81ftivQ/UjSvgGW/pdAFRb0sI91H8V5WQmzRS7i+5+04sKQpfRVGHD/bzci4FV HCSxuXJew6nHhDOWBQILa2yDUlhgCiPcIf918q1+5LRaqxQfaLtR3zi19DbZN+VF Y3qVwMcwhwcXeWqmaJDTvd/Uy6z1ujGJgmu2sUjdpURRfNc0VIW3yNees1DfcaYN WuMc4oDydq6rSpmrRARgDVdvyBRIpB9PVBKYpM+kVHqWlGZbozmLLJe7zwYm6G/w EfJr1Xz2MMu1imJafEdc+9OcjEsfDGruhd2D2WInJxgGsMSUP+eJdJaLCYxZQigV NF7rlGPd1nCY/31e90P7OLcRCx1WmKB0aAV5Lp7mcdppFSSguHLAsXAX31J5Hsuf uJ6V3xwhaGU4k0CPwLQA =S5a6 -----END PGP SIGNATURE----- --Apple-Mail=_01FE2B0D-D145-486D-BF18-16B2E2AF2D46-- From owner-freebsd-net@FreeBSD.ORG Wed Feb 25 18:47:48 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DE1E74B7 for ; Wed, 25 Feb 2015 18:47:48 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [IPv6:2001:4f8:3:ffe0:406a:0:50:2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BEBB3377 for ; Wed, 25 Feb 2015 18:47:48 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [127.0.1.5]) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9) with ESMTP id t1PIlmhe062587 for ; Wed, 25 Feb 2015 18:47:48 GMT (envelope-from root@phabric-backend.isc.freebsd.org) Received: (from root@localhost) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9/Submit) id t1PIlmOG062586; Wed, 25 Feb 2015 18:47:48 GMT (envelope-from root) Date: Wed, 25 Feb 2015 18:47:48 +0000 To: freebsd-net@freebsd.org From: "erj (Eric Joyner)" Subject: [Differential] [Changed Subscribers] D1965: Add extended media types to if_media.h and ifconfig Message-ID: <4f52006a04d325d7cc9e88a55757934c@localhost.localdomain> X-Priority: 3 Thread-Topic: D1965: Add extended media types to if_media.h and ifconfig X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: In-Reply-To: References: Thread-Index: YmU0NTlhN2Q2ZDc3OTdjODA5NmQwOTI1OTFkIFTuGNQ= X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2015 18:47:49 -0000 erj added a subscriber: freebsd-net. REVISION DETAIL https://reviews.freebsd.org/D1965 To: erj, gnn, jfvogel Cc: freebsd-net From owner-freebsd-net@FreeBSD.ORG Wed Feb 25 20:29:58 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 02216FF for ; Wed, 25 Feb 2015 20:29:58 +0000 (UTC) Received: from mail-lb0-x241.google.com (mail-lb0-x241.google.com [IPv6:2a00:1450:4010:c04::241]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5E1102A1 for ; Wed, 25 Feb 2015 20:29:57 +0000 (UTC) Received: by lbiw7 with SMTP id w7so1140230lbi.3 for ; Wed, 25 Feb 2015 12:29:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=Wf1Mxg5Y3PQTTQX3A2s84efepqh79OZVdHOdofzKZlA=; b=UQk5VVkprLI8SaulXgBbSqPierROMrkw7sbXoARS1kuJuqyEm90Q2DFChoVGx4K6PY s2XoxHYQfoAXolIOPrh83ouTXLyRSbvZUe8ssTavbl/XscXwUX9+yN1L69K5tSjLj+KG orLsdNvhyPGKQ/kXBqR/OjHB9xU2vvF2Bt/88Nrux8ehrhZRRl29/bbDA8a2rSGSuEZa Ic4urUJiMae/bfWbYqwXpffo0ssw+yP1Qx7yklCEWgP7Fx48H/aUbVlX2+lgoxJaUlym HxMFz61navLMRab3tVTd7jcQ6cZN9sJWtqDJd2V5a/brODhC3ncQtLOehrnG7OdKrWB2 7nFA== MIME-Version: 1.0 X-Received: by 10.153.5.33 with SMTP id cj1mr4576250lad.66.1424896195237; Wed, 25 Feb 2015 12:29:55 -0800 (PST) Received: by 10.112.136.5 with HTTP; Wed, 25 Feb 2015 12:29:55 -0800 (PST) Date: Wed, 25 Feb 2015 22:29:55 +0200 Message-ID: Subject: Cant Access Web site From: Ercan Deger To: freebsd-net@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2015 20:29:58 -0000 Dear All, I am using freebsd 7 as router, I have strange problem while accessing a site when I try to access via browser from windows pc behind freebsd kernel nat waiting and not opening site, I can access site from other places I can access via telnet [root@proxy ]# telnet 85.96.190.177 81 Trying 85.96.190.177... Connected to 85.96.190.177.static.ttnet.com.tr. Escape character is '^]'. I added second wan (DSL) and route to site from dsl link I can access also without problem tcpdump log is; 21:38:00.903848 IP (tos 0x0, ttl 51, id 0, offset 0, flags [DF], proto TCP (6), length 52) 85.96.xx.xx.81 > 213.14.xx.xx.64886: S, cksum 0x874b (correct), 2875126544:2875126544(0) ack 2564833079 win 14600 21:38:00.904910 IP (tos 0x0, ttl 51, id 0, offset 0, flags [DF], proto TCP (6), length 40) 85.96.xx.xx.81 > 213.14.xx.xx.64883: R, cksum 0xda56 (correct), 3424964916:3424964916(0) win 0 21:38:00.904954 IP (tos 0x0, ttl 51, id 0, offset 0, flags [DF], proto TCP (6), length 52) 85.96.xx.xx.81 > 213.14.xx.xx.64887: S, cksum 0x8afa (correct), 2281919230:2281919230(0) ack 4103381824 win 14600 21:38:01.207088 IP (tos 0x0, ttl 51, id 48134, offset 0, flags [DF], proto TCP (6), length 40) 85.96.xx.xx.81 > 213.14.xx.xx.64886: ., cksum 0xf175 (correct), 1:1(0) ack 354 win 3650 21:38:01.357109 IP (tos 0x0, ttl 51, id 48135, offset 0, flags [DF], proto TCP (6), length 57) 85.96.xx.xx.81 > 213.14.xx.xx.64886: P, cksum 0x3198 (correct), 1:18(17) ack 354 win 3650 21:38:01.389613 IP (tos 0x0, ttl 51, id 48136, offset 0, flags [+], proto TCP (6), length 1492) 85.96.xx.xx.81 > 213.14.xx.xx.64886: . 18:1470(1452) ack 354 win 3650 21:38:01.389701 IP (tos 0x0, ttl 51, id 48136, offset 1472, flags [none], proto TCP (6), length 28) 85.96.xx.xx > 213.14.xx.xx: tcp 21:38:01.402883 IP (tos 0x0, ttl 51, id 48137, offset 0, flags [+], proto TCP (6), length 1492) 85.96.xx.xx.81 > 213.14.xx.xx.64886: . 1478:2930(1452) ack 354 win 3650 21:38:01.403904 IP (tos 0x0, ttl 51, id 48137, offset 1472, flags [none], proto TCP (6), length 28) 85.96.xx.xx > 213.14.xx.xx: tcp 21:38:01.417175 IP (tos 0x0, ttl 51, id 48138, offset 0, flags [+], proto TCP (6), length 1492) 85.96.xx.xx.81 > 213.14.xx.xx.64886: . 2938:4390(1452) ack 354 win 3650 21:38:01.418196 IP (tos 0x0, ttl 51, id 48138, offset 1472, flags [none], proto TCP (6), length 28) 85.96.xx.xx > 213.14.xx.xx: tcp 21:38:01.431607 IP (tos 0x0, ttl 51, id 48139, offset 0, flags [+], proto TCP (6), length 1492) 85.96.xx.xx.81 > 213.14.xx.xx.64886: . 4398:5850(1452) ack 354 win 3650 21:38:01.431657 IP (tos 0x0, ttl 51, id 48139, offset 1472, flags [none], proto TCP (6), length 28) 85.96.xx.xx > 213.14.xx.xx: tcp 21:38:01.445760 IP (tos 0x0, ttl 51, id 48140, offset 0, flags [+], proto TCP (6), length 1492) 85.96.xx.xx.81 > 213.14.xx.xx.64886: . 5858:7310(1452) ack 354 win 3650 21:38:01.445811 IP (tos 0x0, ttl 51, id 48140, offset 1472, flags [none], proto TCP (6), length 28) 85.96.xx.xx > 213.14.xx.xx: tcp 21:38:01.460053 IP (tos 0x0, ttl 51, id 48141, offset 0, flags [+], proto TCP (6), length 1492) 85.96.xx.xx.81 > 213.14.xx.xx.64886: . 7318:8770(1452) ack 354 win 3650 21:38:01.460103 IP (tos 0x0, ttl 51, id 48141, offset 1472, flags [none], proto TCP (6), length 28) 85.96.xx.xx > 213.14.xx.xx: tcp 21:38:01.473314 IP (tos 0x0, ttl 51, id 48142, offset 0, flags [+], proto TCP (6), length 1492) 85.96.xx.xx.81 > 213.14.xx.xx.64886: . 8778:10230(1452) ack 354 win 3650 21:38:01.474336 IP (tos 0x0, ttl 51, id 48142, offset 1472, flags [none], proto TCP (6), length 28) 85.96.xx.xx > 213.14.xx.xx: tcp 21:38:01.480460 IP (tos 0x0, ttl 51, id 48143, offset 0, flags [DF], proto TCP (6), length 664) 85.96.xx.xx.81 > 213.14.xx.xx.64886: FP 10238:10862(624) ack 354 win 3650 21:38:02.044104 IP (tos 0x0, ttl 51, id 48144, offset 0, flags [+], proto TCP (6), length 1492) 85.96.xx.xx.81 > 213.14.xx.xx.64886: . 18:1470(1452) ack 354 win 3650 21:38:02.044215 IP (tos 0x0, ttl 51, id 48144, offset 1472, flags [none], proto TCP (6), length 28) 85.96.xx.xx > 213.14.xx.xx: tcp 21:38:03.063015 IP (tos 0x0, ttl 51, id 48145, offset 0, flags [+], proto TCP (6), length 1492) 85.96.xx.xx.81 > 213.14.xx.xx.64886: . 18:1470(1452) ack 354 win 3650 21:38:03.064036 IP (tos 0x0, ttl 51, id 48145, offset 1472, flags [none], proto TCP (6), length 28) 85.96.xx.xx > 213.14.xx.xx: tcp 21:38:05.103427 IP (tos 0x0, ttl 51, id 48146, offset 0, flags [+], proto TCP (6), length 1492) 85.96.xx.xx.81 > 213.14.xx.xx.64886: . 18:1470(1452) ack 354 win 3650 21:38:05.104448 IP (tos 0x0, ttl 51, id 48146, offset 1472, flags [none], proto TCP (6), length 28) 85.96.xx.xx > 213.14.xx.xx: tcp 21:38:09.193863 IP (tos 0x0, ttl 51, id 48147, offset 0, flags [+], proto TCP (6), length 1492) 85.96.xx.xx.81 > 213.14.xx.xx.64886: . 18:1470(1452) ack 354 win 3650 21:38:09.193915 IP (tos 0x0, ttl 51, id 48147, offset 1472, flags [none], proto TCP (6), length 28) 85.96.xx.xx > 213.14.xx.xx: tcp 21:38:17.363864 IP (tos 0x0, ttl 51, id 48148, offset 0, flags [+], proto TCP (6), length 1492) 85.96.xx.xx.81 > 213.14.xx.xx.64886: . 18:1470(1452) ack 354 win 3650 21:38:17.364885 IP (tos 0x0, ttl 51, id 48148, offset 1472, flags [none], proto TCP (6), length 28) 85.96.xx.xx > 213.14.xx.xx: tcp 21:38:20.954233 IP (tos 0x0, ttl 51, id 7656, offset 0, flags [DF], proto TCP (6), length 626) 85.96.xx.xx.81 > 213.14.xx.xx.64887: P 1:587(586) ack 2 win 3650 21:38:20.954278 IP (tos 0x0, ttl 51, id 7657, offset 0, flags [DF], proto TCP (6), length 40) 85.96.xx.xx.81 > 213.14.xx.xx.64887: F, cksum 0xf439 (correct), 587:587(0) ack 2 win 3650 what can be the problem? Best Regards, From owner-freebsd-net@FreeBSD.ORG Wed Feb 25 21:41:58 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 007172EF for ; Wed, 25 Feb 2015 21:41:57 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [IPv6:2001:4f8:3:ffe0:406a:0:50:2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CE41BD24 for ; Wed, 25 Feb 2015 21:41:57 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [127.0.1.5]) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9) with ESMTP id t1PLfvUL045962 for ; Wed, 25 Feb 2015 21:41:57 GMT (envelope-from root@phabric-backend.isc.freebsd.org) Received: (from root@localhost) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9/Submit) id t1PLfvrs045961; Wed, 25 Feb 2015 21:41:57 GMT (envelope-from root) Date: Wed, 25 Feb 2015 21:41:57 +0000 To: freebsd-net@freebsd.org From: "erj (Eric Joyner)" Subject: [Differential] [Updated] D1965: Add extended media types to if_media.h and ifconfig Message-ID: X-Priority: 3 Thread-Topic: D1965: Add extended media types to if_media.h and ifconfig X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: In-Reply-To: References: Thread-Index: YmU0NTlhN2Q2ZDc3OTdjODA5NmQwOTI1OTFkIFTuQaU= X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2015 21:41:58 -0000 erj added a reviewer: adrian. REVISION DETAIL https://reviews.freebsd.org/D1965 To: erj, gnn, jfvogel, adrian Cc: freebsd-net From owner-freebsd-net@FreeBSD.ORG Wed Feb 25 21:51:13 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E34D9D22 for ; Wed, 25 Feb 2015 21:51:13 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [IPv6:2001:4f8:3:ffe0:406a:0:50:2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C139BE6C for ; Wed, 25 Feb 2015 21:51:13 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [127.0.1.5]) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9) with ESMTP id t1PLpDPc055732 for ; Wed, 25 Feb 2015 21:51:13 GMT (envelope-from root@phabric-backend.isc.freebsd.org) Received: (from root@localhost) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9/Submit) id t1PLpDl0055731; Wed, 25 Feb 2015 21:51:13 GMT (envelope-from root) Date: Wed, 25 Feb 2015 21:51:13 +0000 To: freebsd-net@freebsd.org From: "glebius (Gleb Smirnoff)" Subject: [Differential] [Changed Subscribers] D1965: Add extended media types to if_media.h and ifconfig Message-ID: X-Priority: 3 Thread-Topic: D1965: Add extended media types to if_media.h and ifconfig X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: X-Phabricator-Cc: In-Reply-To: References: Thread-Index: YmU0NTlhN2Q2ZDc3OTdjODA5NmQwOTI1OTFkIFTuQ9E= X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2015 21:51:14 -0000 glebius added a subscriber: glebius. glebius added a comment. We can't and don't plan to preserve the driver KPI for the 11 branch. We actually plan to change it very much, in sake of keeping it stable there on. Please see on what is going on in the projects/ifnet branch. So, I'd suggest to introduce new 'struct ifmedia' with enough space, and of course put extra space in there. Give a new value to SIOCGIFMEDIA. Write a new clear code to handle it, without any extended bit tricks. For the sake of userland API, save old current 'struct ifmedia' as 'struct oifmedia', and take old value of ioctl to OSIOCIGIFMEDIA. Keep current code for old APU, only renaming the struct and ioctl value. REVISION DETAIL https://reviews.freebsd.org/D1965 To: erj, gnn, jfvogel, adrian Cc: glebius, freebsd-net From owner-freebsd-net@FreeBSD.ORG Wed Feb 25 21:52:36 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9D4FEEB9 for ; Wed, 25 Feb 2015 21:52:36 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [IPv6:2001:4f8:3:ffe0:406a:0:50:2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7C865E99 for ; Wed, 25 Feb 2015 21:52:36 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [127.0.1.5]) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9) with ESMTP id t1PLqawe057283 for ; Wed, 25 Feb 2015 21:52:36 GMT (envelope-from root@phabric-backend.isc.freebsd.org) Received: (from root@localhost) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9/Submit) id t1PLqa9V057282; Wed, 25 Feb 2015 21:52:36 GMT (envelope-from root) Date: Wed, 25 Feb 2015 21:52:36 +0000 To: freebsd-net@freebsd.org From: "glebius (Gleb Smirnoff)" Subject: [Differential] [Commented On] D1965: Add extended media types to if_media.h and ifconfig Message-ID: <24e7fc1283ca5a16b4ae40c3f5cc4a46@localhost.localdomain> X-Priority: 3 Thread-Topic: D1965: Add extended media types to if_media.h and ifconfig X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: X-Phabricator-Cc: In-Reply-To: References: Thread-Index: YmU0NTlhN2Q2ZDc3OTdjODA5NmQwOTI1OTFkIFTuRCQ= X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2015 21:52:36 -0000 glebius added a comment. Is it possible to add Mike Karels to this discussion? REVISION DETAIL https://reviews.freebsd.org/D1965 To: erj, gnn, jfvogel, adrian Cc: glebius, freebsd-net From owner-freebsd-net@FreeBSD.ORG Wed Feb 25 22:08:25 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 779F468E; Wed, 25 Feb 2015 22:08:25 +0000 (UTC) Received: from khavrinen.csail.mit.edu (khavrinen.csail.mit.edu [IPv6:2001:470:8b2d:1e1c:21b:21ff:feb8:d7b0]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "khavrinen.csail.mit.edu", Issuer "Client CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 45106B1; Wed, 25 Feb 2015 22:08:25 +0000 (UTC) Received: from khavrinen.csail.mit.edu (localhost [127.0.0.1]) by khavrinen.csail.mit.edu (8.14.9/8.14.9) with ESMTP id t1PM8NSC060838 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL CN=khavrinen.csail.mit.edu issuer=Client+20CA); Wed, 25 Feb 2015 17:08:23 -0500 (EST) (envelope-from wollman@khavrinen.csail.mit.edu) Received: (from wollman@localhost) by khavrinen.csail.mit.edu (8.14.9/8.14.9/Submit) id t1PM8MGf060835; Wed, 25 Feb 2015 17:08:22 -0500 (EST) (envelope-from wollman) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <21742.18390.976511.707403@khavrinen.csail.mit.edu> Date: Wed, 25 Feb 2015 17:08:22 -0500 From: Garrett Wollman To: freebsd-fs@freebsd.org Subject: Implementing backpressure in the NFS server X-Mailer: VM 7.17 under 21.4 (patch 22) "Instant Classic" XEmacs Lucid X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (khavrinen.csail.mit.edu [127.0.0.1]); Wed, 25 Feb 2015 17:08:23 -0500 (EST) Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2015 22:08:25 -0000 Here's the scenario: 1) A small number of (Linux) clients run a large number of processes (compute jobs) that read large files sequentially out of an NFS filesystem. Each process is reading from a different file. 2) The clients are behind a network bottleneck. 3) The Linux NFS client will issue NFS3PROC_READ RPCs (potentially including read-ahead) independently for each process. 4) The network bottleneck does not serve to limit the rate at which read RPCs can be issued, because the requests are small (it's only the responses that are large). 5) Even if the responses are delayed, causing one process to block, there are sufficient other processes that are still runnable to allow more reads to be issued. 6) On the server side, because these are requests for different file handles, they will get steered to different NFS service threads by the generic RPC queueing code. 7) Each service thread will process the read to completion, and then block when the reply is transmitted because the socket buffer is full. 8) As more reads continue to be issued by the clients, more and more service threads are stuck waiting for the socket buffer until all of the nfsd threads are blocked. 9) The server is now almost completely idle. Incoming requests can only be serviced when one of the nfsd threads finally manages to put its pending reply on the socket send queue, at which point it can return to the RPC code and pick up one request -- which, because the incoming queues are full of pending reads from the problem clients, is likely to get stuck in the same place. Lather, rinse, repeat. What should happen here? As an administrator, I can certainly increase the number of NFS service threads until there are sufficient threads available to handle all of the offered load -- but the load varies widely over time, and it's likely that I would run into other resource constraints if I did this without limit. (Is 1000 threads practical? What happens when a different mix of RPCs comes in -- will it livelock the server?) I'm of the opinion that we need at least one of the following things to mitigate this issue, but I don't have a good knowledge of the RPC code to have an idea how feasible this is: a) Admission control. RPCs should not be removed from the receive queue if the transmit queue is over some high-water mark. This will ensure that a problem client behind a network bottleneck like this one will eventually feel backpressure via TCP window contraction if nothing else. This will also make it more likely that other clients will still get their RPCs processed even if most service threads are taken up by the problem clients. b) Fairness scheduling. There should be some parameter, configurable by the administrator, that restricts the number of nfsd threads any one client can occupy, independent of how many requests it has pending. A really advanced scheduler would allow bursting over the limit for some small number of requests. Does anyone else have thoughts, or even implementation ideas, on this? -GAWollman From owner-freebsd-net@FreeBSD.ORG Wed Feb 25 22:11:27 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E78598E2; Wed, 25 Feb 2015 22:11:26 +0000 (UTC) Received: from cell.glebius.int.ru (glebius.int.ru [81.19.69.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "cell.glebius.int.ru", Issuer "cell.glebius.int.ru" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 6488A142; Wed, 25 Feb 2015 22:11:25 +0000 (UTC) Received: from cell.glebius.int.ru (localhost [127.0.0.1]) by cell.glebius.int.ru (8.14.9/8.14.9) with ESMTP id t1PMBL4B025160 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 26 Feb 2015 01:11:21 +0300 (MSK) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebius.int.ru (8.14.9/8.14.9/Submit) id t1PMBLut025159; Thu, 26 Feb 2015 01:11:21 +0300 (MSK) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebius.int.ru: glebius set sender to glebius@FreeBSD.org using -f Date: Thu, 26 Feb 2015 01:11:21 +0300 From: Gleb Smirnoff To: Mike Karels Subject: Re: Adding new media types to if_media.h Message-ID: <20150225221120.GC17947@FreeBSD.org> References: <201502170150.t1H1ouxM020621@mail.karels.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201502170150.t1H1ouxM020621@mail.karels.net> User-Agent: Mutt/1.5.23 (2014-03-12) Cc: "freebsd-net@freebsd.org" , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2015 22:11:27 -0000 On Mon, Feb 16, 2015 at 07:50:56PM -0600, Mike Karels wrote: M> Well, I developed the prototype as I had planned, using a 64-bit media M> word, and found that I got about 100 files in GENERIC that didn't compile; M> they attempted to store "media words" in an int. My kingdom for a typedef. M> That didn't meet my goal of KPI compatibility, so I went to Plan B. M> M> Plan B is to steal an unused bit (RFU) to indicate an "extended" media M> type. I then used the variant/subtype field to store the extended type. M> Effectively, the previously unused bit doubles the effective size of the M> subtype field. Given that the previous 5-bit field lasted us 18 years, M> I figured that doubling it would last a while. I also changed the M> SIOGGIFMEDIA ioctl, splitting it for binary compatibility; extended M> types are all mapped to IFM_OTHER (31) using the old interface, but M> are visible using the new one. M> M> With these changes, I modified one driver (vtnet) to use an extended type, M> and the rest of GENERIC is happy. The changes to ifconfig are also fairly M> small. The patch is appended, where email programs will screw it up, M> or at ftp://ftp.karels.net/outgoing/if_media.patch. M> M> The VFAST subtype is a throw-away for testing. M> M> This seems like a reasonably pragmatic change to support the new 40 Gb/s M> media types until someone wants to design an improved but non-backward- M> compatible interface. I think it meets the goal of suitability for M> back-porting; it could be MFCed. I will dare to vote against the crowd. We can't and don't plan to preserve the driver KPI for the 11 branch. The plan, that I hope to accomplish by 11 is to provide a driver KPI, where drivers do not about struct ifnet, and other network stack stuff. Of course, that's a huge change in KPI. But we do it for the sake to avoid future changes. So, all this tricks with one extra bit seem unnecessary to me. I'd suggest to introduce new 'struct ifmedia' with enough space, and of course put extra space in there. Give a new value to SIOCGIFMEDIA. Write a new clear code to handle it, without any extended bit tricks. For the sake of userland API, save old current 'struct ifmedia' as 'struct oifmedia', and take old value of ioctl to OSIOCIGIFMEDIA. Write a function under BURN_BRIDGES that handles OSIOCIGIFMEDIA and tries to convert from ifmedia to oifmedia, To summarise: the patch adds tricks to just double the ifmedia name space, not solving the problem forever. New API is introduced, but old limited one doesn't have foreseable obsolete plan, since new is tied to it. All tricks are performed for the sake of driver KPI stability, which isn't planned to be kept for this major release cycle. -- Totus tuus, Glebius. From owner-freebsd-net@FreeBSD.ORG Wed Feb 25 22:13:15 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C49EDCA2 for ; Wed, 25 Feb 2015 22:13:15 +0000 (UTC) Received: from serv5.pstu.ru (serv5.pstu.ru [195.19.162.243]) by mx1.freebsd.org (Postfix) with ESMTP id DD6D31B2 for ; Wed, 25 Feb 2015 22:13:14 +0000 (UTC) Received: from black.berezniki.ru (host.bf.pstu.ru [195.19.182.2] (may be forged)) by serv5.pstu.ru (8.14.5/8.14.2) with ESMTP id t1PAsELS082122 for ; Wed, 25 Feb 2015 15:54:14 +0500 (GMT-5) (envelope-from kvas@bf.pstu.ru) Received: from 10.1.1.100 (kvas_int [10.1.1.100]) by black.berezniki.ru (Postfix) with ESMTP id 10BD62E06C for ; Wed, 25 Feb 2015 16:52:16 +0600 (YEKT) Date: Wed, 25 Feb 2015 15:47:02 +0600 From: kvas X-Mailer: The Bat! (v1.60c) Reply-To: kvas Organization: BF PGTU X-Priority: 3 (Normal) Message-ID: <187112505953.20150225154702@bf.pstu.ru> To: FreeBSD Net Subject: options "in_port" and "out_port" equivalent for in-kernel NAT MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2015 22:13:15 -0000 Hello FreeBSD, Using natd it is possible to configure different "in_port" and "out_port" and divert packets to necessary port for both directions. Is there any way diverting a packet to in-kernel NAT to give an advise in which direction packets src/dst addresses should be translated? -- Best regards, kvas mailto:kvas@bf.pstu.ru From owner-freebsd-net@FreeBSD.ORG Wed Feb 25 22:17:47 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 59044F9A; Wed, 25 Feb 2015 22:17:47 +0000 (UTC) Received: from mail-la0-x22e.google.com (mail-la0-x22e.google.com [IPv6:2a00:1450:4010:c03::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BB5A91E9; Wed, 25 Feb 2015 22:17:46 +0000 (UTC) Received: by lams18 with SMTP id s18so7119192lam.11; Wed, 25 Feb 2015 14:17:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=ehVDh+dZz5XzbqTPgyNVhQMDB6B0hGYGlnQNl5aXvzg=; b=J0pFXpfd+Niavq4VJufPKlkKMhkXmHkT6YrZv+EETpcZUTlOiYILMgidLMNd8CSaVh C4MywqW9MlpvNgLqjmqhDB+ya/3j0AJJOLOuGTTzCOXY40+YpyIJl4AD6IAtpHLC3aMR M8fQtXH2pLQjE7UVjkzUuzJnGZ0+jNn6Ifgirp93SEDb8JF7bhQL2rFTRsIXmAoOq0Gh dcXvIbd7CNTBdbEbeaj2+CrZd19golYkhg9Z41quD2RxHD7V7+kdvM+LhJodmqzM/9sb PAbOFZsMAelfr7YaK3kYF7MCPCC/ze/irU3LcB2EFn68Zk6HcMuWw0rUZ2EHgWC/F7gi Sv/A== MIME-Version: 1.0 X-Received: by 10.112.169.39 with SMTP id ab7mr4662205lbc.85.1424902664816; Wed, 25 Feb 2015 14:17:44 -0800 (PST) Received: by 10.112.173.199 with HTTP; Wed, 25 Feb 2015 14:17:44 -0800 (PST) In-Reply-To: <20150225221120.GC17947@FreeBSD.org> References: <201502170150.t1H1ouxM020621@mail.karels.net> <20150225221120.GC17947@FreeBSD.org> Date: Wed, 25 Feb 2015 14:17:44 -0800 Message-ID: Subject: Re: Adding new media types to if_media.h From: Jack Vogel To: Gleb Smirnoff Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: "freebsd-net@freebsd.org" , Mike Karels , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2015 22:17:47 -0000 So we have products coming soon that need to extend the media, if you have grandiose plans in the future you can worry about things then, Linux can handle the extended media TODAY, wouldn't it be nice to do so as well? Also, this solution is something that can be MFC'd into 10 STABLE, if you do something big in the 11 stream I will be that wont be possible, so I'd say lets do this as a first step. Jack On Wed, Feb 25, 2015 at 2:11 PM, Gleb Smirnoff wrote: > On Mon, Feb 16, 2015 at 07:50:56PM -0600, Mike Karels wrote: > M> Well, I developed the prototype as I had planned, using a 64-bit media > M> word, and found that I got about 100 files in GENERIC that didn't > compile; > M> they attempted to store "media words" in an int. My kingdom for a > typedef. > M> That didn't meet my goal of KPI compatibility, so I went to Plan B. > M> > M> Plan B is to steal an unused bit (RFU) to indicate an "extended" media > M> type. I then used the variant/subtype field to store the extended type. > M> Effectively, the previously unused bit doubles the effective size of the > M> subtype field. Given that the previous 5-bit field lasted us 18 years, > M> I figured that doubling it would last a while. I also changed the > M> SIOGGIFMEDIA ioctl, splitting it for binary compatibility; extended > M> types are all mapped to IFM_OTHER (31) using the old interface, but > M> are visible using the new one. > M> > M> With these changes, I modified one driver (vtnet) to use an extended > type, > M> and the rest of GENERIC is happy. The changes to ifconfig are also > fairly > M> small. The patch is appended, where email programs will screw it up, > M> or at ftp://ftp.karels.net/outgoing/if_media.patch. > M> > M> The VFAST subtype is a throw-away for testing. > M> > M> This seems like a reasonably pragmatic change to support the new 40 Gb/s > M> media types until someone wants to design an improved but non-backward- > M> compatible interface. I think it meets the goal of suitability for > M> back-porting; it could be MFCed. > > I will dare to vote against the crowd. > > We can't and don't plan to preserve the driver KPI for the 11 branch. The > plan, that I hope to accomplish by 11 is to provide a driver KPI, where > drivers do not about struct ifnet, and other network stack stuff. Of > course, that's a huge change in KPI. But we do it for the sake to avoid > future changes. > > So, all this tricks with one extra bit seem unnecessary to me. I'd suggest > to introduce new 'struct ifmedia' with enough space, and of course put > extra > space in there. Give a new value to SIOCGIFMEDIA. Write a new clear code > to handle it, without any extended bit tricks. > > For the sake of userland API, save old current 'struct ifmedia' as > 'struct oifmedia', and take old value of ioctl to OSIOCIGIFMEDIA. > Write a function under BURN_BRIDGES that handles OSIOCIGIFMEDIA and > tries to convert from ifmedia to oifmedia, > > To summarise: the patch adds tricks to just double the ifmedia name space, > not solving the problem forever. New API is introduced, but old limited one > doesn't have foreseable obsolete plan, since new is tied to it. All tricks > are performed for the sake of driver KPI stability, which isn't planned > to be kept for this major release cycle. > > -- > Totus tuus, Glebius. > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Wed Feb 25 22:26:14 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EE68C510 for ; Wed, 25 Feb 2015 22:26:14 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [IPv6:2001:4f8:3:ffe0:406a:0:50:2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CC050334 for ; Wed, 25 Feb 2015 22:26:14 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [127.0.1.5]) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9) with ESMTP id t1PMQEbt096089 for ; Wed, 25 Feb 2015 22:26:14 GMT (envelope-from root@phabric-backend.isc.freebsd.org) Received: (from root@localhost) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9/Submit) id t1PMQEfo096088; Wed, 25 Feb 2015 22:26:14 GMT (envelope-from root) Date: Wed, 25 Feb 2015 22:26:14 +0000 To: freebsd-net@freebsd.org From: "jfvogel (Jack Vogel)" Subject: [Differential] [Accepted] D1965: Add extended media types to if_media.h and ifconfig Message-ID: <6d346adf9ad98adfad1a4d64cb790c08@localhost.localdomain> X-Priority: 3 Thread-Topic: D1965: Add extended media types to if_media.h and ifconfig X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: X-Phabricator-Cc: In-Reply-To: References: Thread-Index: YmU0NTlhN2Q2ZDc3OTdjODA5NmQwOTI1OTFkIFTuTAY= X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2015 22:26:15 -0000 jfvogel accepted this revision. This revision is now accepted and ready to land. BRANCH /head REVISION DETAIL https://reviews.freebsd.org/D1965 To: erj, gnn, adrian, jfvogel Cc: glebius, freebsd-net From owner-freebsd-net@FreeBSD.ORG Wed Feb 25 22:29:05 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DB48E661; Wed, 25 Feb 2015 22:29:05 +0000 (UTC) Received: from cell.glebius.int.ru (glebius.int.ru [81.19.69.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "cell.glebius.int.ru", Issuer "cell.glebius.int.ru" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 59F9135C; Wed, 25 Feb 2015 22:29:04 +0000 (UTC) Received: from cell.glebius.int.ru (localhost [127.0.0.1]) by cell.glebius.int.ru (8.14.9/8.14.9) with ESMTP id t1PMT2EP025280 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 26 Feb 2015 01:29:02 +0300 (MSK) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebius.int.ru (8.14.9/8.14.9/Submit) id t1PMT2Z6025279; Thu, 26 Feb 2015 01:29:02 +0300 (MSK) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebius.int.ru: glebius set sender to glebius@FreeBSD.org using -f Date: Thu, 26 Feb 2015 01:29:02 +0300 From: Gleb Smirnoff To: Jack Vogel Subject: Re: Adding new media types to if_media.h Message-ID: <20150225222902.GD17947@glebius.int.ru> References: <201502170150.t1H1ouxM020621@mail.karels.net> <20150225221120.GC17947@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: "freebsd-net@freebsd.org" , Mike Karels , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2015 22:29:06 -0000 On Wed, Feb 25, 2015 at 02:17:44PM -0800, Jack Vogel wrote: J> So we have products coming soon that need to extend the media, if you have J> grandiose plans in the future you can worry about things then, Linux can J> handle the extended media TODAY, wouldn't it be nice to do so as well? I didn't suggest to wait, did I? J> Also, this solution is something that can be MFC'd into 10 STABLE, J> if you do something big in the 11 stream I will be that wont be possible, J> so I'd say lets do this as a first step. That would mean that new SIOCXGIFMEDIA needs to be supported from now and forever, still using the old structure and still limited to 62 types. Alternative is: In head: - new if_media, new value for SIOCGIFMEDIA, old API preserved via keeping oif_media, OSIOCGIFMEDIA, keeping it contained under ifdef BURN_BRIDGES. With a possibility to remove it some foreseable future. Merging to stable/10: - new if_media by the name of nif_media, new SIOCGIFMEDIA by name of NSIOCGIFMEDIA - old API untouched This way we still are able to introduce new feature to stable/10. But the ugly compat code is put into the branch with limited lifetime, instead of carrying it to future. -- Totus tuus, Glebius. From owner-freebsd-net@FreeBSD.ORG Wed Feb 25 22:42:57 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A6198D5D; Wed, 25 Feb 2015 22:42:57 +0000 (UTC) Received: from mail-pa0-x231.google.com (mail-pa0-x231.google.com [IPv6:2607:f8b0:400e:c03::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6E1A6784; Wed, 25 Feb 2015 22:42:57 +0000 (UTC) Received: by pabkx10 with SMTP id kx10so8785779pab.0; Wed, 25 Feb 2015 14:42:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=synNKbI10QmCXiRS13RbWOTJKfeuehlof2Uwl9DaGbQ=; b=Ffy0o7Zk6e8wOYkUJNMxojCvdcn3Gy2GvdtAjOgEFDWMd3WX+CL+5vLjNzs2jrslX8 b9yjC4+rtce5c65fnMd6V/yQ2TmU+dB8n/DcmIKJtQzJJnCGUktdHMx5y2qTMdDIj5gL jaPKiE3Kp56P3wtU8kYCaRHMPM8oSrlgXSY1HO7qwnn1+zNY+qe4/IKta6dmciUDnozc 1dALhF/lSLN3cKiY+59xlIpXb14Smq9OYCz5E43mL5CLiqih7fWEzAn7kvhVi8ugQNdk wvTRKy3z2oAlaU/dqWKzi+lBk840mCVS30Yn6OHJNZDyTxYvIrOzu5PwLNcchXM3aNol K7Fw== X-Received: by 10.70.46.65 with SMTP id t1mr9378775pdm.128.1424904176925; Wed, 25 Feb 2015 14:42:56 -0800 (PST) MIME-Version: 1.0 Received: by 10.70.67.104 with HTTP; Wed, 25 Feb 2015 14:42:16 -0800 (PST) In-Reply-To: <20150225222902.GD17947@glebius.int.ru> References: <201502170150.t1H1ouxM020621@mail.karels.net> <20150225221120.GC17947@FreeBSD.org> <20150225222902.GD17947@glebius.int.ru> From: Eric Joyner Date: Wed, 25 Feb 2015 14:42:16 -0800 Message-ID: Subject: Re: Adding new media types to if_media.h To: Gleb Smirnoff Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: "freebsd-net@freebsd.org" , Jack Vogel , Mike Karels , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2015 22:42:57 -0000 Tbh, I respect Gleb's approach, but developing such a thing would take a while; the fix Mike proposed would be a fix now. I mean, I'd like to see a decoupling of media types and speeds from "standard" names, and maybe have both an ability to query what modules a device supports and what speeds it supports given its current module, instead of relying on the clunky media type list now. And then it'd be nice to set flow control from ifconfig, too, without having to couple those modes to media types. I'm still all for having a new system in the future. But in changing the KPI so much, it'd be important to consider the "do we need an ethtool-equivalent" discussion. And I think we've lost a bunch of people who were in the original discussion from the to:/cc: list. --- - Eric Joyner On Wed, Feb 25, 2015 at 2:29 PM, Gleb Smirnoff wrote: > On Wed, Feb 25, 2015 at 02:17:44PM -0800, Jack Vogel wrote: > J> So we have products coming soon that need to extend the media, if you > have > J> grandiose plans in the future you can worry about things then, Linux can > J> handle the extended media TODAY, wouldn't it be nice to do so as well? > > I didn't suggest to wait, did I? > > J> Also, this solution is something that can be MFC'd into 10 STABLE, > J> if you do something big in the 11 stream I will be that wont be > possible, > J> so I'd say lets do this as a first step. > > That would mean that new SIOCXGIFMEDIA needs to be supported from now and > forever, still using the old structure and still limited to 62 types. > > Alternative is: > > In head: > - new if_media, new value for SIOCGIFMEDIA, old API preserved via > keeping oif_media, OSIOCGIFMEDIA, keeping it contained under > ifdef BURN_BRIDGES. With a possibility to remove it some foreseable > future. > > Merging to stable/10: > - new if_media by the name of nif_media, new SIOCGIFMEDIA by name of > NSIOCGIFMEDIA > - old API untouched > > This way we still are able to introduce new feature to stable/10. But > the ugly compat code is put into the branch with limited lifetime, > instead of carrying it to future. > > -- > Totus tuus, Glebius. > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Wed Feb 25 22:50:54 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 62F2ACB; Wed, 25 Feb 2015 22:50:54 +0000 (UTC) Received: from cell.glebius.int.ru (glebius.int.ru [81.19.69.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "cell.glebius.int.ru", Issuer "cell.glebius.int.ru" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id B8C79833; Wed, 25 Feb 2015 22:50:53 +0000 (UTC) Received: from cell.glebius.int.ru (localhost [127.0.0.1]) by cell.glebius.int.ru (8.14.9/8.14.9) with ESMTP id t1PMop0u025388 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 26 Feb 2015 01:50:51 +0300 (MSK) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebius.int.ru (8.14.9/8.14.9/Submit) id t1PMopYp025387; Thu, 26 Feb 2015 01:50:51 +0300 (MSK) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebius.int.ru: glebius set sender to glebius@FreeBSD.org using -f Date: Thu, 26 Feb 2015 01:50:51 +0300 From: Gleb Smirnoff To: Eric Joyner Subject: Re: Adding new media types to if_media.h Message-ID: <20150225225051.GE17947@glebius.int.ru> References: <201502170150.t1H1ouxM020621@mail.karels.net> <20150225221120.GC17947@FreeBSD.org> <20150225222902.GD17947@glebius.int.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: "freebsd-net@freebsd.org" , Jack Vogel , Mike Karels , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2015 22:50:54 -0000 On Wed, Feb 25, 2015 at 02:42:16PM -0800, Eric Joyner wrote: E> Tbh, I respect Gleb's approach, but developing such a thing would take a E> while; the fix Mike proposed would be a fix now. E> E> I mean, I'd like to see a decoupling of media types and speeds from E> "standard" names, and maybe have both an ability to query what modules a E> device supports and what speeds it supports given its current module, E> instead of relying on the clunky media type list now. And then it'd be nice E> to set flow control from ifconfig, too, without having to couple those E> modes to media types. I'm still all for having a new system in the future. E> E> But in changing the KPI so much, it'd be important to consider the "do we E> need an ethtool-equivalent" discussion. E> E> And I think we've lost a bunch of people who were in the original E> discussion from the to:/cc: list. Actually the amount of code for my approach is approximately the same as with Mike's. The only thing we must sit down and think without a hurry are the required and spare fields for new if_media. We definitely need input from Adrian on his net80211 requirements, and input from all involved parties. I'm willing to code this if we all agree on the topic, so that you will code all done and commited before 10.2-RELEASE. -- Totus tuus, Glebius. From owner-freebsd-net@FreeBSD.ORG Wed Feb 25 23:07:47 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 712833AC; Wed, 25 Feb 2015 23:07:47 +0000 (UTC) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 5A6649FA; Wed, 25 Feb 2015 23:07:47 +0000 (UTC) Received: from AlfredMacbookAir.local (3.sub-70-208-71.myvzw.com [70.208.71.3]) by elvis.mu.org (Postfix) with ESMTPSA id C34CC341F90F; Wed, 25 Feb 2015 15:07:39 -0800 (PST) Message-ID: <54EE5692.5070402@mu.org> Date: Wed, 25 Feb 2015 18:11:14 -0500 From: Alfred Perlstein User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 MIME-Version: 1.0 To: Gleb Smirnoff , Mike Karels Subject: Re: Adding new media types to if_media.h References: <201502170150.t1H1ouxM020621@mail.karels.net> <20150225221120.GC17947@FreeBSD.org> In-Reply-To: <20150225221120.GC17947@FreeBSD.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Cc: "freebsd-net@freebsd.org" , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2015 23:07:47 -0000 On 2/25/15 5:11 PM, Gleb Smirnoff wrote: > On Mon, Feb 16, 2015 at 07:50:56PM -0600, Mike Karels wrote: > M> Well, I developed the prototype as I had planned, using a 64-bit media > M> word, and found that I got about 100 files in GENERIC that didn't compile; > M> they attempted to store "media words" in an int. My kingdom for a typedef. > M> That didn't meet my goal of KPI compatibility, so I went to Plan B. > M> > M> Plan B is to steal an unused bit (RFU) to indicate an "extended" media > M> type. I then used the variant/subtype field to store the extended type. > M> Effectively, the previously unused bit doubles the effective size of the > M> subtype field. Given that the previous 5-bit field lasted us 18 years, > M> I figured that doubling it would last a while. I also changed the > M> SIOGGIFMEDIA ioctl, splitting it for binary compatibility; extended > M> types are all mapped to IFM_OTHER (31) using the old interface, but > M> are visible using the new one. > M> > M> With these changes, I modified one driver (vtnet) to use an extended type, > M> and the rest of GENERIC is happy. The changes to ifconfig are also fairly > M> small. The patch is appended, where email programs will screw it up, > M> or at ftp://ftp.karels.net/outgoing/if_media.patch. > M> > M> The VFAST subtype is a throw-away for testing. > M> > M> This seems like a reasonably pragmatic change to support the new 40 Gb/s > M> media types until someone wants to design an improved but non-backward- > M> compatible interface. I think it meets the goal of suitability for > M> back-porting; it could be MFCed. > > I will dare to vote against the crowd. > > We can't and don't plan to preserve the driver KPI for the 11 branch. The > plan, that I hope to accomplish by 11 is to provide a driver KPI, where > drivers do not about struct ifnet, and other network stack stuff. Of > course, that's a huge change in KPI. But we do it for the sake to avoid > future changes. > > So, all this tricks with one extra bit seem unnecessary to me. I'd suggest > to introduce new 'struct ifmedia' with enough space, and of course put extra > space in there. Give a new value to SIOCGIFMEDIA. Write a new clear code > to handle it, without any extended bit tricks. > > For the sake of userland API, save old current 'struct ifmedia' as > 'struct oifmedia', and take old value of ioctl to OSIOCIGIFMEDIA. > Write a function under BURN_BRIDGES that handles OSIOCIGIFMEDIA and > tries to convert from ifmedia to oifmedia, > > To summarise: the patch adds tricks to just double the ifmedia name space, > not solving the problem forever. New API is introduced, but old limited one > doesn't have foreseable obsolete plan, since new is tied to it. All tricks > are performed for the sake of driver KPI stability, which isn't planned > to be kept for this major release cycle. +1, rip the bandaid off. -Alfred From owner-freebsd-net@FreeBSD.ORG Wed Feb 25 23:26:14 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6A9906CF; Wed, 25 Feb 2015 23:26:14 +0000 (UTC) Received: from elvis.mu.org (elvis.mu.org [IPv6:2001:470:1f05:b76::196]) by mx1.freebsd.org (Postfix) with ESMTP id 55B52C02; Wed, 25 Feb 2015 23:26:14 +0000 (UTC) Received: from AlfredMacbookAir.local (3.sub-70-208-71.myvzw.com [70.208.71.3]) by elvis.mu.org (Postfix) with ESMTPSA id DBE70341F912; Wed, 25 Feb 2015 15:26:12 -0800 (PST) Message-ID: <54EE5AE9.1000908@freebsd.org> Date: Wed, 25 Feb 2015 18:29:45 -0500 From: Alfred Perlstein Organization: FreeBSD User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 MIME-Version: 1.0 To: Garrett Wollman , freebsd-fs@freebsd.org Subject: Re: Implementing backpressure in the NFS server References: <21742.18390.976511.707403@khavrinen.csail.mit.edu> In-Reply-To: <21742.18390.976511.707403@khavrinen.csail.mit.edu> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2015 23:26:14 -0000 On 2/25/15 5:08 PM, Garrett Wollman wrote: > Here's the scenario: > > 1) A small number of (Linux) clients run a large number of processes > (compute jobs) that read large files sequentially out of an NFS > filesystem. Each process is reading from a different file. > > 2) The clients are behind a network bottleneck. > > 3) The Linux NFS client will issue NFS3PROC_READ RPCs (potentially > including read-ahead) independently for each process. > > 4) The network bottleneck does not serve to limit the rate at which > read RPCs can be issued, because the requests are small (it's only the > responses that are large). > > 5) Even if the responses are delayed, causing one process to block, > there are sufficient other processes that are still runnable to allow > more reads to be issued. > > 6) On the server side, because these are requests for different file > handles, they will get steered to different NFS service threads by the > generic RPC queueing code. > > 7) Each service thread will process the read to completion, and then > block when the reply is transmitted because the socket buffer is full. > > 8) As more reads continue to be issued by the clients, more and more > service threads are stuck waiting for the socket buffer until all of > the nfsd threads are blocked. > > 9) The server is now almost completely idle. Incoming requests can > only be serviced when one of the nfsd threads finally manages to put > its pending reply on the socket send queue, at which point it can > return to the RPC code and pick up one request -- which, because the > incoming queues are full of pending reads from the problem clients, is > likely to get stuck in the same place. Lather, rinse, repeat. > > What should happen here? As an administrator, I can certainly > increase the number of NFS service threads until there are sufficient > threads available to handle all of the offered load -- but the load > varies widely over time, and it's likely that I would run into other > resource constraints if I did this without limit. (Is 1000 threads > practical? What happens when a different mix of RPCs comes in -- will > it livelock the server?) > > I'm of the opinion that we need at least one of the following things > to mitigate this issue, but I don't have a good knowledge of the RPC > code to have an idea how feasible this is: > > a) Admission control. RPCs should not be removed from the receive > queue if the transmit queue is over some high-water mark. This will > ensure that a problem client behind a network bottleneck like this one > will eventually feel backpressure via TCP window contraction if > nothing else. This will also make it more likely that other clients > will still get their RPCs processed even if most service threads are > taken up by the problem clients. > > b) Fairness scheduling. There should be some parameter, configurable > by the administrator, that restricts the number of nfsd threads any > one client can occupy, independent of how many requests it has > pending. A really advanced scheduler would allow bursting over the > limit for some small number of requests. > > Does anyone else have thoughts, or even implementation ideas, on this? The default number of threads is insanely low, the only reason I didn't bump them to FreeNAS levels (or higher) was because of the inevitable bikeshed/cryfest about Alfred touching defaults so I didn't bother. I kept them really small, because y'know people whine, and they are capped at ncpu * 8, it really should be higher imo. Just increase the nfs servers to something higher, I think we were at 256 threads in FreeNAS and it did us just fine. Higher seemed ok, except we lost a bit of performance. The only problem you might see is on SMALL machines where people will complain. So probably want an arch specific override or perhaps a memory based sliding scale. If that could become a FreeBSD default (with overrides for small memory machines and arches) that would be even better. I think your other suggestions are fine, however the problem is that: 1) they seem complex for an edge case 2) turning them on may tank performance for no good reason if the heuristic is met but we're not in the bad situation That said if you want to pursue those options, by all means please do. -Alfred From owner-freebsd-net@FreeBSD.ORG Wed Feb 25 23:36:28 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9F3118F1; Wed, 25 Feb 2015 23:36:28 +0000 (UTC) Received: from khavrinen.csail.mit.edu (khavrinen.csail.mit.edu [IPv6:2001:470:8b2d:1e1c:21b:21ff:feb8:d7b0]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "khavrinen.csail.mit.edu", Issuer "Client CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 25F54CEC; Wed, 25 Feb 2015 23:36:28 +0000 (UTC) Received: from khavrinen.csail.mit.edu (localhost [127.0.0.1]) by khavrinen.csail.mit.edu (8.14.9/8.14.9) with ESMTP id t1PNaQ0b062094 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL CN=khavrinen.csail.mit.edu issuer=Client+20CA); Wed, 25 Feb 2015 18:36:26 -0500 (EST) (envelope-from wollman@khavrinen.csail.mit.edu) Received: (from wollman@localhost) by khavrinen.csail.mit.edu (8.14.9/8.14.9/Submit) id t1PNaQbl062091; Wed, 25 Feb 2015 18:36:26 -0500 (EST) (envelope-from wollman) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <21742.23674.220013.63261@khavrinen.csail.mit.edu> Date: Wed, 25 Feb 2015 18:36:26 -0500 From: Garrett Wollman To: Alfred Perlstein Subject: Re: Implementing backpressure in the NFS server In-Reply-To: <54EE5AE9.1000908@freebsd.org> References: <21742.18390.976511.707403@khavrinen.csail.mit.edu> <54EE5AE9.1000908@freebsd.org> X-Mailer: VM 7.17 under 21.4 (patch 22) "Instant Classic" XEmacs Lucid X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (khavrinen.csail.mit.edu [127.0.0.1]); Wed, 25 Feb 2015 18:36:26 -0500 (EST) Cc: freebsd-fs@freebsd.org, freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2015 23:36:28 -0000 < said: > I think your other suggestions are fine, however the problem is that: > 1) they seem complex for an edge case > 2) turning them on may tank performance for no good reason if the > heuristic is met but we're not in the bad situation I'm OK with trading off performance for one user against availability for the other 800. I'm pleased to hear that FreeNAS was fine with 256 threads as the default; I'm going to try running with that to see if we encounter any other scaling issues as a result. (Our servers are all 12-core, 24-thread systems with buckets of memory, and I remember increasing the thread count as high as 128 previously, but I also remember having to back it down to 64 for some reason.) -GAWollman From owner-freebsd-net@FreeBSD.ORG Thu Feb 26 00:27:38 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CDF5342C; Thu, 26 Feb 2015 00:27:38 +0000 (UTC) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 77B9D214; Thu, 26 Feb 2015 00:27:37 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2CSBQBbZe5U/95baINbg1RaBIMFwBUKhSdJAoFoAQEBAQEBfIQPAQEBAwEBAQEgBCcgCwUWGAICDRkCKQEJJgYIBwQBHASIBggNvEWZIQEBAQEBBQEBAQEBAQEbgSGIdH6EDAsFAgEbNAeCaIFDBYpQgw2FZoNGgzo5gmGCUUuIMoM+IoF9AgMcgW4gMQd7AQQbIn8BAQE X-IronPort-AV: E=Sophos;i="5.09,648,1418101200"; d="scan'208";a="194869451" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-annu.net.uoguelph.ca with ESMTP; 25 Feb 2015 19:27:30 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 9642CB3F01; Wed, 25 Feb 2015 19:27:30 -0500 (EST) Date: Wed, 25 Feb 2015 19:27:30 -0500 (EST) From: Rick Macklem To: Alfred Perlstein Message-ID: <2109245074.408271.1424910450599.JavaMail.root@uoguelph.ca> In-Reply-To: <54EE5AE9.1000908@freebsd.org> Subject: Re: Implementing backpressure in the NFS server MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.95.12] X-Mailer: Zimbra 7.2.6_GA_2926 (ZimbraWebClient - FF3.0 (Win)/7.2.6_GA_2926) Cc: freebsd-fs@freebsd.org, freebsd-net@freebsd.org, Garrett Wollman X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Feb 2015 00:27:38 -0000 Alfred Perlstein wrote: > > On 2/25/15 5:08 PM, Garrett Wollman wrote: > > Here's the scenario: > > > > 1) A small number of (Linux) clients run a large number of > > processes > > (compute jobs) that read large files sequentially out of an NFS > > filesystem. Each process is reading from a different file. > > > > 2) The clients are behind a network bottleneck. > > > > 3) The Linux NFS client will issue NFS3PROC_READ RPCs (potentially > > including read-ahead) independently for each process. > > > > 4) The network bottleneck does not serve to limit the rate at which > > read RPCs can be issued, because the requests are small (it's only > > the > > responses that are large). > > > > 5) Even if the responses are delayed, causing one process to block, > > there are sufficient other processes that are still runnable to > > allow > > more reads to be issued. > > > > 6) On the server side, because these are requests for different > > file > > handles, they will get steered to different NFS service threads by > > the > > generic RPC queueing code. > > > > 7) Each service thread will process the read to completion, and > > then > > block when the reply is transmitted because the socket buffer is > > full. > > > > 8) As more reads continue to be issued by the clients, more and > > more > > service threads are stuck waiting for the socket buffer until all > > of > > the nfsd threads are blocked. > > > > 9) The server is now almost completely idle. Incoming requests can > > only be serviced when one of the nfsd threads finally manages to > > put > > its pending reply on the socket send queue, at which point it can > > return to the RPC code and pick up one request -- which, because > > the > > incoming queues are full of pending reads from the problem clients, > > is > > likely to get stuck in the same place. Lather, rinse, repeat. > > > > What should happen here? As an administrator, I can certainly > > increase the number of NFS service threads until there are > > sufficient > > threads available to handle all of the offered load -- but the load > > varies widely over time, and it's likely that I would run into > > other > > resource constraints if I did this without limit. (Is 1000 threads > > practical? What happens when a different mix of RPCs comes in -- > > will > > it livelock the server?) > > As far as I know, even 256 is an arbitrary limit left from when a server was typically a single core i386. Since they are just kernel threads, the idle ones are very little overhead. There is a 256 limit wired into the sources, but you can increase this and recompile. (MAXNFSDCNT in nfsd.c) I can't think of why 1000 threads isn't practical for server hardware of the size you run. > > I'm of the opinion that we need at least one of the following > > things > > to mitigate this issue, but I don't have a good knowledge of the > > RPC > > code to have an idea how feasible this is: > > > > a) Admission control. RPCs should not be removed from the receive > > queue if the transmit queue is over some high-water mark. This > > will > > ensure that a problem client behind a network bottleneck like this > > one > > will eventually feel backpressure via TCP window contraction if > > nothing else. This will also make it more likely that other > > clients > > will still get their RPCs processed even if most service threads > > are > > taken up by the problem clients. > > This sounds like a good idea in theory. However, I'm not sure how you can implement it. As you mention, Read requests are small. However, Write requests are fairly large. --> one 64K write request will result in as many bytes in the socket's receive queue as something like 700 Read requests. As such, using the socket receive queue's sb_cc isn't going to work. Since TCP doesn't have record marks, there is no way to know how many RPC requests are in the queue until they are processed. But, by the time the krpc has parsed out an RPC request from the socket's receive queue, it is pretty must "too late". For NFSv3, doing something like file handle affinity does, which pre-parses the RPC request, then putting it in some other queue instead of handing it to an nfsd right away, might be feasible. Also, lots of Getattr and/or Lookup isn't the same as lots of Reads. Then you'd have to decide how long to delay the RPC. If you use TCP send queue length, then what about cases where there are a lot of TCP connections for the clients on the other side of the network bottleneck? (However, since all NFSv4 RPCs are the same, ie Compound, this doesn't work for NFSv4, since it is very hard to parse and determine what an NFSv4 compound does without parsing it completely. The current code parses it as the Ops in it are done.) I think just having lots of nfsd threads and letting a bunch of them block on the socket send queue is much simpler to me. One of the nice things about using TCP transport is that it will apply backpressure at this level. > > b) Fairness scheduling. There should be some parameter, > > configurable > > by the administrator, that restricts the number of nfsd threads any > > one client can occupy, independent of how many requests it has > > pending. A really advanced scheduler would allow bursting over the > > limit for some small number of requests. > > I've thought about this and so long as you go with "per TCP connection" instead of per-client (which I think is close to the same thing in practice), it may be a good idea. I suspect the main problem with this is that it will negatively impact clients when most other clients are idle. (The worst case is when one client wants to do lots of reading when no other client mount is active.) I thought of something like a running estimate of "active clients" and then divide that into the total # of nfsd threads, but then estimating "active clients" will be a heuristic at best. Also, what about the case of many clients each doing some reads behind the network bottleneck instead of fewer clients doing lots of reads behind the network bottleneck? > > Does anyone else have thoughts, or even implementation ideas, on > > this? > The default number of threads is insanely low, the only reason I > didn't > bump them to FreeNAS levels (or higher) was because of the inevitable > bikeshed/cryfest about Alfred touching defaults so I didn't bother. > I > kept them really small, because y'know people whine, and they are > capped > at ncpu * 8, it really should be higher imo. > Heh, heh. That's why I never change defaults. Also, just fyi, I got email complaining about this and asking it be reverted back to "4" threads total by default. I just suggested they contact you;-) > Just increase the nfs servers to something higher, I think we were at > 256 threads in FreeNAS and it did us just fine. Higher seemed ok, > except we lost a bit of performance. > Yep, that's what I'd suggest too. If you can't get this to work well, then looking more closely at implementing one of your other suggestions. (I'd also recompile nfsd, so that you can go past 256 if you need to.) Good luck with it, rick > The only problem you might see is on SMALL machines where people will > complain. So probably want an arch specific override or perhaps a > memory based sliding scale. > > If that could become a FreeBSD default (with overrides for small > memory > machines and arches) that would be even better. > > I think your other suggestions are fine, however the problem is that: > 1) they seem complex for an edge case > 2) turning them on may tank performance for no good reason if the > heuristic is met but we're not in the bad situation > > That said if you want to pursue those options, by all means please > do. > > -Alfred > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Thu Feb 26 02:58:22 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2B744252; Thu, 26 Feb 2015 02:58:22 +0000 (UTC) Received: from mail.karels.net (mail.karels.net [63.231.190.5]) by mx1.freebsd.org (Postfix) with ESMTP id AC4299DA; Thu, 26 Feb 2015 02:58:21 +0000 (UTC) Received: from mail.karels.net (localhost [127.0.0.1]) by mail.karels.net (8.14.7/8.14.7) with ESMTP id t1Q2wKNK054143; Wed, 25 Feb 2015 20:58:20 -0600 (CST) (envelope-from mike@karels.net) Message-Id: <201502260258.t1Q2wKNK054143@mail.karels.net> To: Gleb Smirnoff From: Mike Karels Reply-to: mike@karels.net Subject: Re: Adding new media types to if_media.h In-reply-to: Your message of Thu, 26 Feb 2015 01:50:51 +0300. <20150225225051.GE17947@glebius.int.ru> Date: Wed, 25 Feb 2015 20:58:20 -0600 Cc: "freebsd-net@freebsd.org" , Eric Joyner , Jack Vogel , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Feb 2015 02:58:22 -0000 > On Wed, Feb 25, 2015 at 02:42:16PM -0800, Eric Joyner wrote: > E> Tbh, I respect Gleb's approach, but developing such a thing would take a > E> while; the fix Mike proposed would be a fix now. > E> > E> I mean, I'd like to see a decoupling of media types and speeds from > E> "standard" names, and maybe have both an ability to query what modules a > E> device supports and what speeds it supports given its current module, > E> instead of relying on the clunky media type list now. And then it'd be nice > E> to set flow control from ifconfig, too, without having to couple those > E> modes to media types. I'm still all for having a new system in the future. > E> > E> But in changing the KPI so much, it'd be important to consider the "do we > E> need an ethtool-equivalent" discussion. > E> > E> And I think we've lost a bunch of people who were in the original > E> discussion from the to:/cc: list. > Actually the amount of code for my approach is approximately the same > as with Mike's. The only thing we must sit down and think without a hurry > are the required and spare fields for new if_media. We definitely need > input from Adrian on his net80211 requirements, and input from all involved > parties. I'm not sure what would be different about your approach; you mentioned "n" versions rather than "x" versions of the ioctls, but I don't know what you have in mind for encoding. Any compatible version would be limited to int. In terms of a "real" fix ("ripping the bandaid off"), I think that if_media is basically wrong, and widening it won't fix it. There should be a generic structure that reports the media type (e.g. Ethernet), perhaps the speed and some generic status ("active"). Then there should be media-specific structures that encode the appropriate things including attachment type. 802.11 apparently already has an extension, and Ethernet should have a similar extension. The KPI should be media-type-specific. I don't see something like this being designed soon, and certainly wouldn't be able to be MFC'd. Meanwhile, many of us need to support 40 Gb/s Ethernet on non-current (or non-future) systems. > I'm willing to code this if we all agree on the topic, so that you will > code all done and commited before 10.2-RELEASE. I'd be interested in a sketch or more extended description sometime before 10.2. Back on the subject of the review: the VFAST entries should be removed, and the other entries should be moved down. Mike From owner-freebsd-net@FreeBSD.ORG Thu Feb 26 03:55:38 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E53F9E21; Thu, 26 Feb 2015 03:55:37 +0000 (UTC) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 81B78A3; Thu, 26 Feb 2015 03:55:36 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2CSBQD3l+5U/95baINbDoNGWgSDBcAVCoUnSQKBbQEBAQEBAXyEEAEBBAEBASArIAsFFhgCAg0ZAikBCSYGCAcEARwEiA4NvBmZCAEBAQEBAQEDAQEBAQEBAQEagSGJcoQdAQEbNAeCaIFDBYpQiHODRoM6OYUyiH2DPiKDMVsgMQEBAQSBBDl/AQEB X-IronPort-AV: E=Sophos;i="5.09,650,1418101200"; d="scan'208";a="194891817" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-annu.net.uoguelph.ca with ESMTP; 25 Feb 2015 22:55:35 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 8624BB3F85; Wed, 25 Feb 2015 22:55:35 -0500 (EST) Date: Wed, 25 Feb 2015 22:55:35 -0500 (EST) From: Rick Macklem To: Garrett Wollman Message-ID: <1930269117.485441.1424922935533.JavaMail.root@uoguelph.ca> In-Reply-To: <201502250244.t1P2iu6N094346@hergotha.csail.mit.edu> Subject: Re: NFS: kernel modules (loading/unloading) and scheduling MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.95.11] X-Mailer: Zimbra 7.2.6_GA_2926 (ZimbraWebClient - FF3.0 (Win)/7.2.6_GA_2926) Cc: freebsd-fs@freebsd.org, freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Feb 2015 03:55:38 -0000 Garrett Wollman wrote: > In article > <388835013.10159778.1424820357923.JavaMail.root@uoguelph.ca>, > rmacklem@uoguelph.ca writes: > > >I tend to think that a bias towards doing Getattr/Lookup over > >Read/Write > >may help performance (the old "shortest job first" principal), I'm > >not > >sure you'll have a big enough queue of outstanding RPCs under normal > >load > >for this to make a real difference. > > I don't think this is a particularly relevant condition here. There > are lots of ways RPCs can pile up where you really need to do better > work-sharing than the current implementation does. One example is a > client that issues lots of concurrent reads (e.g., a compute node > running dozens of parallel jobs). Two such systems on gigabit NICs > can easily issue large reads fast enough to cause 64 nfsd service > threads to blocked while waiting for the socket send buffer to drain. > Meanwhile, the file server is completely idle, but unable to respond > to incoming requests, and the other users get angry. Rather than > assigning new threads to requests from the slow clients, it would be > better to let the requests sit until the send buffer drains, and > process other clients' requests instead of letting the resources get > monopolized by a single user. > > Lest you think this is purely hypothetical: we actually experienced > this problem today, and I verified with "procstat -kk" that all of > the > nfsd threads were in fact blocked waiting for send buffer space to > open up. I was able to restore service immediately by increasing the > number of nfsd threads, but I'm unsure to what extent I can do this > without breaking other things or hitting other bottlenecks.[1] So I > have a user asking me why I haven't enable fair-share scheduling for > NFS, and I'm going to have to tell him the answer is "no such thing". > > -GAWollman > > [1] What would the right number actually be? We could potentially > have many thousands of threads in a compute cluster all operating > simultaneously on the same filesystem, well within the I/O capacity > of > the server, and we'd really like to degrade gracefully rather than > falling over when a single slow client soaks up all of the nfsd > worker > threads. Well, each of these threads have two structures allocated to it. 1 - The kthread info (sched_sizeof_thread() <-- struct thread + the scheduler info one) 2 - A structure used by the krpc for each thread. Since allocating two moderate sized structures isn't a lot of kernel memory, I would think a server like yours would be fine with several thousand nfsd threads. What would be interesting would be the receive queue lengths for the sockets for NFS client TCP connections when the server is running normally. (This would be an indication of how many outstanding RPC requests any scheduling effort would select between.) I'll admit (given basic queuing theory) I would have expected these receive queues to be small unless the server is overloaded. Oh, and I now realize my response related to your first idea "Admission" was way off and didn't make much sense. Somehow, I thought receive queue when you were talking about send queue. (Basically, just ignore my response.) However, given the different sizes of RPC replies, it might be hard to come up with a reasonable high water mark for the send queue. Also, the networking code would have to do some sort of upcall to the krpc when the send queue shrinks. (So, still not trivial to implement, I think?) I do agree with Alfred, in that I think you are experiencing nfsd thread starvation and increasing the number of nfsd threads a lot is the simple way to resolve this. rick > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to > "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Thu Feb 26 06:06:29 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C141CEB9 for ; Thu, 26 Feb 2015 06:06:29 +0000 (UTC) Received: from mail-ig0-x233.google.com (mail-ig0-x233.google.com [IPv6:2607:f8b0:4001:c05::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7F5EEDC2 for ; Thu, 26 Feb 2015 06:06:29 +0000 (UTC) Received: by mail-ig0-f179.google.com with SMTP id l13so12118134iga.0 for ; Wed, 25 Feb 2015 22:06:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=4yYx55r02xXGhHJ2izESX8dS9CESHGXbYQbWVbmnFFM=; b=JfgPWA4KRn5+0unqUxa7PAd5ffXUG1roLezhYAwsSu/FZbBSOg3oi3o9FhR6cdgANb nR+cbVg6rbSNrmSmLvfgf9Y24jRVmOCmyQKuNUeIfi9qlpUAKKPeWCV4EGOVCeMZiDu/ ADfZWsq/vnAvYhL/HNmoq3EuulIYRdfnNh2WpuHvHTdiPK5y6LoaLIkTpAPkApiUnm49 SzUH9M5uB8oP0deoMCWH4KkSIPqIzippsldPbE5CBGqy3y8yEggslpLUgMyOIVkKWzux T4GZLWjUoQUGM//XWogE72+LH/kJcDaFMzg31Nwt7qyi1e2VzoaVSoOZVaBzL2SDZI3C uE2w== MIME-Version: 1.0 X-Received: by 10.107.25.72 with SMTP id 69mr9962593ioz.44.1424930788973; Wed, 25 Feb 2015 22:06:28 -0800 (PST) Sender: kob6558@gmail.com Received: by 10.107.174.86 with HTTP; Wed, 25 Feb 2015 22:06:28 -0800 (PST) In-Reply-To: References: Date: Wed, 25 Feb 2015 22:06:28 -0800 X-Google-Sender-Auth: S21pI2em1NreQHv2EBIJ2lfb1II Message-ID: Subject: Re: Cant Access Web site From: Kevin Oberman To: Ercan Deger Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Feb 2015 06:06:29 -0000 On Wed, Feb 25, 2015 at 12:29 PM, Ercan Deger wrote: > Dear All, > > I am using freebsd 7 as router, I have strange problem while accessing a > site > > when I try to access via browser from windows pc behind freebsd kernel nat > waiting and not opening site, I can access site from other places > > I can access via telnet > > [root@proxy ]# telnet 85.96.190.177 81 > Trying 85.96.190.177... > Connected to 85.96.190.177.static.ttnet.com.tr. > Escape character is '^]'. > > > I added second wan (DSL) and route to site from dsl link I can access also > without problem > > tcpdump log is; > > 21:38:00.903848 IP (tos 0x0, ttl 51, id 0, offset 0, flags [DF], proto TCP > (6), length 52) 85.96.xx.xx.81 > 213.14.xx.xx.64886: S, cksum 0x874b > (correct), 2875126544:2875126544(0) ack 2564833079 win 14600 1452,nop,nop,sackOK,nop,wscale 2> > 21:38:00.904910 IP (tos 0x0, ttl 51, id 0, offset 0, flags [DF], proto TCP > (6), length 40) 85.96.xx.xx.81 > 213.14.xx.xx.64883: R, cksum 0xda56 > (correct), 3424964916:3424964916(0) win 0 > 21:38:00.904954 IP (tos 0x0, ttl 51, id 0, offset 0, flags [DF], proto TCP > (6), length 52) 85.96.xx.xx.81 > 213.14.xx.xx.64887: S, cksum 0x8afa > (correct), 2281919230:2281919230(0) ack 4103381824 win 14600 1452,nop,nop,sackOK,nop,wscale 2> > 21:38:01.207088 IP (tos 0x0, ttl 51, id 48134, offset 0, flags [DF], proto > TCP (6), length 40) 85.96.xx.xx.81 > 213.14.xx.xx.64886: ., cksum 0xf175 > (correct), 1:1(0) ack 354 win 3650 > 21:38:01.357109 IP (tos 0x0, ttl 51, id 48135, offset 0, flags [DF], proto > TCP (6), length 57) 85.96.xx.xx.81 > 213.14.xx.xx.64886: P, cksum 0x3198 > (correct), 1:18(17) ack 354 win 3650 > 21:38:01.389613 IP (tos 0x0, ttl 51, id 48136, offset 0, flags [+], proto > TCP (6), length 1492) 85.96.xx.xx.81 > 213.14.xx.xx.64886: . 18:1470(1452) > ack 354 win 3650 > 21:38:01.389701 IP (tos 0x0, ttl 51, id 48136, offset 1472, flags [none], > proto TCP (6), length 28) 85.96.xx.xx > 213.14.xx.xx: tcp > 21:38:01.402883 IP (tos 0x0, ttl 51, id 48137, offset 0, flags [+], proto > TCP (6), length 1492) 85.96.xx.xx.81 > 213.14.xx.xx.64886: . > 1478:2930(1452) ack 354 win 3650 > 21:38:01.403904 IP (tos 0x0, ttl 51, id 48137, offset 1472, flags [none], > proto TCP (6), length 28) 85.96.xx.xx > 213.14.xx.xx: tcp > 21:38:01.417175 IP (tos 0x0, ttl 51, id 48138, offset 0, flags [+], proto > TCP (6), length 1492) 85.96.xx.xx.81 > 213.14.xx.xx.64886: . > 2938:4390(1452) ack 354 win 3650 > 21:38:01.418196 IP (tos 0x0, ttl 51, id 48138, offset 1472, flags [none], > proto TCP (6), length 28) 85.96.xx.xx > 213.14.xx.xx: tcp > 21:38:01.431607 IP (tos 0x0, ttl 51, id 48139, offset 0, flags [+], proto > TCP (6), length 1492) 85.96.xx.xx.81 > 213.14.xx.xx.64886: . > 4398:5850(1452) ack 354 win 3650 > 21:38:01.431657 IP (tos 0x0, ttl 51, id 48139, offset 1472, flags [none], > proto TCP (6), length 28) 85.96.xx.xx > 213.14.xx.xx: tcp > 21:38:01.445760 IP (tos 0x0, ttl 51, id 48140, offset 0, flags [+], proto > TCP (6), length 1492) 85.96.xx.xx.81 > 213.14.xx.xx.64886: . > 5858:7310(1452) ack 354 win 3650 > 21:38:01.445811 IP (tos 0x0, ttl 51, id 48140, offset 1472, flags [none], > proto TCP (6), length 28) 85.96.xx.xx > 213.14.xx.xx: tcp > 21:38:01.460053 IP (tos 0x0, ttl 51, id 48141, offset 0, flags [+], proto > TCP (6), length 1492) 85.96.xx.xx.81 > 213.14.xx.xx.64886: . > 7318:8770(1452) ack 354 win 3650 > 21:38:01.460103 IP (tos 0x0, ttl 51, id 48141, offset 1472, flags [none], > proto TCP (6), length 28) 85.96.xx.xx > 213.14.xx.xx: tcp > 21:38:01.473314 IP (tos 0x0, ttl 51, id 48142, offset 0, flags [+], proto > TCP (6), length 1492) 85.96.xx.xx.81 > 213.14.xx.xx.64886: . > 8778:10230(1452) ack 354 win 3650 > 21:38:01.474336 IP (tos 0x0, ttl 51, id 48142, offset 1472, flags [none], > proto TCP (6), length 28) 85.96.xx.xx > 213.14.xx.xx: tcp > 21:38:01.480460 IP (tos 0x0, ttl 51, id 48143, offset 0, flags [DF], proto > TCP (6), length 664) 85.96.xx.xx.81 > 213.14.xx.xx.64886: FP > 10238:10862(624) ack 354 win 3650 > 21:38:02.044104 IP (tos 0x0, ttl 51, id 48144, offset 0, flags [+], proto > TCP (6), length 1492) 85.96.xx.xx.81 > 213.14.xx.xx.64886: . 18:1470(1452) > ack 354 win 3650 > 21:38:02.044215 IP (tos 0x0, ttl 51, id 48144, offset 1472, flags [none], > proto TCP (6), length 28) 85.96.xx.xx > 213.14.xx.xx: tcp > 21:38:03.063015 IP (tos 0x0, ttl 51, id 48145, offset 0, flags [+], proto > TCP (6), length 1492) 85.96.xx.xx.81 > 213.14.xx.xx.64886: . 18:1470(1452) > ack 354 win 3650 > 21:38:03.064036 IP (tos 0x0, ttl 51, id 48145, offset 1472, flags [none], > proto TCP (6), length 28) 85.96.xx.xx > 213.14.xx.xx: tcp > 21:38:05.103427 IP (tos 0x0, ttl 51, id 48146, offset 0, flags [+], proto > TCP (6), length 1492) 85.96.xx.xx.81 > 213.14.xx.xx.64886: . 18:1470(1452) > ack 354 win 3650 > 21:38:05.104448 IP (tos 0x0, ttl 51, id 48146, offset 1472, flags [none], > proto TCP (6), length 28) 85.96.xx.xx > 213.14.xx.xx: tcp > 21:38:09.193863 IP (tos 0x0, ttl 51, id 48147, offset 0, flags [+], proto > TCP (6), length 1492) 85.96.xx.xx.81 > 213.14.xx.xx.64886: . 18:1470(1452) > ack 354 win 3650 > 21:38:09.193915 IP (tos 0x0, ttl 51, id 48147, offset 1472, flags [none], > proto TCP (6), length 28) 85.96.xx.xx > 213.14.xx.xx: tcp > 21:38:17.363864 IP (tos 0x0, ttl 51, id 48148, offset 0, flags [+], proto > TCP (6), length 1492) 85.96.xx.xx.81 > 213.14.xx.xx.64886: . 18:1470(1452) > ack 354 win 3650 > 21:38:17.364885 IP (tos 0x0, ttl 51, id 48148, offset 1472, flags [none], > proto TCP (6), length 28) 85.96.xx.xx > 213.14.xx.xx: tcp > 21:38:20.954233 IP (tos 0x0, ttl 51, id 7656, offset 0, flags [DF], proto > TCP (6), length 626) 85.96.xx.xx.81 > 213.14.xx.xx.64887: P 1:587(586) ack > 2 win 3650 > 21:38:20.954278 IP (tos 0x0, ttl 51, id 7657, offset 0, flags [DF], proto > TCP (6), length 40) 85.96.xx.xx.81 > 213.14.xx.xx.64887: F, cksum 0xf439 > (correct), 587:587(0) ack 2 win 3650 > > what can be the problem? > > Best Regards, > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > The data you provided shows little. It would be nice to see both sides of the session. I really suggest that you capture the whole session. Something like "tcpdump -p -s0 -w some_file.bpf tcp port 81 and host 85.96.xx.xx" will save the packet data to "some_file.bpf" and then you can use other tools to analyze it. I use wireshark. It's free, in ports and works quite well. It will flag packets which have issues and handle breaking down the prorocols. Since port 81 is not the "normal" http port, you will need to tell wireshark that. I have not had to do that in a while, but it was pretty obvious. wireshark can also do the capture directly, but I'll admit that after years of using tcpdump (two of the authors used to sit down the hall from me), I have never tried to use wireshark for packet capture. -- Kevin Oberman, Network Engineer, Retired E-mail: rkoberman@gmail.com From owner-freebsd-net@FreeBSD.ORG Thu Feb 26 08:26:53 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D0754F7D for ; Thu, 26 Feb 2015 08:26:53 +0000 (UTC) Received: from mail-la0-x22e.google.com (mail-la0-x22e.google.com [IPv6:2a00:1450:4010:c03::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 08D26E13 for ; Thu, 26 Feb 2015 08:26:53 +0000 (UTC) Received: by labhv19 with SMTP id hv19so9322424lab.10 for ; Thu, 26 Feb 2015 00:26:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=6KNdXXC782o8hl/Iqf90+8gA7nUuPaEt5KoJF/IaAgs=; b=0Aq80dig0uwDus22E7BYQfB5ijz4jeiS2YwOwxi60Iyap0FAhWYEt3c1e/WTBRIdKC QYFN+yZkNh5oTH/SJ0corl8HueEqW94SK/XxQOkmu4G/2oiXCv2/jIsJ+bp70OhtYkSi TzTJA8EGda3gkuT33AhOSMGwkmt/L8aJ1gVXPQ6kJCbDMctPZofQMp1iiRj88zwo9BKe EGR4Yf0o8hJ/6E9MtY5xU+EAXhkvZaM6R38PPnNzOO1mmd9k2mRLXZfH+A0sc3QA8pu4 o0E4iFomC8L71XaUMbg16sJzCzTf4fs0tKpg2jbUrmJ4SEtTa0FtfOUcOJeS1Pch/E/I dS1g== MIME-Version: 1.0 X-Received: by 10.112.53.137 with SMTP id b9mr6408726lbp.66.1424939211195; Thu, 26 Feb 2015 00:26:51 -0800 (PST) Received: by 10.112.136.5 with HTTP; Thu, 26 Feb 2015 00:26:50 -0800 (PST) In-Reply-To: References: Date: Thu, 26 Feb 2015 10:26:50 +0200 Message-ID: Subject: Re: Cant Access Web site From: Ercan Deger To: Kevin Oberman Content-Type: multipart/mixed; boundary=001a11c3a2c282d52c050ff983f0 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Feb 2015 08:26:53 -0000 --001a11c3a2c282d52c050ff983f0 Content-Type: text/plain; charset=UTF-8 I added captured packed, when I open this file via wireshark It shows lots of Fragmented IP Protocol (proto=TCP 6, off=0, id=cbxx) when I change the outside interface mtu to 1492 I can access via squid this site but still cant access without squid I think MTU specific problem but cant solve :( >> > The data you provided shows little. It would be nice to see both sides of > the session. > > I really suggest that you capture the whole session. Something like > "tcpdump -p -s0 -w some_file.bpf tcp port 81 and host 85.96.xx.xx" will > save the packet data to "some_file.bpf" and then you can use other tools to > analyze it. I use wireshark. It's free, in ports and works quite well. It > will flag packets which have issues and handle breaking down the prorocols. > Since port 81 is not the "normal" http port, you will need to tell > wireshark that. I have not had to do that in a while, but it was pretty > obvious. > > wireshark can also do the capture directly, but I'll admit that after > years of using tcpdump (two of the authors used to sit down the hall from > me), I have never tried to use wireshark for packet capture. > -- > Kevin Oberman, Network Engineer, Retired > E-mail: rkoberman@gmail.com > --001a11c3a2c282d52c050ff983f0 Content-Type: application/octet-stream; name="dumped.bpf" Content-Disposition: attachment; filename="dumped.bpf" Content-Transfer-Encoding: base64 X-Attachment-Id: f_i6lvi90d0 1MOyoQIABAAAAAAAAAAAAP//AAABAAAAZsXuVFeADQA8AAAAPAAAAAAEdsgDBwAaTU98EQgARQAA KGehQACABr2qwKgAylVgvrHIoQBRy4rhvnzAS2xQEUE1WrEAAAAAAAAAAGbF7lT+hQ0AQgAAAEIA AAAABHbIAwcAGk1PfBEIAEUAADRnokAAgAa9ncCoAMpVYL6xyLgAUYdTjN8AAAAAgAIgAJxVAAAC BAW0AQMDAgEBBAJmxe5UDrENADYAAAA2AAAAABpNT3wRAAR2yAMHCABFAAAoAABAADUGcExVYL6x wKgAygBRyKF8wEtsAAAAAFAEAABJPQAAZsXuVNq2DQBCAAAAQgAAAAAaTU98EQAEdsgDBwgARQAA NAAAQAA1BnBAVWC+scCoAMoAUci4e1aSoodTjOCAEjkIdUsAAAIEBawBAQQCAQMDAmbF7lQFuA0A PAAAADwAAAAABHbIAwcAGk1PfBEIAEUAAChno0AAgAa9qMCoAMpVYL6xyLgAUYdTjOB7VpKjUBBB Oq3eAAAAAAAAAABmxe5UV74NAOEBAADhAQAAAAR2yAMHABpNT3wRCABFAAHTZ6RAAIAGu/zAqADK VWC+sci4AFGHU4zge1aSo1AYQTrprwAAR0VUIC9sb2dpbi5yc3AgSFRUUC8xLjENCkhvc3Q6IDg1 Ljk2LjE5MC4xNzc6ODENCkNvbm5lY3Rpb246IGtlZXAtYWxpdmUNCkNhY2hlLUNvbnRyb2w6IG1h eC1hZ2U9MA0KQWNjZXB0OiB0ZXh0L2h0bWwsYXBwbGljYXRpb24veGh0bWwreG1sLGFwcGxpY2F0 aW9uL3htbDtxPTAuOSxpbWFnZS93ZWJwLCovKjtxPTAuOA0KVXNlci1BZ2VudDogTW96aWxsYS81 LjAgKFdpbmRvd3MgTlQgNi4zOyBXT1c2NCkgQXBwbGVXZWJLaXQvNTM3LjM2IChLSFRNTCwgbGlr ZSBHZWNrbykgQ2hyb21lLzQwLjAuMjIxNC4xMTUgU2FmYXJpLzUzNy4zNg0KQWNjZXB0LUVuY29k aW5nOiBnemlwLCBkZWZsYXRlLCBzZGNoDQpBY2NlcHQtTGFuZ3VhZ2U6IHRyLVRSLHRyO3E9MC44 LGVuLVVTO3E9MC42LGVuO3E9MC40DQpDb29raWU6IGxhbmd1YWdlX2NoPWZhbHNlDQoNCmbF7lRb 8w0ANgAAADYAAAAAGk1PfBEABHbIAwcIAEUAACiNwkAANQbiiVVgvrHAqADKAFHIuHtWkqOHU46L UBAPTt4fAABnxe5UEpUIAEcAAABHAAAAABpNT3wRAAR2yAMHCABFAAA5jcNAADUG4ndVYL6xwKgA ygBRyLh7VpKjh1OOi1AYD04eQgAASFRUUC8xLjAgMjAwIE9LDQpnxe5U3QkJAOIFAADiBQAAABpN T3wRAAR2yAMHCABFAAXUjcQgADUG/NtVYL6xwKgAygBRyLh7VpK0h1OOi1AQD071ugAAQ29udGVu dC10eXBlOiB0ZXh0L2h0bWw7IGNoYXJzZXQ9VVRGLTgKU2VydmVyOiBHTlUgcnNwLzEuMApDb250 ZW50LUxhbmd1YWdlOiBlbgpFeHBpcmVzOiAtMQpQM1A6IENQPSdJREMgRFNQIENPUiBBRE0gREVW aSBUQUlpIFBTQSBQU0QgSVZBaSBJVkRpIENPTmkgSElTIE9VUiBJTkQgQ05UJwpDYWNoZS1Db250 cm9sOiBwcml2YXRlLCBtYXgtYWdlPTAKQ29ubmVjdGlvbjogY2xvc2UKRGF0ZTogVGh1LCAyNiBG ZWIgMjAxNSAwNzowMjozMCBHTVQKCu+7vzwhRE9DVFlQRSBodG1sIFBVQkxJQyAiLS8vVzNDLy9E VEQgWEhUTUwgMS4wIFRyYW5zaXRpb25hbC8vRU4iICJodHRwOi8vd3d3LnczLm9yZy9UUi94aHRt bDEvRFREL3hodG1sMS10cmFuc2l0aW9uYWwuZHRkIj4NCjxodG1sIHhtbG5zPSJodHRwOi8vd3d3 LnczLm9yZy8xOTk5L3hodG1sIj4NCjxoZWFkPg0KICAgIDx0aXRsZT5EVlIgR8SwUsSwxZ48L3Rp dGxlPg0KICAgIDxsaW5rIGhyZWY9Ii4vSEFQUExFL3BhZ2VzL1N0eWxlcy9Mb2dpbk5ldy5jc3M/ djEiIHJlbD0ic3R5bGVzaGVldCIgdHlwZT0idGV4dC9jc3MiIC8+DQogICAgPHNjcmlwdCBzcmM9 Ii4vSEFQUExFL2pzL2pxdWVyeS0xLjcuMS5taW4uanMiIHR5cGU9InRleHQvamF2YXNjcmlwdCI+ PC9zY3JpcHQ+DQo8c2NyaXB0IHR5cGU9InRleHQvamF2YXNjcmlwdCIgc3JjPSIuL0hBUFBMRS9q cy9yc3BGcmFtZS5qcz92MTk1Ij48L3NjcmlwdD4NCiAgICA8c2NyaXB0IHNyYz0iLi9IQVBQTEUv cGFnZXMvSlMvbG9naW4uanM/VjE2MSIgdHlwZT0idGV4dC9qYXZhc2NyaXB0Ij48L3NjcmlwdD4N CiAgICA8c2NyaXB0IHR5cGU9InRleHQvamF2YXNjcmlwdCIgbGFuZ3VhZ2U9ImphdmFzY3JpcHQi Pg0KICAgICAgICBmdW5jdGlvbiBJRU9SRk9SRUZPWF9sb2coKSB7DQogICAgICAgICAgICB2YXIg aGVpZ2h0ID0gMDsNCiAgICAgICAgICAgIGlmICh3aW5kb3cuWE1MSHR0cFJlcXVlc3QpLy9EZXRl cm1pbmUgd2hldGhlciB0aGUgYnJvd3NlciBNb3ppbGxhLCBTb2ZhcmkNCiAgICAgICAgICAgIHsN CiAgICAgICAgICAgICAgICBoZWlnaHQgPSAoJCh3aW5kb3cpLmhlaWdodCgpIC0gMzk2KSAvIDI7 DQogICAgICAgICAgICAgICAgJCgiLmRpdkxvZ2luT25lIikuY3NzKCJtYXJnaW4tdG9wIiwgaGVp Z2h0ICsgInB4Iik7DQogICAgICAgICAgICB9DQogICAgICAgICAgICBlbHNlIGlmICh3aW5kb3cu QWN0aXZlWE9iamVjdCkvLyBEZXRlcm1pbmUgd2hldGhlciBJRSBicm93c2VyDQogICAgICAgICAg ICB7DQogICAgICAgICAgICAgICAgdmFyIGJyb3dzZXIgPSBuYXZpZ2F0b3IuYXBwTmFtZTsNCiAg ICAgICAgICAgICAgICB2YXIgYl92ZXJzaW9uID0gbmF2aWdhdG9yLmFwcFZlcnNpb247DQogICAg ICAgICAgICAgICAgdmFyIHZlcnNpb24gPSBiX3ZlcnNpb24uc3BsaXQoIjsiKTsNCiAgICAgICAg ICAgICAgICB2YXIgdHJpbV9WZXJzZ8XuVIFACQDiBQAA4gUAAAAaTU98EQAEdsgDBwgARQAF1I3F IAA1BvzaVWC+scCoAMoAUci4e1aYaIdTjotQEA9O5HoAAHJzaW9uWzFdLnJlcGxhY2UoL1sgXS9n LCAiIik7DQogICAgICAgICAgICAgICAgaGVpZ2h0ID0gKCQod2luZG93KS5oZWlnaHQoKSAtIDM5 NikgLyAyOw0KICAgICAgICAgICAgICAgIGlmIChicm93c2VyID09ICJNaWNyb3NvZnQgSW50ZXJu ZXQgRXhwbG9yZXIiICYmIHRyaW1fVmVyc2lvbiA9PSAiTVNJRTYuMCIpIHsNCiAgICAgICAgICAg ICAgICAgICAgalF1ZXJ5KCIuZGl2TG9naW5PbmUiKS5jc3MoIm1hcmdpbi10b3AiLCBoZWlnaHQg KyAicHgiKTsNCiAgICAgICAgICAgICAgICB9DQogICAgICAgICAgICAgICAgZWxzZSB7DQogICAg ICAgICAgICAgICAgICAgICQoIiNkaXZPbmUiKS5jc3MoIm1hcmdpbi10b3AiLCBoZWlnaHQgLSAx NSArICJweCIpOw0KICAgICAgICAgICAgICAgIH0NCiAgICAgICAgICAgIH0NCiAgICAgICAgfQ0K ICAgICAgICB3aW5kb3cub25yZXNpemUgPSBmdW5jdGlvbigpIHtzZXRUaW1lb3V0KCJJRU9SRk9S RUZPWF9sb2coKSIsMTAwKTtwYWdlT25TaXplTW92ZU1lbnUoKTt9Ow0KDQogICAgICAgIGZ1bmN0 aW9uIHN1Ym1pdCgpIHsNCiAgICAgICAgICAgIGRvY3VtZW50LmdldEVsZW1lbnRCeUlkKCJsb2dp bmZvcm0iKS5zdWJtaXQoKTsgcmV0dXJuIGZhbHNlOw0KICAgICAgICB9DQogICAgICAgICQoZG9j dW1lbnQpLmtleWRvd24oZnVuY3Rpb24oZXZlbnQpIHsgICAvL0VudGVyIGV2ZW50DQogICAgICAg ICAgICB2YXIga2V5Y29kZSA9IGV2ZW50LndoaWNoOw0KICAgICAgICAgICAgaWYgKGtleWNvZGUg PT0gMTMpIHsNCgkJCQkNCgkJCQlpZihkb2N1bWVudC5nZXRFbGVtZW50QnlJZCgidXNlcnB3ZDEi KSl7ZG9jdW1lbnQuZ2V0RWxlbWVudEJ5SWQoInVzZXJwd2QxIikuaWQ9InVzZXJwd2QiO30NCgkJ CQlpZihkb2N1bWVudC5nZXRFbGVtZW50QnlJZCgidXNlcnB3ZCIpLnZhbHVlPT0nUGFyb2xhJykN CgkJCQl7DQoJCQkJICAgICQoIi5kaXZfcmlnaHRfdHdvX2lucHV0X2JvcmRlciIpLmh0bWwoIjxp bnB1dCB0eXBlPSdwYXNzd29yZCcgbWF4bGVuZ3RoPScxNicgaWQ9J3VzZXJwd2QnIG5hbWU9J3Vz ZXJwd2QnIHZhbHVlPScnIC8+Iik7DQoJCQkJfQ0KCQkJCWlmKGRvY3VtZW50LmdldEVsZW1lbnRC eUlkKCJ1c2VybmFtZSIpLnZhbHVlPT0nS3VsbGFuxLFjxLEgQWTEsScpDQoJCQkJew0KCQkJCQlk b2N1bWVudC5nZXRFbGVtZW50QnlJZCgidXNlcm5hbWUiKS52YWx1ZT0iIjsNCgkJCQl9DQoNCgkJ CQlpZihqUXVlcnkoIiN1c2VybmFtZSIpLnZhbCgpIT0iS3VsbGFuxLFjxLEgQWTEsSImJmpRdWVy eSgiI3VzZXJwd2QiKS52YWwoKSE9IlBhcm9sYSIpew0KICAgICAgICAgICAgICAgIAlkb2N1bWVu dC5nZXRFbGVtZW50QnlJZCgibG9naW5mb3JtIikuc3VibWl0KCk7DQoJCQkJfQ0KICAgICAgICAg ICAgfQ0KICAgICAgICB9KTsNCgkJDQogICAgPC9zY3JpcHQ+DQo8L2hlYWQ+DQo8IS0tW2lmIElF IDZdPmfF7lTNWwkAPAAAADwAAAAABHbIAwcAGk1PfBEIAEUAAChnpUAAgAa9psCoAMpVYL6xyLgA UYdTjot7VpK0UBBBNawnAAAAAAAAAABnxe5UJ3cJAOIFAADiBQAAABpNT3wRAAR2yAMHCABFAAXU jcYgADUG/NlVYL6xwKgAygBRyLh7Vp4ch1OOi1AQD054PAAAdCBsYW5ndWFnZT0iSmF2YVNjcmlw dCIgdHlwZT0idGV4dC9qYXZhc2NyaXB0IiBzcmM9Ii4vSEFQUExFL3BhZ2VzL0pTL19wbmcuanMi Pjwvc2NyaXB0Pg0KPHNjcmlwdD4NCkREX2JlbGF0ZWRQTkcuZml4KCdpbWcsLnBuZywubWVudSBs aScpOw0KPC9zY3JpcHQ+DQo8IVtlbmRpZl0tLT4NCjxib2R5IGlkPSJsb2dpbl9ib2R5IiBvbmxv YWQ9IklFT1JGT1JFRk9YX2xvZygpIj4NCg0KPGZvcm0gbmFtZT0ibG9naW5mb3JtIiBpZD0ibG9n aW5mb3JtIiBhY3Rpb249Ii4vbG9naW5jaGVjay5yc3A/dHlwZT0xIiBtZXRob2Q9InBvc3QiIG9u c3VibWl0PSJyZXR1cm4gUk0uTE9HSU4uY2hlY2tMb2dpbklucHV0KCkiPg0KICAgIDxkaXYgY2xh c3M9ImRpdkxvZ2luT25lIHBuZyIgaWQ9ImRpdk9uZSI+DQogICAgICAgIDxkaXYgY2xhc3M9ImRp dl9sZWZ0Qm90dG9tIj4NCiAgICAgICAgICAgIDxkaXYgY2xhc3M9J2xvZy1kdnItbG9nbyc+DQog ICAgICAgICAgICAgICAgPGRpdiBzdHlsZT0iIHBhZGRpbmctdG9wOjI2cHg7IHRleHQtYWxpZ246 Y2VudGVyOyB3aWR0aDoxMDAlIj4NCiAgICAgICAgICAgICAgICAgICAgPGltZyBpZD0ibG9naW5f bG9nb2ltZyIgc3JjPSIuL0hBUFBMRS9sb2dpbmxvZ28vSUVfMDQuNzk4X2xvZ2luLnBuZyIgY2xh c3M9InBuZyIgLz4NCiAgICAgICAgICAgICAgICA8L2Rpdj4NCiAgICAgICAgICAgIDwvZGl2Pg0K ICAgICAgICAgICAgPGRpdiBjbGFzcz0nbG9nLWxhbmd1YWdlJz48bGFiZWwgaWQ9J2xvZ19sYW5n dWFnZV9ib3JkZXInIGNsYXNzPSdsb2ctbGFuZ3VhZ2Utb3V0JyAgb25tb3VzZW92ZXI9J21vdXNl T25MYWJlbCgpOycgb25tb3VzZW91dD0ibW91c2VMZWF2ZUxhYmVsKCk7Ij5EaWwgc2XDp2ltaTwv bGFiZWw+PC9kaXY+DQogICAgICAgICAgIA0KICAgICAgICA8L2Rpdj4NCiAgICAgICAgPGRpdiBj bGFzcz0iZGl2X2xvZ2luX3JpZ2h0Ij4NCiAgICAgICAgICAgIDxkaXYgY2xhc3M9ImRpdl9yaWdo dF90b3BfZm9udCI+DQogICAgICAgICAgICAgICAgPGRpdiBjbGFzcz0iZGl2X3JpZ2h0X3RpdGxl X2ZvbnQiPg0KICAgICAgICAgICAgICAgICAgICA8Yj5EVlIgR8SwUsSwxZ48L2I+DQogICAgICAg ICAgICAgICAgPC9kaXY+DQogICAgICAgICAgICA8L2Rpdj4NCiAgICAgICAgICAgIDxkaXYgY2xh c3M9ImRpdl9yaWdodF9vbmUgcG5nIj4NCiAgICAgICAgICAgICAgICA8ZGl2IGNsYXNzPSJkaXZf cmlnaHRfdHdvX3VuYW1lX2JvcmRlciI+DQogICAgICAgICAgICAgICAgICAgIDxpbnB1dCB0eXBl PSJ0ZXh0IiBpZD0idXNlcm5hbWUiICBuYW1lPSJ1c2VybmFtZSIgbWF4bGVuZ3RoPSIxNiIgc3R5 bGU9ImNvbG9yOmdyYXk7IGZvbnQtc3R5bGU6aXRhbGljOyIgdmFsdWU9Ikt1bGxhbsSxY8SxIEFk xLEiIG9uZm9jdXM9ImlmKHRoaXMudmFsdWU9PSdLdWxsYW7EsWPEsSBBZMSxJyl7dGhpcy52YWx1 ZT0nJ31lZ8XuVLaxCQDiBQAA4gUAAAAaTU98EQAEdsgDBwgARQAF1I3HIAA1BvzYVWC+scCoAMoA Uci4e1aj0IdTjotQEA9OXpgAAC5zZWxlY3QoKTt9O3RoaXMuc3R5bGUuY29sb3I9J2JsYWNrJzt1 cGRhdGVGb250U3R5bGUodGhpcy5pZCk7IiBvbmJsdXI9InVwZGF0ZVBhc3N3b3JkKCkiLz48L2Rp dj4NCiAgICAgICAgICAgICAgICA8L2Rpdj4NCiAgICAgICAgICAgIDxkaXYgY2xhc3M9ImRpdl9y aWdodF90d28gcG5nIj4NCiAgICAgICAgICAgICAgICA8ZGl2IGNsYXNzPSJkaXZfcmlnaHRfdHdv X2lucHV0X2JvcmRlciI+DQogICAgICAgICAgICAgICAgICAgIDxpbnB1dCB0eXBlPSJ0ZXh0IiBp ZD0idXNlcnB3ZDEiIG5hbWU9InVzZXJwd2QiIG1heGxlbmd0aD0iMTYiIHN0eWxlPSJjb2xvcjpn cmF5OyBmb250LXN0eWxlOml0YWxpYzsiIHZhbHVlPSJQYXJvbGEiIG9uZm9jdXM9InVwZGF0ZVBh c3N3b3JkKCk7IiAvPg0KICAgICAgICAgICAgICAgIDwvZGl2Pg0KICAgICAgICAgICAgICAgIDxk aXYgY2xhc3M9ImRpdl9yaWdodF90aGlyZSBwbmciIG9ubW91c2Vkb3duPSJ0aGlzLmNsYXNzTmFt ZT0nZGlkaXZfcmlnaHRfdGhpcmVfVHdvJzt1cGRhdGVQYXNzd29yZCgpO3N1Ym1pdCgpIiBvbm1v dXNldXA9InRoaXMuY2xhc3NOYW1lPSdkaXZfcmlnaHRfdGhpcmUnIj48L2Rpdj4NCiAgICAgICAg ICAgIDwvZGl2Pg0KICAgICAgICAgICAgPGRpdiBjbGFzcz0iZGl2X3JpZ2h0X2xhc3QiPg0KICAg ICAgICAgICAgICAgIDxpbnB1dCBpZD0iYXV0b2xvZ2luIiBuYW1lPSJhdXRvbG9naW4iIHR5cGU9 ImNoZWNrYm94IiBvbmZvY3VzPSJ0aGlzLmJsdXIoKSIgdmFsdWU9IjEiLz5CZW5pIGhhdMSxcmxh DQogICAgICAgICAgICA8L2Rpdj4NCiAgICAgICAgPC9kaXY+DQogICAgICAgIDxkaXYgY2xhc3M9 ImNsZWFyIj48L2Rpdj4NCiAgICA8L2Rpdj4NCjwvZm9ybT4NCjxzY3JpcHQgdHlwZT0idGV4dC9q YXZhc2NyaXB0Ij4NCglmdW5jdGlvbiB1cGRhdGVQYXNzd29yZCgpew0KCQkNCgkJaWYoZG9jdW1l bnQuZ2V0RWxlbWVudEJ5SWQoInVzZXJwd2QxIikpe2RvY3VtZW50LmdldEVsZW1lbnRCeUlkKCJ1 c2VycHdkMSIpLmlkPSJ1c2VycHdkIjt9DQoJCWlmKGRvY3VtZW50LmdldEVsZW1lbnRCeUlkKCJ1 c2VycHdkIikudmFsdWU9PSdQYXJvbGEnKQ0KCQl7DQoJCQkkKCIuZGl2X3JpZ2h0X3R3b19pbnB1 dF9ib3JkZXIiKS5odG1sKCI8aW5wdXQgdHlwZT0ncGFzc3dvcmQnIGlkPSd1c2VycHdkJyBuYW1l PSd1c2VycHdkJyBtYXhsZW5ndGg9JzE2JyB2YWx1ZT0nJyAvPiIpOw0KCQl9DQoJCWVsc2UNCgkJ ew0KCQkJJCgiLmRpdl9yaWdodF90d29faW5wdXRfYm9yZGVyIikuaHRtbCgiPGlucHV0IHR5cGU9 J3Bhc3N3b3JkJyBpZD0ndXNlcnB3ZCcgbmFtZT0ndXNlcnB3ZCcgbWF4bGVuZ3RoPScxNicgdmFs dWU9JyIrZG9jdW1lbnQuZ2V0RWxlbWVudEJ5SWQoJ3VzZXJwd2QnKS52YWx1ZSsiJyBvbmZvY3Vz PSd0aGlzLnNlbGVjdCgpJy8+Iik7DQoJCQkNCgkJfQ0KCQlpZihkb2N1bWVudGfF7lRp5gkA4gUA AOIFAAAAGk1PfBEABHbIAwcIAEUABdSNyCAANQb811VgvrHAqADKAFHIuHtWqYSHU46LUBAPTiov AABlbnRCeUlkKCJ1c2VybmFtZSIpLnZhbHVlPT0nS3VsbGFuxLFjxLEgQWTEsScpDQoJCXsNCgkJ CWRvY3VtZW50LmdldEVsZW1lbnRCeUlkKCJ1c2VybmFtZSIpLnZhbHVlPSIiOw0KCQl9DQoJCXNl dFRpbWVvdXQoImRvY3VtZW50LmdldEVsZW1lbnRCeUlkKCd1c2VycHdkJykuZm9jdXMoKSIsMTAw KTsNCgl9DQoJZnVuY3Rpb24gdXBkYXRlRm9udFN0eWxlKGlkKXsNCgkJalF1ZXJ5KCIjIitpZCsi IikuY3NzKCJmb250LXN0eWxlIiwibm9ybWFsIikvL2luaGVyaXQNCgkJLy9kb2N1bWVudC5nZXRF bGVtZW50QnlJZChpZCkuc3R5bGUuZm9uPSJmb250LXN0eWxlOmluaGVyaXQiOw0KCX0NCgkNCgl2 YXIgbGFuQ2FzaCA9IFt7IkNGR19GSUxFIjoidWkvbGFuZ3VhZ2UvY2hpbmVzZS5pbmkiLCJGT05U X0xJQiI6IkZaWWFvVGkiLCJGT05UX0xJQl9GSUxFIjoiZm9udHMvRlpZVEsuVFRGIiwiSUNPTl9E SVIiOiJ1aS9pY29ucy9jaGluZXNlIiwiTEFORyI6IuS4reaWhyIsIlNFTEVDVCI6ImZhbHNlIiwi VkFMVUUiOjB9LHsiQ0ZHX0ZJTEUiOiJ1aS9sYW5ndWFnZS9lbmdsaXNoLmluaSIsIkZPTlRfTElC IjoiQ0FMSUJSSSIsIkZPTlRfTElCX0ZJTEUiOiJmb250cy9DQUxJQlJJLlRURiIsIkxBTkciOiJF bmdsaXNoIiwiU0VMRUNUIjoidHJ1ZSIsIlZBTFVFIjoxfSx7IkNGR19GSUxFIjoidWkvbGFuZ3Vh Z2Uva29yZWFuLmluaSIsIkZPTlRfTElCIjoiTmV3IEd1bGltIiwiRk9OVF9MSUJfRklMRSI6ImZv bnRzL05ld0d1bGltLnR0ZiIsIklDT05fRElSIjoidWkvaWNvbnMva29yZWFuIiwiTEFORyI6Iu2V nOq1reyWtCIsIlNFTEVDVCI6ImZhbHNlIiwiVkFMVUUiOjJ9LHsiQ0ZHX0ZJTEUiOiJ1aS9sYW5n dWFnZS9pdGFsaWFuLmluaSIsIkZPTlRfTElCIjoiQ0FMSUJSSSIsIkZPTlRfTElCX0ZJTEUiOiJm b250cy9DQUxJQlJJLlRURiIsIklDT05fRElSIjoidWkvaWNvbnMvaXRhbGlhbiIsIkxBTkciOiJJ dGFsaWFubyIsIlNFTEVDVCI6ImZhbHNlIiwiVkFMVUUiOjN9LHsiQ0ZHX0ZJTEUiOiJ1aS9sYW5n dWFnZS9nZXJtYW4uaW5pIiwiRk9OVF9MSUIiOiJDQUxJQlJJIiwiRk9OVF9MSUJfRklMRSI6ImZv bnRzL0NBTElCUkkuVFRGIiwiSUNPTl9ESVIiOiJ1aS9pY29ucy9nZXJtYW4iLCJMQU5HIjoiRGV1 dHNjaCIsIlNFTEVDVCI6InRydWUiLCJWQUxVRSI6NH0seyJDRkdfRklMRSI6InVpL2xhbmd1YWdl L3RoYWkuaW5pIiwiRk9OVF9MSUIiOiJUYWhvbWEiLCJGT05UX0xJQl9GSUxFIjoiZm9udHMvdGFo b21hLnR0ZiIsIklDT05fRElSIjoidWkvaWNvbnMvdGhhaSIsIkxBTkciOiLguYTguJfguKIiLCJT RUxFQ1QiOiJmYWxzZSIsIlZBTFVFIjo1fSx7IkNGR19GSUxFIjoidWkvbGFuZ3VhZ2UvdHVya2lz aC5pbmkiLCJGT05UX0xJQiI6IkNBTElCUkkiLCJGT05UX0xJQl9GSUxFIjoiZm9udHMvQ0FMSUJS SS5UVEYiLCJJQ09OX0RJUiI6InVpL2ljb25zL3Rnxe5UDx0KAOIFAADiBQAAABpNT3wRAAR2yAMH CABFAAXUjckgADUG/NZVYL6xwKgAygBRyLh7Vq84h1OOi1AQD07WvQAAIkxBTkciOiJUw7xya8On ZSIsIlNFTEVDVCI6InRydWUiLCJWQUxVRSI6Nn0seyJDRkdfRklMRSI6InVpL2xhbmd1YWdlL3Bv cnR1Z3Vlc2UuaW5pIiwiRk9OVF9MSUIiOiJDQUxJQlJJIiwiRk9OVF9MSUJfRklMRSI6ImZvbnRz L0NBTElCUkkuVFRGIiwiSUNPTl9ESVIiOiJ1aS9pY29ucy9wb3J0dWd1ZXNlIiwiTEFORyI6IlBv cnR1Z3XDqnMiLCJTRUxFQ1QiOiJmYWxzZSIsIlZBTFVFIjo3fSx7IkNGR19GSUxFIjoidWkvbGFu Z3VhZ2Uvc3Bhbmxpc2guaW5pIiwiRk9OVF9MSUIiOiJDQUxJQlJJIiwiRk9OVF9MSUJfRklMRSI6 ImZvbnRzL0NBTElCUkkuVFRGIiwiSUNPTl9ESVIiOiJ1aS9pY29ucy9zcGFubGlzaCIsIkxBTkci OiJFc3Bhw7FvbCIsIlNFTEVDVCI6ImZhbHNlIiwiVkFMVUUiOjh9LHsiQ0ZHX0ZJTEUiOiJ1aS9s YW5ndWFnZS9yb21hbmlhbi5pbmkiLCJGT05UX0xJQiI6IkNBTElCUkkiLCJGT05UX0xJQl9GSUxF IjoiZm9udHMvQ0FMSUJSSS5UVEYiLCJJQ09OX0RJUiI6InVpL2ljb25zL3JvbWFuaWFuIiwiTEFO RyI6IlJvbcOibsSDIiwiU0VMRUNUIjoiZmFsc2UiLCJWQUxVRSI6OX0seyJDRkdfRklMRSI6InVp L2xhbmd1YWdlL2dyZWVrLmluaSIsIkZPTlRfTElCIjoiQ0FMSUJSSSIsIkZPTlRfTElCX0ZJTEUi OiJmb250cy9DQUxJQlJJLlRURiIsIkxBTkciOiLOlc67zrvOt869zrnOus6sIiwiU0VMRUNUIjoi ZmFsc2UiLCJWQUxVRSI6MTB9LHsiQ0ZHX0ZJTEUiOiJ1aS9sYW5ndWFnZS9mcmVuY2guaW5pIiwi Rk9OVF9MSUIiOiJDQUxJQlJJIiwiRk9OVF9MSUJfRklMRSI6ImZvbnRzL0NBTElCUkkuVFRGIiwi SUNPTl9ESVIiOiJ1aS9pY29ucy9mcmVuY2giLCJMQU5HIjoiRnJhbsOnYWlzIiwiU0VMRUNUIjoi dHJ1ZSIsIlZBTFVFIjoxMX0seyJDRkdfRklMRSI6InVpL2xhbmd1YWdlL3J1c3NpYW4uaW5pIiwi Rk9OVF9MSUIiOiJDQUxJQlJJIiwiRk9OVF9MSUJfRklMRSI6ImZvbnRzL0NBTElCUkkuVFRGIiwi SUNPTl9ESVIiOiJ1aS9pY29ucy9ydXNzaWFuIiwiTEFORyI6ItCg0YPRgdGB0LrQuNC5IiwiU0VM RUNUIjoiZmFsc2UiLCJWQUxVRSI6MTJ9LHsiQ0ZHX0ZJTEUiOiJ1aS9sYW5ndWFnZS9uZWRlcmxh bmRzLmluaSIsIkZPTlRfTElCIjoiQ0FMSUJSSSIsIkZPTlRfTElCX0ZJTEUiOiJmb250cy9DQUxJ QlJJLlRURiIsIkxBTkciOiJOZWRlcmxhbmRzIiwiU0VMRUNUIjoiZmFsc2UiLCJWQUxVRSI6MTN9 LHsiQ0ZHX0ZJTEUiOiJ1aS9sYW5ndWFnZS9oZWJyZXcuaW5pIiwiRk9OVF9MSUIiOiJBcmlhbCIs IkZPTlRfTElCX0ZJTEUiOiJmb250cy9hcmlhbC50dGYiLCJJQ09OX0RJUiI6InVpL2ljb25zL2hl YnJldyIsIkxBTkciOiLXoteR16jXmdeqIiwiU0VMRUNUIjoiZmFsc2UiLCJWQUxVRSI6MTR9LHsi Q0ZHX0ZJTEUiOiJ1aS9sYW5ndWFnZS9jaGluZXNldHJhZC5pbmkiLCJGT05UX0xJQiI6Im1zZiIs IkZPTlRfTElCX0ZJZ8XuVJJZCgDiBQAA4gUAAAAaTU98EQAEdsgDBwgARQAF1I3KIAA1BvzVVWC+ scCoAMoAUci4e1a07IdTjotQEA9OfcwAAHRzL21zZi50dGYiLCJJQ09OX0RJUiI6InVpL2ljb25z L3RyYWRpdGlvbmFsIiwiTEFORyI6IuS4reaWh+e5geS9kyIsIlNFTEVDVCI6ImZhbHNlIiwiVkFM VUUiOjE1fSx7IkNGR19GSUxFIjoidWkvbGFuZ3VhZ2UvcG9saXNoLmluaSIsIkZPTlRfTElCIjoi Q0FMSUJSSSIsIkZPTlRfTElCX0ZJTEUiOiJmb250cy9DQUxJQlJJLlRURiIsIklDT05fRElSIjoi dWkvaWNvbnMvcG9saXNoIiwiTEFORyI6IlBvbHNraSIsIlNFTEVDVCI6ImZhbHNlIiwiVkFMVUUi OjE2fSx7IkNGR19GSUxFIjoidWkvbGFuZ3VhZ2UvQ2hlY2hueWEuaW5pIiwiRk9OVF9MSUIiOiJD QUxJQlJJIiwiRk9OVF9MSUJfRklMRSI6ImZvbnRzL0NBTElCUkkuVFRGIiwiSUNPTl9ESVIiOiJ1 aS9pY29ucy9DaGVjaG55YSIsIkxBTkciOiLEjGXFoXRpbmEiLCJTRUxFQ1QiOiJmYWxzZSIsIlZB TFVFIjoxN30seyJDRkdfRklMRSI6InVpL2xhbmd1YWdlL3ZpZXRuYW0uaW5pIiwiRk9OVF9MSUIi OiJUaW1lcyIsIkZPTlRfTElCX0ZJTEUiOiJmb250cy9UaW1lcy50dGYiLCJJQ09OX0RJUiI6InVp L2ljb25zL3ZpZXRuYW0iLCJMQU5HIjoiVmnhu4d0IE5hbSIsIlNFTEVDVCI6ImZhbHNlIiwiVkFM VUUiOjE4fSx7IkNGR19GSUxFIjoidWkvbGFuZ3VhZ2UvamFwYW5lc2UuaW5pIiwiRk9OVF9MSUIi OiJNUyBHb3RoaWMiLCJGT05UX0xJQl9GSUxFIjoiZm9udHMvTVMtR290aGljLnR0ZiIsIklDT05f RElSIjoidWkvaWNvbnMvSmFwYW5lc2UiLCJMQU5HIjoi5pel5pys6KqeIiwiU0VMRUNUIjoiZmFs c2UiLCJWQUxVRSI6MTl9LHsiQ0ZHX0ZJTEUiOiJ1aS9sYW5ndWFnZS9QZXJzaWFuLmluaSIsIkZP TlRfTElCIjoiQXJpYWwiLCJGT05UX0xJQl9GSUxFIjoiZm9udHMvYXJpYWwudHRmIiwiSUNPTl9E SVIiOiJ1aS9pY29ucy9QZXJzaWFuIiwiTEFORyI6ItmB2KfYsdiz24wiLCJTRUxFQ1QiOiJmYWxz ZSIsIlZBTFVFIjoyMH0seyJDRkdfRklMRSI6InVpL2xhbmd1YWdlL2FyYXBjYS5pbmkiLCJGT05U X0xJQiI6IkFyaWFsIiwiRk9OVF9MSUJfRklMRSI6ImZvbnRzL2FyaWFsLnR0ZiIsIklDT05fRElS IjoidWkvaWNvbnMvYXJhcGNhIiwiTEFORyI6Itin2YTYudix2KjZitipIiwiU0VMRUNUIjoidHJ1 ZSIsIlZBTFVFIjoyMX1dCjsNCgl2YXIgcHJvZHVjdF90eXBlPSc3MTE2WFFfVkEnOw0KCWN1c3RO YW1lPScxMDQnOw0KDQoJdmFyIHJpbmcgPSBmYWxzZTsNCiAgICB2YXIgbGFuZ3VhZ2VBcnIgPSBb XTsNCiAgICBmb3IgKHZhciBpID0gMDsgaSA8IGxhbkNhc2gubGVuZ3RoOyBpKyspIHsNCiAgICAg ICAgaWYgKGxhbkNhc2hbaV0uU0VMRUNUICsgIiIgPT0gInRydWUiKSB7DQoJCQlpZighcmluZykN CgkJCXsNCgkJCQlpZigobGFuQ2FzaFtpXS5MQU5HID09ICLkuK3mlociICYmIChwcm9kdWN0X3R5 cGUuc3BsaXQoIl8iKVswXSE9IjcyMTZYRyIpICYmIChwcm9kdWN0X3R5cGUuc3BsaXQoImfF7lTV dgoAzAIAAMwCAAAAGk1PfBEABHbIAwcIAEUAAr6Ny0AANQbf6lVgvrHAqADKAFHIuHtWuqCHU46L UBkPTrMsAAAiODYwOFhHIikgJiYgKHByb2R1Y3RfdHlwZS5zcGxpdCgiXyIpWzBdIT0iODYwNFhH IikpIHx8IChsYW5DYXNoW2ldLlZBTFVFID09IDE1ICYmIGN1c3ROYW1lID09ICIxMDMiKSkNCgkJ CQl7DQoJCQkJCXJpbmcgPSB0cnVlOw0KCQkJCQlyc3Bfc2V0Q29va2llKCJsYW5ndWFnZV9jaCIs InRydWUiKTsNCgkJCQl9DQoJCQkJZWxzZQ0KCQkJCXsNCgkJCQkJcnNwX3NldENvb2tpZSgibGFu Z3VhZ2VfY2giLCJmYWxzZSIpOw0KCQkJCX0NCgkJCX0NCiAgICAgICAgICAgIGxhbmd1YWdlQXJy W2xhbmd1YWdlQXJyLmxlbmd0aF0gPSB7InRleHQiOmxhbkNhc2hbaV0uTEFORywidmFsdWUiOmxh bkNhc2hbaV0uVkFMVUV9Oy8vbGFuQWxsW2xhbkNhc2hbaV0uTEFOR107DQoJCQkvL2xhbmd1YWdl QXJyW2xhbmd1YWdlQXJyLmxlbmd0aF0gPXsidGV4dCI6IiIrbGFuQ2FzaFtpXS5MQU5HKyIiLCJ2 YWx1ZSI6bGFuQ2FzaFtpXS5MQU5HfTsNCiAgICAgICAgfQ0KICAgIH0NCiAgICBnX2xhbm1lbnUg PSBuZXcgcnNwTGFuZ3VhZ2VNZW51KCJsYW5ndWFnZSIsICJsYW5ndWFnZSIsIGxhbmd1YWdlQXJy KTsNCiAgICAkKCdib2R5JykuYXBwZW5kKGdfbGFubWVudS5nZXRIdG1sKCkpOw0KPC9zY3JpcHQ+ DQogDQogDQogDQogDQogDQogDQogDQoNCjwvYm9keT4NCjwvaHRtbD4NCmfF7lTmeAoAQgAAAEIA AAAABHbIAwcAGk1PfBEIAEUAADRnpkAAgAa9mcCoAMpVYL6xyLgAUYdTjot7VpK0gBBBNQeMAAAB AQUKe1a6oHtWvTZnxe5U6tMNAOIFAADiBQAAABpNT3wRAAR2yAMHCABFAAXUjcwgADUG/NNVYL6x wKgAygBRyLh7VpK0h1OOi1AQD071ugAAQ29udGVudC10eXBlOiB0ZXh0L2h0bWw7IGNoYXJzZXQ9 VVRGLTgKU2VydmVyOiBHTlUgcnNwLzEuMApDb250ZW50LUxhbmd1YWdlOiBlbgpFeHBpcmVzOiAt MQpQM1A6IENQPSdJREMgRFNQIENPUiBBRE0gREVWaSBUQUlpIFBTQSBQU0QgSVZBaSBJVkRpIENP TmkgSElTIE9VUiBJTkQgQ05UJwpDYWNoZS1Db250cm9sOiBwcml2YXRlLCBtYXgtYWdlPTAKQ29u bmVjdGlvbjogY2xvc2UKRGF0ZTogVGh1LCAyNiBGZWIgMjAxNSAwNzowMjozMCBHTVQKCu+7vzwh RE9DVFlQRSBodG1sIFBVQkxJQyAiLS8vVzNDLy9EVEQgWEhUTUwgMS4wIFRyYW5zaXRpb25hbC8v RU4iICJodHRwOi8vd3d3LnczLm9yZy9UUi94aHRtbDEvRFREL3hodG1sMS10cmFuc2l0aW9uYWwu ZHRkIj4NCjxodG1sIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8xOTk5L3hodG1sIj4NCjxoZWFk Pg0KICAgIDx0aXRsZT5EVlIgR8SwUsSwxZ48L3RpdGxlPg0KICAgIDxsaW5rIGhyZWY9Ii4vSEFQ UExFL3BhZ2VzL1N0eWxlcy9Mb2dpbk5ldy5jc3M/djEiIHJlbD0ic3R5bGVzaGVldCIgdHlwZT0i dGV4dC9jc3MiIC8+DQogICAgPHNjcmlwdCBzcmM9Ii4vSEFQUExFL2pzL2pxdWVyeS0xLjcuMS5t aW4uanMiIHR5cGU9InRleHQvamF2YXNjcmlwdCI+PC9zY3JpcHQ+DQo8c2NyaXB0IHR5cGU9InRl eHQvamF2YXNjcmlwdCIgc3JjPSIuL0hBUFBMRS9qcy9yc3BGcmFtZS5qcz92MTk1Ij48L3Njcmlw dD4NCiAgICA8c2NyaXB0IHNyYz0iLi9IQVBQTEUvcGFnZXMvSlMvbG9naW4uanM/VjE2MSIgdHlw ZT0idGV4dC9qYXZhc2NyaXB0Ij48L3NjcmlwdD4NCiAgICA8c2NyaXB0IHR5cGU9InRleHQvamF2 YXNjcmlwdCIgbGFuZ3VhZ2U9ImphdmFzY3JpcHQiPg0KICAgICAgICBmdW5jdGlvbiBJRU9SRk9S RUZPWF9sb2coKSB7DQogICAgICAgICAgICB2YXIgaGVpZ2h0ID0gMDsNCiAgICAgICAgICAgIGlm ICh3aW5kb3cuWE1MSHR0cFJlcXVlc3QpLy9EZXRlcm1pbmUgd2hldGhlciB0aGUgYnJvd3NlciBN b3ppbGxhLCBTb2ZhcmkNCiAgICAgICAgICAgIHsNCiAgICAgICAgICAgICAgICBoZWlnaHQgPSAo JCh3aW5kb3cpLmhlaWdodCgpIC0gMzk2KSAvIDI7DQogICAgICAgICAgICAgICAgJCgiLmRpdkxv Z2luT25lIikuY3NzKCJtYXJnaW4tdG9wIiwgaGVpZ2h0ICsgInB4Iik7DQogICAgICAgICAgICB9 DQogICAgICAgICAgICBlbHNlIGlmICh3aW5kb3cuQWN0aXZlWE9iamVjdCkvLyBEZXRlcm1pbmUg d2hldGhlciBJRSBicm93c2VyDQogICAgICAgICAgICB7DQogICAgICAgICAgICAgICAgdmFyIGJy b3dzZXIgPSBuYXZpZ2F0b3IuYXBwTmFtZTsNCiAgICAgICAgICAgICAgICB2YXIgYl92ZXJzaW9u ID0gbmF2aWdhdG9yLmFwcFZlcnNpb247DQogICAgICAgICAgICAgICAgdmFyIHZlcnNpb24gPSBi X3ZlcnNpb24uc3BsaXQoIjsiKTsNCiAgICAgICAgICAgICAgICB2YXIgdHJpbV9WZXJzaMXuVGTZ BgDiBQAA4gUAAAAaTU98EQAEdsgDBwgARQAF1I3NIAA1BvzSVWC+scCoAMoAUci4e1aStIdTjotQ EA9O9boAAENvbnRlbnQtdHlwZTogdGV4dC9odG1sOyBjaGFyc2V0PVVURi04ClNlcnZlcjogR05V IHJzcC8xLjAKQ29udGVudC1MYW5ndWFnZTogZW4KRXhwaXJlczogLTEKUDNQOiBDUD0nSURDIERT UCBDT1IgQURNIERFVmkgVEFJaSBQU0EgUFNEIElWQWkgSVZEaSBDT05pIEhJUyBPVVIgSU5EIENO VCcKQ2FjaGUtQ29udHJvbDogcHJpdmF0ZSwgbWF4LWFnZT0wCkNvbm5lY3Rpb246IGNsb3NlCkRh dGU6IFRodSwgMjYgRmViIDIwMTUgMDc6MDI6MzAgR01UCgrvu788IURPQ1RZUEUgaHRtbCBQVUJM SUMgIi0vL1czQy8vRFREIFhIVE1MIDEuMCBUcmFuc2l0aW9uYWwvL0VOIiAiaHR0cDovL3d3dy53 My5vcmcvVFIveGh0bWwxL0RURC94aHRtbDEtdHJhbnNpdGlvbmFsLmR0ZCI+DQo8aHRtbCB4bWxu cz0iaHR0cDovL3d3dy53My5vcmcvMTk5OS94aHRtbCI+DQo8aGVhZD4NCiAgICA8dGl0bGU+RFZS IEfEsFLEsMWePC90aXRsZT4NCiAgICA8bGluayBocmVmPSIuL0hBUFBMRS9wYWdlcy9TdHlsZXMv TG9naW5OZXcuY3NzP3YxIiByZWw9InN0eWxlc2hlZXQiIHR5cGU9InRleHQvY3NzIiAvPg0KICAg IDxzY3JpcHQgc3JjPSIuL0hBUFBMRS9qcy9qcXVlcnktMS43LjEubWluLmpzIiB0eXBlPSJ0ZXh0 L2phdmFzY3JpcHQiPjwvc2NyaXB0Pg0KPHNjcmlwdCB0eXBlPSJ0ZXh0L2phdmFzY3JpcHQiIHNy Yz0iLi9IQVBQTEUvanMvcnNwRnJhbWUuanM/djE5NSI+PC9zY3JpcHQ+DQogICAgPHNjcmlwdCBz cmM9Ii4vSEFQUExFL3BhZ2VzL0pTL2xvZ2luLmpzP1YxNjEiIHR5cGU9InRleHQvamF2YXNjcmlw dCI+PC9zY3JpcHQ+DQogICAgPHNjcmlwdCB0eXBlPSJ0ZXh0L2phdmFzY3JpcHQiIGxhbmd1YWdl PSJqYXZhc2NyaXB0Ij4NCiAgICAgICAgZnVuY3Rpb24gSUVPUkZPUkVGT1hfbG9nKCkgew0KICAg ICAgICAgICAgdmFyIGhlaWdodCA9IDA7DQogICAgICAgICAgICBpZiAod2luZG93LlhNTEh0dHBS ZXF1ZXN0KS8vRGV0ZXJtaW5lIHdoZXRoZXIgdGhlIGJyb3dzZXIgTW96aWxsYSwgU29mYXJpDQog ICAgICAgICAgICB7DQogICAgICAgICAgICAgICAgaGVpZ2h0ID0gKCQod2luZG93KS5oZWlnaHQo KSAtIDM5NikgLyAyOw0KICAgICAgICAgICAgICAgICQoIi5kaXZMb2dpbk9uZSIpLmNzcygibWFy Z2luLXRvcCIsIGhlaWdodCArICJweCIpOw0KICAgICAgICAgICAgfQ0KICAgICAgICAgICAgZWxz ZSBpZiAod2luZG93LkFjdGl2ZVhPYmplY3QpLy8gRGV0ZXJtaW5lIHdoZXRoZXIgSUUgYnJvd3Nl cg0KICAgICAgICAgICAgew0KICAgICAgICAgICAgICAgIHZhciBicm93c2VyID0gbmF2aWdhdG9y LmFwcE5hbWU7DQogICAgICAgICAgICAgICAgdmFyIGJfdmVyc2lvbiA9IG5hdmlnYXRvci5hcHBW ZXJzaW9uOw0KICAgICAgICAgICAgICAgIHZhciB2ZXJzaW9uID0gYl92ZXJzaW9uLnNwbGl0KCI7 Iik7DQogICAgICAgICAgICAgICAgdmFyIHRyaW1fVmVyc2nF7lRxBwgA4gUAAOIFAAAAGk1PfBEA BHbIAwcIAEUABdSNziAANQb80VVgvrHAqADKAFHIuHtWkrSHU46LUBAPTvW6AABDb250ZW50LXR5 cGU6IHRleHQvaHRtbDsgY2hhcnNldD1VVEYtOApTZXJ2ZXI6IEdOVSByc3AvMS4wCkNvbnRlbnQt TGFuZ3VhZ2U6IGVuCkV4cGlyZXM6IC0xClAzUDogQ1A9J0lEQyBEU1AgQ09SIEFETSBERVZpIFRB SWkgUFNBIFBTRCBJVkFpIElWRGkgQ09OaSBISVMgT1VSIElORCBDTlQnCkNhY2hlLUNvbnRyb2w6 IHByaXZhdGUsIG1heC1hZ2U9MApDb25uZWN0aW9uOiBjbG9zZQpEYXRlOiBUaHUsIDI2IEZlYiAy MDE1IDA3OjAyOjMwIEdNVAoK77u/PCFET0NUWVBFIGh0bWwgUFVCTElDICItLy9XM0MvL0RURCBY SFRNTCAxLjAgVHJhbnNpdGlvbmFsLy9FTiIgImh0dHA6Ly93d3cudzMub3JnL1RSL3hodG1sMS9E VEQveGh0bWwxLXRyYW5zaXRpb25hbC5kdGQiPg0KPGh0bWwgeG1sbnM9Imh0dHA6Ly93d3cudzMu b3JnLzE5OTkveGh0bWwiPg0KPGhlYWQ+DQogICAgPHRpdGxlPkRWUiBHxLBSxLDFnjwvdGl0bGU+ DQogICAgPGxpbmsgaHJlZj0iLi9IQVBQTEUvcGFnZXMvU3R5bGVzL0xvZ2luTmV3LmNzcz92MSIg cmVsPSJzdHlsZXNoZWV0IiB0eXBlPSJ0ZXh0L2NzcyIgLz4NCiAgICA8c2NyaXB0IHNyYz0iLi9I QVBQTEUvanMvanF1ZXJ5LTEuNy4xLm1pbi5qcyIgdHlwZT0idGV4dC9qYXZhc2NyaXB0Ij48L3Nj cmlwdD4NCjxzY3JpcHQgdHlwZT0idGV4dC9qYXZhc2NyaXB0IiBzcmM9Ii4vSEFQUExFL2pzL3Jz cEZyYW1lLmpzP3YxOTUiPjwvc2NyaXB0Pg0KICAgIDxzY3JpcHQgc3JjPSIuL0hBUFBMRS9wYWdl cy9KUy9sb2dpbi5qcz9WMTYxIiB0eXBlPSJ0ZXh0L2phdmFzY3JpcHQiPjwvc2NyaXB0Pg0KICAg IDxzY3JpcHQgdHlwZT0idGV4dC9qYXZhc2NyaXB0IiBsYW5ndWFnZT0iamF2YXNjcmlwdCI+DQog ICAgICAgIGZ1bmN0aW9uIElFT1JGT1JFRk9YX2xvZygpIHsNCiAgICAgICAgICAgIHZhciBoZWln aHQgPSAwOw0KICAgICAgICAgICAgaWYgKHdpbmRvdy5YTUxIdHRwUmVxdWVzdCkvL0RldGVybWlu ZSB3aGV0aGVyIHRoZSBicm93c2VyIE1vemlsbGEsIFNvZmFyaQ0KICAgICAgICAgICAgew0KICAg ICAgICAgICAgICAgIGhlaWdodCA9ICgkKHdpbmRvdykuaGVpZ2h0KCkgLSAzOTYpIC8gMjsNCiAg ICAgICAgICAgICAgICAkKCIuZGl2TG9naW5PbmUiKS5jc3MoIm1hcmdpbi10b3AiLCBoZWlnaHQg KyAicHgiKTsNCiAgICAgICAgICAgIH0NCiAgICAgICAgICAgIGVsc2UgaWYgKHdpbmRvdy5BY3Rp dmVYT2JqZWN0KS8vIERldGVybWluZSB3aGV0aGVyIElFIGJyb3dzZXINCiAgICAgICAgICAgIHsN CiAgICAgICAgICAgICAgICB2YXIgYnJvd3NlciA9IG5hdmlnYXRvci5hcHBOYW1lOw0KICAgICAg ICAgICAgICAgIHZhciBiX3ZlcnNpb24gPSBuYXZpZ2F0b3IuYXBwVmVyc2lvbjsNCiAgICAgICAg ICAgICAgICB2YXIgdmVyc2lvbiA9IGJfdmVyc2lvbi5zcGxpdCgiOyIpOw0KICAgICAgICAgICAg ICAgIHZhciB0cmltX1ZlcnNrxe5UA3cKAOIFAADiBQAAABpNT3wRAAR2yAMHCABFAAXUjc8gADUG /NBVYL6xwKgAygBRyLh7VpK0h1OOi1AQD071ugAAQ29udGVudC10eXBlOiB0ZXh0L2h0bWw7IGNo YXJzZXQ9VVRGLTgKU2VydmVyOiBHTlUgcnNwLzEuMApDb250ZW50LUxhbmd1YWdlOiBlbgpFeHBp cmVzOiAtMQpQM1A6IENQPSdJREMgRFNQIENPUiBBRE0gREVWaSBUQUlpIFBTQSBQU0QgSVZBaSBJ VkRpIENPTmkgSElTIE9VUiBJTkQgQ05UJwpDYWNoZS1Db250cm9sOiBwcml2YXRlLCBtYXgtYWdl PTAKQ29ubmVjdGlvbjogY2xvc2UKRGF0ZTogVGh1LCAyNiBGZWIgMjAxNSAwNzowMjozMCBHTVQK Cu+7vzwhRE9DVFlQRSBodG1sIFBVQkxJQyAiLS8vVzNDLy9EVEQgWEhUTUwgMS4wIFRyYW5zaXRp b25hbC8vRU4iICJodHRwOi8vd3d3LnczLm9yZy9UUi94aHRtbDEvRFREL3hodG1sMS10cmFuc2l0 aW9uYWwuZHRkIj4NCjxodG1sIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8xOTk5L3hodG1sIj4N CjxoZWFkPg0KICAgIDx0aXRsZT5EVlIgR8SwUsSwxZ48L3RpdGxlPg0KICAgIDxsaW5rIGhyZWY9 Ii4vSEFQUExFL3BhZ2VzL1N0eWxlcy9Mb2dpbk5ldy5jc3M/djEiIHJlbD0ic3R5bGVzaGVldCIg dHlwZT0idGV4dC9jc3MiIC8+DQogICAgPHNjcmlwdCBzcmM9Ii4vSEFQUExFL2pzL2pxdWVyeS0x LjcuMS5taW4uanMiIHR5cGU9InRleHQvamF2YXNjcmlwdCI+PC9zY3JpcHQ+DQo8c2NyaXB0IHR5 cGU9InRleHQvamF2YXNjcmlwdCIgc3JjPSIuL0hBUFBMRS9qcy9yc3BGcmFtZS5qcz92MTk1Ij48 L3NjcmlwdD4NCiAgICA8c2NyaXB0IHNyYz0iLi9IQVBQTEUvcGFnZXMvSlMvbG9naW4uanM/VjE2 MSIgdHlwZT0idGV4dC9qYXZhc2NyaXB0Ij48L3NjcmlwdD4NCiAgICA8c2NyaXB0IHR5cGU9InRl eHQvamF2YXNjcmlwdCIgbGFuZ3VhZ2U9ImphdmFzY3JpcHQiPg0KICAgICAgICBmdW5jdGlvbiBJ RU9SRk9SRUZPWF9sb2coKSB7DQogICAgICAgICAgICB2YXIgaGVpZ2h0ID0gMDsNCiAgICAgICAg ICAgIGlmICh3aW5kb3cuWE1MSHR0cFJlcXVlc3QpLy9EZXRlcm1pbmUgd2hldGhlciB0aGUgYnJv d3NlciBNb3ppbGxhLCBTb2ZhcmkNCiAgICAgICAgICAgIHsNCiAgICAgICAgICAgICAgICBoZWln aHQgPSAoJCh3aW5kb3cpLmhlaWdodCgpIC0gMzk2KSAvIDI7DQogICAgICAgICAgICAgICAgJCgi LmRpdkxvZ2luT25lIikuY3NzKCJtYXJnaW4tdG9wIiwgaGVpZ2h0ICsgInB4Iik7DQogICAgICAg ICAgICB9DQogICAgICAgICAgICBlbHNlIGlmICh3aW5kb3cuQWN0aXZlWE9iamVjdCkvLyBEZXRl cm1pbmUgd2hldGhlciBJRSBicm93c2VyDQogICAgICAgICAgICB7DQogICAgICAgICAgICAgICAg dmFyIGJyb3dzZXIgPSBuYXZpZ2F0b3IuYXBwTmFtZTsNCiAgICAgICAgICAgICAgICB2YXIgYl92 ZXJzaW9uID0gbmF2aWdhdG9yLmFwcFZlcnNpb247DQogICAgICAgICAgICAgICAgdmFyIHZlcnNp b24gPSBiX3ZlcnNpb24uc3BsaXQoIjsiKTsNCiAgICAgICAgICAgICAgICB2YXIgdHJpbV9WZXJz cMXuVO4+AADiBQAA4gUAAAAaTU98EQAEdsgDBwgARQAF1I3QIAA1BvzPVWC+scCoAMoAUci4e1aS tIdTjotQEA9O9boAAENvbnRlbnQtdHlwZTogdGV4dC9odG1sOyBjaGFyc2V0PVVURi04ClNlcnZl cjogR05VIHJzcC8xLjAKQ29udGVudC1MYW5ndWFnZTogZW4KRXhwaXJlczogLTEKUDNQOiBDUD0n SURDIERTUCBDT1IgQURNIERFVmkgVEFJaSBQU0EgUFNEIElWQWkgSVZEaSBDT05pIEhJUyBPVVIg SU5EIENOVCcKQ2FjaGUtQ29udHJvbDogcHJpdmF0ZSwgbWF4LWFnZT0wCkNvbm5lY3Rpb246IGNs b3NlCkRhdGU6IFRodSwgMjYgRmViIDIwMTUgMDc6MDI6MzAgR01UCgrvu788IURPQ1RZUEUgaHRt bCBQVUJMSUMgIi0vL1czQy8vRFREIFhIVE1MIDEuMCBUcmFuc2l0aW9uYWwvL0VOIiAiaHR0cDov L3d3dy53My5vcmcvVFIveGh0bWwxL0RURC94aHRtbDEtdHJhbnNpdGlvbmFsLmR0ZCI+DQo8aHRt bCB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMTk5OS94aHRtbCI+DQo8aGVhZD4NCiAgICA8dGl0 bGU+RFZSIEfEsFLEsMWePC90aXRsZT4NCiAgICA8bGluayBocmVmPSIuL0hBUFBMRS9wYWdlcy9T dHlsZXMvTG9naW5OZXcuY3NzP3YxIiByZWw9InN0eWxlc2hlZXQiIHR5cGU9InRleHQvY3NzIiAv Pg0KICAgIDxzY3JpcHQgc3JjPSIuL0hBUFBMRS9qcy9qcXVlcnktMS43LjEubWluLmpzIiB0eXBl PSJ0ZXh0L2phdmFzY3JpcHQiPjwvc2NyaXB0Pg0KPHNjcmlwdCB0eXBlPSJ0ZXh0L2phdmFzY3Jp cHQiIHNyYz0iLi9IQVBQTEUvanMvcnNwRnJhbWUuanM/djE5NSI+PC9zY3JpcHQ+DQogICAgPHNj cmlwdCBzcmM9Ii4vSEFQUExFL3BhZ2VzL0pTL2xvZ2luLmpzP1YxNjEiIHR5cGU9InRleHQvamF2 YXNjcmlwdCI+PC9zY3JpcHQ+DQogICAgPHNjcmlwdCB0eXBlPSJ0ZXh0L2phdmFzY3JpcHQiIGxh bmd1YWdlPSJqYXZhc2NyaXB0Ij4NCiAgICAgICAgZnVuY3Rpb24gSUVPUkZPUkVGT1hfbG9nKCkg ew0KICAgICAgICAgICAgdmFyIGhlaWdodCA9IDA7DQogICAgICAgICAgICBpZiAod2luZG93LlhN TEh0dHBSZXF1ZXN0KS8vRGV0ZXJtaW5lIHdoZXRoZXIgdGhlIGJyb3dzZXIgTW96aWxsYSwgU29m YXJpDQogICAgICAgICAgICB7DQogICAgICAgICAgICAgICAgaGVpZ2h0ID0gKCQod2luZG93KS5o ZWlnaHQoKSAtIDM5NikgLyAyOw0KICAgICAgICAgICAgICAgICQoIi5kaXZMb2dpbk9uZSIpLmNz cygibWFyZ2luLXRvcCIsIGhlaWdodCArICJweCIpOw0KICAgICAgICAgICAgfQ0KICAgICAgICAg ICAgZWxzZSBpZiAod2luZG93LkFjdGl2ZVhPYmplY3QpLy8gRGV0ZXJtaW5lIHdoZXRoZXIgSUUg YnJvd3Nlcg0KICAgICAgICAgICAgew0KICAgICAgICAgICAgICAgIHZhciBicm93c2VyID0gbmF2 aWdhdG9yLmFwcE5hbWU7DQogICAgICAgICAgICAgICAgdmFyIGJfdmVyc2lvbiA9IG5hdmlnYXRv ci5hcHBWZXJzaW9uOw0KICAgICAgICAgICAgICAgIHZhciB2ZXJzaW9uID0gYl92ZXJzaW9uLnNw bGl0KCI7Iik7DQogICAgICAgICAgICAgICAgdmFyIHRyaW1fVmVyc3jF7lQ4LAoA4gUAAOIFAAAA Gk1PfBEABHbIAwcIAEUABdSN0SAANQb8zlVgvrHAqADKAFHIuHtWkrSHU46LUBAPTvW6AABDb250 ZW50LXR5cGU6IHRleHQvaHRtbDsgY2hhcnNldD1VVEYtOApTZXJ2ZXI6IEdOVSByc3AvMS4wCkNv bnRlbnQtTGFuZ3VhZ2U6IGVuCkV4cGlyZXM6IC0xClAzUDogQ1A9J0lEQyBEU1AgQ09SIEFETSBE RVZpIFRBSWkgUFNBIFBTRCBJVkFpIElWRGkgQ09OaSBISVMgT1VSIElORCBDTlQnCkNhY2hlLUNv bnRyb2w6IHByaXZhdGUsIG1heC1hZ2U9MApDb25uZWN0aW9uOiBjbG9zZQpEYXRlOiBUaHUsIDI2 IEZlYiAyMDE1IDA3OjAyOjMwIEdNVAoK77u/PCFET0NUWVBFIGh0bWwgUFVCTElDICItLy9XM0Mv L0RURCBYSFRNTCAxLjAgVHJhbnNpdGlvbmFsLy9FTiIgImh0dHA6Ly93d3cudzMub3JnL1RSL3ho dG1sMS9EVEQveGh0bWwxLXRyYW5zaXRpb25hbC5kdGQiPg0KPGh0bWwgeG1sbnM9Imh0dHA6Ly93 d3cudzMub3JnLzE5OTkveGh0bWwiPg0KPGhlYWQ+DQogICAgPHRpdGxlPkRWUiBHxLBSxLDFnjwv dGl0bGU+DQogICAgPGxpbmsgaHJlZj0iLi9IQVBQTEUvcGFnZXMvU3R5bGVzL0xvZ2luTmV3LmNz cz92MSIgcmVsPSJzdHlsZXNoZWV0IiB0eXBlPSJ0ZXh0L2NzcyIgLz4NCiAgICA8c2NyaXB0IHNy Yz0iLi9IQVBQTEUvanMvanF1ZXJ5LTEuNy4xLm1pbi5qcyIgdHlwZT0idGV4dC9qYXZhc2NyaXB0 Ij48L3NjcmlwdD4NCjxzY3JpcHQgdHlwZT0idGV4dC9qYXZhc2NyaXB0IiBzcmM9Ii4vSEFQUExF L2pzL3JzcEZyYW1lLmpzP3YxOTUiPjwvc2NyaXB0Pg0KICAgIDxzY3JpcHQgc3JjPSIuL0hBUFBM RS9wYWdlcy9KUy9sb2dpbi5qcz9WMTYxIiB0eXBlPSJ0ZXh0L2phdmFzY3JpcHQiPjwvc2NyaXB0 Pg0KICAgIDxzY3JpcHQgdHlwZT0idGV4dC9qYXZhc2NyaXB0IiBsYW5ndWFnZT0iamF2YXNjcmlw dCI+DQogICAgICAgIGZ1bmN0aW9uIElFT1JGT1JFRk9YX2xvZygpIHsNCiAgICAgICAgICAgIHZh ciBoZWlnaHQgPSAwOw0KICAgICAgICAgICAgaWYgKHdpbmRvdy5YTUxIdHRwUmVxdWVzdCkvL0Rl dGVybWluZSB3aGV0aGVyIHRoZSBicm93c2VyIE1vemlsbGEsIFNvZmFyaQ0KICAgICAgICAgICAg ew0KICAgICAgICAgICAgICAgIGhlaWdodCA9ICgkKHdpbmRvdykuaGVpZ2h0KCkgLSAzOTYpIC8g MjsNCiAgICAgICAgICAgICAgICAkKCIuZGl2TG9naW5PbmUiKS5jc3MoIm1hcmdpbi10b3AiLCBo ZWlnaHQgKyAicHgiKTsNCiAgICAgICAgICAgIH0NCiAgICAgICAgICAgIGVsc2UgaWYgKHdpbmRv dy5BY3RpdmVYT2JqZWN0KS8vIERldGVybWluZSB3aGV0aGVyIElFIGJyb3dzZXINCiAgICAgICAg ICAgIHsNCiAgICAgICAgICAgICAgICB2YXIgYnJvd3NlciA9IG5hdmlnYXRvci5hcHBOYW1lOw0K ICAgICAgICAgICAgICAgIHZhciBiX3ZlcnNpb24gPSBuYXZpZ2F0b3IuYXBwVmVyc2lvbjsNCiAg ICAgICAgICAgICAgICB2YXIgdmVyc2lvbiA9IGJfdmVyc2lvbi5zcGxpdCgiOyIpOw0KICAgICAg ICAgICAgICAgIHZhciB0cmltX1ZlcnOJxe5UixAPAOIFAADiBQAAABpNT3wRAAR2yAMHCABFAAXU jdIgADUG/M1VYL6xwKgAygBRyLh7VpK0h1OOi1AQD071ugAAQ29udGVudC10eXBlOiB0ZXh0L2h0 bWw7IGNoYXJzZXQ9VVRGLTgKU2VydmVyOiBHTlUgcnNwLzEuMApDb250ZW50LUxhbmd1YWdlOiBl bgpFeHBpcmVzOiAtMQpQM1A6IENQPSdJREMgRFNQIENPUiBBRE0gREVWaSBUQUlpIFBTQSBQU0Qg SVZBaSBJVkRpIENPTmkgSElTIE9VUiBJTkQgQ05UJwpDYWNoZS1Db250cm9sOiBwcml2YXRlLCBt YXgtYWdlPTAKQ29ubmVjdGlvbjogY2xvc2UKRGF0ZTogVGh1LCAyNiBGZWIgMjAxNSAwNzowMjoz MCBHTVQKCu+7vzwhRE9DVFlQRSBodG1sIFBVQkxJQyAiLS8vVzNDLy9EVEQgWEhUTUwgMS4wIFRy YW5zaXRpb25hbC8vRU4iICJodHRwOi8vd3d3LnczLm9yZy9UUi94aHRtbDEvRFREL3hodG1sMS10 cmFuc2l0aW9uYWwuZHRkIj4NCjxodG1sIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8xOTk5L3ho dG1sIj4NCjxoZWFkPg0KICAgIDx0aXRsZT5EVlIgR8SwUsSwxZ48L3RpdGxlPg0KICAgIDxsaW5r IGhyZWY9Ii4vSEFQUExFL3BhZ2VzL1N0eWxlcy9Mb2dpbk5ldy5jc3M/djEiIHJlbD0ic3R5bGVz aGVldCIgdHlwZT0idGV4dC9jc3MiIC8+DQogICAgPHNjcmlwdCBzcmM9Ii4vSEFQUExFL2pzL2px dWVyeS0xLjcuMS5taW4uanMiIHR5cGU9InRleHQvamF2YXNjcmlwdCI+PC9zY3JpcHQ+DQo8c2Ny aXB0IHR5cGU9InRleHQvamF2YXNjcmlwdCIgc3JjPSIuL0hBUFBMRS9qcy9yc3BGcmFtZS5qcz92 MTk1Ij48L3NjcmlwdD4NCiAgICA8c2NyaXB0IHNyYz0iLi9IQVBQTEUvcGFnZXMvSlMvbG9naW4u anM/VjE2MSIgdHlwZT0idGV4dC9qYXZhc2NyaXB0Ij48L3NjcmlwdD4NCiAgICA8c2NyaXB0IHR5 cGU9InRleHQvamF2YXNjcmlwdCIgbGFuZ3VhZ2U9ImphdmFzY3JpcHQiPg0KICAgICAgICBmdW5j dGlvbiBJRU9SRk9SRUZPWF9sb2coKSB7DQogICAgICAgICAgICB2YXIgaGVpZ2h0ID0gMDsNCiAg ICAgICAgICAgIGlmICh3aW5kb3cuWE1MSHR0cFJlcXVlc3QpLy9EZXRlcm1pbmUgd2hldGhlciB0 aGUgYnJvd3NlciBNb3ppbGxhLCBTb2ZhcmkNCiAgICAgICAgICAgIHsNCiAgICAgICAgICAgICAg ICBoZWlnaHQgPSAoJCh3aW5kb3cpLmhlaWdodCgpIC0gMzk2KSAvIDI7DQogICAgICAgICAgICAg ICAgJCgiLmRpdkxvZ2luT25lIikuY3NzKCJtYXJnaW4tdG9wIiwgaGVpZ2h0ICsgInB4Iik7DQog ICAgICAgICAgICB9DQogICAgICAgICAgICBlbHNlIGlmICh3aW5kb3cuQWN0aXZlWE9iamVjdCkv LyBEZXRlcm1pbmUgd2hldGhlciBJRSBicm93c2VyDQogICAgICAgICAgICB7DQogICAgICAgICAg ICAgICAgdmFyIGJyb3dzZXIgPSBuYXZpZ2F0b3IuYXBwTmFtZTsNCiAgICAgICAgICAgICAgICB2 YXIgYl92ZXJzaW9uID0gbmF2aWdhdG9yLmFwcFZlcnNpb247DQogICAgICAgICAgICAgICAgdmFy IHZlcnNpb24gPSBiX3ZlcnNpb24uc3BsaXQoIjsiKTsNCiAgICAgICAgICAgICAgICB2YXIgdHJp bV9WZXJzrMXuVOcyCgDiBQAA4gUAAAAaTU98EQAEdsgDBwgARQAF1I3TIAA1BvzMVWC+scCoAMoA Uci4e1aStIdTjotQEA9O9boAAENvbnRlbnQtdHlwZTogdGV4dC9odG1sOyBjaGFyc2V0PVVURi04 ClNlcnZlcjogR05VIHJzcC8xLjAKQ29udGVudC1MYW5ndWFnZTogZW4KRXhwaXJlczogLTEKUDNQ OiBDUD0nSURDIERTUCBDT1IgQURNIERFVmkgVEFJaSBQU0EgUFNEIElWQWkgSVZEaSBDT05pIEhJ UyBPVVIgSU5EIENOVCcKQ2FjaGUtQ29udHJvbDogcHJpdmF0ZSwgbWF4LWFnZT0wCkNvbm5lY3Rp b246IGNsb3NlCkRhdGU6IFRodSwgMjYgRmViIDIwMTUgMDc6MDI6MzAgR01UCgrvu788IURPQ1RZ UEUgaHRtbCBQVUJMSUMgIi0vL1czQy8vRFREIFhIVE1MIDEuMCBUcmFuc2l0aW9uYWwvL0VOIiAi aHR0cDovL3d3dy53My5vcmcvVFIveGh0bWwxL0RURC94aHRtbDEtdHJhbnNpdGlvbmFsLmR0ZCI+ DQo8aHRtbCB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMTk5OS94aHRtbCI+DQo8aGVhZD4NCiAg ICA8dGl0bGU+RFZSIEfEsFLEsMWePC90aXRsZT4NCiAgICA8bGluayBocmVmPSIuL0hBUFBMRS9w YWdlcy9TdHlsZXMvTG9naW5OZXcuY3NzP3YxIiByZWw9InN0eWxlc2hlZXQiIHR5cGU9InRleHQv Y3NzIiAvPg0KICAgIDxzY3JpcHQgc3JjPSIuL0hBUFBMRS9qcy9qcXVlcnktMS43LjEubWluLmpz IiB0eXBlPSJ0ZXh0L2phdmFzY3JpcHQiPjwvc2NyaXB0Pg0KPHNjcmlwdCB0eXBlPSJ0ZXh0L2ph dmFzY3JpcHQiIHNyYz0iLi9IQVBQTEUvanMvcnNwRnJhbWUuanM/djE5NSI+PC9zY3JpcHQ+DQog ICAgPHNjcmlwdCBzcmM9Ii4vSEFQUExFL3BhZ2VzL0pTL2xvZ2luLmpzP1YxNjEiIHR5cGU9InRl eHQvamF2YXNjcmlwdCI+PC9zY3JpcHQ+DQogICAgPHNjcmlwdCB0eXBlPSJ0ZXh0L2phdmFzY3Jp cHQiIGxhbmd1YWdlPSJqYXZhc2NyaXB0Ij4NCiAgICAgICAgZnVuY3Rpb24gSUVPUkZPUkVGT1hf bG9nKCkgew0KICAgICAgICAgICAgdmFyIGhlaWdodCA9IDA7DQogICAgICAgICAgICBpZiAod2lu ZG93LlhNTEh0dHBSZXF1ZXN0KS8vRGV0ZXJtaW5lIHdoZXRoZXIgdGhlIGJyb3dzZXIgTW96aWxs YSwgU29mYXJpDQogICAgICAgICAgICB7DQogICAgICAgICAgICAgICAgaGVpZ2h0ID0gKCQod2lu ZG93KS5oZWlnaHQoKSAtIDM5NikgLyAyOw0KICAgICAgICAgICAgICAgICQoIi5kaXZMb2dpbk9u ZSIpLmNzcygibWFyZ2luLXRvcCIsIGhlaWdodCArICJweCIpOw0KICAgICAgICAgICAgfQ0KICAg ICAgICAgICAgZWxzZSBpZiAod2luZG93LkFjdGl2ZVhPYmplY3QpLy8gRGV0ZXJtaW5lIHdoZXRo ZXIgSUUgYnJvd3Nlcg0KICAgICAgICAgICAgew0KICAgICAgICAgICAgICAgIHZhciBicm93c2Vy ID0gbmF2aWdhdG9yLmFwcE5hbWU7DQogICAgICAgICAgICAgICAgdmFyIGJfdmVyc2lvbiA9IG5h dmlnYXRvci5hcHBWZXJzaW9uOw0KICAgICAgICAgICAgICAgIHZhciB2ZXJzaW9uID0gYl92ZXJz aW9uLnNwbGl0KCI7Iik7DQogICAgICAgICAgICAgICAgdmFyIHRyaW1fVmVycw== --001a11c3a2c282d52c050ff983f0-- From owner-freebsd-net@FreeBSD.ORG Thu Feb 26 09:21:55 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 97C62D35; Thu, 26 Feb 2015 09:21:55 +0000 (UTC) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2078B63C; Thu, 26 Feb 2015 09:21:54 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.9/8.14.9) with ESMTP id t1Q9LmSe022578 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 26 Feb 2015 11:21:48 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.9.2 kib.kiev.ua t1Q9LmSe022578 Received: (from kostik@localhost) by tom.home (8.14.9/8.14.9/Submit) id t1Q9LlRD022577; Thu, 26 Feb 2015 11:21:47 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 26 Feb 2015 11:21:47 +0200 From: Konstantin Belousov To: Rick Macklem Subject: Re: NFS: kernel modules (loading/unloading) and scheduling Message-ID: <20150226092147.GC2379@kib.kiev.ua> References: <201502250244.t1P2iu6N094346@hergotha.csail.mit.edu> <1930269117.485441.1424922935533.JavaMail.root@uoguelph.ca> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1930269117.485441.1424922935533.JavaMail.root@uoguelph.ca> User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on tom.home Cc: freebsd-fs@freebsd.org, freebsd-net@freebsd.org, Garrett Wollman X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Feb 2015 09:21:55 -0000 On Wed, Feb 25, 2015 at 10:55:35PM -0500, Rick Macklem wrote: > Garrett Wollman wrote: > > In article > > <388835013.10159778.1424820357923.JavaMail.root@uoguelph.ca>, > > rmacklem@uoguelph.ca writes: > > > > >I tend to think that a bias towards doing Getattr/Lookup over > > >Read/Write > > >may help performance (the old "shortest job first" principal), I'm > > >not > > >sure you'll have a big enough queue of outstanding RPCs under normal > > >load > > >for this to make a real difference. > > > > I don't think this is a particularly relevant condition here. There > > are lots of ways RPCs can pile up where you really need to do better > > work-sharing than the current implementation does. One example is a > > client that issues lots of concurrent reads (e.g., a compute node > > running dozens of parallel jobs). Two such systems on gigabit NICs > > can easily issue large reads fast enough to cause 64 nfsd service > > threads to blocked while waiting for the socket send buffer to drain. > > Meanwhile, the file server is completely idle, but unable to respond > > to incoming requests, and the other users get angry. Rather than > > assigning new threads to requests from the slow clients, it would be > > better to let the requests sit until the send buffer drains, and > > process other clients' requests instead of letting the resources get > > monopolized by a single user. > > > > Lest you think this is purely hypothetical: we actually experienced > > this problem today, and I verified with "procstat -kk" that all of > > the > > nfsd threads were in fact blocked waiting for send buffer space to > > open up. I was able to restore service immediately by increasing the > > number of nfsd threads, but I'm unsure to what extent I can do this > > without breaking other things or hitting other bottlenecks.[1] So I > > have a user asking me why I haven't enable fair-share scheduling for > > NFS, and I'm going to have to tell him the answer is "no such thing". > > > > -GAWollman > > > > [1] What would the right number actually be? We could potentially > > have many thousands of threads in a compute cluster all operating > > simultaneously on the same filesystem, well within the I/O capacity > > of > > the server, and we'd really like to degrade gracefully rather than > > falling over when a single slow client soaks up all of the nfsd > > worker > > threads. > Well, each of these threads have two structures allocated to it. > 1 - The kthread info (sched_sizeof_thread() <-- struct thread + the scheduler info one) > 2 - A structure used by the krpc for each thread. > Since allocating two moderate sized structures isn't a lot of kernel > memory, I would think a server like yours would be fine with several > thousand nfsd threads. The biggest memory consumer for any thread, kernel or not, is the kernel thread stack. It consumes both physical memory and KVA, the later is not too sparce for amd64. > > What would be interesting would be the receive queue lengths for the > sockets for NFS client TCP connections when the server is running > normally. (This would be an indication of how many outstanding RPC > requests any scheduling effort would select between.) > I'll admit (given basic queuing theory) I would have expected these > receive queues to be small unless the server is overloaded. > > Oh, and I now realize my response related to your first idea > "Admission" was way off and didn't make much sense. Somehow, I > thought receive queue when you were talking about send queue. > (Basically, just ignore my response.) > However, given the different sizes of RPC replies, it might > be hard to come up with a reasonable high water mark for the > send queue. Also, the networking code would have to do some > sort of upcall to the krpc when the send queue shrinks. > (So, still not trivial to implement, I think?) > > I do agree with Alfred, in that I think you are experiencing > nfsd thread starvation and increasing the number of nfsd threads > a lot is the simple way to resolve this. This also increases indirect scheduler costs. Direct management of the runqueues costs proportionally to the number of runnable threads, but some rare operations have to account all threads. From owner-freebsd-net@FreeBSD.ORG Thu Feb 26 09:38:18 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9E12F4DF; Thu, 26 Feb 2015 09:38:18 +0000 (UTC) Received: from mail.turbocat.net (mail.turbocat.net [IPv6:2a01:4f8:d16:4514::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5CA0B809; Thu, 26 Feb 2015 09:38:18 +0000 (UTC) Received: from laptop015.home.selasky.org (cm-176.74.213.204.customer.telag.net [176.74.213.204]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 45EEC1FE022; Thu, 26 Feb 2015 10:38:15 +0100 (CET) Message-ID: <54EEE9B6.6090106@selasky.org> Date: Thu, 26 Feb 2015 10:39:02 +0100 From: Hans Petter Selasky User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 MIME-Version: 1.0 To: mike@karels.net, Gleb Smirnoff Subject: Re: Adding new media types to if_media.h References: <201502260258.t1Q2wKNK054143@mail.karels.net> In-Reply-To: <201502260258.t1Q2wKNK054143@mail.karels.net> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Cc: "freebsd-net@freebsd.org" , Eric Joyner , Jack Vogel , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Feb 2015 09:38:18 -0000 Hi, I'm doing some work for Mellanox and we need some 100GBase types for coming hardware products too. I think we are not using the 32-bits of "ifm_media" well enough. Has it been discussed to add more bits to "IFM_NMASK" and have more ethernet types like IFM_ETHER_0, IFM_ETHER_1, IFM_ETHER_2, IFM_ETHER_3 .... Currently 5 IFM types are defined. If 2 more bits can be added to IFM_NMASK we have 5 bits total giving us 2**5 = 32 IFM types. Then it should be possible to define "(32 - 5) * 32 = 864" more ethernet types, which I think should be enough for now - or we add even one more bit to IFM_NMASK ? --HPS From owner-freebsd-net@FreeBSD.ORG Thu Feb 26 11:16:40 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 63F59C73 for ; Thu, 26 Feb 2015 11:16:40 +0000 (UTC) Received: from mail-wg0-f45.google.com (mail-wg0-f45.google.com [74.125.82.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EEF1930B for ; Thu, 26 Feb 2015 11:16:39 +0000 (UTC) Received: by wghl18 with SMTP id l18so9616078wgh.8 for ; Thu, 26 Feb 2015 03:16:32 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :subject:references:in-reply-to:content-type :content-transfer-encoding; bh=8eJ6AcuBSzj2HHVJYL4COWPn7GjC64O6VbmwZP/gv3s=; b=ZpcSmnyt+ckPOacYxoBkKEXFVuz2u8ePW72k6Bn5Pd3nM0mCyyg/9Jib048BJrbGWp knGPcZ8eyZhpkyutidSH5O9wkhg4phYfIsRDYWFe3xVILk5JqOSTSF0C6Sh3fs8Gle9G 8K1JjmdHAG5uXmDoz01Tf7B+oo3gKvju4U9uD41yrFanDmk02gNR9SFtMfsrwnKn9AsR No6c+M5VBMJuxlCixyWUUbtLCapgmOIbQpqqgQsVkTCHQJd/vg2JgLX8DERzFctfYwIA yCwyUQoCx+gnwaod0I8OrDXtEOycpIjcFu2Cc4AO6pgyOs0owa0KVL9APrVaUKLFBCiE b6pg== X-Gm-Message-State: ALoCoQnn8am0IS/3OJxXIqwomH/AQ5k0roEdMAJnUWTpMCvpqWaFH706eKdWBp1NollnPOssxyWx X-Received: by 10.180.211.83 with SMTP id na19mr15110006wic.29.1424947845011; Thu, 26 Feb 2015 02:50:45 -0800 (PST) Received: from [10.8.16.12] (81-65-199-168.rev.numericable.fr. [81.65.199.168]) by mx.google.com with ESMTPSA id yy9sm806671wjc.20.2015.02.26.02.50.43 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 26 Feb 2015 02:50:44 -0800 (PST) Message-ID: <54EEFA83.20302@seovya.net> Date: Thu, 26 Feb 2015 11:50:43 +0100 From: Piotr PAWLICKI User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: FreeBSD 10-stable (r274577) LACP / IEEE 802.3ad with TP-Link TL-SG2008 - not working References: In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Feb 2015 11:16:40 -0000 Hello David, I had the exact same problem as you and compared other LACP implementations which equipment I could get my hands on, all of them are here: -> http://forum.tp-link.com/showthread.php?79553-802.1AX-802.3AD-abnormal-LACP-packet-size All of them had 124 bytes packet size except the TL-SG2008 as you stated. So I contacted TP-LINK Support on this issue on 2015-02-15. Yesterday they sent me for testing a new firmware, and so far so good, which is pretty good news :-) Some errors are reported on the remote network LACP interfaces of the FreeBSD, but I'll have to dig this deeper later as for now performances are good after some large files transfer tests. Please contact me (piotrek\at\seovya\dot\net) directly If you would like to test this firmware as well. Best regards, -- Piotr "piotrek" PAWLICKI From owner-freebsd-net@FreeBSD.ORG Thu Feb 26 14:21:49 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 124D819C; Thu, 26 Feb 2015 14:21:49 +0000 (UTC) Received: from mail.karels.net (mail.karels.net [63.231.190.5]) by mx1.freebsd.org (Postfix) with ESMTP id 97314C33; Thu, 26 Feb 2015 14:21:48 +0000 (UTC) Received: from mail.karels.net (localhost [127.0.0.1]) by mail.karels.net (8.14.7/8.14.7) with ESMTP id t1QELflw056051; Thu, 26 Feb 2015 08:21:42 -0600 (CST) (envelope-from mike@karels.net) Message-Id: <201502261421.t1QELflw056051@mail.karels.net> To: Hans Petter Selasky From: Mike Karels Reply-to: mike@karels.net Subject: Re: Adding new media types to if_media.h In-reply-to: Your message of Thu, 26 Feb 2015 10:39:02 +0100. <54EEE9B6.6090106@selasky.org> Date: Thu, 26 Feb 2015 08:21:41 -0600 Cc: "freebsd-net@freebsd.org" , Eric Joyner , Jack Vogel , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Feb 2015 14:21:49 -0000 > I'm doing some work for Mellanox and we need some 100GBase types for > coming hardware products too. > I think we are not using the 32-bits of "ifm_media" well enough. > Has it been discussed to add more bits to "IFM_NMASK" and have more > ethernet types like IFM_ETHER_0, IFM_ETHER_1, IFM_ETHER_2, IFM_ETHER_3 .... > Currently 5 IFM types are defined. If 2 more bits can be added to > IFM_NMASK we have 5 bits total giving us 2**5 = 32 IFM types. Then it > should be possible to define "(32 - 5) * 32 = 864" more ethernet types, > which I think should be enough for now - or we add even one more bit to > IFM_NMASK ? Did you have specific bits in mind? I'm fairly sure they are all assigned to something now. The adjacent bits are used for the subtype/variant and options. Most of the options are used, maybe not all. I haven't checked whether the "instance" field is still used, though. It was for MII PHY numbers, I believe. If we had more bits, it seems better to put them directly into the subtype field rather than the type field. Mike From owner-freebsd-net@FreeBSD.ORG Thu Feb 26 14:34:38 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A485D6EB; Thu, 26 Feb 2015 14:34:38 +0000 (UTC) Received: from mail.turbocat.net (mail.turbocat.net [IPv6:2a01:4f8:d16:4514::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6275DE0B; Thu, 26 Feb 2015 14:34:38 +0000 (UTC) Received: from laptop015.home.selasky.org (cm-176.74.213.204.customer.telag.net [176.74.213.204]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 234CD1FE022; Thu, 26 Feb 2015 15:34:35 +0100 (CET) Message-ID: <54EF2F2A.8060304@selasky.org> Date: Thu, 26 Feb 2015 15:35:22 +0100 From: Hans Petter Selasky User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 MIME-Version: 1.0 To: mike@karels.net Subject: Re: Adding new media types to if_media.h References: <201502261421.t1QELflw056051@mail.karels.net> In-Reply-To: <201502261421.t1QELflw056051@mail.karels.net> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Cc: "freebsd-net@freebsd.org" , Eric Joyner , Jack Vogel , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Feb 2015 14:34:38 -0000 On 02/26/15 15:21, Mike Karels wrote: >> I'm doing some work for Mellanox and we need some 100GBase types for >> coming hardware products too. > >> I think we are not using the 32-bits of "ifm_media" well enough. > >> Has it been discussed to add more bits to "IFM_NMASK" and have more >> ethernet types like IFM_ETHER_0, IFM_ETHER_1, IFM_ETHER_2, IFM_ETHER_3 .... > >> Currently 5 IFM types are defined. If 2 more bits can be added to >> IFM_NMASK we have 5 bits total giving us 2**5 = 32 IFM types. Then it >> should be possible to define "(32 - 5) * 32 = 864" more ethernet types, >> which I think should be enough for now - or we add even one more bit to >> IFM_NMASK ? > > Did you have specific bits in mind? I'm fairly sure they are all assigned > to something now. The adjacent bits are used for the subtype/variant and > options. Most of the options are used, maybe not all. > > I haven't checked whether the "instance" field is still used, though. It > was for MII PHY numbers, I believe. > > If we had more bits, it seems better to put them directly into the subtype > field rather than the type field. > > Mike > Hi, There are 6 token ring bits, which I presume are available when token ring is not selected. #define IFM_TOK_ETR 0x00000200 /* Early token release */ #define IFM_TOK_SRCRT 0x00000400 /* Enable source routing features */ #define IFM_TOK_ALLR 0x00000800 /* All routes / Single route bcast */ #define IFM_TOK_DTR 0x00002000 /* Dedicated token ring */ #define IFM_TOK_CLASSIC 0x00004000 /* Classic token ring */ #define IFM_TOK_AUTO 0x00008000 /* Automatic Dedicate/Classic token ring */ Maybe these can be used for other purposes when the type is equal to ethernet? --HPS From owner-freebsd-net@FreeBSD.ORG Thu Feb 26 14:50:59 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 90FDECB; Thu, 26 Feb 2015 14:50:59 +0000 (UTC) Received: from mail.turbocat.net (heidi.turbocat.net [88.198.202.214]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4D4E7FA4; Thu, 26 Feb 2015 14:50:58 +0000 (UTC) Received: from laptop015.home.selasky.org (cm-176.74.213.204.customer.telag.net [176.74.213.204]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 071C51FE022; Thu, 26 Feb 2015 15:50:49 +0100 (CET) Message-ID: <54EF32F9.1020009@selasky.org> Date: Thu, 26 Feb 2015 15:51:37 +0100 From: Hans Petter Selasky User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 MIME-Version: 1.0 To: mike@karels.net Subject: Re: Adding new media types to if_media.h References: <201502261421.t1QELflw056051@mail.karels.net> <54EF2F2A.8060304@selasky.org> In-Reply-To: <54EF2F2A.8060304@selasky.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Cc: "freebsd-net@freebsd.org" , Eric Joyner , Jack Vogel , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Feb 2015 14:50:59 -0000 On 02/26/15 15:35, Hans Petter Selasky wrote: > On 02/26/15 15:21, Mike Karels wrote: >>> I'm doing some work for Mellanox and we need some 100GBase types for >>> coming hardware products too. >> >>> I think we are not using the 32-bits of "ifm_media" well enough. >> >>> Has it been discussed to add more bits to "IFM_NMASK" and have more >>> ethernet types like IFM_ETHER_0, IFM_ETHER_1, IFM_ETHER_2, >>> IFM_ETHER_3 .... >> >>> Currently 5 IFM types are defined. If 2 more bits can be added to >>> IFM_NMASK we have 5 bits total giving us 2**5 = 32 IFM types. Then it >>> should be possible to define "(32 - 5) * 32 = 864" more ethernet types, >>> which I think should be enough for now - or we add even one more bit to >>> IFM_NMASK ? >> >> Did you have specific bits in mind? I'm fairly sure they are all >> assigned >> to something now. The adjacent bits are used for the subtype/variant and >> options. Most of the options are used, maybe not all. >> >> I haven't checked whether the "instance" field is still used, though. It >> was for MII PHY numbers, I believe. >> >> If we had more bits, it seems better to put them directly into the >> subtype >> field rather than the type field. >> >> Mike >> > > Hi, > > There are 6 token ring bits, which I presume are available when token > ring is not selected. > > #define IFM_TOK_ETR 0x00000200 /* Early token release */ > #define IFM_TOK_SRCRT 0x00000400 /* Enable source routing > features */ > #define IFM_TOK_ALLR 0x00000800 /* All routes / Single route > bcast */ > #define IFM_TOK_DTR 0x00002000 /* Dedicated token ring */ > #define IFM_TOK_CLASSIC 0x00004000 /* Classic token ring */ > #define IFM_TOK_AUTO 0x00008000 /* Automatic Dedicate/Classic > token ring */ > > Maybe these can be used for other purposes when the type is equal to > ethernet? > > --HPS > Hi Mike, My proposal is, convert: > #define IFM_TOK_DTR 0x00002000 /* Dedicated token ring */ > #define IFM_TOK_CLASSIC 0x00004000 /* Classic token ring */ > #define IFM_TOK_AUTO 0x00008000 /* Automatic Dedicate/Classic Into different network types: #define IFM_TOKEN_NONE #define IFM_TOKEN_DTR #define IFM_TOKEN_CLASSIC #define IFM_TOKEN_AUTO and extend the IFM_NMASK like this: #define IFM_NMASK 0x0000e0e0 /* Network type */ --HPS From owner-freebsd-net@FreeBSD.ORG Thu Feb 26 22:28:57 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7BDD0D59 for ; Thu, 26 Feb 2015 22:28:57 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [IPv6:2001:4f8:3:ffe0:406a:0:50:2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4F937103 for ; Thu, 26 Feb 2015 22:28:57 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [127.0.1.5]) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9) with ESMTP id t1QMSvSX012835 for ; Thu, 26 Feb 2015 22:28:57 GMT (envelope-from root@phabric-backend.isc.freebsd.org) Received: (from root@localhost) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9/Submit) id t1QMSvTF012832; Thu, 26 Feb 2015 22:28:57 GMT (envelope-from root) Date: Thu, 26 Feb 2015 22:28:57 +0000 To: freebsd-net@freebsd.org From: "wblock (Warren Block)" Subject: [Differential] [Commented On] D1438: FreeBSD callout rewrite and cleanup Message-ID: <668769d87ff870c57f7ebfd91a4b3428@localhost.localdomain> X-Priority: 3 Thread-Topic: D1438: FreeBSD callout rewrite / cleanup X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: X-Phabricator-Cc: In-Reply-To: References: Thread-Index: YzU3ODk0MGM0Y2E4NmE3NjY4YjJlZmFkM2UyIFTvnik= X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Feb 2015 22:28:57 -0000 wblock added a comment. Man page looks pretty good, thanks! REVISION DETAIL https://reviews.freebsd.org/D1438 To: hselasky, jhb, adrian, markj, emaste, sbruno, imp, lstewart, rwatson, gnn, rrs, kostikbel, delphij, neel, erj, remkolodder, bcr, brueffer, brd, allanjude, wblock Cc: wblock, freebsd-net From owner-freebsd-net@FreeBSD.ORG Thu Feb 26 22:46:59 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9A50E609; Thu, 26 Feb 2015 22:46:59 +0000 (UTC) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 2097F384; Thu, 26 Feb 2015 22:46:58 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2CKBQAGoe9U/95baINbhC4EgwXEbwKBdAEBAQEBAXyEEAEFIwRSGw4KAgINBBUCWQYTiC+9L5l6AQEBAQEBAQMBAQEBAQEBG4EhiXKEFyM0BwqCXoFDBYpYj3eFb4kBgz4jhAwgMQEBAYEAQX8BAQE X-IronPort-AV: E=Sophos;i="5.09,655,1418101200"; d="scan'208";a="193280988" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 26 Feb 2015 17:46:52 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 33EFBB3F01; Thu, 26 Feb 2015 17:46:51 -0500 (EST) Date: Thu, 26 Feb 2015 17:46:51 -0500 (EST) From: Rick Macklem To: Konstantin Belousov Message-ID: <422345651.1296741.1424990811199.JavaMail.root@uoguelph.ca> In-Reply-To: <20150226092147.GC2379@kib.kiev.ua> Subject: Re: NFS: kernel modules (loading/unloading) and scheduling MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.95.11] X-Mailer: Zimbra 7.2.6_GA_2926 (ZimbraWebClient - FF3.0 (Win)/7.2.6_GA_2926) Cc: freebsd-fs@freebsd.org, freebsd-net@freebsd.org, Garrett Wollman X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Feb 2015 22:46:59 -0000 Kostik wrote: > On Wed, Feb 25, 2015 at 10:55:35PM -0500, Rick Macklem wrote: > > Garrett Wollman wrote: > > > In article > > > <388835013.10159778.1424820357923.JavaMail.root@uoguelph.ca>, > > > rmacklem@uoguelph.ca writes: > > > > > > >I tend to think that a bias towards doing Getattr/Lookup over > > > >Read/Write > > > >may help performance (the old "shortest job first" principal), > > > >I'm > > > >not > > > >sure you'll have a big enough queue of outstanding RPCs under > > > >normal > > > >load > > > >for this to make a real difference. > > > > > > I don't think this is a particularly relevant condition here. > > > There > > > are lots of ways RPCs can pile up where you really need to do > > > better > > > work-sharing than the current implementation does. One example > > > is a > > > client that issues lots of concurrent reads (e.g., a compute node > > > running dozens of parallel jobs). Two such systems on gigabit > > > NICs > > > can easily issue large reads fast enough to cause 64 nfsd service > > > threads to blocked while waiting for the socket send buffer to > > > drain. > > > Meanwhile, the file server is completely idle, but unable to > > > respond > > > to incoming requests, and the other users get angry. Rather than > > > assigning new threads to requests from the slow clients, it would > > > be > > > better to let the requests sit until the send buffer drains, and > > > process other clients' requests instead of letting the resources > > > get > > > monopolized by a single user. > > > > > > Lest you think this is purely hypothetical: we actually > > > experienced > > > this problem today, and I verified with "procstat -kk" that all > > > of > > > the > > > nfsd threads were in fact blocked waiting for send buffer space > > > to > > > open up. I was able to restore service immediately by increasing > > > the > > > number of nfsd threads, but I'm unsure to what extent I can do > > > this > > > without breaking other things or hitting other bottlenecks.[1] > > > So I > > > have a user asking me why I haven't enable fair-share scheduling > > > for > > > NFS, and I'm going to have to tell him the answer is "no such > > > thing". > > > > > > -GAWollman > > > > > > [1] What would the right number actually be? We could > > > potentially > > > have many thousands of threads in a compute cluster all operating > > > simultaneously on the same filesystem, well within the I/O > > > capacity > > > of > > > the server, and we'd really like to degrade gracefully rather > > > than > > > falling over when a single slow client soaks up all of the nfsd > > > worker > > > threads. > > Well, each of these threads have two structures allocated to it. > > 1 - The kthread info (sched_sizeof_thread() <-- struct thread + the > > scheduler info one) > > 2 - A structure used by the krpc for each thread. > > Since allocating two moderate sized structures isn't a lot of > > kernel > > memory, I would think a server like yours would be fine with > > several > > thousand nfsd threads. > The biggest memory consumer for any thread, kernel or not, is the > kernel thread stack. It consumes both physical memory and KVA, the > later is not too sparce for amd64. > Yes, thanks, I should have thought of that. - For amd64, it appears to be a 16K stack (plus a KVA page to catch stack overflows?) --> Figure around 16Mbytes for 1000 kernel threads. I don't think this would be an issue for a 64bit arch with quite a few Gbytes of RAM? - For i386, it appears to be a 8K stack (plus a KVA page to catch stack overflows?) --> Figure 8Mbytes for 1000 kernel threads. Still sounds ok to me, although I think the KVA limit is about 430Mbytes by default. I'd guess that KVA exhaustion due to mbuf and other malloc() allocations will happen before having some extra nfsd threads, will occur. I have succeeded in exhausting KVA with the server running on a 256Mbyte i386, but it took several days of heavy load to get it to happen and I never found a reliable way to do it. > > > > What would be interesting would be the receive queue lengths for > > the > > sockets for NFS client TCP connections when the server is running > > normally. (This would be an indication of how many outstanding RPC > > requests any scheduling effort would select between.) > > I'll admit (given basic queuing theory) I would have expected these > > receive queues to be small unless the server is overloaded. > > > > Oh, and I now realize my response related to your first idea > > "Admission" was way off and didn't make much sense. Somehow, I > > thought receive queue when you were talking about send queue. > > (Basically, just ignore my response.) > > However, given the different sizes of RPC replies, it might > > be hard to come up with a reasonable high water mark for the > > send queue. Also, the networking code would have to do some > > sort of upcall to the krpc when the send queue shrinks. > > (So, still not trivial to implement, I think?) > > > > I do agree with Alfred, in that I think you are experiencing > > nfsd thread starvation and increasing the number of nfsd threads > > a lot is the simple way to resolve this. > > This also increases indirect scheduler costs. Direct management of > the runqueues costs proportionally to the number of runnable threads, > but some rare operations have to account all threads. > Since the extra nfsd threads won't be runable, I'd guess that the rare operations will be the only ones affected. Thanks for pointing this out, rick ps: Does increasing the MAXNFSDCNT to something like 2048 sound reasonable for FreeBSD11? (I'm not talking default, but the maximum a server can be set to.) From owner-freebsd-net@FreeBSD.ORG Thu Feb 26 23:00:35 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D3FDB9F3; Thu, 26 Feb 2015 23:00:35 +0000 (UTC) Received: from cell.glebius.int.ru (glebius.int.ru [81.19.69.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "cell.glebius.int.ru", Issuer "cell.glebius.int.ru" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 5941F69D; Thu, 26 Feb 2015 23:00:34 +0000 (UTC) Received: from cell.glebius.int.ru (localhost [127.0.0.1]) by cell.glebius.int.ru (8.14.9/8.14.9) with ESMTP id t1QN0WMC031904 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 27 Feb 2015 02:00:32 +0300 (MSK) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebius.int.ru (8.14.9/8.14.9/Submit) id t1QN0Vxt031903; Fri, 27 Feb 2015 02:00:31 +0300 (MSK) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebius.int.ru: glebius set sender to glebius@FreeBSD.org using -f Date: Fri, 27 Feb 2015 02:00:31 +0300 From: Gleb Smirnoff To: Mike Karels Subject: Re: Adding new media types to if_media.h Message-ID: <20150226230031.GN17947@glebius.int.ru> References: <20150225225051.GE17947@glebius.int.ru> <201502260258.t1Q2wKNK054143@mail.karels.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201502260258.t1Q2wKNK054143@mail.karels.net> User-Agent: Mutt/1.5.23 (2014-03-12) Cc: "freebsd-net@freebsd.org" , Eric Joyner , Jack Vogel , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Feb 2015 23:00:36 -0000 On Wed, Feb 25, 2015 at 08:58:20PM -0600, Mike Karels wrote: M> > On Wed, Feb 25, 2015 at 02:42:16PM -0800, Eric Joyner wrote: M> > E> Tbh, I respect Gleb's approach, but developing such a thing would take a M> > E> while; the fix Mike proposed would be a fix now. M> > E> M> > E> I mean, I'd like to see a decoupling of media types and speeds from M> > E> "standard" names, and maybe have both an ability to query what modules a M> > E> device supports and what speeds it supports given its current module, M> > E> instead of relying on the clunky media type list now. And then it'd be nice M> > E> to set flow control from ifconfig, too, without having to couple those M> > E> modes to media types. I'm still all for having a new system in the future. M> > E> M> > E> But in changing the KPI so much, it'd be important to consider the "do we M> > E> need an ethtool-equivalent" discussion. M> > E> M> > E> And I think we've lost a bunch of people who were in the original M> > E> discussion from the to:/cc: list. M> M> > Actually the amount of code for my approach is approximately the same M> > as with Mike's. The only thing we must sit down and think without a hurry M> > are the required and spare fields for new if_media. We definitely need M> > input from Adrian on his net80211 requirements, and input from all involved M> > parties. M> M> I'm not sure what would be different about your approach; you mentioned "n" M> versions rather than "x" versions of the ioctls, but I don't know what you M> have in mind for encoding. Any compatible version would be limited to int. The difference is that I suggest to go with a completely new interface. Yep, as you say, if_media is basically wrong. So new ioctl will use new non-wrong structure as argument. And we achieve new feature in 10.2 by merging new ioctl back there, where it will coexist with old unmodified interface. While in head, we no longer need to carry forth the wrong if_media. M> In terms of a "real" fix ("ripping the bandaid off"), I think that M> if_media is basically wrong, and widening it won't fix it. There should M> be a generic structure that reports the media type (e.g. Ethernet), M> perhaps the speed and some generic status ("active"). Then there should M> be media-specific structures that encode the appropriate things including M> attachment type. 802.11 apparently already has an extension, and Ethernet M> should have a similar extension. The KPI should be media-type-specific. M> I don't see something like this being designed soon, and certainly wouldn't M> be able to be MFC'd. Meanwhile, many of us need to support 40 Gb/s Ethernet M> on non-current (or non-future) systems. M> M> > I'm willing to code this if we all agree on the topic, so that you will M> > code all done and commited before 10.2-RELEASE. M> M> I'd be interested in a sketch or more extended description sometime before M> 10.2. I will try to show smth soon. -- Totus tuus, Glebius. From owner-freebsd-net@FreeBSD.ORG Fri Feb 27 04:11:33 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 54977946; Fri, 27 Feb 2015 04:11:33 +0000 (UTC) Received: from mail.karels.net (mail.karels.net [63.231.190.5]) by mx1.freebsd.org (Postfix) with ESMTP id E0E49BC5; Fri, 27 Feb 2015 04:11:32 +0000 (UTC) Received: from mail.karels.net (localhost [127.0.0.1]) by mail.karels.net (8.14.7/8.14.7) with ESMTP id t1R4BTee058023; Thu, 26 Feb 2015 22:11:30 -0600 (CST) (envelope-from mike@karels.net) Message-Id: <201502270411.t1R4BTee058023@mail.karels.net> To: Hans Petter Selasky From: Mike Karels Reply-to: mike@karels.net Subject: Re: Adding new media types to if_media.h In-reply-to: Your message of Thu, 26 Feb 2015 15:51:37 +0100. <54EF32F9.1020009@selasky.org> Date: Thu, 26 Feb 2015 22:11:29 -0600 Cc: "freebsd-net@freebsd.org" , Eric Joyner , Jack Vogel , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Feb 2015 04:11:33 -0000 > > Hi, > > > > There are 6 token ring bits, which I presume are available when token > > ring is not selected. > > > > #define IFM_TOK_ETR 0x00000200 /* Early token release */ > > #define IFM_TOK_SRCRT 0x00000400 /* Enable source routing > > features */ > > #define IFM_TOK_ALLR 0x00000800 /* All routes / Single route > > bcast */ > > #define IFM_TOK_DTR 0x00002000 /* Dedicated token ring */ > > #define IFM_TOK_CLASSIC 0x00004000 /* Classic token ring */ > > #define IFM_TOK_AUTO 0x00008000 /* Automatic Dedicate/Classic > > token ring */ > > > > Maybe these can be used for other purposes when the type is equal to > > ethernet? These are the "type-specific options". Ethernet uses three of these (see IFM_ETH_*). I hadn't thought about it, but some of the remaining five bits could be used for extended Ethernet subtypes. > Hi Mike, > My proposal is, convert: > > #define IFM_TOK_DTR 0x00002000 /* Dedicated token ring */ > > #define IFM_TOK_CLASSIC 0x00004000 /* Classic token ring */ > > #define IFM_TOK_AUTO 0x00008000 /* Automatic Dedicate/Classic > Into different network types: > #define IFM_TOKEN_NONE > #define IFM_TOKEN_DTR > #define IFM_TOKEN_CLASSIC > #define IFM_TOKEN_AUTO > and extend the IFM_NMASK like this: > #define IFM_NMASK 0x0000e0e0 /* Network type */ I don't think any change is required for token ring. Finding hardware to test would be a challenge in any case. Simply using some of the options bits for subtype would be simpler. I could outline this if anyone wants. Mike From owner-freebsd-net@FreeBSD.ORG Fri Feb 27 04:17:04 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2D6FAA9D; Fri, 27 Feb 2015 04:17:04 +0000 (UTC) Received: from mail.karels.net (mail.karels.net [63.231.190.5]) by mx1.freebsd.org (Postfix) with ESMTP id BBCB6BE7; Fri, 27 Feb 2015 04:17:03 +0000 (UTC) Received: from mail.karels.net (localhost [127.0.0.1]) by mail.karels.net (8.14.7/8.14.7) with ESMTP id t1R4H37Y058057; Thu, 26 Feb 2015 22:17:03 -0600 (CST) (envelope-from mike@karels.net) Message-Id: <201502270417.t1R4H37Y058057@mail.karels.net> To: Gleb Smirnoff From: Mike Karels Reply-to: mike@karels.net Subject: Re: Adding new media types to if_media.h In-reply-to: Your message of Fri, 27 Feb 2015 02:00:31 +0300. <20150226230031.GN17947@glebius.int.ru> Date: Thu, 26 Feb 2015 22:17:03 -0600 Cc: "freebsd-net@freebsd.org" , Eric Joyner , Jack Vogel , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Feb 2015 04:17:04 -0000 > M> I'm not sure what would be different about your approach; you mentioned "n" > M> versions rather than "x" versions of the ioctls, but I don't know what you > M> have in mind for encoding. Any compatible version would be limited to int. > The difference is that I suggest to go with a completely new interface. Yep, > as you say, if_media is basically wrong. So new ioctl will use new non-wrong > structure as argument. I think that part of the wrongness of if_media is to try to create a one-size-fits-all-network-types interface. If the replacement is a single ioctl, I don't think it's enough of an improvement to break the driver KPI. > And we achieve new feature in 10.2 by merging new ioctl back there, where > it will coexist with old unmodified interface. While in head, we no longer > need to carry forth the wrong if_media. I would think that this would be a huge problem for driver modules. And I think the old if_media will need to be supported for the user-level API for some time, unless someone is going to fix a lot of ports. Also, many of us are backporting much before 10.2. Mike From owner-freebsd-net@FreeBSD.ORG Fri Feb 27 04:26:00 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C5291C3F; Fri, 27 Feb 2015 04:26:00 +0000 (UTC) Received: from mail-ie0-x236.google.com (mail-ie0-x236.google.com [IPv6:2607:f8b0:4001:c03::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 89C97CB9; Fri, 27 Feb 2015 04:26:00 +0000 (UTC) Received: by iecrp18 with SMTP id rp18so25578697iec.9; Thu, 26 Feb 2015 20:26:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=eQ7qCGqmKvS8aCXViddXsmkEP9pmB7eBbMcA2OU7M8I=; b=zGqIq1aGvxP1meZUYAwiGyNk1SXvJJJMpXnzeqzYQklNhQi/rQFtutECbvs1Twq1S9 GUH8mz05MTFBsuvrdfk6eomJYbsE2J6Epo4dMxGj92uGMUteL16MEb3HK51A4R+UER/0 tpXgFJy+ZPU+vSuG46+ER+x7D8YpUkfZQNDYWoqCaYRUZImc4XpSdfnWGDJ2AYcoiEuq NcUb6IQoVJ/O8rTA6jU3niYWllc4OgbawMXsekzWsQK7k4wllM5IzwP+Ms+XOrutJEPK S6LZCHpXmnw92X51kPVMEyb7pmZLODSf65thiqefkrrRx+E1OZPnXRWg/s6pXiic34ie GGGg== MIME-Version: 1.0 X-Received: by 10.50.66.170 with SMTP id g10mr484956igt.49.1425011160083; Thu, 26 Feb 2015 20:26:00 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.36.17.66 with HTTP; Thu, 26 Feb 2015 20:25:59 -0800 (PST) In-Reply-To: <201502270417.t1R4H37Y058057@mail.karels.net> References: <20150226230031.GN17947@glebius.int.ru> <201502270417.t1R4H37Y058057@mail.karels.net> Date: Thu, 26 Feb 2015 20:25:59 -0800 X-Google-Sender-Auth: niK7omPtjjM7VOy2dINawBj6xxw Message-ID: Subject: Re: Adding new media types to if_media.h From: Adrian Chadd To: mike@karels.net Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-net@freebsd.org" , Eric Joyner , Jack Vogel , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Feb 2015 04:26:00 -0000 [snip] I think Mike's approach is good - it makes it easy to MFC to 10.2 since there's extended lifecycle stuff to do there - and then we can plan out how do the "betterer" fix after it's landed and churned things. -a From owner-freebsd-net@FreeBSD.ORG Fri Feb 27 05:56:36 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2620D861; Fri, 27 Feb 2015 05:56:36 +0000 (UTC) Received: from mail-we0-x22e.google.com (mail-we0-x22e.google.com [IPv6:2a00:1450:400c:c03::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A8B6E6E0; Fri, 27 Feb 2015 05:56:35 +0000 (UTC) Received: by wesp10 with SMTP id p10so15237352wes.12; Thu, 26 Feb 2015 21:56:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=ZW4Szw2euCIuED6BM8osB3JmUUvhYMIBLB7qFEx2GfY=; b=c4NPLOEJqabcrF9Keuqllg6ZXW1xWxG6jEwVWjxvKUOZPnfmz0Shzoz6k+/PnFs1W5 nrvWzHb1N5dw0B8KxrDPZhGeQg6CNByPWCx7OV9qwrhf3EW4lDcdjvuaSP+JKeCTr+C4 7VahJseHjqqrI3htbxxgX/oH3hlsLFAhf5TVCTloxKZ5MJzsikkl72ztwCWnVk4GqUbW m7dTPw9RVsrRBwtFs75bNsfj/SR4u6HxrAF1RxHnahpb8P5iQY+DQiZ5M9R9b1mXjynm 7wuQA7/avrdUf+7PCH5YAgBCPltry0By3Mhvr3hFoAahk8kxrQKrZllQcpKx8iYgyE77 j63w== MIME-Version: 1.0 X-Received: by 10.180.38.76 with SMTP id e12mr2989387wik.76.1425016594048; Thu, 26 Feb 2015 21:56:34 -0800 (PST) Received: by 10.194.101.106 with HTTP; Thu, 26 Feb 2015 21:56:33 -0800 (PST) In-Reply-To: References: <20150226230031.GN17947@glebius.int.ru> <201502270417.t1R4H37Y058057@mail.karels.net> Date: Thu, 26 Feb 2015 21:56:33 -0800 Message-ID: Subject: Re: Adding new media types to if_media.h From: Jack Vogel To: Adrian Chadd Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: "freebsd-net@freebsd.org" , Eric Joyner , Mike Karels , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Feb 2015 05:56:36 -0000 I agree, like I said, we have hardware coming this year that will need support, and as we just saw 11 isn't until 2016. Eric tested Mike's code with our needed types and it worked. If something truly revolutionary wants to be done for 11 it can be done after this and that not backed into the 10.X stream. Jack On Thu, Feb 26, 2015 at 8:25 PM, Adrian Chadd wrote: > [snip] > > I think Mike's approach is good - it makes it easy to MFC to 10.2 > since there's extended lifecycle stuff to do there - and then we can > plan out how do the "betterer" fix after it's landed and churned > things. > > > > -a > From owner-freebsd-net@FreeBSD.ORG Fri Feb 27 15:21:21 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 247453FE; Fri, 27 Feb 2015 15:21:21 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id F11A1A59; Fri, 27 Feb 2015 15:21:20 +0000 (UTC) Received: from ralph.baldwin.cx (pool-173-54-116-245.nwrknj.fios.verizon.net [173.54.116.245]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 90323B915; Fri, 27 Feb 2015 10:21:19 -0500 (EST) From: John Baldwin To: freebsd-net@freebsd.org Subject: Re: Accessing socket APIs soon after accept Date: Fri, 27 Feb 2015 10:20:52 -0500 Message-ID: <4258761.a1pROkzJe6@ralph.baldwin.cx> User-Agent: KMail/4.14.2 (FreeBSD/10.1-STABLE; KDE/4.14.2; amd64; ; ) In-Reply-To: <1421428048190.48193@netapp.com> References: <1421339375968.94209@netapp.com> <1421428048190.48193@netapp.com> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Fri, 27 Feb 2015 10:21:19 -0500 (EST) Cc: Adrian Chadd , "Quattlebaum, Ryan" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Feb 2015 15:21:21 -0000 On Friday, January 16, 2015 05:07:28 PM Quattlebaum, Ryan wrote: > Hi, Adrian. Thanks for taking a look at this. > > We're using FreeBSD 8.2 and httpd-2.4.10 with arp-1.5.1 and apr-util-1.5.4. > > The problem we're seeing is pretty intermittent, so I hope this test case > can shine a little bit of light on the > problem. We tried debugging this on our own by adding calls to > getsockname() right after the accept call (in > srclib/apr/network_io/unix/sockets.c: apr_socket_accept()) and logging the > output. That's where we saw invalid data. > > I took a look at the source code for the TCP syncache module and the accept > syscall. It looks like the new child socket is available for the > application to accept after the call to sonewconn returns, but the address > information isn't set until further down in the function. Wouldn't this > open a window where an application could accept on a socket that the > syncache code isn't done configuring? This is a bug in 8.x it seems. It was fixed in HEAD in this commit: ------------------------------------------------------------------------ r261242 | gnn | 2014-01-28 15:28:32 -0500 (Tue, 28 Jan 2014) | 10 lines Decrease lock contention within the TCP accept case by removing the INP_INFO lock from tcp_usr_accept. As the PR/patch states this was following the advice already in the code. See the PR below for a full disucssion of this change and its measured effects. PR: 183659 Submitted by: Julian Charbon Reviewed by: jhb In particular, that commit changed the syncache code to not place the socket in the queue until the end of the function via soisconnected(). You can probably merge the tcp_syncache.c portion of that change back to 8.x without any ill effects and it should fix your problem. -- John Baldwin From owner-freebsd-net@FreeBSD.ORG Fri Feb 27 15:28:00 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 83E2564C; Fri, 27 Feb 2015 15:28:00 +0000 (UTC) Received: from mx143.netapp.com (mx143.netapp.com [216.240.21.24]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "mx143.netapp.com", Issuer "VeriSign Class 3 International Server CA - G3" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 3255DABC; Fri, 27 Feb 2015 15:27:59 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.09,660,1418112000"; d="scan'208";a="26388997" Received: from hioexcmbx06-prd.hq.netapp.com ([10.122.105.39]) by mx143-out.netapp.com with ESMTP; 27 Feb 2015 07:22:44 -0800 Received: from HIOEXCMBX02-PRD.hq.netapp.com (10.122.105.35) by hioexcmbx06-prd.hq.netapp.com (10.122.105.39) with Microsoft SMTP Server (TLS) id 15.0.995.29; Fri, 27 Feb 2015 07:22:44 -0800 Received: from HIOEXCMBX02-PRD.hq.netapp.com ([fe80::98d0:d570:a8a3:beae]) by hioexcmbx02-prd.hq.netapp.com ([fe80::98d0:d570:a8a3:beae%21]) with mapi id 15.00.0995.031; Fri, 27 Feb 2015 07:22:43 -0800 From: "Quattlebaum, Ryan" To: John Baldwin , "freebsd-net@freebsd.org" Subject: RE: Accessing socket APIs soon after accept Thread-Topic: Accessing socket APIs soon after accept Thread-Index: AQHQMN+s1D+Mgs6QWka0EkIfHwtzNpzCBCMAgAD35iGAQmpFAP//ekh/ Date: Fri, 27 Feb 2015 15:22:43 +0000 Message-ID: <1425050565215.33356@netapp.com> References: <1421339375968.94209@netapp.com> <1421428048190.48193@netapp.com>,<4258761.a1pROkzJe6@ralph.baldwin.cx> In-Reply-To: <4258761.a1pROkzJe6@ralph.baldwin.cx> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.122.56.79] Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: Adrian Chadd X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Feb 2015 15:28:00 -0000 Thanks, John. That's almost exactly the approach we were considering.=0A= =0A= - Ryan Q=0A= ________________________________________=0A= From: John Baldwin =0A= Sent: Friday, February 27, 2015 10:20 AM=0A= To: freebsd-net@freebsd.org=0A= Cc: Quattlebaum, Ryan; Adrian Chadd=0A= Subject: Re: Accessing socket APIs soon after accept=0A= =0A= On Friday, January 16, 2015 05:07:28 PM Quattlebaum, Ryan wrote:=0A= > Hi, Adrian. Thanks for taking a look at this.=0A= >=0A= > We're using FreeBSD 8.2 and httpd-2.4.10 with arp-1.5.1 and apr-util-1.5.= 4.=0A= >=0A= > The problem we're seeing is pretty intermittent, so I hope this test case= =0A= > can shine a little bit of light on t= he=0A= > problem. We tried debugging this on our own by adding calls to=0A= > getsockname() right after the accept call (in=0A= > srclib/apr/network_io/unix/sockets.c: apr_socket_accept()) and logging th= e=0A= > output. That's where we saw invalid data.=0A= >=0A= > I took a look at the source code for the TCP syncache module and the acce= pt=0A= > syscall. It looks like the new child socket is available for the=0A= > application to accept after the call to sonewconn returns, but the addres= s=0A= > information isn't set until further down in the function. Wouldn't this= =0A= > open a window where an application could accept on a socket that the=0A= > syncache code isn't done configuring?=0A= =0A= This is a bug in 8.x it seems. It was fixed in HEAD in this commit:=0A= =0A= ------------------------------------------------------------------------=0A= r261242 | gnn | 2014-01-28 15:28:32 -0500 (Tue, 28 Jan 2014) | 10 lines=0A= =0A= Decrease lock contention within the TCP accept case by removing=0A= the INP_INFO lock from tcp_usr_accept. As the PR/patch states=0A= this was following the advice already in the code.=0A= See the PR below for a full disucssion of this change and its=0A= measured effects.=0A= =0A= PR: 183659=0A= Submitted by: Julian Charbon=0A= Reviewed by: jhb=0A= =0A= In particular, that commit changed the syncache code to not place the socke= t=0A= in the queue until the end of the function via soisconnected().=0A= =0A= You can probably merge the tcp_syncache.c portion of that change back to 8.= x=0A= without any ill effects and it should fix your problem.=0A= =0A= --=0A= John Baldwin=0A= From owner-freebsd-net@FreeBSD.ORG Fri Feb 27 17:23:21 2015 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2E775F1C for ; Fri, 27 Feb 2015 17:23:21 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 14854A32 for ; Fri, 27 Feb 2015 17:23:21 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id t1RHNK5H063635 for ; Fri, 27 Feb 2015 17:23:20 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 165622] [ndis][panic][patch] Unregistered use of FPU in kernel on amd64 Date: Fri, 27 Feb 2015 17:23:21 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: jhb@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Feb 2015 17:23:21 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=165622 John Baldwin changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jhb@FreeBSD.org --- Comment #4 from John Baldwin --- Comment on attachment 122422 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=122422 fpu_patch3.txt Did you consider removing the O(n) lookup by keeping separate "busy" and "free" lists? You could then also assert that that the busy list was empty in the windrv_libfini() routine. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Fri Feb 27 18:03:34 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 478FCA1A; Fri, 27 Feb 2015 18:03:34 +0000 (UTC) Received: from mail-ie0-x233.google.com (mail-ie0-x233.google.com [IPv6:2607:f8b0:4001:c03::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0B5C9ED4; Fri, 27 Feb 2015 18:03:34 +0000 (UTC) Received: by iecar1 with SMTP id ar1so32953736iec.0; Fri, 27 Feb 2015 10:03:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=Niehm15p6jlbOyfLV4wsMIOpUljRi8NxSWU+DUXDF1Q=; b=L5bdsQzZs7zFtTTlzwmZMH0Vi9JjiCQn+EATkN8RRKaVYTLzQiKLVg3QlE3zsC5ngb 3IUWc7fraS38yKnKfi5726M8K8FzjePDMILRH11ZlZHWMp4r4chpwWtJO+KlEdNyDHqw DftW5cecU/RyZeZ74v3WM618HkOqakgCT3D7UBluaPjwhtIftCbXGQvQtyTmF7/mg65c wtOsH2pyJbm6AOjBpj1NZH3lQNcmVP/YB6lJmK8rwrxBf8TzxxUi1CZIOZlPru4uYUZj Ovhmk5vjKA3RhTwLlroLaXHQlYBhSuQnD0ld2G6A0apySweGT8WA2EuPPGR1lRw4IXXz MuWQ== MIME-Version: 1.0 X-Received: by 10.42.201.78 with SMTP id ez14mr17014873icb.22.1425060213391; Fri, 27 Feb 2015 10:03:33 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.36.17.66 with HTTP; Fri, 27 Feb 2015 10:03:33 -0800 (PST) In-Reply-To: <1425050565215.33356@netapp.com> References: <1421339375968.94209@netapp.com> <1421428048190.48193@netapp.com> <4258761.a1pROkzJe6@ralph.baldwin.cx> <1425050565215.33356@netapp.com> Date: Fri, 27 Feb 2015 10:03:33 -0800 X-Google-Sender-Auth: HDnpBR4QqQZqCQhJQE8KKLpVBuc Message-ID: Subject: Re: Accessing socket APIs soon after accept From: Adrian Chadd To: "Quattlebaum, Ryan" Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-net@freebsd.org" , John Baldwin X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Feb 2015 18:03:34 -0000 Is this also a bug on -9 and -10? -a On 27 February 2015 at 07:22, Quattlebaum, Ryan wrote: > Thanks, John. That's almost exactly the approach we were considering. > > - Ryan Q > ________________________________________ > From: John Baldwin > Sent: Friday, February 27, 2015 10:20 AM > To: freebsd-net@freebsd.org > Cc: Quattlebaum, Ryan; Adrian Chadd > Subject: Re: Accessing socket APIs soon after accept > > On Friday, January 16, 2015 05:07:28 PM Quattlebaum, Ryan wrote: >> Hi, Adrian. Thanks for taking a look at this. >> >> We're using FreeBSD 8.2 and httpd-2.4.10 with arp-1.5.1 and apr-util-1.5.4. >> >> The problem we're seeing is pretty intermittent, so I hope this test case >> can shine a little bit of light on the >> problem. We tried debugging this on our own by adding calls to >> getsockname() right after the accept call (in >> srclib/apr/network_io/unix/sockets.c: apr_socket_accept()) and logging the >> output. That's where we saw invalid data. >> >> I took a look at the source code for the TCP syncache module and the accept >> syscall. It looks like the new child socket is available for the >> application to accept after the call to sonewconn returns, but the address >> information isn't set until further down in the function. Wouldn't this >> open a window where an application could accept on a socket that the >> syncache code isn't done configuring? > > This is a bug in 8.x it seems. It was fixed in HEAD in this commit: > > ------------------------------------------------------------------------ > r261242 | gnn | 2014-01-28 15:28:32 -0500 (Tue, 28 Jan 2014) | 10 lines > > Decrease lock contention within the TCP accept case by removing > the INP_INFO lock from tcp_usr_accept. As the PR/patch states > this was following the advice already in the code. > See the PR below for a full disucssion of this change and its > measured effects. > > PR: 183659 > Submitted by: Julian Charbon > Reviewed by: jhb > > In particular, that commit changed the syncache code to not place the socket > in the queue until the end of the function via soisconnected(). > > You can probably merge the tcp_syncache.c portion of that change back to 8.x > without any ill effects and it should fix your problem. > > -- > John Baldwin From owner-freebsd-net@FreeBSD.ORG Fri Feb 27 18:15:09 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3876AF94; Fri, 27 Feb 2015 18:15:09 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0EC73E2; Fri, 27 Feb 2015 18:15:09 +0000 (UTC) Received: from ralph.baldwin.cx (pool-173-54-116-245.nwrknj.fios.verizon.net [173.54.116.245]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 1D642B918; Fri, 27 Feb 2015 13:15:07 -0500 (EST) From: John Baldwin To: Adrian Chadd Subject: Re: Accessing socket APIs soon after accept Date: Fri, 27 Feb 2015 13:07:33 -0500 Message-ID: <4083712.jb7qREZuG6@ralph.baldwin.cx> User-Agent: KMail/4.14.2 (FreeBSD/10.1-STABLE; KDE/4.14.2; amd64; ; ) In-Reply-To: References: <1421339375968.94209@netapp.com> <1425050565215.33356@netapp.com> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Fri, 27 Feb 2015 13:15:07 -0500 (EST) Cc: "freebsd-net@freebsd.org" , "Quattlebaum, Ryan" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Feb 2015 18:15:09 -0000 On Friday, February 27, 2015 10:03:33 AM Adrian Chadd wrote: > Is this also a bug on -9 and -10? Yes. I may merge just the tcp_syncache.c part of this change down to stable branches. > > -a > > > On 27 February 2015 at 07:22, Quattlebaum, Ryan > > wrote: > > Thanks, John. That's almost exactly the approach we were considering. > > > > - Ryan Q > > ________________________________________ > > From: John Baldwin > > Sent: Friday, February 27, 2015 10:20 AM > > To: freebsd-net@freebsd.org > > Cc: Quattlebaum, Ryan; Adrian Chadd > > Subject: Re: Accessing socket APIs soon after accept > > > > On Friday, January 16, 2015 05:07:28 PM Quattlebaum, Ryan wrote: > >> Hi, Adrian. Thanks for taking a look at this. > >> > >> We're using FreeBSD 8.2 and httpd-2.4.10 with arp-1.5.1 and > >> apr-util-1.5.4. > >> > >> The problem we're seeing is pretty intermittent, so I hope this test case > >> can shine a little bit of light on > >> the > >> problem. We tried debugging this on our own by adding calls to > >> getsockname() right after the accept call (in > >> srclib/apr/network_io/unix/sockets.c: apr_socket_accept()) and logging > >> the > >> output. That's where we saw invalid data. > >> > >> I took a look at the source code for the TCP syncache module and the > >> accept > >> syscall. It looks like the new child socket is available for the > >> application to accept after the call to sonewconn returns, but the > >> address > >> information isn't set until further down in the function. Wouldn't this > >> open a window where an application could accept on a socket that the > >> syncache code isn't done configuring? > > > > This is a bug in 8.x it seems. It was fixed in HEAD in this commit: > > > > ------------------------------------------------------------------------ > > r261242 | gnn | 2014-01-28 15:28:32 -0500 (Tue, 28 Jan 2014) | 10 lines > > > > Decrease lock contention within the TCP accept case by removing > > the INP_INFO lock from tcp_usr_accept. As the PR/patch states > > this was following the advice already in the code. > > See the PR below for a full disucssion of this change and its > > measured effects. > > > > PR: 183659 > > Submitted by: Julian Charbon > > Reviewed by: jhb > > > > In particular, that commit changed the syncache code to not place the > > socket in the queue until the end of the function via soisconnected(). > > > > You can probably merge the tcp_syncache.c portion of that change back to > > 8.x without any ill effects and it should fix your problem. > > > > -- > > John Baldwin -- John Baldwin From owner-freebsd-net@FreeBSD.ORG Fri Feb 27 18:32:18 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 01B877FD; Fri, 27 Feb 2015 18:32:18 +0000 (UTC) Received: from mail-ie0-x22e.google.com (mail-ie0-x22e.google.com [IPv6:2607:f8b0:4001:c03::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BA0FB3A0; Fri, 27 Feb 2015 18:32:17 +0000 (UTC) Received: by iecrd18 with SMTP id rd18so33019119iec.8; Fri, 27 Feb 2015 10:32:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=J0VgLpIkEsS7f8Eu9NwOKlNznpvuQxy5v/WzKvx2D7o=; b=uoQcJGowskWZI59SpAZ66Bs99luEV5VwrEXXVDWTs9WtiArXwudGsclEvg/aXYjC1k yjklymT0/Tl+uyuxOEEctHc8B379yJwREA2x6cO2nrTCVBf4x+8bsF72nK7aGyPS4ttF 9SBPzeQvFkhoWC/6pVOESyPJf2+uOP4UDq8GIf8DkIYg9vzJei6B/rFy9+sDHl9QZOT+ TZc89obSBBGnKoV3X1Q3H5FrmAWI0ezh1LmQwhHwuCmhLkjaFxVtfSAlGd12hRbxne+t foZJTNdkH4iCw8v6Fd93tQIhmNiVs/wm0fWwVuscx8VShRCl2j5Ab4ctD5WGZVPfc9EI lXuA== MIME-Version: 1.0 X-Received: by 10.50.66.170 with SMTP id g10mr4730189igt.49.1425061937115; Fri, 27 Feb 2015 10:32:17 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.36.17.66 with HTTP; Fri, 27 Feb 2015 10:32:17 -0800 (PST) In-Reply-To: <4083712.jb7qREZuG6@ralph.baldwin.cx> References: <1421339375968.94209@netapp.com> <1425050565215.33356@netapp.com> <4083712.jb7qREZuG6@ralph.baldwin.cx> Date: Fri, 27 Feb 2015 10:32:17 -0800 X-Google-Sender-Auth: yOL2EVvA_85t0uyrFunBgqRDjVQ Message-ID: Subject: Re: Accessing socket APIs soon after accept From: Adrian Chadd To: John Baldwin Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-net@freebsd.org" , "Quattlebaum, Ryan" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Feb 2015 18:32:18 -0000 On 27 February 2015 at 10:07, John Baldwin wrote: > On Friday, February 27, 2015 10:03:33 AM Adrian Chadd wrote: >> Is this also a bug on -9 and -10? > > Yes. I may merge just the tcp_syncache.c part of this change down to stable > branches. Cool, thanks. Placing half-completed connections on the queue always looked a bit odd to me.. -a From owner-freebsd-net@FreeBSD.ORG Fri Feb 27 19:19:04 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 355328B4; Fri, 27 Feb 2015 19:19:04 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0B57CBC2; Fri, 27 Feb 2015 19:19:04 +0000 (UTC) Received: from ralph.baldwin.cx (pool-173-54-116-245.nwrknj.fios.verizon.net [173.54.116.245]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 3D8E8B945; Fri, 27 Feb 2015 14:19:02 -0500 (EST) From: John Baldwin To: Adrian Chadd Subject: Re: Accessing socket APIs soon after accept Date: Fri, 27 Feb 2015 14:18:21 -0500 Message-ID: <4615961.47iyoSO4QG@ralph.baldwin.cx> User-Agent: KMail/4.14.2 (FreeBSD/10.1-STABLE; KDE/4.14.2; amd64; ; ) In-Reply-To: References: <1421339375968.94209@netapp.com> <4083712.jb7qREZuG6@ralph.baldwin.cx> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Fri, 27 Feb 2015 14:19:02 -0500 (EST) Cc: "freebsd-net@freebsd.org" , 'Robert Watson' , "Quattlebaum, Ryan" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Feb 2015 19:19:04 -0000 On Friday, February 27, 2015 10:32:17 AM Adrian Chadd wrote: > On 27 February 2015 at 10:07, John Baldwin wrote: > > On Friday, February 27, 2015 10:03:33 AM Adrian Chadd wrote: > >> Is this also a bug on -9 and -10? > > > > Yes. I may merge just the tcp_syncache.c part of this change down to > > stable branches. > > Cool, thanks. > > Placing half-completed connections on the queue always looked a bit odd to > me.. So this appears stranger. Supposedly, the tcbinfo global lock should have fixed this race. In particular, in 8.x, tcp_intput holds a write lock on the tcbinfo lock around all of syncache_expand() including all of syncache_socket() from sonewconn() on down to the end of the function not releasing it until after the addresses are all set, etc. tcp_usr_accept() on 8.x acquires a read lock on the tcbinfo global lock, so if accept() races with syncache_socket(), even though accept() might dequeue the socket from sq_comp before the socket is fully constructed, the call to soaccept() inside of accept() should call tcp_usr_accept() which will try to read-lock the tcbinfo lock and will thus block until syncache_socket() has completed. Thus, you shouldn't be able to have accept() return before syncache_socket() has finished. -- John Baldwin From owner-freebsd-net@FreeBSD.ORG Fri Feb 27 19:19:20 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EBD2F94B; Fri, 27 Feb 2015 19:19:20 +0000 (UTC) Received: from cell.glebius.int.ru (glebius.int.ru [81.19.69.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "cell.glebius.int.ru", Issuer "cell.glebius.int.ru" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 72FECBD1; Fri, 27 Feb 2015 19:19:19 +0000 (UTC) Received: from cell.glebius.int.ru (localhost [127.0.0.1]) by cell.glebius.int.ru (8.14.9/8.14.9) with ESMTP id t1RJJHQe036826 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 27 Feb 2015 22:19:17 +0300 (MSK) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebius.int.ru (8.14.9/8.14.9/Submit) id t1RJJGWq036825; Fri, 27 Feb 2015 22:19:16 +0300 (MSK) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebius.int.ru: glebius set sender to glebius@FreeBSD.org using -f Date: Fri, 27 Feb 2015 22:19:16 +0300 From: Gleb Smirnoff To: Mike Karels Subject: Re: Adding new media types to if_media.h Message-ID: <20150227191916.GQ17947@glebius.int.ru> References: <20150226230031.GN17947@glebius.int.ru> <201502270417.t1R4H37Y058057@mail.karels.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201502270417.t1R4H37Y058057@mail.karels.net> User-Agent: Mutt/1.5.23 (2014-03-12) Cc: "freebsd-net@freebsd.org" , Eric Joyner , Jack Vogel , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Feb 2015 19:19:21 -0000 On Thu, Feb 26, 2015 at 10:17:03PM -0600, Mike Karels wrote: M> > M> I'm not sure what would be different about your approach; you mentioned "n" M> > M> versions rather than "x" versions of the ioctls, but I don't know what you M> > M> have in mind for encoding. Any compatible version would be limited to int. M> M> > The difference is that I suggest to go with a completely new interface. Yep, M> > as you say, if_media is basically wrong. So new ioctl will use new non-wrong M> > structure as argument. M> M> I think that part of the wrongness of if_media is to try to create a M> one-size-fits-all-network-types interface. If the replacement is a M> single ioctl, I don't think it's enough of an improvement to break M> the driver KPI. The KPI for drivers in 11 will be changed very significantly anyway[1], so right now we got a chance to make everything properly. Why not to utilize this chance for ifmedia? Imagine you do this from scratch, and implement it the right way. And this way it will go into 11. Then shims for 10-merge can be done. And it would be okay if shims look ugly, since we won't take them with us into the future. M> > And we achieve new feature in 10.2 by merging new ioctl back there, where M> > it will coexist with old unmodified interface. While in head, we no longer M> > need to carry forth the wrong if_media. M> M> I would think that this would be a huge problem for driver modules. Why huge? The new functionality is required for a couple of drivers only. And module KPI is guaranteed to be forward compatible only: one can load older module on newer kernel, but not vice versa. So, if newer module requires some functionality from the kernel, its entirely okay. M> And I think the old if_media will need to be supported for the user-level M> API for some time, unless someone is going to fix a lot of ports. M> M> Also, many of us are backporting much before 10.2. Of course old if_media API should remain for ports. [1] See projects/ifnet subversion branch. -- Totus tuus, Glebius. From owner-freebsd-net@FreeBSD.ORG Fri Feb 27 19:23:12 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E451CC50; Fri, 27 Feb 2015 19:23:12 +0000 (UTC) Received: from cell.glebius.int.ru (glebius.int.ru [81.19.69.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "cell.glebius.int.ru", Issuer "cell.glebius.int.ru" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 6B878CA5; Fri, 27 Feb 2015 19:23:11 +0000 (UTC) Received: from cell.glebius.int.ru (localhost [127.0.0.1]) by cell.glebius.int.ru (8.14.9/8.14.9) with ESMTP id t1RJNAwQ036864 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 27 Feb 2015 22:23:10 +0300 (MSK) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebius.int.ru (8.14.9/8.14.9/Submit) id t1RJNAVU036863; Fri, 27 Feb 2015 22:23:10 +0300 (MSK) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebius.int.ru: glebius set sender to glebius@FreeBSD.org using -f Date: Fri, 27 Feb 2015 22:23:10 +0300 From: Gleb Smirnoff To: Adrian Chadd Subject: Re: Adding new media types to if_media.h Message-ID: <20150227192310.GR17947@glebius.int.ru> References: <20150226230031.GN17947@glebius.int.ru> <201502270417.t1R4H37Y058057@mail.karels.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: "freebsd-net@freebsd.org" , Eric Joyner , Jack Vogel , mike@karels.net, "freebsd-arch@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Feb 2015 19:23:13 -0000 On Thu, Feb 26, 2015 at 08:25:59PM -0800, Adrian Chadd wrote: A> [snip] A> A> I think Mike's approach is good - it makes it easy to MFC to 10.2 A> since there's extended lifecycle stuff to do there - and then we can A> plan out how do the "betterer" fix after it's landed and churned A> things. ... and we will be ought to support the "betterer" fix along with the "not so betterer" for a very long time. The rock on which we split in this argument is that some developers write their code for stable/x and then forward-port it to head, focused on quality of result for stable/x; while other developers do the opposite: write code to head, then consider or not consider merging it stable/x. -- Totus tuus, Glebius. From owner-freebsd-net@FreeBSD.ORG Fri Feb 27 19:28:06 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 270A7E4A for ; Fri, 27 Feb 2015 19:28:06 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [IPv6:2001:4f8:3:ffe0:406a:0:50:2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 03E1DCFD for ; Fri, 27 Feb 2015 19:28:06 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [127.0.1.5]) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9) with ESMTP id t1RJS5nV032573 for ; Fri, 27 Feb 2015 19:28:05 GMT (envelope-from root@phabric-backend.isc.freebsd.org) Received: (from root@localhost) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9/Submit) id t1RJS5NM032572; Fri, 27 Feb 2015 19:28:05 GMT (envelope-from root) Date: Fri, 27 Feb 2015 19:28:05 +0000 To: freebsd-net@freebsd.org From: "glebius (Gleb Smirnoff)" Subject: [Differential] [Updated] D1944: PF and VIMAGE fixes Message-ID: <2ee64a84c0e5f3ab551e5034e87677b6@localhost.localdomain> X-Priority: 3 Thread-Topic: D1944: PF and VIMAGE fixes X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: X-Phabricator-Cc: X-Phabricator-Cc: In-Reply-To: References: Thread-Index: NDc2NzM0MzY4OTdiYThiNTU1MjY2ZDZmMTJiIFTwxUU= X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Feb 2015 19:28:06 -0000 glebius added a comment. Nikos, acking that I see the patches. Right now I'm waiting for pf to stablize after recent patches to fragment handling. Kristof is working on the known problem. Meanwhile you can finish your patch moving from "almost there" to "there" :) If you got any questions about pf or FreeBSD kernel interfaces, feel free to ask me via email. REVISION DETAIL https://reviews.freebsd.org/D1944 To: nvass-gmx.com, gnn, bz, zec, trociny, rodrigc, glebius Cc: freebsd-virtualization, freebsd-pf, freebsd-net From owner-freebsd-net@FreeBSD.ORG Fri Feb 27 21:58:28 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9CAAFFFC; Fri, 27 Feb 2015 21:58:28 +0000 (UTC) Received: from mail-wg0-x232.google.com (mail-wg0-x232.google.com [IPv6:2a00:1450:400c:c00::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2B0D3F6B; Fri, 27 Feb 2015 21:58:28 +0000 (UTC) Received: by wgha1 with SMTP id a1so22938255wgh.12; Fri, 27 Feb 2015 13:58:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=bZpkrHrM/VKQFp5R5pZ95k9XQsUoWNg60KMvdAkvR4I=; b=jqe2Af445fTtxB9pSgVCgheqJy2S2UkifLHG8ugqLaX4qqYWbyo9TcgFuz0LFSqNS7 LCpymDDj/YVtI64AFX3dll2N6qQn7cuOinUbAMvoD6EVMTHP1v23rKfaxYujmZy4WCkH Fmeh1s5CYCMEc6a//TpVujFjXDLq390fxbexG4GK21B4YzId4wK2YXleNEH5LF0+RKTA TFdB7Us71nWOgiA5j2dq+7yzRu7kXbqTqxbNvUePSlrVIbEI0n/eEWxDzZwwx03r6j1S ZAutwcC5pYtEbm8klwuVPrlcOhPESGByajOWEdLRj4DXmPyOyuMN/mlsdRmwZGa4EHwV umOg== MIME-Version: 1.0 X-Received: by 10.180.38.76 with SMTP id e12mr10565678wik.76.1425074306512; Fri, 27 Feb 2015 13:58:26 -0800 (PST) Received: by 10.194.101.106 with HTTP; Fri, 27 Feb 2015 13:58:26 -0800 (PST) In-Reply-To: <20150227192310.GR17947@glebius.int.ru> References: <20150226230031.GN17947@glebius.int.ru> <201502270417.t1R4H37Y058057@mail.karels.net> <20150227192310.GR17947@glebius.int.ru> Date: Fri, 27 Feb 2015 13:58:26 -0800 Message-ID: Subject: Re: Adding new media types to if_media.h From: Jack Vogel To: Gleb Smirnoff Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: "freebsd-net@freebsd.org" , Adrian Chadd , Eric Joyner , Mike Karels , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Feb 2015 21:58:28 -0000 On Fri, Feb 27, 2015 at 11:23 AM, Gleb Smirnoff wrote: > On Thu, Feb 26, 2015 at 08:25:59PM -0800, Adrian Chadd wrote: > A> [snip] > A> > A> I think Mike's approach is good - it makes it easy to MFC to 10.2 > A> since there's extended lifecycle stuff to do there - and then we can > A> plan out how do the "betterer" fix after it's landed and churned > A> things. > > ... and we will be ought to support the "betterer" fix along with > the "not so betterer" for a very long time. > > The rock on which we split in this argument is that some developers > write their code for stable/x and then forward-port it to head, > focused on quality of result for stable/x; while other developers > do the opposite: write code to head, then consider or not consider > merging it stable/x. > > I think this is oversimplified. In my 10 years in this role at Intel I've had cases when, in response to a customer issue, I had to work from an existing code base to solve a problem, or add support for a new feature. But then there are other times when I've been working on a new driver, and its been totally developed from HEAD. It depends on what's right for the circumstance, but as I said, on this issue we have real product/customer needs that are short term, and the competition (Linux and Windows) is prepared to handle the new media today, I think its in FreeBSD's interest to address this ASAP. Jack From owner-freebsd-net@FreeBSD.ORG Fri Feb 27 23:15:17 2015 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0D7DF8D8 for ; Fri, 27 Feb 2015 23:15:17 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E7FB4A60 for ; Fri, 27 Feb 2015 23:15:16 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id t1RNFGP7066231 for ; Fri, 27 Feb 2015 23:15:16 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 193802] tso seems broken on RELENG10 for version 7.4.2 of em driver Date: Fri, 27 Feb 2015 23:15:17 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: hiren@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Feb 2015 23:15:17 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=193802 Hiren Panchasara changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |erj@freebsd.org, | |jfv@FreeBSD.org Assignee|freebsd-bugs@FreeBSD.org |freebsd-net@FreeBSD.org --- Comment #3 from Hiren Panchasara --- Moving to -net and CCing Jack and Eric from Intel. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Sat Feb 28 04:29:03 2015 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D5567249 for ; Sat, 28 Feb 2015 04:29:03 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BB2B3E87 for ; Sat, 28 Feb 2015 04:29:03 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id t1S4T3bF089900 for ; Sat, 28 Feb 2015 04:29:03 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 197997] [panic] ng_pppoe sometimes panics with trap 12 when server drops session Date: Sat, 28 Feb 2015 04:29:03 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Feb 2015 04:29:03 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=197997 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-net@FreeBSD.org -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Sat Feb 28 04:31:10 2015 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D5733395 for ; Sat, 28 Feb 2015 04:31:10 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BB2ECF31 for ; Sat, 28 Feb 2015 04:31:10 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id t1S4VAAT094884 for ; Sat, 28 Feb 2015 04:31:10 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 197887] msk0 watchdog timeout 88E8071 PCI-E Gigabit Ethernet Controller Date: Sat, 28 Feb 2015 04:31:10 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Feb 2015 04:31:10 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=197887 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-net@FreeBSD.org -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Sat Feb 28 04:31:26 2015 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1106E495 for ; Sat, 28 Feb 2015 04:31:26 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E26FEF55 for ; Sat, 28 Feb 2015 04:31:25 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id t1S4VPJi095663 for ; Sat, 28 Feb 2015 04:31:25 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 197882] [panic] kernel panics in soreceive_dgram Date: Sat, 28 Feb 2015 04:31:25 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Feb 2015 04:31:26 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=197882 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-net@FreeBSD.org -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Sat Feb 28 12:41:30 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8053BD5E; Sat, 28 Feb 2015 12:41:30 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5699BB4; Sat, 28 Feb 2015 12:41:30 +0000 (UTC) Received: from ralph.baldwin.cx (pool-173-54-116-245.nwrknj.fios.verizon.net [173.54.116.245]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 5401FB9A3; Sat, 28 Feb 2015 07:41:29 -0500 (EST) From: John Baldwin To: freebsd-arch@freebsd.org Subject: Re: Adding new media types to if_media.h Date: Sat, 28 Feb 2015 07:28:14 -0500 Message-ID: <1919032.aFEK3un8ig@ralph.baldwin.cx> User-Agent: KMail/4.14.2 (FreeBSD/10.1-STABLE; KDE/4.14.2; amd64; ; ) In-Reply-To: <20150227192310.GR17947@glebius.int.ru> References: <20150226230031.GN17947@glebius.int.ru> <20150227192310.GR17947@glebius.int.ru> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Sat, 28 Feb 2015 07:41:29 -0500 (EST) Cc: Adrian Chadd , mike@karels.net, "freebsd-net@freebsd.org" , Jack Vogel , Eric Joyner X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Feb 2015 12:41:30 -0000 On Friday, February 27, 2015 10:23:10 PM Gleb Smirnoff wrote: > On Thu, Feb 26, 2015 at 08:25:59PM -0800, Adrian Chadd wrote: > A> [snip] > A> > A> I think Mike's approach is good - it makes it easy to MFC to 10.2 > A> since there's extended lifecycle stuff to do there - and then we can > A> plan out how do the "betterer" fix after it's landed and churned > A> things. > > ... and we will be ought to support the "betterer" fix along with > the "not so betterer" for a very long time. > > The rock on which we split in this argument is that some developers > write their code for stable/x and then forward-port it to head, > focused on quality of result for stable/x; while other developers > do the opposite: write code to head, then consider or not consider > merging it stable/x. No, this is not quite true. Some folks have to write drivers on HEAD but also support running those drivers on older branches. The MFC's get harder when you have very different APIs on the different branches. It's already harder to test stat changes now since it requires completely different patches for <= 10 (the only thing people are supposed to use in production) vs head due to if_getcounter() and friends. Also, since 11 won't be out until 2016, that is far, far too long to wait for more media types. The stuff we need to support is already shipping in products today. We can't not support these in 10 (and possibly 9). -- John Baldwin From owner-freebsd-net@FreeBSD.ORG Sat Feb 28 13:56:05 2015 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1B45B971 for ; Sat, 28 Feb 2015 13:56:05 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 020EB8D3 for ; Sat, 28 Feb 2015 13:56:05 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id t1SDu4ub013961 for ; Sat, 28 Feb 2015 13:56:04 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 197997] [panic] ng_pppoe sometimes panics with trap 12 when server drops session Date: Sat, 28 Feb 2015 13:56:05 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: eugen@grosbein.net X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Feb 2015 13:56:05 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=197997 eugen@grosbein.net changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |eugen@grosbein.net --- Comment #1 from eugen@grosbein.net --- Do you have crashdump and kgdb backtrace? -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Sat Feb 28 14:10:15 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6D0C6ADD for ; Sat, 28 Feb 2015 14:10:15 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [IPv6:2001:4f8:3:ffe0:406a:0:50:2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4A9899C3 for ; Sat, 28 Feb 2015 14:10:15 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [127.0.1.5]) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9) with ESMTP id t1SEAFDU069788 for ; Sat, 28 Feb 2015 14:10:15 GMT (envelope-from root@phabric-backend.isc.freebsd.org) Received: (from root@localhost) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9/Submit) id t1SEAFPl069787; Sat, 28 Feb 2015 14:10:15 GMT (envelope-from root) Date: Sat, 28 Feb 2015 14:10:15 +0000 To: freebsd-net@freebsd.org From: "nvass-gmx.com (Nikos Vassiliadis)" Subject: [Differential] [Commented On] D1944: PF and VIMAGE fixes Message-ID: X-Priority: 3 Thread-Topic: D1944: PF and VIMAGE fixes X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: X-Phabricator-Cc: X-Phabricator-Cc: In-Reply-To: References: Thread-Index: NDc2NzM0MzY4OTdiYThiNTU1MjY2ZDZmMTJiIFTxzEc= X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Feb 2015 14:10:15 -0000 nvass-gmx.com added a comment. >>! In D1944#8, @glebius wrote: > Nikos, > > acking that I see the patches. Right now I'm waiting for pf to stablize after > recent patches to fragment handling. Kristof is working on the known problem. > Meanwhile you can finish your patch moving from "almost there" to "there" :) Yes, currently working on it. > If you got any questions about pf or FreeBSD kernel interfaces, feel free > to ask me via email. Sure, thanks! REVISION DETAIL https://reviews.freebsd.org/D1944 To: nvass-gmx.com, gnn, bz, zec, trociny, rodrigc, glebius Cc: freebsd-virtualization, freebsd-pf, freebsd-net From owner-freebsd-net@FreeBSD.ORG Sat Feb 28 15:16:05 2015 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7C5FFC0B for ; Sat, 28 Feb 2015 15:16:05 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 62BEB8 for ; Sat, 28 Feb 2015 15:16:05 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id t1SFG5j9096455 for ; Sat, 28 Feb 2015 15:16:05 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 193802] tso seems broken on RELENG10 for version 7.4.2 of em driver Date: Sat, 28 Feb 2015 15:16:04 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: mike@sentex.net X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Feb 2015 15:16:05 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=193802 --- Comment #4 from mike@sentex.net --- https://lists.freebsd.org/pipermail/freebsd-stable/2014-September/080088.html has some insights / discussion on it from Rick M -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Sat Feb 28 16:19:25 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 20057BC9 for ; Sat, 28 Feb 2015 16:19:25 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [IPv6:2001:4f8:3:ffe0:406a:0:50:2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0065D983 for ; Sat, 28 Feb 2015 16:19:24 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [127.0.1.5]) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9) with ESMTP id t1SGJOSM010995 for ; Sat, 28 Feb 2015 16:19:24 GMT (envelope-from root@phabric-backend.isc.freebsd.org) Received: (from root@localhost) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9/Submit) id t1SGJOif010994; Sat, 28 Feb 2015 16:19:24 GMT (envelope-from root) Date: Sat, 28 Feb 2015 16:19:24 +0000 To: freebsd-net@freebsd.org From: "hselasky (Hans Petter Selasky)" Subject: [Differential] [Request, 513 lines] D1987: Factor out MBUF hashing code for Ethernet and TCP/IP. Message-ID: X-Priority: 3 Thread-Topic: D1987: Factor out MBUF hashing code for Ethernet and TCP/IP. X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: Thread-Index: ODZmZDU0NzdlNGFlMDVlYzU4N2Q1NTMzODk3 X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , , , MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Feb 2015 16:19:25 -0000 hselasky created this revision. hselasky added reviewers: glebius, ken, adrian. hselasky added a subscriber: freebsd-net. hselasky set the repository for this revision to rS (FreeBSD src repository). REVISION SUMMARY Factor out mbuf hashing code instead of copying it around. REVISION DETAIL https://reviews.freebsd.org/D1987 AFFECTED FILES sys/conf/files sys/kern/uipc_mbufhash.c sys/net/if_lagg.c sys/ofed/drivers/net/mlx4/en_tx.c sys/ofed/drivers/net/mlx4/utils.c sys/ofed/drivers/net/mlx4/utils.h sys/sys/mbuf.h To: hselasky, glebius, ken, adrian Cc: freebsd-net From owner-freebsd-net@FreeBSD.ORG Sat Feb 28 16:19:37 2015 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 616A1C51 for ; Sat, 28 Feb 2015 16:19:37 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 47B7398C for ; Sat, 28 Feb 2015 16:19:37 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id t1SGJbMt094303 for ; Sat, 28 Feb 2015 16:19:37 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 197882] [panic] kernel panics in soreceive_dgram Date: Sat, 28 Feb 2015 16:19:37 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: ae@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: ae@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Feb 2015 16:19:37 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=197882 Andrey V. Elsukov changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-net@FreeBSD.org |ae@FreeBSD.org -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Sat Feb 28 16:20:29 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C6DB5CE9 for ; Sat, 28 Feb 2015 16:20:29 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [IPv6:2001:4f8:3:ffe0:406a:0:50:2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A5FB599E for ; Sat, 28 Feb 2015 16:20:29 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [127.0.1.5]) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9) with ESMTP id t1SGKTtE012495 for ; Sat, 28 Feb 2015 16:20:29 GMT (envelope-from root@phabric-backend.isc.freebsd.org) Received: (from root@localhost) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9/Submit) id t1SGKTOO012494; Sat, 28 Feb 2015 16:20:29 GMT (envelope-from root) Date: Sat, 28 Feb 2015 16:20:29 +0000 To: freebsd-net@freebsd.org From: "hselasky (Hans Petter Selasky)" Subject: [Differential] [Updated, 513 lines] D1987: Factor out MBUF hashing code for Ethernet and TCP/IP. Message-ID: X-Priority: 3 Thread-Topic: D1987: Factor out MBUF hashing code for Ethernet and TCP/IP. X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: In-Reply-To: References: Thread-Index: ODZmZDU0NzdlNGFlMDVlYzU4N2Q1NTMzODk3IFTx6s0= X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Feb 2015 16:20:29 -0000 hselasky updated this revision to Diff 4030. hselasky added a comment. Add full patch context. CHANGES SINCE LAST UPDATE https://reviews.freebsd.org/D1987?vs=4029&id=4030 REVISION DETAIL https://reviews.freebsd.org/D1987 AFFECTED FILES sys/conf/files sys/kern/uipc_mbufhash.c sys/net/if_lagg.c sys/ofed/drivers/net/mlx4/en_tx.c sys/ofed/drivers/net/mlx4/utils.c sys/ofed/drivers/net/mlx4/utils.h sys/sys/mbuf.h To: hselasky, glebius, ken, adrian Cc: freebsd-net From owner-freebsd-net@FreeBSD.ORG Sat Feb 28 19:14:48 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8E1F3535 for ; Sat, 28 Feb 2015 19:14:48 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [IPv6:2001:4f8:3:ffe0:406a:0:50:2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6D878C9A for ; Sat, 28 Feb 2015 19:14:48 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [127.0.1.5]) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9) with ESMTP id t1SJEmTo097134 for ; Sat, 28 Feb 2015 19:14:48 GMT (envelope-from root@phabric-backend.isc.freebsd.org) Received: (from root@localhost) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9/Submit) id t1SJEmOY097133; Sat, 28 Feb 2015 19:14:48 GMT (envelope-from root) Date: Sat, 28 Feb 2015 19:14:48 +0000 To: freebsd-net@freebsd.org From: "hselasky (Hans Petter Selasky)" Subject: [Differential] [Updated, 1, 385 lines] D1987: Factor out MBUF hashing code for Ethernet and TCP/IP. Message-ID: <52019cf0c62ff7aef1001d2980bc95d0@localhost.localdomain> X-Priority: 3 Thread-Topic: D1987: Factor out MBUF hashing code for Ethernet and TCP/IP. X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: In-Reply-To: References: Thread-Index: ODZmZDU0NzdlNGFlMDVlYzU4N2Q1NTMzODk3IFTyE6g= X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Feb 2015 19:14:48 -0000 hselasky updated this revision to Diff 4032. hselasky added a comment. Factor out more code. CHANGES SINCE LAST UPDATE https://reviews.freebsd.org/D1987?vs=4030&id=4032 REVISION DETAIL https://reviews.freebsd.org/D1987 AFFECTED FILES share/man/man9/Makefile share/man/man9/timeout.9 sys/conf/files sys/kern/uipc_mbufhash.c sys/net/if_lagg.c sys/ofed/drivers/net/mlx4/en_tx.c sys/ofed/drivers/net/mlx4/utils.c sys/ofed/drivers/net/mlx4/utils.h sys/sys/mbuf.h To: hselasky, glebius, ken, adrian Cc: freebsd-net From owner-freebsd-net@FreeBSD.ORG Sat Feb 28 19:15:34 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5E8335D4 for ; Sat, 28 Feb 2015 19:15:34 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [IPv6:2001:4f8:3:ffe0:406a:0:50:2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3749BCAA for ; Sat, 28 Feb 2015 19:15:34 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [127.0.1.5]) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9) with ESMTP id t1SJFXra097802 for ; Sat, 28 Feb 2015 19:15:33 GMT (envelope-from root@phabric-backend.isc.freebsd.org) Received: (from root@localhost) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9/Submit) id t1SJFXEJ097801; Sat, 28 Feb 2015 19:15:33 GMT (envelope-from root) Date: Sat, 28 Feb 2015 19:15:33 +0000 To: freebsd-net@freebsd.org From: "hselasky (Hans Petter Selasky)" Subject: [Differential] [Updated, 518 lines] D1987: Factor out MBUF hashing code for Ethernet and TCP/IP. Message-ID: <33524bced9b7caae4164055c2c389c43@localhost.localdomain> X-Priority: 3 Thread-Topic: D1987: Factor out MBUF hashing code for Ethernet and TCP/IP. X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: In-Reply-To: References: Thread-Index: ODZmZDU0NzdlNGFlMDVlYzU4N2Q1NTMzODk3IFTyE9U= X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Feb 2015 19:15:34 -0000 hselasky updated this revision to Diff 4033. hselasky added a comment. Remove patches not belonging to this issue. CHANGES SINCE LAST UPDATE https://reviews.freebsd.org/D1987?vs=4032&id=4033 REVISION DETAIL https://reviews.freebsd.org/D1987 AFFECTED FILES sys/conf/files sys/kern/uipc_mbufhash.c sys/net/if_lagg.c sys/ofed/drivers/net/mlx4/en_tx.c sys/ofed/drivers/net/mlx4/utils.c sys/ofed/drivers/net/mlx4/utils.h sys/sys/mbuf.h To: hselasky, glebius, ken, adrian Cc: freebsd-net From owner-freebsd-net@FreeBSD.ORG Sat Feb 28 19:37:38 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 76632958 for ; Sat, 28 Feb 2015 19:37:38 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [IPv6:2001:4f8:3:ffe0:406a:0:50:2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5155CE59 for ; Sat, 28 Feb 2015 19:37:38 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [127.0.1.5]) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9) with ESMTP id t1SJbcc1020700 for ; Sat, 28 Feb 2015 19:37:38 GMT (envelope-from root@phabric-backend.isc.freebsd.org) Received: (from root@localhost) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9/Submit) id t1SJbc0w020699; Sat, 28 Feb 2015 19:37:38 GMT (envelope-from root) Date: Sat, 28 Feb 2015 19:37:38 +0000 To: freebsd-net@freebsd.org From: "hselasky (Hans Petter Selasky)" Subject: [Differential] [Commented On] D1761: Extend LRO support to accumulate more than 65535 bytes Message-ID: <8aef3f10266d8838ef4025464b8c585d@localhost.localdomain> X-Priority: 3 Thread-Topic: D1761: Extend LRO support to accumulate more than 65535 bytes X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: In-Reply-To: References: Thread-Index: M2NjZGMxNGQwNjQ0ZTg4NzgyYzE1NGYxMTJmIFTyGQI= X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Feb 2015 19:37:38 -0000 hselasky added a comment. Hi, Adrian: Would a set of macros hiding how we set and clear the LRO_TCP "flag" be acceptable, then we can later resolve that minor detail, exactly how the bit is encoded? #define M_SET_LRO_TCP(m) ... #define M_CLR_LRO_TCP(m) ... #define M_GET_LRO_TCP(m) ... --HPS REVISION DETAIL https://reviews.freebsd.org/D1761 To: hselasky, rrs, glebius, gnn, emaste, lstewart, rwatson, bz, imp, np, adrian Cc: freebsd-net From owner-freebsd-net@FreeBSD.ORG Sat Feb 28 19:40:53 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5A614A1D for ; Sat, 28 Feb 2015 19:40:53 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [IPv6:2001:4f8:3:ffe0:406a:0:50:2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 34DF2E7B for ; Sat, 28 Feb 2015 19:40:53 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [127.0.1.5]) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9) with ESMTP id t1SJeqU4024525 for ; Sat, 28 Feb 2015 19:40:52 GMT (envelope-from root@phabric-backend.isc.freebsd.org) Received: (from root@localhost) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9/Submit) id t1SJeq5q024522; Sat, 28 Feb 2015 19:40:52 GMT (envelope-from root) Date: Sat, 28 Feb 2015 19:40:52 +0000 To: freebsd-net@freebsd.org From: "hselasky (Hans Petter Selasky)" Subject: [Differential] [Updated] D1761: Extend LRO support to accumulate more than 65535 bytes Message-ID: X-Priority: 3 Thread-Topic: D1761: Extend LRO support to accumulate more than 65535 bytes X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: In-Reply-To: References: Thread-Index: M2NjZGMxNGQwNjQ0ZTg4NzgyYzE1NGYxMTJmIFTyGcQ= X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Feb 2015 19:40:53 -0000 hselasky added a reviewer: jfvogel. REVISION DETAIL https://reviews.freebsd.org/D1761 To: hselasky, rrs, glebius, gnn, emaste, lstewart, rwatson, bz, imp, np, adrian, jfvogel Cc: freebsd-net From owner-freebsd-net@FreeBSD.ORG Sat Feb 28 19:41:01 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3E12FABE for ; Sat, 28 Feb 2015 19:41:01 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [IPv6:2001:4f8:3:ffe0:406a:0:50:2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 18A91E8F for ; Sat, 28 Feb 2015 19:41:01 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [127.0.1.5]) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9) with ESMTP id t1SJf0hL024609 for ; Sat, 28 Feb 2015 19:41:00 GMT (envelope-from root@phabric-backend.isc.freebsd.org) Received: (from root@localhost) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9/Submit) id t1SJf0mR024608; Sat, 28 Feb 2015 19:41:00 GMT (envelope-from root) Date: Sat, 28 Feb 2015 19:41:00 +0000 To: freebsd-net@freebsd.org From: "hselasky (Hans Petter Selasky)" Subject: [Differential] [Updated] D1761: Extend LRO support to accumulate more than 65535 bytes Message-ID: <78a1c6a13485c7283e4f0a7a37a27c3c@localhost.localdomain> X-Priority: 3 Thread-Topic: D1761: Extend LRO support to accumulate more than 65535 bytes X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: In-Reply-To: References: Thread-Index: M2NjZGMxNGQwNjQ0ZTg4NzgyYzE1NGYxMTJmIFTyGcw= X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Feb 2015 19:41:01 -0000 hselasky set the repository for this revision to rS (FreeBSD src repository). REVISION DETAIL https://reviews.freebsd.org/D1761 To: hselasky, rrs, glebius, gnn, emaste, lstewart, rwatson, bz, imp, np, adrian, jfvogel Cc: freebsd-net From owner-freebsd-net@FreeBSD.ORG Sat Feb 28 19:41:23 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8B83FB51 for ; Sat, 28 Feb 2015 19:41:23 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [IPv6:2001:4f8:3:ffe0:406a:0:50:2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6B26FE9B for ; Sat, 28 Feb 2015 19:41:23 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [127.0.1.5]) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9) with ESMTP id t1SJfNh3025986 for ; Sat, 28 Feb 2015 19:41:23 GMT (envelope-from root@phabric-backend.isc.freebsd.org) Received: (from root@localhost) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9/Submit) id t1SJfNDq025985; Sat, 28 Feb 2015 19:41:23 GMT (envelope-from root) Date: Sat, 28 Feb 2015 19:41:23 +0000 To: freebsd-net@freebsd.org From: "hselasky (Hans Petter Selasky)" Subject: [Differential] [Updated] D1987: Factor out MBUF hashing code for Ethernet and TCP/IP. Message-ID: <779ec120d93fba73e65ad9aacc0b370d@localhost.localdomain> X-Priority: 3 Thread-Topic: D1987: Factor out MBUF hashing code for Ethernet and TCP/IP. X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: In-Reply-To: References: Thread-Index: ODZmZDU0NzdlNGFlMDVlYzU4N2Q1NTMzODk3IFTyGeM= X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Feb 2015 19:41:23 -0000 hselasky set the repository for this revision to rS (FreeBSD src repository). REVISION DETAIL https://reviews.freebsd.org/D1987 To: hselasky, glebius, ken, adrian Cc: freebsd-net From owner-freebsd-net@FreeBSD.ORG Sat Feb 28 20:53:36 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DD1F2ADD for ; Sat, 28 Feb 2015 20:53:36 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [IPv6:2001:4f8:3:ffe0:406a:0:50:2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9D6B382D for ; Sat, 28 Feb 2015 20:53:36 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [127.0.1.5]) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9) with ESMTP id t1SKra2w008359 for ; Sat, 28 Feb 2015 20:53:36 GMT (envelope-from root@phabric-backend.isc.freebsd.org) Received: (from root@localhost) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9/Submit) id t1SKra3Z008358; Sat, 28 Feb 2015 20:53:36 GMT (envelope-from root) Date: Sat, 28 Feb 2015 20:53:36 +0000 To: freebsd-net@freebsd.org From: "adrian (Adrian Chadd)" Subject: [Differential] [Commented On] D1761: Extend LRO support to accumulate more than 65535 bytes Message-ID: X-Priority: 3 Thread-Topic: D1761: Extend LRO support to accumulate more than 65535 bytes X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: In-Reply-To: References: Thread-Index: M2NjZGMxNGQwNjQ0ZTg4NzgyYzE1NGYxMTJmIFTyKtA= X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Feb 2015 20:53:36 -0000 adrian added a comment. I don't think we should be using the hash bits like this. There's even more hash types to add (all the variations on symmetric and non-RSS hashing; hashing XOR versus RSS, different fields, etc). I'd rather we don't make things more complicated by having the flowid/hashtype fields get more overloaded than they already are. Let's see if we can do something more sensible by adding more flags; we can always look at growing the mbuf to include these new features. As I said before, there will be more features than this one coming in the pipeline so we will be better off if we list stuff that's in the pipeline so we can all figure out what new bits and what shuffling we need to do. There's also robert's mbuf shuffling that needs to finish; once that's finished we may be in a much better situation to add fields. :-) REVISION DETAIL https://reviews.freebsd.org/D1761 To: hselasky, rrs, glebius, gnn, emaste, lstewart, rwatson, bz, imp, np, adrian, jfvogel Cc: freebsd-net From owner-freebsd-net@FreeBSD.ORG Sat Feb 28 22:14:07 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 70EFF313 for ; Sat, 28 Feb 2015 22:14:07 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [IPv6:2001:4f8:3:ffe0:406a:0:50:2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4F018F3D for ; Sat, 28 Feb 2015 22:14:07 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [127.0.1.5]) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9) with ESMTP id t1SME79T093197 for ; Sat, 28 Feb 2015 22:14:07 GMT (envelope-from root@phabric-backend.isc.freebsd.org) Received: (from root@localhost) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9/Submit) id t1SME7l8093196; Sat, 28 Feb 2015 22:14:07 GMT (envelope-from root) Date: Sat, 28 Feb 2015 22:14:07 +0000 To: freebsd-net@freebsd.org From: "gnn (George Neville-Neil)" Subject: [Differential] [Accepted] D1965: Add extended media types to if_media.h and ifconfig Message-ID: <8818bbf5229e45122045171a2e3f5a8a@localhost.localdomain> X-Priority: 3 Thread-Topic: D1965: Add extended media types to if_media.h and ifconfig X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: X-Phabricator-Cc: In-Reply-To: References: Thread-Index: YmU0NTlhN2Q2ZDc3OTdjODA5NmQwOTI1OTFkIFTyPa8= X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , , MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Feb 2015 22:14:07 -0000 gnn accepted this revision. gnn added a comment. BTW Mike Karels was in favor of this in an email thread. He's not yet on phabricator but I'll ask him here as well. BRANCH /head REVISION DETAIL https://reviews.freebsd.org/D1965 To: erj, adrian, jfvogel, gnn Cc: glebius, freebsd-net From owner-freebsd-net@FreeBSD.ORG Sat Feb 28 22:20:13 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 49314639 for ; Sat, 28 Feb 2015 22:20:13 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [IPv6:2001:4f8:3:ffe0:406a:0:50:2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2B1A2F97 for ; Sat, 28 Feb 2015 22:20:13 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [127.0.1.5]) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9) with ESMTP id t1SMKCVd099405 for ; Sat, 28 Feb 2015 22:20:12 GMT (envelope-from root@phabric-backend.isc.freebsd.org) Received: (from root@localhost) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9/Submit) id t1SMKCr4099404; Sat, 28 Feb 2015 22:20:12 GMT (envelope-from root) Date: Sat, 28 Feb 2015 22:20:12 +0000 To: freebsd-net@freebsd.org From: "rstone (Ryan Stone)" Subject: [Differential] [Changed Subscribers] D1986: Teach lagg(4) to change MTU Message-ID: <8cf49571db825eb9278361cf240d5f83@localhost.localdomain> X-Priority: 3 Thread-Topic: D1986: Teach lagg(4) to change MTU X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: In-Reply-To: References: Thread-Index: ODZhMzNlYThiYzMxOTgzYmRhMDE5M2Q2Yzk4IFTyPxw= X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Feb 2015 22:20:13 -0000 rstone added a subscriber: freebsd-net. REVISION DETAIL https://reviews.freebsd.org/D1986 To: rpokala-panasas.com, rstone Cc: freebsd-net