From owner-freebsd-net@freebsd.org Sun Jan 15 00:56:04 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C4101CA478E for ; Sun, 15 Jan 2017 00:56:04 +0000 (UTC) (envelope-from jeffrey.e.pieper@intel.com) Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "ORSMGA103.jf.intel.com", Issuer "Intel External Issuing CA 6B" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 88E4317E3 for ; Sun, 15 Jan 2017 00:56:03 +0000 (UTC) (envelope-from jeffrey.e.pieper@intel.com) Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga103.jf.intel.com with ESMTP; 14 Jan 2017 16:54:55 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.33,230,1477983600"; d="scan'208";a="922644841" Received: from orsmsx104.amr.corp.intel.com ([10.22.225.131]) by orsmga003.jf.intel.com with ESMTP; 14 Jan 2017 16:54:55 -0800 Received: from orsmsx111.amr.corp.intel.com ([169.254.12.23]) by ORSMSX104.amr.corp.intel.com ([169.254.4.69]) with mapi id 14.03.0248.002; Sat, 14 Jan 2017 16:54:54 -0800 From: "Pieper, Jeffrey E" To: Kevin Bowling , "freebsd-net@freebsd.org" CC: "Cramer, Jeb J" , "Joyner, Eric" Subject: RE: ixgbe PF and ixv driver Thread-Topic: ixgbe PF and ixv driver Thread-Index: AQHSbpggPXIVWDMuuU6NrjX/McIlYqE4taWg Date: Sun, 15 Jan 2017 00:54:54 +0000 Message-ID: <2A35EA60C3C77D438915767F458D656883A01EE5@ORSMSX111.amr.corp.intel.com> References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.22.254.140] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Jan 2017 00:56:04 -0000 Kevin, Currently there is some missing PCIOV functionality in ix. It is something = we are working on fixing. Thanks, Jeff -----Original Message----- From: owner-freebsd-net@freebsd.org [mailto:owner-freebsd-net@freebsd.org] = On Behalf Of Kevin Bowling Sent: Saturday, January 14, 2017 10:58 AM To: freebsd-net@freebsd.org Subject: ixgbe PF and ixv driver Hi, I've been trying to test SR-IOV on -CURRENT and 11.0-RELEASE. Physical NIC is an X552. The ixgbe PF has to be limited to 1-4 queues in order to create VFs or you will get ENOSPC from iovctl. Either letting ixv try to attach on the host, or creating a passthrough and allowing pfsense 2.4.0 beta to try to attach yields this failure to attach: m 0xc0104000-0xc0107fff,0xc0108000-0xc010bfff at device 7.0 on pci0 ixv0: Using MSIX interrupts with 2 vectors ixv0: ixgbe_reset_hw() failed with error -10 device_attach: ixv0 attach returned 5 Another issue, the PF is unable to pass traffic after creating a VF. 'iovctl -D -d ix0; service netif restart; service routing restart' will restore connectivity. Regards, Kevin _______________________________________________ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@freebsd.org Sun Jan 15 09:32:58 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 145F7CA4457 for ; Sun, 15 Jan 2017 09:32:58 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from onlyone.not-for.work (onlyone.not-for.work [148.251.9.81]) by mx1.freebsd.org (Postfix) with ESMTP id D797A1973 for ; Sun, 15 Jan 2017 09:32:57 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from lion.home.serebryakov.spb.ru (unknown [IPv6:2001:470:923f:1:e0f4:994:662:862]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.not-for.work (Postfix) with ESMTPSA id F188B39D for ; Sun, 15 Jan 2017 12:32:49 +0300 (MSK) Date: Sun, 15 Jan 2017 12:32:52 +0300 From: Lev Serebryakov X-Priority: 3 (Normal) Message-ID: <1773236257.20170115123252@serebryakov.spb.ru> To: freebsd-net@freebsd.org Subject: =?windows-1251?Q?10Gb_on_budged_with_fiber_=97_what_should_IP_choose=3F?= MIME-Version: 1.0 Content-Type: text/plain; charset=windows-1251 Content-Transfer-Encoding: base64 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Jan 2017 09:32:58 -0000 SGVsbG8gRnJlZWJzZC1uZXQsDQoNCiAgSSB3YW50IHRvIGF0dGFjaCBteSBESVktTkFTIChG cmVlQlNELWJhc2VkKSB0byBteSBkZXNrdG9wIChXaW5kb3dzLWJhc2VkKQ0KIHdpdGggMTBH YiBsaW5rLiBJIGNvdWxkIG5vdCBhZmZvcmQgMTBHYiBzd2l0Y2ggZm9yIHN1cmUuIFNvLCBp dCB3aWxsIGJlDQogcG9pbnQtdG8tcG9pbnQgY29ubmVjdGlvbiBvbmx5IGZvciBteSBkZXNr dG9wIGFuZCBub3Qgb3RoZXIgY29tcHV0ZXJzIGluIG15DQogaG91c2UuDQoNCiAgQWxzbywg SSBjb3VsZCBub3QgdXNlIG15IGN1cnJlbnQgdHdpc3RlZCBwYWlyIHdpcmluZywgYXMgSSBu ZWVkIGNvbm5lY3Rpb24NCiB0byByb3V0ZXIgZm9yIG15IGRlc2t0b3AgdG9vIDopIENhYmxl IGR1Y3RzIGlvbiBteSBob3VzZSBpcyBvdmVyY3JvdWRlZCwNCiBzbywgSSB0aGluayBvbmx5 IGZpYmVyIGlzIHdheSB0byBnbywgdGhlcmUgaXMgbm8gY2hhbmNlIHRvIGFkZCBDQVQ2IGNh YmxlDQogYmV0d2VlbiBteSByb29tIGFuZCBzdG9yYWdlIHJvb20gd2l0aCBzZXJ2ZXIsIGFu ZCBjb3BwZXIgU0ZQLXRvLVNGUA0KIHBhdGNoY29yZCBpcyBvdXQgb2YgcXVlc3Rpb24hDQoN CiAgU28sIEkgbmVlZCBjaGVhcCBQQ0llIGNhcmRzIHdpdGggU0ZQKyBzbG90cyB3aGljaCBp cyBub3QgdmVyeSBwaWNreSBhYm91dA0KIG1vZHVsZXMgKEknbSBvbiBidWRnZXQhKSwgd2l0 aCBnb29kIEZyZWVCU0Qgc3VwcG9ydCBhbmQgZHJpdmVycyBmb3INCiBXaW4nMTAgKHdoaWNo IGlzIGFub3RoZXIgcHJvYmxlbSB3aXRoIHZlcnkgb2xkIGNhcmRzKS4NCg0KICBXaGF0IHNo b3VsZCBJIGNob29zZT8gVGhlcmUgYXJlIEEgTE9UIG9mIG9sZCAxMEdiIGNhcmRzIG9uIGVC YXkuIFNob3VsZCBJDQogY2hvb3NlIE1lbGxhbm94IENvbm5lY3RpWC0yPyBDaGVsc2lvLVQz PyBTb21ldGhpbmcgZWxzZT8gSW50ZWwgWDUyMCBsb29rcw0KIGFzIHNhZmUgdmFyaWFudCwg YnV0IGl0IGlzIG11Y2ggbW9yZSBleHBlbnNpdmUgYW5kLCBhcyBmYXIgYXMgSSBrbm93LCBp dA0KIHN1cHBvcnRzIG9ubHkgZXhwZW5zaXZlIGJyYW5kZWQgU0ZQKyBtb2R1bGVzLCB0b28u DQoNCg0KUC5TLiBZb3UgY291bGQgbmV2ZXIgc3RvcCB1cGdyYWRpbmcgeW91ciBob21lIE5B Uy4NCg0KLS0gDQpCZXN0IHJlZ2FyZHMsDQogTGV2ICAgICAgICAgICAgICAgICAgICAgICAg ICBtYWlsdG86bGV2QHNlcmVicnlha292LnNwYi5ydQ== From owner-freebsd-net@freebsd.org Sun Jan 15 10:03:17 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6C397CA51E5 for ; Sun, 15 Jan 2017 10:03:17 +0000 (UTC) (envelope-from kevin.bowling@kev009.com) Received: from mail-oi0-x22d.google.com (mail-oi0-x22d.google.com [IPv6:2607:f8b0:4003:c06::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 341741E4D for ; Sun, 15 Jan 2017 10:03:17 +0000 (UTC) (envelope-from kevin.bowling@kev009.com) Received: by mail-oi0-x22d.google.com with SMTP id j15so86410294oih.2 for ; Sun, 15 Jan 2017 02:03:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kev009.com; s=google; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=0Rn1Rp1m1se9OGMWxz1bXnA0z+2W89sxQ9u2bH/mKqY=; b=UF2tUfJT6ncZTl40FSNqLFehfh4YLzGzVR2+F12+HoWEn2gf7ovqSD94n4d5SIR7m2 h17DDLnVAAzyQCIh5+mW0ti5XkklzbJbk1O41vPif3T9iMr2efAlh6dRO3IHOMgt3X3u uSZGaC5jdVXNAy+IyfcdtwcAtfP/gdCXNgIXo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=0Rn1Rp1m1se9OGMWxz1bXnA0z+2W89sxQ9u2bH/mKqY=; b=oFr1YkTVsJNUFxhdIIQoCcUiFUy4UUiuLBCPZX3lJl6kzoAsseH/3dC7I9Lj2r3ei/ S1dUJ+nf2xa6853gLRfhoyQZx8qKztYHZu6oPVIQLbZdK2/NAe+5ix2Iedtl0KvsJP0t PqwezlUVdPvG6NHxqmAGOap6/K1VxtvAPlmsLeq8wE8wRByovkT1Z01gsfKcD8f6ef1t KI0JtLdUxx79GUG/SzeCkLX/MULNhWJIViCTJTdzCBG0NpoQCFz6pGxAf13ahnplz3Gt 2TLUJxrPNwO/Pk1rHGVzybjSECiMcyYUW/PN27o1Mh+KJqO/qUR4fZXoO8i59fuv/Uqj zTBw== X-Gm-Message-State: AIkVDXLU2crzHF+sHySOnDMKoEziAEBTDUx4rJjpHd1YId6kyxsON/DwsxV62BhdYNx+WhNtT676FVLp8Lis1A== X-Received: by 10.202.175.68 with SMTP id y65mr15028869oie.187.1484474596500; Sun, 15 Jan 2017 02:03:16 -0800 (PST) MIME-Version: 1.0 Received: by 10.157.23.208 with HTTP; Sun, 15 Jan 2017 02:03:16 -0800 (PST) In-Reply-To: <1773236257.20170115123252@serebryakov.spb.ru> References: <1773236257.20170115123252@serebryakov.spb.ru> From: Kevin Bowling Date: Sun, 15 Jan 2017 03:03:16 -0700 Message-ID: Subject: =?UTF-8?Q?Re=3A_10Gb_on_budged_with_fiber_=E2=80=94_what_should_IP_cho?= =?UTF-8?Q?ose=3F?= To: Lev Serebryakov Cc: freebsd-net@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Jan 2017 10:03:17 -0000 Intel is overpriced and still riding their Ethernet reputation from well over a decade ago with little to show for it. I would recommend cxgbe anything, so Chelsio T420 and up. The T520-SO-CR is within reason for demanding home office/small office users to purchase new and I've deployed over a thousand of these at work with flawless performance. T420 are a bit cheaper on i.e. US ebay if that is viable to you but not by too much. I bought a T420-BT that way for home use and have no complaints. For non-TOE use at 10g, the difference between T4 and T5 doesn't matter too much, pcie 2.0 vs 3.0. You can use DAC twinax cables to connect 10G SFP+ ports together. This will save several watts of power and heat on each side vs 10G Base-T. Sometimes DACs have interop problems, usually with switches. Optics are more forgiving, although again the switch side may have extra hoops like approved vendors. As far as a switch, I got a good deal on the beta UniFi version of https://www.ubnt.com/edgemax/edgeswitch-16-xg/ for my home network. Now that it is full price, I would recommend the EdgeMax version unless you have other UniFi gear. They are inexpensive as far as the physical connectivity versus the rest of the market but not quite cheap yet. These aren't exactly pro grade, the interop seems pretty poor in general, but that's the price you pay for the price you didn't pay :) Regards, On Sun, Jan 15, 2017 at 2:32 AM, Lev Serebryakov wrote: > Hello Freebsd-net, > > I want to attach my DIY-NAS (FreeBSD-based) to my desktop (Windows-based) > with 10Gb link. I could not afford 10Gb switch for sure. So, it will be > point-to-point connection only for my desktop and not other computers in > my > house. > > Also, I could not use my current twisted pair wiring, as I need > connection > to router for my desktop too :) Cable ducts ion my house is overcrouded, > so, I think only fiber is way to go, there is no chance to add CAT6 cable > between my room and storage room with server, and copper SFP-to-SFP > patchcord is out of question! > > So, I need cheap PCIe cards with SFP+ slots which is not very picky about > modules (I'm on budget!), with good FreeBSD support and drivers for > Win'10 (which is another problem with very old cards). > > What should I choose? There are A LOT of old 10Gb cards on eBay. Should I > choose Mellanox ConnectiX-2? Chelsio-T3? Something else? Intel X520 looks > as safe variant, but it is much more expensive and, as far as I know, it > supports only expensive branded SFP+ modules, too. > > > P.S. You could never stop upgrading your home NAS. > > -- > Best regards, > Lev mailto:lev@serebryakov.spb.ru > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@freebsd.org Sun Jan 15 12:33:37 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C6E4ACAE6D5 for ; Sun, 15 Jan 2017 12:33:37 +0000 (UTC) (envelope-from wonko@4amlunch.net) Received: from mail-yw0-x231.google.com (mail-yw0-x231.google.com [IPv6:2607:f8b0:4002:c05::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8995E1486 for ; Sun, 15 Jan 2017 12:33:37 +0000 (UTC) (envelope-from wonko@4amlunch.net) Received: by mail-yw0-x231.google.com with SMTP id w75so55165000ywg.1 for ; Sun, 15 Jan 2017 04:33:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=4amlunch-net.20150623.gappssmtp.com; s=20150623; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=L1IUdiRkqKIyVOMt8+TEhOvn7kLvW0A4kyapYEGCu8o=; b=fveAL3kj6zcYhOiBPNixpRDS4dgL4ronPAHTT3RMDVa+FNDljxXXWki8pEVcIPN4SI aopAkRm2kcuUf+OhjqB7yGkUoC6KnjY4ulbaeMkFhvelME5DTTBt6qdFtt0fg2wBjJmB FbTawhLm81HQflYVi/YZQXgbQwq/Y7JG93DaS5SDwFm8yID2yrkjLVcxu3KMFvqzmeVH y63+1GNE9Tnqlz34O7gW8/D3SDh8LRxij8XLduvVkIuZIijQtMig+jKzL2BMFICUX5cs adrwx1TnZCLjS9Eh/vOblahBFwFlSWiCoM3f6b+JwgvXXO/nuVa4R31sV4tray1Sh5tK k8KQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=L1IUdiRkqKIyVOMt8+TEhOvn7kLvW0A4kyapYEGCu8o=; b=sRLWMjiom9RMGGLGLrVqaA+gGeuWDjBj5T7HzAX8NSUMcGX607LHDkWYjp6Yg6lWyO XVV1flJRB3abtbyF/BIC4EaSfzZrtJXvIGRUrlAiIE236Bum2YST3hTa/szrMF+Btu2U dDu7uGnMSU4HBLdS3TSU+Q3f4C9vx56KWKekiinT5fLuxoBk2ZJ3cWZfkMw02BiZu7NY GVs9J8mM2mbEOhGoUXOQkX1lPcGTzOuQEGBA8moRPu6vgF/KXnWaAnl3CvE5iIhlgnWN pUjyq1Jk/utQQM1m3HPhjSEv9YKPv91/curOG6Gd/9iNyf9vSiZv/ERsThUircgVk0h9 lqqw== X-Gm-Message-State: AIkVDXKo4c37VtUeSjP+4Yx2isRiY0zjbDPiaVcs2MzgVqLSwM9tU4XSZHWAntzZEj3m8g== X-Received: by 10.13.239.198 with SMTP id y189mr20885121ywe.202.1484483616423; Sun, 15 Jan 2017 04:33:36 -0800 (PST) Received: from [192.168.190.250] (50-204-9-73-static.hfc.comcastbusiness.net. [50.204.9.73]) by smtp.gmail.com with ESMTPSA id r17sm5099342ywa.24.2017.01.15.04.33.35 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 15 Jan 2017 04:33:35 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (1.0) Subject: =?utf-8?Q?Re:_10Gb_on_budged_with_fiber_=E2=80=94_what_should_IP?= =?utf-8?Q?_choose=3F?= From: wonko@4amlunch.net X-Mailer: iPhone Mail (14C92) In-Reply-To: Date: Sun, 15 Jan 2017 07:33:35 -0500 Cc: Lev Serebryakov , freebsd-net@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <6FCA135B-B7F6-4148-B1C3-FE0D4470A568@4amlunch.net> References: <1773236257.20170115123252@serebryakov.spb.ru> To: Kevin Bowling X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Jan 2017 12:33:37 -0000 How good is FreeBSD's mellanox support? You can get a ConnectX-2 EN for $15 o= n eBay and they are amazing cards. -brian > On Jan 15, 2017, at 05:03, Kevin Bowling wrote:= >=20 > Intel is overpriced and still riding their Ethernet reputation from well > over a decade ago with little to show for it. I would recommend cxgbe > anything, so Chelsio T420 and up. The T520-SO-CR is within reason for > demanding home office/small office users to purchase new and I've deployed= > over a thousand of these at work with flawless performance. T420 are a bi= t > cheaper on i.e. US ebay if that is viable to you but not by too much. I > bought a T420-BT that way for home use and have no complaints. For non-TO= E > use at 10g, the difference between T4 and T5 doesn't matter too much, pcie= > 2.0 vs 3.0. >=20 > You can use DAC twinax cables to connect 10G SFP+ ports together. This > will save several watts of power and heat on each side vs 10G Base-T. > Sometimes DACs have interop problems, usually with switches. Optics are > more forgiving, although again the switch side may have extra hoops like > approved vendors. >=20 > As far as a switch, I got a good deal on the beta UniFi version of > https://www.ubnt.com/edgemax/edgeswitch-16-xg/ for my home network. Now > that it is full price, I would recommend the EdgeMax version unless you > have other UniFi gear. They are inexpensive as far as the physical > connectivity versus the rest of the market but not quite cheap yet. These= > aren't exactly pro grade, the interop seems pretty poor in general, but > that's the price you pay for the price you didn't pay :) >=20 > Regards, >=20 >> On Sun, Jan 15, 2017 at 2:32 AM, Lev Serebryakov wrote:= >>=20 >> Hello Freebsd-net, >>=20 >> I want to attach my DIY-NAS (FreeBSD-based) to my desktop (Windows-based= ) >> with 10Gb link. I could not afford 10Gb switch for sure. So, it will be >> point-to-point connection only for my desktop and not other computers in >> my >> house. >>=20 >> Also, I could not use my current twisted pair wiring, as I need >> connection >> to router for my desktop too :) Cable ducts ion my house is overcrouded, >> so, I think only fiber is way to go, there is no chance to add CAT6 cable= >> between my room and storage room with server, and copper SFP-to-SFP >> patchcord is out of question! >>=20 >> So, I need cheap PCIe cards with SFP+ slots which is not very picky abou= t >> modules (I'm on budget!), with good FreeBSD support and drivers for >> Win'10 (which is another problem with very old cards). >>=20 >> What should I choose? There are A LOT of old 10Gb cards on eBay. Should I= >> choose Mellanox ConnectiX-2? Chelsio-T3? Something else? Intel X520 looks= >> as safe variant, but it is much more expensive and, as far as I know, it >> supports only expensive branded SFP+ modules, too. >>=20 >>=20 >> P.S. You could never stop upgrading your home NAS. >>=20 >> -- >> Best regards, >> Lev mailto:lev@serebryakov.spb.ru >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >>=20 > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@freebsd.org Sun Jan 15 13:22:46 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D081ACB145F for ; Sun, 15 Jan 2017 13:22:46 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: from mail-wm0-x241.google.com (mail-wm0-x241.google.com [IPv6:2a00:1450:400c:c09::241]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 60F211E50 for ; Sun, 15 Jan 2017 13:22:46 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: by mail-wm0-x241.google.com with SMTP id c85so23846062wmi.1 for ; Sun, 15 Jan 2017 05:22:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=XRm6tzymczxZMEvtDIzOdVGfz1yruS/jVNwD2cSrVao=; b=aD7aA1TnAzhYTh7uGNdD5TuJJIccfoRPTxIE2P9XXHTm9eaVwRspjcPQ0xrGZt1eMY R0iIV5i+U1l3NPNoUJrIM5Vz85IW/+9zRMoE1gqlxfaK2VtxM8U/bNe0Ecc1sRmONZZk Fv/DUiiXIKWb5mkpACDs1rJnxvp+aJobivD39i27/bZkOQkhfrKhSvDTDPIt7Skj8wgZ HEOCr5UmCFutnARJ0HLIa2uhDHMbg0FIO/vKh4HOEaS89c2Z+te17fptOGtcJGP8Z/gf 5sOiH7RHTkx2J19RbOeObC1zlgbgcOWwBrJFPB4eLENYgCtdnDzQYBOXdBNsCTacuUkM rGQQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=XRm6tzymczxZMEvtDIzOdVGfz1yruS/jVNwD2cSrVao=; b=SJmS+NYDuYohcqpyBJu5RqDSHzgiXXPqTzzqA57emjLR8HrhLM4Byu6YabEnOPh+d8 HmRJIeQRoD5Qudki3xntRoln98YWMFhmy/MCNpt979bb0ozVtpoVJkY0dTjsTMt27Brx ku1xqoX1sM1TARSd34emFLZskSTeH/G0+jx4ctgvite26QmZua6tec5jWRV4jNxF8mwc NVKKxSDkgwO1WvMTVxpPSwTuH3Hx/z1Us7WGpi5JRKsGPHJt2bXp37HKeaoAjbkRdzQz RIa4lHg69CYDj4VXSvohLSqMxGRdUoXXNdvfXDHB2fJ0EXGhiJs6/Nim31su33ZjUYqE Pn0g== X-Gm-Message-State: AIkVDXIeFOlMpfwCip+Cbsf/dObXkcwz6l0m/zvJZaFHqNCCPfEmpIPlG6iGgsurC1iRfA== X-Received: by 10.223.135.201 with SMTP id c9mr11696235wrc.139.1484486563902; Sun, 15 Jan 2017 05:22:43 -0800 (PST) Received: from ben.home (LFbn-1-7159-4.w90-116.abo.wanadoo.fr. [90.116.90.4]) by smtp.gmail.com with ESMTPSA id 81sm20902476wmw.7.2017.01.15.05.22.42 for (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Sun, 15 Jan 2017 05:22:43 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: =?utf-8?Q?Re=3A_10Gb_on_budged_with_fiber_=E2=80=94_what_should_?= =?utf-8?Q?IP_choose=3F?= From: Ben RUBSON In-Reply-To: <6FCA135B-B7F6-4148-B1C3-FE0D4470A568@4amlunch.net> Date: Sun, 15 Jan 2017 14:22:41 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <858B840B-95A0-4556-AD77-F3918B51B90A@gmail.com> References: <1773236257.20170115123252@serebryakov.spb.ru> <6FCA135B-B7F6-4148-B1C3-FE0D4470A568@4amlunch.net> To: FreeBSD Net X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Jan 2017 13:22:46 -0000 > On 15 Jan 2017, at 13:33, wonko@4amlunch.net wrote: >=20 > How good is FreeBSD's mellanox support? =46rom my own experience with ConnectX-3 adapters, very good ! Ben From owner-freebsd-net@freebsd.org Sun Jan 15 13:31:18 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0CE9ACB1988 for ; Sun, 15 Jan 2017 13:31:18 +0000 (UTC) (envelope-from wonko@4amlunch.net) Received: from mail-yw0-x229.google.com (mail-yw0-x229.google.com [IPv6:2607:f8b0:4002:c05::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AE98B12B5 for ; Sun, 15 Jan 2017 13:31:17 +0000 (UTC) (envelope-from wonko@4amlunch.net) Received: by mail-yw0-x229.google.com with SMTP id a10so55483045ywa.3 for ; Sun, 15 Jan 2017 05:31:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=4amlunch-net.20150623.gappssmtp.com; s=20150623; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=j6qZ2Sv+e7EeW8LYBMHCSic30QEMXYLDNQ9zSfP2siA=; b=wBPgs49O1NdFhIUgOoz6LGrdoKQ0hLZo2IrnLlfzH71pBgWG9YAXwlemOT/kL8GFtH Qav/89pRfDw07PF/fQzxDMuSktt66pXwRtd55RKlVnCfTCZrEeHm+L84DoYeXtUsEL3v nGTRZYMLgnCgqWtNsuzY8g2Bm4sHJ4MTdwLwNjHAzGh04tphmo0Uz97+rZawMV4XOyiU 5i8ZgnyO4KuP4nYiF603jwoGn3ElzrGIrsQZcBfZXr3KzIdRwPezaR626flS/Wx6H7/P lbLbkrKclOuuRXUropLDkUkFi0Ba3Bpjs8G1CGWZgcKLZ/zrFdkQyDVZ9t5oInFnwgV6 FIqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=j6qZ2Sv+e7EeW8LYBMHCSic30QEMXYLDNQ9zSfP2siA=; b=Fx4dX2eplTny/Bm88KSFA0yifciAIApzrLpnBSlxr0dU+Wt9M6u8o33Yfb8zdg8ypg N8hR13stRQNBMQMJPTAkxBP1Fp5Ka2a1azhxsGfqczB1aQn5RG0qVHp3bgY8NlnoXKBw UrawQwSXml6gC41PvSZVdyCeGonzOKnUR9Ewx2pbcvbqRn589GDAuZ1Jz/R9zHT4/OS4 sMVswHWRHXt0n5q6AGqwVI473SUWsYX+x47a0cbxFxIkM/06Myle+l767S/dLiHvsZng 8w6f6e0AASU1igyalSdk5WgDy32LMR4PvCJfK87qK1SKbpoWITKSBHtcC3ize8KUQKAF +HqA== X-Gm-Message-State: AIkVDXL8zzw4kXmkUO7mQW9jsYw1OR0pikEyDlw7jLdvc+lHF035zSzlJUc181NZHi8i2g== X-Received: by 10.13.212.85 with SMTP id w82mr21184733ywd.245.1484487076692; Sun, 15 Jan 2017 05:31:16 -0800 (PST) Received: from [192.168.190.250] (50-204-9-73-static.hfc.comcastbusiness.net. [50.204.9.73]) by smtp.gmail.com with ESMTPSA id g18sm8028908ywb.37.2017.01.15.05.31.16 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 15 Jan 2017 05:31:16 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (1.0) Subject: =?utf-8?Q?Re:_10Gb_on_budged_with_fiber_=E2=80=94_what_should_IP?= =?utf-8?Q?_choose=3F?= From: wonko@4amlunch.net X-Mailer: iPhone Mail (14C92) In-Reply-To: <858B840B-95A0-4556-AD77-F3918B51B90A@gmail.com> Date: Sun, 15 Jan 2017 08:31:15 -0500 Cc: FreeBSD Net Content-Transfer-Encoding: 7bit Message-Id: References: <1773236257.20170115123252@serebryakov.spb.ru> <6FCA135B-B7F6-4148-B1C3-FE0D4470A568@4amlunch.net> <858B840B-95A0-4556-AD77-F3918B51B90A@gmail.com> To: Ben RUBSON X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Jan 2017 13:31:18 -0000 SRP target support yet? Last time I looked there wasn't any. :( -brian On Jan 15, 2017, at 08:22, Ben RUBSON wrote: >> On 15 Jan 2017, at 13:33, wonko@4amlunch.net wrote: >> >> How good is FreeBSD's mellanox support? > > From my own experience with ConnectX-3 adapters, very good ! > > Ben > > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@freebsd.org Sun Jan 15 13:37:21 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 38492CB1B9F for ; Sun, 15 Jan 2017 13:37:21 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: from mail-wm0-x235.google.com (mail-wm0-x235.google.com [IPv6:2a00:1450:400c:c09::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BF43817D0 for ; Sun, 15 Jan 2017 13:37:20 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: by mail-wm0-x235.google.com with SMTP id r144so137616774wme.1 for ; Sun, 15 Jan 2017 05:37:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=AjEjKiVGMBUeM6T+N/pIhS/x8pQvBe0ECcAs35e0AgU=; b=uvgAJp4Y//RahEfBNaLTtXWyj05k7E7c5KeuyMaU+AqGm4g1KkTS7GmC7rS9UuOmPp o4jBudupojWXKMSDt2dBIXzZwh1M8XS6SL69a/n1v/hXNYXjQEsEheB4QjlXqLIohYFt KJmm9UarlCt4EDFNq4w6D7A7iAE8aT+GNk1FWt0aHP/jShBz3AgTRu/2vf2PHli846Kb bo8xavRN/3urEVR/FL+QEBapqplAtYaVcoiLLlBsRHK4OyTrB2//yFg7yD+TpvPZmRyP SMVGyMj49t6HOwzg2gQNmmrUs6v9Ba/5phf+LBThhlVdHFlIEqYPUIeXa0kM0ndqCHLT YEyw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=AjEjKiVGMBUeM6T+N/pIhS/x8pQvBe0ECcAs35e0AgU=; b=JMnVoJG+V3ltvskUko2kP01/PjmlFz6ae92xwqZrpNwXZJaL3sWt2ZtyMnuAtqW3CR CU+KduzX8CzKl0IArTPecsxsvFmMaxsa+zCwBNEFmVBJn+9Z4IJmsFn3x/NP2SGIT/FM yzbNl/Gc5BF6n+pUGp4/sz54HPdLAGAtGw1smOK6DP1Pq+opqNtR6QrUaf4vvjBq+u+X RQPw1hkJ2akSfF0yFwqtEGXD7dDKeuUCFh5UyrBKuxdYgkOdMFniE95O4PIcpc1a6tPM utzmz7GJ2YkkRIrs/KRrd6BTjh1UKuV70I/BevIwAjWOKAXr9CUTOPwNLII2sB0NWCfc ETcA== X-Gm-Message-State: AIkVDXIt4FrLHG55Jg+l6FGYqrbOx6cFjpHTxhTIBBZopAD44HUNULxHZQUJOwI0C/mJng== X-Received: by 10.223.174.1 with SMTP id x1mr19454807wrc.126.1484487438869; Sun, 15 Jan 2017 05:37:18 -0800 (PST) Received: from ben.home (LFbn-1-7159-4.w90-116.abo.wanadoo.fr. [90.116.90.4]) by smtp.gmail.com with ESMTPSA id w197sm20883491wmd.11.2017.01.15.05.37.17 for (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Sun, 15 Jan 2017 05:37:18 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: =?utf-8?Q?Re=3A_10Gb_on_budged_with_fiber_=E2=80=94_what_should_?= =?utf-8?Q?IP_choose=3F?= From: Ben RUBSON In-Reply-To: Date: Sun, 15 Jan 2017 14:37:15 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <374DD84B-40E7-4B8A-A03B-68FDAD0EF911@gmail.com> References: <1773236257.20170115123252@serebryakov.spb.ru> <6FCA135B-B7F6-4148-B1C3-FE0D4470A568@4amlunch.net> <858B840B-95A0-4556-AD77-F3918B51B90A@gmail.com> To: FreeBSD Net X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Jan 2017 13:37:21 -0000 iSER is supported as initiator, however iSER target is missing, you then = have to run a non-FreeBSD iSER target :-/ I'm not sure for SRP. Ben > On 15 Jan 2017, at 14:31, wonko@4amlunch.net wrote: >=20 > SRP target support yet? Last time I looked there wasn't any. :( >=20 > -brian >=20 > On Jan 15, 2017, at 08:22, Ben RUBSON wrote: >=20 >>> On 15 Jan 2017, at 13:33, wonko@4amlunch.net wrote: >>>=20 >>> How good is FreeBSD's mellanox support? >>=20 >> =46rom my own experience with ConnectX-3 adapters, very good ! >>=20 >> Ben From owner-freebsd-net@freebsd.org Sun Jan 15 13:47:49 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DA937CB1FAF for ; Sun, 15 Jan 2017 13:47:49 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from onlyone.not-for.work (onlyone.not-for.work [IPv6:2a01:4f8:201:6350::2]) by mx1.freebsd.org (Postfix) with ESMTP id A98E71FC4 for ; Sun, 15 Jan 2017 13:47:49 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from lion.home.serebryakov.spb.ru (unknown [IPv6:2001:470:923f:1:e0f4:994:662:862]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.not-for.work (Postfix) with ESMTPSA id 4BC3A3E1; Sun, 15 Jan 2017 16:47:48 +0300 (MSK) Date: Sun, 15 Jan 2017 16:47:51 +0300 From: Lev Serebryakov Reply-To: Lev Serebryakov Organization: FreeBSD X-Priority: 3 (Normal) Message-ID: <1985967690.20170115164751@serebryakov.spb.ru> To: wonko@4amlunch.net, Kevin Bowling CC: freebsd-net@freebsd.org Subject: =?windows-1250?Q?Re:_10Gb_on_budged_with_fiber_=97_what_should_IP_choose=3F?= In-Reply-To: <6FCA135B-B7F6-4148-B1C3-FE0D4470A568@4amlunch.net> References: <1773236257.20170115123252@serebryakov.spb.ru> <6FCA135B-B7F6-4148-B1C3-FE0D4470A568@4amlunch.net> MIME-Version: 1.0 Content-Type: text/plain; charset=windows-1250 Content-Transfer-Encoding: base64 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Jan 2017 13:47:50 -0000 SGVsbG8gV29ua28sDQoNClN1bmRheSwgSmFudWFyeSAxNSwgMjAxNywgMzozMzozNSBQTSwg eW91IHdyb3RlOg0KDQogSXQgaXMgd2hhdCBJIHdhbnQgdG8ga25vdyEgOikNCg0KPiBIb3cg Z29vZCBpcyBGcmVlQlNEJ3MgbWVsbGFub3ggc3VwcG9ydD8gWW91IGNhbiBnZXQgYSBDb25u ZWN0WC0yIEVOIGZvcg0KPiAkMTUgb24gZUJheSBhbmQgdGhleSBhcmUgYW1hemluZyBjYXJk cy4NCg0KLS0gDQpCZXN0IHJlZ2FyZHMsDQogTGV2ICAgICAgICAgICAgICAgICAgICAgICAg ICAgIG1haWx0bzpsZXZARnJlZUJTRC5vcmc= From owner-freebsd-net@freebsd.org Sun Jan 15 13:47:56 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 787DACB1FC4 for ; Sun, 15 Jan 2017 13:47:56 +0000 (UTC) (envelope-from lev@freebsd.org) Received: from onlyone.not-for.work (onlyone.not-for.work [148.251.9.81]) by mx1.freebsd.org (Postfix) with ESMTP id 465EA1FF6 for ; Sun, 15 Jan 2017 13:47:55 +0000 (UTC) (envelope-from lev@freebsd.org) Received: from lion.home.serebryakov.spb.ru (unknown [IPv6:2001:470:923f:1:e0f4:994:662:862]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.not-for.work (Postfix) with ESMTPSA id BA8283E2; Sun, 15 Jan 2017 16:47:54 +0300 (MSK) Date: Sun, 15 Jan 2017 16:47:57 +0300 From: Lev Serebryakov X-Priority: 3 (Normal) Message-ID: <173736586.20170115164757@serebryakov.spb.ru> To: Kevin Bowling CC: freebsd-net@freebsd.org Subject: =?windows-1250?Q?Re:_10Gb_on_budged_with_fiber_=97_what_should_IP_choose=3F?= In-Reply-To: References: <1773236257.20170115123252@serebryakov.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=windows-1250 Content-Transfer-Encoding: base64 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Jan 2017 13:47:56 -0000 SGVsbG8gS2V2aW4sDQoNClN1bmRheSwgSmFudWFyeSAxNSwgMjAxNywgMTowMzoxNiBQTSwg eW91IHdyb3RlOg0KDQo+IEludGVsIGlzIG92ZXJwcmljZWQgYW5kIHN0aWxsIHJpZGluZyB0 aGVpciBFdGhlcm5ldCByZXB1dGF0aW9uIGZyb20gd2VsbA0KPiBvdmVyIGEgZGVjYWRlIGFn byB3aXRoIGxpdHRsZSB0byBzaG93IGZvciBpdC4gIEkgd291bGQgcmVjb21tZW5kIGN4Z2Jl DQo+IGFueXRoaW5nLCBzbyBDaGVsc2lvIFQ0MjAgYW5kIHVwLiAgVGhlIFQ1MjAtU08tQ1Ig aXMgd2l0aGluIHJlYXNvbiBmb3INCj4gZGVtYW5kaW5nIGhvbWUgb2ZmaWNlL3NtYWxsIG9m ZmljZSB1c2VycyB0byBwdXJjaGFzZSBuZXcgYW5kIEkndmUgZGVwbG95ZWQNCj4gb3ZlciBh IHRob3VzYW5kIG9mIHRoZXNlIGF0IHdvcmsgd2l0aCBmbGF3bGVzcyBwZXJmb3JtYW5jZS4g IFQ0MjAgYXJlIGEgYml0DQo+IGNoZWFwZXIgb24gaS5lLiBVUyBlYmF5IGlmIHRoYXQgaXMg dmlhYmxlIHRvIHlvdSBidXQgbm90IGJ5IHRvbyBtdWNoLiAgSQ0KPiBib3VnaHQgYSBUNDIw LUJUIHRoYXQgd2F5IGZvciBob21lIHVzZSBhbmQgaGF2ZSBubyBjb21wbGFpbnRzLg0KICBM b29rcyBsaWtlIG91ciB2aWV3cyBvZiB0ZXJtICJidWRnZXQiIGlzIHZlcnkgZGlmZmVyZW50 IDopLiBJZiBJIGNvdWxkIHNwZW5kDQogJDIwMCsvY2FyZCB0aGVuIGl0IHdvdWxkIGJlIG5v by1yYWluZXIuDQoNCj4gWW91IGNhbiB1c2UgREFDIHR3aW5heCBjYWJsZXMgdG8gY29ubmVj dCAxMEcgU0ZQKyBwb3J0cyB0b2dldGhlci4gIFRoaXMNCj4gd2lsbCBzYXZlIHNldmVyYWwg d2F0dHMgb2YgcG93ZXIgYW5kIGhlYXQgb24gZWFjaCBzaWRlIHZzIDEwRyBCYXNlLVQuDQog VHdpbmF4IGlzIG5vdCB3aGF0IEkgY291bGQgZml0IGluIG15IG9sZCBob3VzZSdzIChub3Qg ZGF0YWNlbnRlciEpIGNhYmxlIGR1Y3RzDQogOiggQW5kIG5ldyBjYWJsZSBkdWN0cyBtZWFu cyBhIExPVCBvZiBkdXN0eSB3b3JrIChyZWluZm9yY2VkIGNvbmNyZXRlDQogd2FsbHMpICYg bmV3IHdhbGxwYXBlcnMgJiBzdHVmZi4NCg0KDQotLSANCkJlc3QgcmVnYXJkcywNCiBMZXYg ICAgICAgICAgICAgICAgICAgICAgICAgICAgbWFpbHRvOmxldkBmcmVlYnNkLm9yZw== From owner-freebsd-net@freebsd.org Sun Jan 15 14:16:10 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8F37FCB186D for ; Sun, 15 Jan 2017 14:16:10 +0000 (UTC) (envelope-from wonko@4amlunch.net) Received: from mail-yw0-x231.google.com (mail-yw0-x231.google.com [IPv6:2607:f8b0:4002:c05::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DCD51247 for ; Sun, 15 Jan 2017 14:16:10 +0000 (UTC) (envelope-from wonko@4amlunch.net) Received: by mail-yw0-x231.google.com with SMTP id a10so55798063ywa.3 for ; Sun, 15 Jan 2017 06:16:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=4amlunch-net.20150623.gappssmtp.com; s=20150623; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=nOzEXkH/2339HCSKuXbK5AkKVR4xcjOIz2Y8HUxlmG4=; b=bTEJC4EsB+fxeY9a91d/gQg94/xtvegMCwIJZbBMvoRkKoKRZ/h9z/9fv20aBoPz8g MYwu8GP+UXClFQU1pCTqOFewoBXF95NwAYKhGYT9Vwhy9AYSOsCQnGjlcdkEydhXf//A 78l4qjOMd8hQi3ssz7dJoaRJDZqF9KqTjOWBdfYSneGCSonf4B1B5d+4bUK6X1ceXyCh ZEszkEgq5FHq8x4lrTP7Q2ouLpxRRizMTZvwmtxBEh1UXeSEAtXLNKbesY0hHeJpTX1p P5RhFAJpX39YKAyhv9Jo87HfMXS6Yx/B70HJIgdLEHnhU65H9O9QEGujA8GPTUMyjXxB ZUTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=nOzEXkH/2339HCSKuXbK5AkKVR4xcjOIz2Y8HUxlmG4=; b=CuZegydLeL8quJbl/Yw/1++3yT1SJf0+O7G4PJ+XaHwlyrMpDwxgfanP9oFZ9MspbH D3hhX6DsTMgpsUK6+7Pp+Y+byBAjwOPUhsWdqG0vot4H+l9aWbuLknHZcZbOHuwAitTj qKSiDUiEV0v3o8pa0yN/7OusGH4uoOBMbYAp/IkG6psEDexpgJYDxFKDpQsRBtVo4I79 InOsGDnOsDPdXkawM4NPp4O9uJcig4/r8bB0bnytWlsDkc0nT4Gij53OuVDd6rS4EAAD AlcqFB9XKZc+2Hq55H6IZ/yOgolNidKF7UJsdPby8Phw6D4Ug+CCtbzl4nJ+wfLcaGs4 ImRQ== X-Gm-Message-State: AIkVDXK5zVqiqywKDSi0676tlrxeODkGmAeP/lABL5jwoAYzbeWExivrvzPW/Sf50B015Q== X-Received: by 10.13.226.147 with SMTP id l141mr24243630ywe.258.1484489769266; Sun, 15 Jan 2017 06:16:09 -0800 (PST) Received: from [192.168.190.250] (50-204-9-73-static.hfc.comcastbusiness.net. [50.204.9.73]) by smtp.gmail.com with ESMTPSA id h3sm5378613ywc.54.2017.01.15.06.16.08 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 15 Jan 2017 06:16:08 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (1.0) Subject: =?utf-8?Q?Re:_10Gb_on_budged_with_fiber_=E2=80=94_what_should_IP?= =?utf-8?Q?_choose=3F?= From: wonko@4amlunch.net X-Mailer: iPhone Mail (14C92) In-Reply-To: <374DD84B-40E7-4B8A-A03B-68FDAD0EF911@gmail.com> Date: Sun, 15 Jan 2017 09:16:08 -0500 Cc: FreeBSD Net Content-Transfer-Encoding: quoted-printable Message-Id: <3DBBD532-67B4-45F0-AE6D-8AFAEBAD2768@4amlunch.net> References: <1773236257.20170115123252@serebryakov.spb.ru> <6FCA135B-B7F6-4148-B1C3-FE0D4470A568@4amlunch.net> <858B840B-95A0-4556-AD77-F3918B51B90A@gmail.com> <374DD84B-40E7-4B8A-A03B-68FDAD0EF911@gmail.com> To: Ben RUBSON X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Jan 2017 14:16:10 -0000 Ok I'm going to guess not then as it didn't last time i looked. Ended up goi= ng to OmniOS for the storage array as COMSTAR has IB SRP target support out o= f the box. -brian > On Jan 15, 2017, at 08:37, Ben RUBSON wrote: >=20 > iSER is supported as initiator, however iSER target is missing, you then h= ave to run a non-FreeBSD iSER target :-/ >=20 > I'm not sure for SRP. >=20 > Ben >=20 >> On 15 Jan 2017, at 14:31, wonko@4amlunch.net wrote: >>=20 >> SRP target support yet? Last time I looked there wasn't any. :( >>=20 >> -brian >>=20 >> On Jan 15, 2017, at 08:22, Ben RUBSON wrote: >>=20 >>>> On 15 Jan 2017, at 13:33, wonko@4amlunch.net wrote: >>>>=20 >>>> How good is FreeBSD's mellanox support? >>>=20 >>> =46rom my own experience with ConnectX-3 adapters, very good ! >>>=20 >>> Ben > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@freebsd.org Sun Jan 15 14:17:08 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A89F2CB192D for ; Sun, 15 Jan 2017 14:17:08 +0000 (UTC) (envelope-from wonko@4amlunch.net) Received: from mail-yw0-x235.google.com (mail-yw0-x235.google.com [IPv6:2607:f8b0:4002:c05::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6474A1432 for ; Sun, 15 Jan 2017 14:17:08 +0000 (UTC) (envelope-from wonko@4amlunch.net) Received: by mail-yw0-x235.google.com with SMTP id l75so55880130ywb.0 for ; Sun, 15 Jan 2017 06:17:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=4amlunch-net.20150623.gappssmtp.com; s=20150623; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=pWduF1DSFlEbMPouZvsu2yXd3Z+nQaDOYuuTP+nUHyI=; b=dJWG6bk3ViaUsdpgqISjZNMd9/i5Dqy95lMYAUdvasmd4P4BaHgZFDQSCTllmK62Tb nhRL2KEde+MLQMubJ+TTswd4tvVEqHxbrZog1jRdiVODApt7MTCcTpg1mGPrWIhf+hgF TGh1qBud+l2uSd55D8inZRFkoFfl6JKC/SNJ96xw4ee/h++kbwvwLIAnlhKkJlQ7YEwH GRvbbeppMNID4AUXzM7ttyGcGeBv7vX1vVCGh0ta0La+ccFctWb1V38kz6v0RtoEvrq6 DBQUHS75bx1dncF2u7rRKpoe8peUJqRM7i0EAoQzhNP/dYXBm4Idz/iDGHA/Nxi+oWxS +o2A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=pWduF1DSFlEbMPouZvsu2yXd3Z+nQaDOYuuTP+nUHyI=; b=ImTUL0oQhgzeA3rjax/nZ4djNzuPvzRMnw0ovVtCl6isUAlzPfLlPO6fmHKYpI+Bfc 6oR+qIhGsOuzzVPdfg4uBK7MF6tmVJN9yhQkQhlUT+j2kpJ0C0ZJKOk417lliYVI7iJu hSXo+4IroRXM2syT/uhlHsGg+0N+lFkUNPNegf7XzhML0nRGea6JkyWUvT+P8pVrYw7j ZbnRrOSgjpowIjwkGCVGC8JY5T/CVQGWzQf9D5xIB8qV82XCXHpIY1toNXFcsAT7KF3v neMhlm4Llvqm809YJ1cXEcC6a9twMlDMsnSTaU/Op/agu95xmwVipLHVC4KUVxYokUMh //4g== X-Gm-Message-State: AIkVDXKsLrKbmAaeXNUXoM01JYaZuqhA0dLLrxxJ42yuN9gaIrInwafvvYxapUi2dOrHbw== X-Received: by 10.13.207.197 with SMTP id r188mr21362785ywd.88.1484489827750; Sun, 15 Jan 2017 06:17:07 -0800 (PST) Received: from [192.168.190.250] (50-204-9-73-static.hfc.comcastbusiness.net. [50.204.9.73]) by smtp.gmail.com with ESMTPSA id d5sm8239344ywe.48.2017.01.15.06.17.07 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 15 Jan 2017 06:17:07 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (1.0) Subject: =?utf-8?Q?Re:_10Gb_on_budged_with_fiber_=E2=80=94_what_should_IP?= =?utf-8?Q?_choose=3F?= From: wonko@4amlunch.net X-Mailer: iPhone Mail (14C92) In-Reply-To: <1985967690.20170115164751@serebryakov.spb.ru> Date: Sun, 15 Jan 2017 09:17:06 -0500 Cc: Kevin Bowling , freebsd-net@freebsd.org Content-Transfer-Encoding: 7bit Message-Id: <76369968-BC74-4FE4-ADF9-16B78545E38B@4amlunch.net> References: <1773236257.20170115123252@serebryakov.spb.ru> <6FCA135B-B7F6-4148-B1C3-FE0D4470A568@4amlunch.net> <1985967690.20170115164751@serebryakov.spb.ru> To: Lev Serebryakov X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Jan 2017 14:17:08 -0000 So it looks like the answer is yes! Great cards. They will work well for you. -brian > On Jan 15, 2017, at 08:47, Lev Serebryakov wrote: > > Hello Wonko, > > Sunday, January 15, 2017, 3:33:35 PM, you wrote: > > It is what I want to know! :) > >> How good is FreeBSD's mellanox support? You can get a ConnectX-2 EN for >> $15 on eBay and they are amazing cards. > > -- > Best regards, > Lev mailto:lev@FreeBSD.org From owner-freebsd-net@freebsd.org Sun Jan 15 15:43:26 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5006ACB148F for ; Sun, 15 Jan 2017 15:43:26 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3F6421E5B for ; Sun, 15 Jan 2017 15:43:26 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v0FFhPi6007153 for ; Sun, 15 Jan 2017 15:43:26 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 213869] when setting an ipsec policy with spdadd src[port], outbound traffic from 2049/tcp is not encrypted Date: Sun, 15 Jan 2017 15:43:26 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: 11.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: commit-hook@freebsd.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Jan 2017 15:43:26 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D213869 --- Comment #11 from commit-hook@freebsd.org --- A commit references this bug: Author: ae Date: Sun Jan 15 15:43:19 UTC 2017 New revision: 312233 URL: https://svnweb.freebsd.org/changeset/base/312233 Log: MFC r311679: Add direction argument to ipsec_setspidx_inpcb() function. This function is used only by ipsec_getpolicybysock() to fill security policy index selector for locally generated packets (that have INPCB). The function incorrectly assumes that spidx is the same for both directions. Fix this by using new direction argument to specify correct INPCB secur= ity policy - sp_in or sp_out. There is no need to fill both policy indeces, because they are overwritten for each packet. This fixes security policy matching for outbound packets when user has specified TCP/UDP ports in the security policy upperspec. PR: 213869 Changes: _U stable/11/ stable/11/sys/netipsec/ipsec.c --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Sun Jan 15 15:43:54 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A9DD6CB150D for ; Sun, 15 Jan 2017 15:43:54 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 992771043 for ; Sun, 15 Jan 2017 15:43:54 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v0FFhs01007992 for ; Sun, 15 Jan 2017 15:43:54 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 213869] when setting an ipsec policy with spdadd src[port], outbound traffic from 2049/tcp is not encrypted Date: Sun, 15 Jan 2017 15:43:54 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: 11.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: ae@FreeBSD.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: FIXED X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_status resolution Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Jan 2017 15:43:54 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D213869 Andrey V. Elsukov changed: What |Removed |Added ---------------------------------------------------------------------------- Status|New |Closed Resolution|--- |FIXED --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Sun Jan 15 17:48:33 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A64A6CB126E for ; Sun, 15 Jan 2017 17:48:33 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: from mail-wm0-x236.google.com (mail-wm0-x236.google.com [IPv6:2a00:1450:400c:c09::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 498791152 for ; Sun, 15 Jan 2017 17:48:33 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: by mail-wm0-x236.google.com with SMTP id c85so130601246wmi.1 for ; Sun, 15 Jan 2017 09:48:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:from:date:message-id:subject:to; bh=T6MTdvb2JZi5qixhpRbnJd/M/yN/CLzPZeVje9bpOO8=; b=kbZ69WJA0UgyObycHWwArG+XTNNW4VSDBiNZMHam2UkaFCHKnlOaKdig3Hvc2DI+2q 58HN7vjKgRlo1mxfkQ1kNSZ2xoDkoLRynjw3rgBhAYR+x8N5TV0Fh8clJ8mJ9QG/+uBq w2kE3DvSCr+Ee67sPQbWtQXpHDz/a6uX25XyZeTw+h1oPqDr3WbG9NUCBK+m7OLcl6vL qBGtUaVGfA0K83xBLxZX2O4yF3VR9uOFbzxqUNv2H3aBJk/Qwe5/zqfN1O3cX2mloOM3 +rlJvtS7HevaJmDxKpmzFVj/qb/3IPEf9dgxRuAG2n9+3gQTRJvhsjVO+ghJUsH2a1lb Qjig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:from:date:message-id:subject :to; bh=T6MTdvb2JZi5qixhpRbnJd/M/yN/CLzPZeVje9bpOO8=; b=LSS7JqUJUb89o5ApTAQ539RJLgSLf/x7oq2+Y41lTC9ynBvNLKcZ1ZtRG+SPuOuoPS tuTxoeO9HWf5D8eFXbGcHxc5f1LQJSC2fj3JJ5Yi1KbC/esJtsmQRQ/Q0q92gGA0XbjS wxI+3zqDKOm5WBG1zF0rJzs/QGG3JCZt2iVE2wCQ0KIGkNtqrroYRnsrZCvaPxzb/imc d7ibpaHWpMydK1lfZwNlIyIN2Z1c+d561pVyiy+WFpKcfmY2rxi/wpYPlrmm/7xDCZlp aSU8X3rVv1krWkbgnYbBS0mbwM9TXVVhK7vCe1j6W3W8sX+gHKD7S8642JRIvpoAea5N MuRg== X-Gm-Message-State: AIkVDXIEW/dT1WWvLmmPT965S8iYKTL1zTs9xogC0mznctjn/pwxs0rb5HybJ69H0jLzspGgxp8QPcedXRsBuA== X-Received: by 10.223.151.18 with SMTP id r18mr19818708wrb.129.1484502511581; Sun, 15 Jan 2017 09:48:31 -0800 (PST) MIME-Version: 1.0 Sender: kob6558@gmail.com Received: by 10.28.41.194 with HTTP; Sun, 15 Jan 2017 09:48:30 -0800 (PST) From: Kevin Oberman Date: Sun, 15 Jan 2017 09:48:30 -0800 X-Google-Sender-Auth: llGHPS1JXmLkZOmSKcvLkIO8lLw Message-ID: Subject: iwn logging "need promiscuous mode update callback" To: "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Jan 2017 17:48:33 -0000 Some time around two months ago I started seeing the message "iwn0: need promiscuous mode update callback" logged every time promiscuous mode was either enabled or disabled. Things still seem to work OK. System is normally built from source and updated about once a month. FreeBSD rogue 11.0-STABLE FreeBSD 11.0-STABLE #0 r311919: Wed Jan 11 09:35:06 PST 2017 root@rogue:/usr/obj/usr/src/sys/GENERIC.4BSD amd64 Added oddness... the message is often split into two log entries: Jan 15 09:41:13 rogue kernel: iwn0: Jan 15 09:41:13 rogue kernel: wlan0: promiscuous mode disabled Jan 15 09:41:13 rogue kernel: need promiscuous mode update callback I don't see how this is even possible as I always thought syslog messages were atomic. Guess not. Again, this is not causing problems that I am aware of, but is a bit disturbing. -- Kevin Oberman, Part time kid herder and retired Network Engineer E-mail: rkoberman@gmail.com PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683 From owner-freebsd-net@freebsd.org Sun Jan 15 21:00:23 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 261ABCB1168 for ; Sun, 15 Jan 2017 21:00:23 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 14955114E for ; Sun, 15 Jan 2017 21:00:23 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v0FL01Ec027183 for ; Sun, 15 Jan 2017 21:00:22 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <201701152100.v0FL01Ec027183@kenobi.freebsd.org> From: bugzilla-noreply@FreeBSD.org To: freebsd-net@FreeBSD.org Subject: Problem reports for freebsd-net@FreeBSD.org that need special attention Date: Sun, 15 Jan 2017 21:00:22 +0000 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Jan 2017 21:00:23 -0000 To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ------------+-----------+--------------------------------------------------- In Progress | 165622 | [ndis][panic][patch] Unregistered use of FPU in k In Progress | 203422 | mpd/ppoe not working with re(4) with revision 285 In Progress | 206581 | bxe_ioctl_nvram handler is faulty New | 204438 | setsockopt() handling of kern.ipc.maxsockbuf limi New | 205592 | TCP processing in IPSec causes kernel panic New | 206053 | kqueue support code of netmap causes panic New | 213410 | [carp] service netif restart causes hang only whe Open | 148807 | [panic] "panic: sbdrop" and "panic: sbsndptr: soc Open | 193452 | Dell PowerEdge 210 II -- Kernel panic bce (broadc Open | 194485 | Userland cannot add IPv6 prefix routes Open | 194515 | Fatal Trap 12 Kernel with vimage Open | 199136 | [if_tap] Added down_on_close sysctl variable to t Open | 202510 | [CARP] advertisements sourced from CARP IP cause Open | 206544 | sendmsg(2) (sendto(2) too?) can fail with EINVAL; Open | 211031 | [panic] in ng_uncallout when argument is NULL Open | 211962 | bxe driver queue soft hangs and flooding tx_soft_ Open | 213257 | Crash in IGB driver with ALTQ 17 problems total for which you should take action. From owner-freebsd-net@freebsd.org Mon Jan 16 00:15:48 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0A82ECAF1F9 for ; Mon, 16 Jan 2017 00:15:48 +0000 (UTC) (envelope-from kevin.bowling@kev009.com) Received: from mail-ot0-x236.google.com (mail-ot0-x236.google.com [IPv6:2607:f8b0:4003:c0f::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CE7041045 for ; Mon, 16 Jan 2017 00:15:47 +0000 (UTC) (envelope-from kevin.bowling@kev009.com) Received: by mail-ot0-x236.google.com with SMTP id 65so35801597otq.2 for ; Sun, 15 Jan 2017 16:15:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kev009.com; s=google; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=jo656u45NKnPd2taMUAawv7lFmUsjNgiR2gB22AjquE=; b=f8Tk1FBknGyZInyGHE3CBV0jLevZpP7kIbZ1OFRtDm37MDxeIOamvX9AWAnz5pkcHo YdQH37oXsEdKW53OZjbZymyzGRT3mqUkcTr78gX6Wm5bIbjQRdWkzfebvy3No3XKrSqj tKUqMcYP6Eeljgtnfm99x6/rjSzmYEGS19n7U= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=jo656u45NKnPd2taMUAawv7lFmUsjNgiR2gB22AjquE=; b=Um9PMxAbR6bcJePslw4l/+sAHqznBO1bZgsgvVJ7bC6MysgeWmOvTYsNMKZd3BgfCY 1cJuaJELAfZ38jpKjzTw8fc/lpC2/2gB9knup7WTRoaBoulx0z3x9ZtUFVSivlmVk59m J6HlfqHe3Yu2HLUV33bELKVB5zkroTINlC2XlyXODUWsXt6EM0U0Fxkhl6Ppmn5vlxyA U4DywvX8tb406xxInO0/zwFyOKIV0yS56wD0JwOU36UH3vy1G6CieWZcQhgHlzj7WkkV WvJmJ7C9Xqtip/vrmINR7D2EyDJUWy9PhVLaEN1Y+irc41H722qE4cDn3bbAvEcGe5TO DvXw== X-Gm-Message-State: AIkVDXIME3bYNAEG1y2KVJ+vOoPHpdZ+nvLFq7oTRgRyxMxOl82umg6Dhk18zXwyQyZ6pteHKACJEQLsVXsMQg== X-Received: by 10.157.26.24 with SMTP id a24mr15907454ote.107.1484525746889; Sun, 15 Jan 2017 16:15:46 -0800 (PST) MIME-Version: 1.0 Received: by 10.157.23.208 with HTTP; Sun, 15 Jan 2017 16:15:46 -0800 (PST) In-Reply-To: <173736586.20170115164757@serebryakov.spb.ru> References: <1773236257.20170115123252@serebryakov.spb.ru> <173736586.20170115164757@serebryakov.spb.ru> From: Kevin Bowling Date: Sun, 15 Jan 2017 17:15:46 -0700 Message-ID: Subject: =?UTF-8?Q?Re=3A_10Gb_on_budged_with_fiber_=E2=80=94_what_should_IP_cho?= =?UTF-8?Q?ose=3F?= To: Lev Serebryakov Cc: freebsd-net@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Jan 2017 00:15:48 -0000 You should be able to find T420-CR or T420-BT for half that. The primary difference is that cxgbe is still a supported driver, and it's been high quality from the start unlike a lot of other vendors. You are engaging in archeology with the other cards you mentioned, and may spend way more value in time than you would in the purchase cost of two NICs :) On Sun, Jan 15, 2017 at 6:47 AM, Lev Serebryakov wrote: > Hello Kevin, > > Sunday, January 15, 2017, 1:03:16 PM, you wrote: > > > Intel is overpriced and still riding their Ethernet reputation from well > > over a decade ago with little to show for it. I would recommend cxgbe > > anything, so Chelsio T420 and up. The T520-SO-CR is within reason for > > demanding home office/small office users to purchase new and I've > deployed > > over a thousand of these at work with flawless performance. T420 are a > bit > > cheaper on i.e. US ebay if that is viable to you but not by too much. I > > bought a T420-BT that way for home use and have no complaints. > Looks like our views of term "budget" is very different :). If I could > spend > $200+/card then it would be noo-rainer. > > > You can use DAC twinax cables to connect 10G SFP+ ports together. This > > will save several watts of power and heat on each side vs 10G Base-T. > Twinax is not what I could fit in my old house's (not datacenter!) cable > ducts > :( And new cable ducts means a LOT of dusty work (reinforced concrete > walls) & new wallpapers & stuff. > > > -- > Best regards, > Lev mailto:lev@freebsd.org From owner-freebsd-net@freebsd.org Mon Jan 16 03:01:38 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CD385CB1365 for ; Mon, 16 Jan 2017 03:01:38 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BC28B1F11 for ; Mon, 16 Jan 2017 03:01:38 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v0G31c1V017570 for ; Mon, 16 Jan 2017 03:01:38 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 215933] SCM_RIGHTS messages being lost, socket data being lost as well (with example code) Date: Mon, 16 Jan 2017 03:01:38 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.3-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: sepherosa@gmail.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Jan 2017 03:01:38 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D215933 Sepherosa Ziehau changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |sepherosa@gmail.com --- Comment #2 from Sepherosa Ziehau --- I believe this is exact same as: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D181741 --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Mon Jan 16 03:04:04 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2B9CCCB16C6 for ; Mon, 16 Jan 2017 03:04:04 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from reviews.nyi.freebsd.org (reviews.nyi.freebsd.org [IPv6:2610:1c1:1:607c::16:b]) by mx1.freebsd.org (Postfix) with ESMTP id E1E0D1351 for ; Mon, 16 Jan 2017 03:04:03 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by reviews.nyi.freebsd.org (Postfix, from userid 1346) id A102936425; Mon, 16 Jan 2017 03:04:03 +0000 (UTC) Date: Mon, 16 Jan 2017 03:04:03 +0000 To: freebsd-net@freebsd.org From: Phabricator Reply-to: D9058+325+b410c05319dac5ee@reviews.freebsd.org Subject: [Differential] D9058: alc: Add Killer E2500 support. Message-ID: <6026021ffbc6e2e10a7c3e10423cf950@localhost.localdomain> X-Priority: 3 X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , Thread-Topic: D9058: alc: Add Killer E2500 support. X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: Precedence: bulk In-Reply-To: References: Thread-Index: MmU5MmIzMzRkNmRiYmM3Y2U1YzM2ZmMxMjQ1IFh8OCM= MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="b1_6026021ffbc6e2e10a7c3e10423cf950" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Jan 2017 03:04:04 -0000 --b1_6026021ffbc6e2e10a7c3e10423cf950 Content-Type: text/plain; charset = "utf-8" Content-Transfer-Encoding: base64 VGhpcyByZXZpc2lvbiB3YXMgYXV0b21hdGljYWxseSB1cGRhdGVkIHRvIHJlZmxlY3QgdGhlIGNv bW1pdHRlZCBjaGFuZ2VzLgpDbG9zZWQgYnkgY29tbWl0IHJTMzEyMjUwOiBhbGM6IEFkZCBLaWxs ZXIgRTI1MDAgc3VwcG9ydCAoYXV0aG9yZWQgYnkgc2VwaGUpLgoKQ0hBTkdFRCBQUklPUiBUTyBD T01NSVQKICBodHRwczovL3Jldmlld3MuZnJlZWJzZC5vcmcvRDkwNTg/dnM9MjM2NDcmaWQ9MjQw NTYjdG9jCgpSRVBPU0lUT1JZCiAgclMgRnJlZUJTRCBzcmMgcmVwb3NpdG9yeQoKQ0hBTkdFUyBT SU5DRSBMQVNUIFVQREFURQogIGh0dHBzOi8vcmV2aWV3cy5mcmVlYnNkLm9yZy9EOTA1OD92cz0y MzY0NyZpZD0yNDA1NgoKUkVWSVNJT04gREVUQUlMCiAgaHR0cHM6Ly9yZXZpZXdzLmZyZWVic2Qu b3JnL0Q5MDU4CgpBRkZFQ1RFRCBGSUxFUwogIGhlYWQvc2hhcmUvbWFuL21hbjQvYWxjLjQKICBo ZWFkL3N5cy9kZXYvYWxjL2lmX2FsYy5jCiAgaGVhZC9zeXMvZGV2L2FsYy9pZl9hbGNyZWcuaAog IGhlYWQvc3lzL2Rldi9wY2kvcGNpLmMKCkVNQUlMIFBSRUZFUkVOQ0VTCiAgaHR0cHM6Ly9yZXZp ZXdzLmZyZWVic2Qub3JnL3NldHRpbmdzL3BhbmVsL2VtYWlscHJlZmVyZW5jZXMvCgpUbzogc2Vw aGVyb3NhX2dtYWlsLmNvbSwgZGVjdWlfbWljcm9zb2Z0LmNvbSwgamhiCkNjOiBmcmVlYnNkLW5l dC1saXN0Cg== --b1_6026021ffbc6e2e10a7c3e10423cf950 Content-Type: text/x-patch; charset=utf-8; name="D9058.24056.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="D9058.24056.patch" ZGlmZiAtLWdpdCBhL2hlYWQvc3lzL2Rldi9wY2kvcGNpLmMgYi9oZWFkL3N5cy9kZXYvcGNpL3Bj aS5jCi0tLSBhL2hlYWQvc3lzL2Rldi9wY2kvcGNpLmMKKysrIGIvaGVhZC9zeXMvZGV2L3BjaS9w Y2kuYwpAQCAtMjgxLDEzICsyODEsMTQgQEAKIAl7IDB4NDM4NTEwMDIsIFBDSV9RVUlSS19VTk1B UF9SRUcsCTB4MTQsCTAgfSwKIAogCS8qCi0JICogQXRoZXJvcyBBUjgxNjEvQVI4MTYyL0UyMjAw L0UyNDAwIEV0aGVybmV0IGNvbnRyb2xsZXJzIGhhdmUgYQotCSAqIGJ1ZyB0aGF0IE1TSSBpbnRl cnJ1cHQgZG9lcyBub3QgYXNzZXJ0IGlmIFBDSU1fQ01EX0lOVHhESVMgYml0CisJICogQXRoZXJv cyBBUjgxNjEvQVI4MTYyL0UyMjAwL0UyNDAwL0UyNTAwIEV0aGVybmV0IGNvbnRyb2xsZXJzIGhh dmUKKwkgKiBhIGJ1ZyB0aGF0IE1TSSBpbnRlcnJ1cHQgZG9lcyBub3QgYXNzZXJ0IGlmIFBDSU1f Q01EX0lOVHhESVMgYml0CiAJICogb2YgdGhlIGNvbW1hbmQgcmVnaXN0ZXIgaXMgc2V0LgogCSAq LwogCXsgMHgxMDkxMTk2OSwgUENJX1FVSVJLX01TSV9JTlRYX0JVRywJMCwJMCB9LAogCXsgMHhF MDkxMTk2OSwgUENJX1FVSVJLX01TSV9JTlRYX0JVRywJMCwJMCB9LAogCXsgMHhFMEExMTk2OSwg UENJX1FVSVJLX01TSV9JTlRYX0JVRywJMCwJMCB9LAorCXsgMHhFMEIxMTk2OSwgUENJX1FVSVJL X01TSV9JTlRYX0JVRywJMCwJMCB9LAogCXsgMHgxMDkwMTk2OSwgUENJX1FVSVJLX01TSV9JTlRY X0JVRywJMCwJMCB9LAogCiAJLyoKZGlmZiAtLWdpdCBhL2hlYWQvc3lzL2Rldi9hbGMvaWZfYWxj cmVnLmggYi9oZWFkL3N5cy9kZXYvYWxjL2lmX2FsY3JlZy5oCi0tLSBhL2hlYWQvc3lzL2Rldi9h bGMvaWZfYWxjcmVnLmgKKysrIGIvaGVhZC9zeXMvZGV2L2FsYy9pZl9hbGNyZWcuaApAQCAtNTAs NiArNTAsNyBAQAogI2RlZmluZQlERVZJQ0VJRF9BVEhFUk9TX0FSODE3MgkJMHgxMEEwCiAjZGVm aW5lCURFVklDRUlEX0FUSEVST1NfRTIyMDAJCTB4RTA5MQogI2RlZmluZQlERVZJQ0VJRF9BVEhF Uk9TX0UyNDAwCQkweEUwQTEKKyNkZWZpbmUJREVWSUNFSURfQVRIRVJPU19FMjUwMAkJMHhFMEIx CiAKICNkZWZpbmUJQVRIRVJPU19BUjgxNTJfQl9WMTAJCTB4QzAKICNkZWZpbmUJQVRIRVJPU19B UjgxNTJfQl9WMTEJCTB4QzEKZGlmZiAtLWdpdCBhL2hlYWQvc3lzL2Rldi9hbGMvaWZfYWxjLmMg Yi9oZWFkL3N5cy9kZXYvYWxjL2lmX2FsYy5jCi0tLSBhL2hlYWQvc3lzL2Rldi9hbGMvaWZfYWxj LmMKKysrIGIvaGVhZC9zeXMvZGV2L2FsYy9pZl9hbGMuYwpAQCAtMTIzLDYgKzEyMyw4IEBACiAJ CSJLaWxsZXIgRTIyMDAgR2lnYWJpdCBFdGhlcm5ldCIgfSwKIAl7IFZFTkRPUklEX0FUSEVST1Ms IERFVklDRUlEX0FUSEVST1NfRTI0MDAsIDkgKiAxMDI0LAogCQkiS2lsbGVyIEUyNDAwIEdpZ2Fi aXQgRXRoZXJuZXQiIH0sCisJeyBWRU5ET1JJRF9BVEhFUk9TLCBERVZJQ0VJRF9BVEhFUk9TX0Uy NTAwLCA5ICogMTAyNCwKKwkJIktpbGxlciBFMjUwMCBHaWdhYml0IEV0aGVybmV0IiB9LAogCXsg MCwgMCwgMCwgTlVMTH0KIH07CiAKQEAgLTEwODMsNiArMTA4NSw3IEBACiAJY2FzZSBERVZJQ0VJ RF9BVEhFUk9TX0FSODE2MToKIAljYXNlIERFVklDRUlEX0FUSEVST1NfRTIyMDA6CiAJY2FzZSBE RVZJQ0VJRF9BVEhFUk9TX0UyNDAwOgorCWNhc2UgREVWSUNFSURfQVRIRVJPU19FMjUwMDoKIAlj YXNlIERFVklDRUlEX0FUSEVST1NfQVI4MTYyOgogCWNhc2UgREVWSUNFSURfQVRIRVJPU19BUjgx NzE6CiAJY2FzZSBERVZJQ0VJRF9BVEhFUk9TX0FSODE3MjoKQEAgLTE0MDIsNiArMTQwNSw3IEBA CiAJc3dpdGNoIChzYy0+YWxjX2lkZW50LT5kZXZpY2VpZCkgewogCWNhc2UgREVWSUNFSURfQVRI RVJPU19FMjIwMDoKIAljYXNlIERFVklDRUlEX0FUSEVST1NfRTI0MDA6CisJY2FzZSBERVZJQ0VJ RF9BVEhFUk9TX0UyNTAwOgogCQlzYy0+YWxjX2ZsYWdzIHw9IEFMQ19GTEFHX0UyWDAwOwogCQkv KiBGQUxMVEhST1VHSCAqLwogCWNhc2UgREVWSUNFSURfQVRIRVJPU19BUjgxNjE6CkBAIC0xNDgw LDcgKzE0ODQsOCBAQAogCQlpZiAoYWxjX2RtYV9idXJzdFtzYy0+YWxjX2RtYV93cl9idXJzdF0g PiAxMDI0KQogCQkJc2MtPmFsY19kbWFfd3JfYnVyc3QgPSAzOwogCQkvKgotCQkgKiBGb3JjZSBt YXhpbXVtIHBheWxvYWQgc2l6ZSB0byAxMjggYnl0ZXMgZm9yIEUyMjAwL0UyNDAwLgorCQkgKiBG b3JjZSBtYXhpbXVtIHBheWxvYWQgc2l6ZSB0byAxMjggYnl0ZXMgZm9yCisJCSAqIEUyMjAwL0Uy NDAwL0UyNTAwLgogCQkgKiBPdGhlcndpc2UgaXQgdHJpZ2dlcnMgRE1BIHdyaXRlIGVycm9yLgog CQkgKi8KIAkJaWYgKChzYy0+YWxjX2ZsYWdzICYgQUxDX0ZMQUdfRTJYMDApICE9IDApCmRpZmYg LS1naXQgYS9oZWFkL3NoYXJlL21hbi9tYW40L2FsYy40IGIvaGVhZC9zaGFyZS9tYW4vbWFuNC9h bGMuNAotLS0gYS9oZWFkL3NoYXJlL21hbi9tYW40L2FsYy40CisrKyBiL2hlYWQvc2hhcmUvbWFu L21hbjQvYWxjLjQKQEAgLTEyNCw2ICsxMjQsOCBAQAogS2lsbGVyIEUyMjAwIEdpZ2FiaXQgRXRo ZXJuZXQgY29udHJvbGxlcgogLkl0CiBLaWxsZXIgRTI0MDAgR2lnYWJpdCBFdGhlcm5ldCBjb250 cm9sbGVyCisuSXQKK0tpbGxlciBFMjUwMCBHaWdhYml0IEV0aGVybmV0IGNvbnRyb2xsZXIKIC5F bAogLlNoIExPQURFUiBUVU5BQkxFUwogVHVuYWJsZXMgY2FuIGJlIHNldCBhdCB0aGUKCg== --b1_6026021ffbc6e2e10a7c3e10423cf950-- From owner-freebsd-net@freebsd.org Mon Jan 16 10:59:56 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A7106CB0738 for ; Mon, 16 Jan 2017 10:59:56 +0000 (UTC) (envelope-from sthaug@nethelp.no) Received: from bizet.nethelp.no (bizet.nethelp.no [IPv6:2001:8c0:9e04:500::1]) by mx1.freebsd.org (Postfix) with ESMTP id 726A01986 for ; Mon, 16 Jan 2017 10:59:56 +0000 (UTC) (envelope-from sthaug@nethelp.no) Received: from localhost (bizet.nethelp.no [IPv6:2001:8c0:9e04:500::1]) by bizet.nethelp.no (Postfix) with ESMTP id 557B4E6065 for ; Mon, 16 Jan 2017 11:59:48 +0100 (CET) Date: Mon, 16 Jan 2017 11:59:48 +0100 (CET) Message-Id: <20170116.115948.74712958.sthaug@nethelp.no> To: freebsd-net@freebsd.org Subject: Has anybody looked at Cisco VPP? From: sthaug@nethelp.no X-Mailer: Mew version 3.3 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Jan 2017 10:59:56 -0000 Has anybody looked at Cisco VPP? http://blogs.cisco.com/sp/a-bigger-helping-of-internet-please It would be interesting to see comparisons with netmap. Steinar Haug, Nethelp consulting, sthaug@nethelp.no From owner-freebsd-net@freebsd.org Mon Jan 16 11:15:51 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4269FCB0FF7 for ; Mon, 16 Jan 2017 11:15:51 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 05F1D17E5 for ; Mon, 16 Jan 2017 11:15:51 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1cT5GN-000NQ5-Nh; Mon, 16 Jan 2017 14:15:47 +0300 Date: Mon, 16 Jan 2017 14:15:47 +0300 From: Slawa Olhovchenkov To: sthaug@nethelp.no Cc: freebsd-net@freebsd.org Subject: Re: Has anybody looked at Cisco VPP? Message-ID: <20170116111547.GC78888@zxy.spb.ru> References: <20170116.115948.74712958.sthaug@nethelp.no> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20170116.115948.74712958.sthaug@nethelp.no> User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Jan 2017 11:15:51 -0000 On Mon, Jan 16, 2017 at 11:59:48AM +0100, sthaug@nethelp.no wrote: > Has anybody looked at Cisco VPP? > > http://blogs.cisco.com/sp/a-bigger-helping-of-internet-please > > It would be interesting to see comparisons with netmap. Unrelated. netmap is a compute-efficient network I/O, like DPDK. VPP work over this layer: "Combined with a compute-efficient network I/O (e.g., DPDK), VPP enables very efficient and high performing design". From owner-freebsd-net@freebsd.org Tue Jan 17 05:03:08 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3F539CB3012 for ; Tue, 17 Jan 2017 05:03:08 +0000 (UTC) (envelope-from 01000159aac969e6-b2fc3913-d04e-42d4-befd-402ed0d830bf-000000@amazonses.com) Received: from a8-13.smtp-out.amazonses.com (a8-13.smtp-out.amazonses.com [54.240.8.13]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 093341C0E for ; Tue, 17 Jan 2017 05:03:07 +0000 (UTC) (envelope-from 01000159aac969e6-b2fc3913-d04e-42d4-befd-402ed0d830bf-000000@amazonses.com) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=vnqrkfnvu6csdl6mwgk5t6ix3nnepx57; d=tarsnap.com; t=1484629043; h=To:From:Subject:Message-ID:Date:MIME-Version:Content-Type:Content-Transfer-Encoding; bh=PwKGFIS+gXQI7q1Yr+8XcTpTpzivkbY/a4eJr1M1gvI=; b=l+Y8vt/bL/XEVP8LziPZDM26OqLj3MLu1tYqT+tOKKuf5j2ReKXIVMPe1JhNWads E1i36g8Y3LWgfwDC8WsPcPkDe1fF2FQJmOrFVUammexftmNDZRm6koyCHTF6npnMq7h VyGJDVVbpZjU8SEyKg9xaOgZKiZqH22KspGT0TPY= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=6gbrjpgwjskckoa6a5zn6fwqkn67xbtw; d=amazonses.com; t=1484629043; h=To:From:Subject:Message-ID:Date:MIME-Version:Content-Type:Content-Transfer-Encoding:Feedback-ID; bh=PwKGFIS+gXQI7q1Yr+8XcTpTpzivkbY/a4eJr1M1gvI=; b=uLORkeykjN1rrCrfkafkYa7WOiRrqiL4xgew7oGeBM8sWJezle07gIzkUM/Jzs/m dA2nKWk80oBod6QseiaNWoTkEr2OpPsm4sKf5hxYqrtp1UeJQrsfynPqIGPSAujOPWE WvZg5XNnn0mbCikP77wsfITB6Jbc1w0p59KJ0024= To: freebsd-net@freebsd.org From: Colin Percival Subject: sosend returning ERESTART Message-ID: <01000159aac969e6-b2fc3913-d04e-42d4-befd-402ed0d830bf-000000@email.amazonses.com> Date: Tue, 17 Jan 2017 04:57:23 +0000 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.5.1 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-SES-Outgoing: 2017.01.17-54.240.8.13 Feedback-ID: 1.us-east-1.Lv9FVjaNvvR5llaqfLoOVbo2VxOELl7cjN0AOyXnPlk=:AmazonSES X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Jan 2017 05:03:08 -0000 I think I've tracked an NFS problem down to sosend returning ERESTART; it looks like it's easy to work around this, but I'm not sure *why* sosend is returning ERESTART... or for that matter *how* since I can't find anywhere in relevant code where that gets returned. Am I missing something obvious? In case it matters, this is NFS making use of RPC code, sosending over a TCP connection which goes out over an ixv interface. -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid From owner-freebsd-net@freebsd.org Tue Jan 17 10:06:44 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2C056CB2366 for ; Tue, 17 Jan 2017 10:06:44 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B94F81BA4 for ; Tue, 17 Jan 2017 10:06:43 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id v0HA6YZZ055708 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 17 Jan 2017 12:06:34 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua v0HA6YZZ055708 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id v0HA6YYo055707; Tue, 17 Jan 2017 12:06:34 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Tue, 17 Jan 2017 12:06:34 +0200 From: Konstantin Belousov To: Colin Percival Cc: freebsd-net@freebsd.org Subject: Re: sosend returning ERESTART Message-ID: <20170117100634.GS2349@kib.kiev.ua> References: <01000159aac969e6-b2fc3913-d04e-42d4-befd-402ed0d830bf-000000@email.amazonses.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <01000159aac969e6-b2fc3913-d04e-42d4-befd-402ed0d830bf-000000@email.amazonses.com> User-Agent: Mutt/1.7.2 (2016-11-26) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Jan 2017 10:06:44 -0000 On Tue, Jan 17, 2017 at 04:57:23AM +0000, Colin Percival wrote: > I think I've tracked an NFS problem down to sosend returning ERESTART; it > looks like it's easy to work around this, but I'm not sure *why* sosend is > returning ERESTART... or for that matter *how* since I can't find anywhere > in relevant code where that gets returned. ERESTART is most likely returned by msleep(9) or similar call down the path when unblocked signal is pending with the restart disposition. Note that any attempt to perform an interruptible sleep would return ERESTART until the signal is either blocked or a trip to userspace is performed to deliver it. > > Am I missing something obvious? In case it matters, this is NFS making use > of RPC code, sosending over a TCP connection which goes out over an ixv > interface. From owner-freebsd-net@freebsd.org Tue Jan 17 19:34:35 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A93EBCB45FF for ; Tue, 17 Jan 2017 19:34:35 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 972D412A9 for ; Tue, 17 Jan 2017 19:34:35 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v0HJYZPp079627 for ; Tue, 17 Jan 2017 19:34:35 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 213869] when setting an ipsec policy with spdadd src[port], outbound traffic from 2049/tcp is not encrypted Date: Tue, 17 Jan 2017 19:34:35 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: 11.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: jasonmader@gmail.com X-Bugzilla-Status: Closed X-Bugzilla-Resolution: FIXED X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Jan 2017 19:34:35 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D213869 --- Comment #12 from Jason Mader --- There might still be a bug even after the patch. I'm using the patch on an NFSv4 server, and encountered a new NFSv4 mount that did not work. So I reverted to [any] on the source in the policy and got the mount to succeed. 19:23:21.702627 IP6 coconut > safe: ESP(spi=3D0xb04a3109,seq=3D0x7), length= 76 19:23:21.702666 IP6 safe.nfsd > coconut.849: Flags [S.], seq 3009240135, ack 1156380226, win 65535, options [mss 1440,nop,wscale 9,sackOK,TS val 1829320= 15 ecr 615648], length 0 --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Tue Jan 17 20:38:30 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5C12ECB4215 for ; Tue, 17 Jan 2017 20:38:30 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 45C1E1D64 for ; Tue, 17 Jan 2017 20:38:30 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v0HKcUb9029708 for ; Tue, 17 Jan 2017 20:38:30 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 213869] when setting an ipsec policy with spdadd src[port], outbound traffic from 2049/tcp is not encrypted Date: Tue, 17 Jan 2017 20:38:30 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: 11.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: ae@FreeBSD.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: FIXED X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Jan 2017 20:38:30 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D213869 --- Comment #13 from Andrey V. Elsukov --- (In reply to Jason Mader from comment #12) > There might still be a bug even after the patch. I'm using the patch on an > NFSv4 server, and encountered a new NFSv4 mount that did not work. So I > reverted to [any] on the source in the policy and got the mount to succee= d. >=20 > 19:23:21.702627 IP6 coconut > safe: ESP(spi=3D0xb04a3109,seq=3D0x7), leng= th 76 > 19:23:21.702666 IP6 safe.nfsd > coconut.849: Flags [S.], seq 3009240135, = ack > 1156380226, win 65535, options [mss 1440,nop,wscale 9,sackOK,TS val > 182932015 ecr 615648], length 0 At least I need the policies list at time when it doesn't work. Do you use = some IKEd? It would be good if you try to reproduce this problem with projects/ipsec. = You can apply the patch to stable/11 with 'svn patch': https://lists.freebsd.org/pipermail/freebsd-net/2017-January/046888.html --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Wed Jan 18 04:37:43 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 13128CB543A for ; Wed, 18 Jan 2017 04:37:43 +0000 (UTC) (envelope-from 01000159afddb4bb-d78bc275-7ce9-41da-93bd-0a7445a74fa0-000000@amazonses.com) Received: from a8-176.smtp-out.amazonses.com (a8-176.smtp-out.amazonses.com [54.240.8.176]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CDFE2106D for ; Wed, 18 Jan 2017 04:37:42 +0000 (UTC) (envelope-from 01000159afddb4bb-d78bc275-7ce9-41da-93bd-0a7445a74fa0-000000@amazonses.com) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=vnqrkfnvu6csdl6mwgk5t6ix3nnepx57; d=tarsnap.com; t=1484714259; h=Subject:To:References:Cc:From:Message-ID:Date:MIME-Version:In-Reply-To:Content-Type:Content-Transfer-Encoding; bh=s9qVqxHiSOPQtViKAeAAyS3J6vqATnmbVkY2D93jRk8=; b=oYXklsHUOWTbJxn+XwWMwfBjDSy1cDe+xXPVvu95/laxf7vfm5Z438J+1Np5BaaX +Hgv/ZRvgQqUQGpCTaWTxbVuggq1ccY3sSJHW0Wyn/Pw4aiQfTGBOER/dWcpg7kH9i+ VonbOsVB4D9X3IemSxZonQzqC+nmn878VYxNzSMI= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=6gbrjpgwjskckoa6a5zn6fwqkn67xbtw; d=amazonses.com; t=1484714259; h=Subject:To:References:Cc:From:Message-ID:Date:MIME-Version:In-Reply-To:Content-Type:Content-Transfer-Encoding:Feedback-ID; bh=s9qVqxHiSOPQtViKAeAAyS3J6vqATnmbVkY2D93jRk8=; b=0GPEbKsv6YN1Xnca4xw1tzKP79CHiTdL1r+B54p3MER38W7o7c4o8KimES2cLjng tWC8oAk7u4i47GoUbVWJePd2fskiStGXPGKCSaBExc5qJSGVOfM4DkZHTrQy6ifrVwF pgXWfTQn3IEhZhbaqCoXEiCSiOfdf8Z7P5gPYsa4= Subject: Re: sosend returning ERESTART To: Konstantin Belousov References: <01000159aac969e6-b2fc3913-d04e-42d4-befd-402ed0d830bf-000000@email.amazonses.com> <20170117100634.GS2349@kib.kiev.ua> Cc: freebsd-net@freebsd.org From: Colin Percival Message-ID: <01000159afddb4bb-d78bc275-7ce9-41da-93bd-0a7445a74fa0-000000@email.amazonses.com> Date: Wed, 18 Jan 2017 04:37:39 +0000 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.5.1 MIME-Version: 1.0 In-Reply-To: <20170117100634.GS2349@kib.kiev.ua> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-SES-Outgoing: 2017.01.18-54.240.8.176 Feedback-ID: 1.us-east-1.Lv9FVjaNvvR5llaqfLoOVbo2VxOELl7cjN0AOyXnPlk=:AmazonSES X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jan 2017 04:37:43 -0000 On 01/17/17 02:06, Konstantin Belousov wrote: > On Tue, Jan 17, 2017 at 04:57:23AM +0000, Colin Percival wrote: >> I think I've tracked an NFS problem down to sosend returning ERESTART; it >> looks like it's easy to work around this, but I'm not sure *why* sosend is >> returning ERESTART... or for that matter *how* since I can't find anywhere >> in relevant code where that gets returned. > ERESTART is most likely returned by msleep(9) or similar call down the > path when unblocked signal is pending with the restart disposition. Thanks, looks like that was exactly it -- if the TCP send buffer was full we would call sbwait, and if a signal arrived it would return ERESTART. It looks like setting the SB_NOINTR flag will prevent this; I'm testing a patch right now. (Google bait in case anyone else trips over this: FreeBSD 11.0 NFS client dropping TCP connections under concurrent I/O load.) -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid From owner-freebsd-net@freebsd.org Wed Jan 18 07:02:46 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E7A60CB4A64 for ; Wed, 18 Jan 2017 07:02:46 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from reviews.nyi.freebsd.org (reviews.nyi.freebsd.org [IPv6:2610:1c1:1:607c::16:b]) by mx1.freebsd.org (Postfix) with ESMTP id BAC7A1DCF for ; Wed, 18 Jan 2017 07:02:46 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by reviews.nyi.freebsd.org (Postfix, from userid 1346) id 291A636BE2; Wed, 18 Jan 2017 07:02:46 +0000 (UTC) Date: Wed, 18 Jan 2017 07:02:46 +0000 To: freebsd-net@freebsd.org From: "sepherosa_gmail.com (Sepherosa Ziehau)" Reply-to: D8963+325+d689c69326b9ae09@reviews.freebsd.org Subject: [Differential] D8963: ifnet: introduce event handlers for ifup/ifdown events Message-ID: X-Priority: 3 X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , Thread-Topic: D8963: ifnet: introduce event handlers for ifup/ifdown events X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: X-Phabricator-Cc: Precedence: bulk In-Reply-To: References: Thread-Index: Y2E4NDdjNWVjOGM5NzQ0Mjk1ZTMzMGM0ZDY2IFh/ExY= MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jan 2017 07:02:47 -0000 c2VwaGVyb3NhX2dtYWlsLmNvbSBhZGRlZCBhIGNvbW1lbnQuCgoKICBJZiBubyBvYmplY3Rpb24g Y29tZXMsIHdlIHBsYW4gdG8gY29tbWl0IHRoaXMgZWFybGllciBuZXh0IHdlZWsuCgpSRVZJU0lP TiBERVRBSUwKICBodHRwczovL3Jldmlld3MuZnJlZWJzZC5vcmcvRDg5NjMKCkVNQUlMIFBSRUZF UkVOQ0VTCiAgaHR0cHM6Ly9yZXZpZXdzLmZyZWVic2Qub3JnL3NldHRpbmdzL3BhbmVsL2VtYWls cHJlZmVyZW5jZXMvCgpUbzogZGVjdWlfbWljcm9zb2Z0LmNvbSwgaHNlbGFza3ksIGNlbSwgbnAs IGttYWN5LCBraWIsIGhvbnpoYW5fbWljcm9zb2Z0LmNvbSwgaG93YXJkMHN1X2dtYWlsLmNvbSwg amhiLCBhZSwgZGVscGhpaiwgcm95Z2VyLCBnbGViaXVzLCBnbm4sIHJ3YXRzb24sIHNlcGhlcm9z YV9nbWFpbC5jb20KQ2M6IGdhcmdhLCBmcmVlYnNkLW5ldC1saXN0Cg== From owner-freebsd-net@freebsd.org Wed Jan 18 09:48:26 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1B9F4CB4492; Wed, 18 Jan 2017 09:48:26 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from elf.hq.norma.perm.ru (mail.norma.perm.ru [IPv6:2a00:7540:1::5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.norma.perm.ru", Issuer "Vivat-Trade UNIX Root CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 6AF181C83; Wed, 18 Jan 2017 09:48:24 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from bsdrookie.norma.com. ([IPv6:fd00::7af]) by elf.hq.norma.perm.ru (8.15.2/8.15.2) with ESMTPS id v0I9mKKS035238 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Wed, 18 Jan 2017 14:48:20 +0500 (YEKT) (envelope-from emz@norma.perm.ru) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=norma.perm.ru; s=key; t=1484732900; bh=VNCZm8mpxxUO8e/n8/e8MCAokyjSuC05SxtB9+PPF7o=; h=To:From:Subject:Date; b=cdaYlZsapSJ5URuQSkwWeyu2Ih7hSraiaFthPCby4dD5sLzp2w/r8EVsqJ3clIlWV wGt+7cFzHk36I8QKudvXpyEnxy4GwhVYgAV2t+8fmTWoqS0Qc2ZGha3gbCgng3RgDL nh90FHLVrlF4VyUfRlM269LR+pMBX6dnGBQ5jgcM= To: freebsd-net@FreeBSD.org, freebsd-stable From: "Eugene M. Zheganin" Subject: decent 40G network adapters Message-ID: <587F39E3.1060608@norma.perm.ru> Date: Wed, 18 Jan 2017 14:48:19 +0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.7.0 MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jan 2017 09:48:26 -0000 Hi. Could someone recommend a decent 40Gbit adapter that are proven to be working under FreeBSD ? The intended purpose - iSCSI traffic, not much pps, but rates definitely above 10G. I've tried Supermicro-manufactured Intel XL710 ones (two boards, different servers - same sad story: packets loss, server unresponsive, spikes), seems like they have a problem in a driver (or firmware), and though Intel support states this is because the Supermicro tampered with the adapter, I'm still suspicious about ixl(4). I've also seen in the ML a guy reported the exact same problem with ixl(4) as I have found. So, what would you say ? Chelsio ? Thanks. Eugene. From owner-freebsd-net@freebsd.org Wed Jan 18 09:52:12 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5FCE1CB47BA; Wed, 18 Jan 2017 09:52:12 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (turbocat.net [88.99.82.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2B8E310FF; Wed, 18 Jan 2017 09:52:11 +0000 (UTC) (envelope-from hps@selasky.org) Received: from hps2016.home.selasky.org (unknown [62.141.129.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 660791FE025; Wed, 18 Jan 2017 10:51:52 +0100 (CET) Subject: Re: decent 40G network adapters To: "Eugene M. Zheganin" , freebsd-net@FreeBSD.org, freebsd-stable , Meny Yossefi References: <587F39E3.1060608@norma.perm.ru> From: Hans Petter Selasky Message-ID: <59b43877-5194-20bc-27b5-486f7f449cf7@selasky.org> Date: Wed, 18 Jan 2017 10:51:36 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <587F39E3.1060608@norma.perm.ru> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jan 2017 09:52:12 -0000 On 01/18/17 10:48, Eugene M. Zheganin wrote: > Hi. > > Could someone recommend a decent 40Gbit adapter that are proven to be > working under FreeBSD ? The intended purpose - iSCSI traffic, not much > pps, but rates definitely above 10G. I've tried Supermicro-manufactured > Intel XL710 ones (two boards, different servers - same sad story: > packets loss, server unresponsive, spikes), seems like they have a > problem in a driver (or firmware), and though Intel support states this > is because the Supermicro tampered with the adapter, I'm still > suspicious about ixl(4). I've also seen in the ML a guy reported the > exact same problem with ixl(4) as I have found. > > So, what would you say ? Chelsio ? > Hi, I think also the Mellanox, mlx4 and mlx5 drivers will support this. Are you using infiniband or TCP for backend? --HPS From owner-freebsd-net@freebsd.org Wed Jan 18 10:02:51 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 133F5CB51AF for ; Wed, 18 Jan 2017 10:02:51 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from elf.hq.norma.perm.ru (mail.norma.perm.ru [IPv6:2a00:7540:1::5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.norma.perm.ru", Issuer "Vivat-Trade UNIX Root CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 6993C1C09 for ; Wed, 18 Jan 2017 10:02:50 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from bsdrookie.norma.com. ([IPv6:fd00::7af]) by elf.hq.norma.perm.ru (8.15.2/8.15.2) with ESMTPS id v0IA2khw036056 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Wed, 18 Jan 2017 15:02:46 +0500 (YEKT) (envelope-from emz@norma.perm.ru) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=norma.perm.ru; s=key; t=1484733767; bh=kLcWsGfpnlCuOazZuIrobclgW1CwNV9x24tZwvMJHo0=; h=Subject:To:References:From:Date:In-Reply-To; b=fze+WMA2/hF/OhUCwoMePeCkP2EbLTKpOM57z4hZkWJp+P+tqthQWdAO9/YJI1kOT Pmb4aoLqKPzMNYyD1kv3RY6NV0XZD93TUrbbtg8Hu4E9qx6fPbJHxe2ZVLn0P4XP1Q FyNRF+stVWvwOswvQq1w7dh6Qdn3HeI/7UpT8IyY= Subject: Re: decent 40G network adapters To: freebsd-net@FreeBSD.org, Meny Yossefi References: <587F39E3.1060608@norma.perm.ru> <59b43877-5194-20bc-27b5-486f7f449cf7@selasky.org> From: "Eugene M. Zheganin" Message-ID: <587F3D46.1010702@norma.perm.ru> Date: Wed, 18 Jan 2017 15:02:46 +0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.7.0 MIME-Version: 1.0 In-Reply-To: <59b43877-5194-20bc-27b5-486f7f449cf7@selasky.org> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jan 2017 10:02:51 -0000 Hi. On 18.01.2017 14:51, Hans Petter Selasky wrote: > On 01/18/17 10:48, Eugene M. Zheganin wrote: >> Hi. >> >> Could someone recommend a decent 40Gbit adapter that are proven to be >> working under FreeBSD ? The intended purpose - iSCSI traffic, not much >> pps, but rates definitely above 10G. I've tried Supermicro-manufactured >> Intel XL710 ones (two boards, different servers - same sad story: >> packets loss, server unresponsive, spikes), seems like they have a >> problem in a driver (or firmware), and though Intel support states this >> is because the Supermicro tampered with the adapter, I'm still >> suspicious about ixl(4). I've also seen in the ML a guy reported the >> exact same problem with ixl(4) as I have found. >> >> So, what would you say ? Chelsio ? >> > > Hi, > > I think also the Mellanox, mlx4 and mlx5 drivers will support this. > Are you using infiniband or TCP for backend? > > Thanks. I'm using TCP. Eugene. From owner-freebsd-net@freebsd.org Wed Jan 18 10:03:56 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8CEE6CB5257; Wed, 18 Jan 2017 10:03:56 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4EA991CE1; Wed, 18 Jan 2017 10:03:56 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1cTn5k-000Au2-Tc; Wed, 18 Jan 2017 13:03:44 +0300 Date: Wed, 18 Jan 2017 13:03:44 +0300 From: Slawa Olhovchenkov To: "Eugene M. Zheganin" Cc: freebsd-net@FreeBSD.org, freebsd-stable Subject: Re: decent 40G network adapters Message-ID: <20170118100344.GH78888@zxy.spb.ru> References: <587F39E3.1060608@norma.perm.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <587F39E3.1060608@norma.perm.ru> User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jan 2017 10:03:56 -0000 On Wed, Jan 18, 2017 at 02:48:19PM +0500, Eugene M. Zheganin wrote: > Hi. > > Could someone recommend a decent 40Gbit adapter that are proven to be > working under FreeBSD ? The intended purpose - iSCSI traffic, not much > pps, but rates definitely above 10G. I've tried Supermicro-manufactured > Intel XL710 ones (two boards, different servers - same sad story: > packets loss, server unresponsive, spikes), seems like they have a > problem in a driver (or firmware), and though Intel support states this > is because the Supermicro tampered with the adapter, I'm still > suspicious about ixl(4). I've also seen in the ML a guy reported the > exact same problem with ixl(4) as I have found. > > So, what would you say ? Chelsio ? I am use Chelsio and Solarflare. Not sure about you workload -- I am have 40K+ TCP connections, you workload need different tuning. Do you planed to utilise both ports? For this case you need PCIe 16x card. This is Chelsio T6 and Solarflare 9200. From owner-freebsd-net@freebsd.org Wed Jan 18 10:11:56 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BB280CB5519; Wed, 18 Jan 2017 10:11:56 +0000 (UTC) (envelope-from arybchik@freebsd.org) Received: from shelob.oktetlabs.ru (shelob.oktetlabs.ru [84.52.89.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7A7FF1189; Wed, 18 Jan 2017 10:11:56 +0000 (UTC) (envelope-from arybchik@freebsd.org) Received: from [192.168.38.17] (aros.oktetlabs.ru [192.168.38.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by shelob.oktetlabs.ru (Postfix) with ESMTPSA id 495757F66A; Wed, 18 Jan 2017 13:03:06 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.9.2 shelob.oktetlabs.ru 495757F66A Authentication-Results: shelob.oktetlabs.ru/495757F66A; dkim=none reason="no signature"; dkim-adsp=none (unprotected policy); dkim-atps=neutral Subject: Re: decent 40G network adapters To: "Eugene M. Zheganin" , freebsd-net@FreeBSD.org, freebsd-stable References: <587F39E3.1060608@norma.perm.ru> From: Andrew Rybchenko Message-ID: <713809d5-a94b-f8e5-6ad2-972f4f704746@freebsd.org> Date: Wed, 18 Jan 2017 13:03:06 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.6.0 MIME-Version: 1.0 In-Reply-To: <587F39E3.1060608@norma.perm.ru> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jan 2017 10:11:56 -0000 On 01/18/2017 12:48 PM, Eugene M. Zheganin wrote: > Hi. > > Could someone recommend a decent 40Gbit adapter that are proven to be > working under FreeBSD ? The intended purpose - iSCSI traffic, not much > pps, but rates definitely above 10G. I've tried Supermicro-manufactured > Intel XL710 ones (two boards, different servers - same sad story: > packets loss, server unresponsive, spikes), seems like they have a > problem in a driver (or firmware), and though Intel support states this > is because the Supermicro tampered with the adapter, I'm still > suspicious about ixl(4). I've also seen in the ML a guy reported the > exact same problem with ixl(4) as I have found. > > So, what would you say ? Chelsio ? Hi, Solarflare SFN8542 with sfxge driver will do the job Andrew. > Thanks. > Eugene. > > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@freebsd.org Wed Jan 18 10:14:00 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B1AFDCB5793; Wed, 18 Jan 2017 10:14:00 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from elf.hq.norma.perm.ru (mail.norma.perm.ru [IPv6:2a00:7540:1::5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.norma.perm.ru", Issuer "Vivat-Trade UNIX Root CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 13ED31598; Wed, 18 Jan 2017 10:13:59 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from bsdrookie.norma.com. ([IPv6:fd00::7af]) by elf.hq.norma.perm.ru (8.15.2/8.15.2) with ESMTPS id v0IADuEQ036613 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Wed, 18 Jan 2017 15:13:56 +0500 (YEKT) (envelope-from emz@norma.perm.ru) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=norma.perm.ru; s=key; t=1484734437; bh=GEI6lwzIzDtoMJKQzTvxs5jMZtSnZFYdAunygVeq2v4=; h=Subject:References:Cc:From:To:Date:In-Reply-To; b=L0cCOgnxU3EenGuxckDHS4UcqEy5/qEedkPpRXyPQUzEhgKXnrrMtRU0g8e6txSaC W26YImQ83kL28NKwGunYiKDDqSiflWwd+dWLfT+swK2MQfVp6ZOV5itDcbN+A/ES+9 6W6aVgK7U33qSSi2TnDedenet5S8z/qXuFPXrFU8= Subject: Re: decent 40G network adapters References: <587F39E3.1060608@norma.perm.ru> <20170118100344.GH78888@zxy.spb.ru> Cc: freebsd-stable From: "Eugene M. Zheganin" To: freebsd-net@FreeBSD.org Message-ID: <587F3FE4.4040902@norma.perm.ru> Date: Wed, 18 Jan 2017 15:13:56 +0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.7.0 MIME-Version: 1.0 In-Reply-To: <20170118100344.GH78888@zxy.spb.ru> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jan 2017 10:14:00 -0000 Hi. On 18.01.2017 15:03, Slawa Olhovchenkov wrote: > I am use Chelsio and Solarflare. > Not sure about you workload -- I am have 40K+ TCP connections, you > workload need different tuning. > Do you planed to utilise both ports? > For this case you need PCIe 16x card. This is Chelsio T6 and > Solarflare 9200.=20 Thanks. No, the number of connections in my case will be small - hundreds, and right now target servers are utilizing 4-5 Gbit/sec bandwidth, so I'm looking forward to something more performing, thats all. The pps number is also way below Mpps - at this time it's about 200 kpps, so I really hope I won't be facing a situation with millions of pps, - though seems like it will be slightly above 1 Mpps. Hopefully with the help of the community I'll be able to tune the servers to handle this ! :) Thanks. Eugene. From owner-freebsd-net@freebsd.org Wed Jan 18 10:36:57 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A78BECB5395 for ; Wed, 18 Jan 2017 10:36:57 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 25E4E14E5 for ; Wed, 18 Jan 2017 10:36:56 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id v0IAaoav040302 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Wed, 18 Jan 2017 12:36:51 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua v0IAaoav040302 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id v0IAaotf040301; Wed, 18 Jan 2017 12:36:50 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 18 Jan 2017 12:36:50 +0200 From: Konstantin Belousov To: Colin Percival Cc: freebsd-net@freebsd.org Subject: Re: sosend returning ERESTART Message-ID: <20170118103650.GE2349@kib.kiev.ua> References: <01000159aac969e6-b2fc3913-d04e-42d4-befd-402ed0d830bf-000000@email.amazonses.com> <20170117100634.GS2349@kib.kiev.ua> <01000159afddb7ce-064a5d17-4b81-4b2c-a9b4-3ddd2ad2e377-000000@email.amazonses.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <01000159afddb7ce-064a5d17-4b81-4b2c-a9b4-3ddd2ad2e377-000000@email.amazonses.com> User-Agent: Mutt/1.7.2 (2016-11-26) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jan 2017 10:36:57 -0000 On Wed, Jan 18, 2017 at 04:37:40AM +0000, Colin Percival wrote: > On 01/17/17 02:06, Konstantin Belousov wrote: > > On Tue, Jan 17, 2017 at 04:57:23AM +0000, Colin Percival wrote: > >> I think I've tracked an NFS problem down to sosend returning ERESTART; it > >> looks like it's easy to work around this, but I'm not sure *why* sosend is > >> returning ERESTART... or for that matter *how* since I can't find anywhere > >> in relevant code where that gets returned. > > ERESTART is most likely returned by msleep(9) or similar call down the > > path when unblocked signal is pending with the restart disposition. > > Thanks, looks like that was exactly it -- if the TCP send buffer was full > we would call sbwait, and if a signal arrived it would return ERESTART. > It looks like setting the SB_NOINTR flag will prevent this; I'm testing a > patch right now. > > (Google bait in case anyone else trips over this: FreeBSD 11.0 NFS client > dropping TCP connections under concurrent I/O load.) Note that passing SB_NOINTR unconditionally or even only for mounts with nointr (default) option is wrong. You make the socket operation uninterruptible, process terminate-ability becomes depended on the external factor, the behaviour of the remote system. From owner-freebsd-net@freebsd.org Wed Jan 18 18:43:33 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 37071CB5DE5 for ; Wed, 18 Jan 2017 18:43:33 +0000 (UTC) (envelope-from stl@wiredrive.com) Received: from mail-wm0-x246.google.com (mail-wm0-x246.google.com [IPv6:2a00:1450:400c:c09::246]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CF6781F21 for ; Wed, 18 Jan 2017 18:43:32 +0000 (UTC) (envelope-from stl@wiredrive.com) Received: by mail-wm0-x246.google.com with SMTP id r126so4740895wmr.2 for ; Wed, 18 Jan 2017 10:43:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wiredrive-com.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=N55jkulxWrT5X1A+hzQvQ5/0VPrZMVloITR2Pyr5AW4=; b=ktDou/aQul6hVClyb7PcVGTkNm0Rjc5G6nd3NyRfZhjs1qVQqwb+CoDDIHrtNCGbzK GBH82N7NccKOBLKSyqA20k3bR/0VNkmqfKGp/CTuPg51OlWKX8ACiT/F4VV/HHdPaDDx LH8B1CG3IDUhe06A1DX1LCLul9zzbUhK6MncZZvC0CLpWRgIyxY0d7cO/HrZR+hvZvxt f3ZY/58RzqwHqHMvaxdiqeD8V+qWJ64NbSg/tZJUyZFO6NiT0NVrfza7lDniKmLrlKAW ilDLPCXtH+qrSWR/ZNJJ/svfsbYMjUTGjtqO0aT8UpOriRccOsM9EoamDNntFr+p9hdn 1zZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=N55jkulxWrT5X1A+hzQvQ5/0VPrZMVloITR2Pyr5AW4=; b=F3orHnc3QnWbW+KTKRpiOHse7RF0MdG/jGm5i70YrvKYAUC2lySbVGnqBtTqRros7U abGjI3r0levfcPhrgA0g3BomeGgrsyuRBn8Q5rpv26xqzoEsQRb7rsW5f5nw84OzBxsi 8OeuUWkuF3ugvw8welCdckdlXLYgwMffrqVv+/guRfpPHhNwa7dS3Nxg2maZHYDKj6OO FyVwVqFvTT63VBbBkOSxfyz9uJ7hLL4GtA3SxBiGJFF7We6C5CKjoXPdjMukp4fuX+vc pcv4nLrcfePOE5ymwdOwElzBboPxBmHeIw92W/822itIBkYp3XJoEa0uuqJ0hE8pZf5E GWCA== X-Gm-Message-State: AIkVDXIrmnSvu3xxLyAYZx/17PbHzZfMDhtaiIsaZBx60cydsiVMUd1A0y2QuWUYci39lNN2X5A2tBptMPkrvAaA X-Received: by 10.223.135.69 with SMTP id 5mr4543803wrz.174.1484765011093; Wed, 18 Jan 2017 10:43:31 -0800 (PST) MIME-Version: 1.0 Received: by 10.194.76.242 with HTTP; Wed, 18 Jan 2017 10:43:30 -0800 (PST) In-Reply-To: <587F39E3.1060608@norma.perm.ru> References: <587F39E3.1060608@norma.perm.ru> From: Scott Larson Date: Wed, 18 Jan 2017 10:43:30 -0800 Message-ID: Subject: Re: decent 40G network adapters To: "Eugene M. Zheganin" Cc: freebsd-net@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jan 2017 18:43:33 -0000 I can vouch for the Mellanox ConnectX-4 cards as working well, along with the Chelsio T5 stuff, both under heavy production usage in both routing and endpoint cases. In my testing the T5 was a slightly better option in situations demanding PPS with a higher ceiling and lower cpu utilization, but they came out essentially in a dead heat when testing for raw throughput. *[image: userimage]Scott Larson[image: los angeles] Lead Systems Administrator[image: wdlogo] [image: linkedin] [image: facebook] [image: twitter] [image: instagram] T 310 823 8238 <310%20823%208238%20x1106> | M 310 904 8818 <310%20904%208818>* On Wed, Jan 18, 2017 at 1:48 AM, Eugene M. Zheganin wrote: > Hi. > > Could someone recommend a decent 40Gbit adapter that are proven to be > working under FreeBSD ? The intended purpose - iSCSI traffic, not much > pps, but rates definitely above 10G. I've tried Supermicro-manufactured > Intel XL710 ones (two boards, different servers - same sad story: > packets loss, server unresponsive, spikes), seems like they have a > problem in a driver (or firmware), and though Intel support states this > is because the Supermicro tampered with the adapter, I'm still > suspicious about ixl(4). I've also seen in the ML a guy reported the > exact same problem with ixl(4) as I have found. > > So, what would you say ? Chelsio ? > > Thanks. > Eugene. > > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@freebsd.org Wed Jan 18 22:01:35 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 38072CB6027 for ; Wed, 18 Jan 2017 22:01:35 +0000 (UTC) (envelope-from 01000159b390c409-5adcb488-67e8-4038-b9b0-5d4f33460205-000000@amazonses.com) Received: from a8-13.smtp-out.amazonses.com (a8-13.smtp-out.amazonses.com [54.240.8.13]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id F1FE11441 for ; Wed, 18 Jan 2017 22:01:33 +0000 (UTC) (envelope-from 01000159b390c409-5adcb488-67e8-4038-b9b0-5d4f33460205-000000@amazonses.com) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=vnqrkfnvu6csdl6mwgk5t6ix3nnepx57; d=tarsnap.com; t=1484776326; h=Subject:To:References:Cc:From:Message-ID:Date:MIME-Version:In-Reply-To:Content-Type:Content-Transfer-Encoding; bh=vtJqVKg3jNMpvgvPT/3oZ82ggVQtkN5NgWFVkj2GAQA=; b=P8if18ktKI7XHRXiw+yX5FGQfxw/uclVsYvdRobDBg0tJKr3giYL7kZjohGPfKl1 PTOirY9X2fsUs6xdch+XAws9x2CaWaJnGLPMWER5HKF7vb5HzVfy95rg6W3K5upasv7 oOBwf3oNDnfk2Kty0GjkY8VX6BGk6yJ9eJ0AUKPQ= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=6gbrjpgwjskckoa6a5zn6fwqkn67xbtw; d=amazonses.com; t=1484776326; h=Subject:To:References:Cc:From:Message-ID:Date:MIME-Version:In-Reply-To:Content-Type:Content-Transfer-Encoding:Feedback-ID; bh=vtJqVKg3jNMpvgvPT/3oZ82ggVQtkN5NgWFVkj2GAQA=; b=NK1bYmEmJoXp+CUWi5AkUg/HSC/N78xqfNLWxV+9tsfaSEOv1rVo0G+bnJuqmZq/ mUSUto2c5j7Vuydy3IUYHGgzzVWnaspbJLqxcjaFcAapEN8oK/RgpoIkHl+ArDGY8sS E2iULdPyq8zoEsSpAK341FiKStnnn8iZ1DTIpWIE= Subject: Re: sosend returning ERESTART To: Konstantin Belousov References: <01000159aac969e6-b2fc3913-d04e-42d4-befd-402ed0d830bf-000000@email.amazonses.com> <20170117100634.GS2349@kib.kiev.ua> <01000159afddb7ce-064a5d17-4b81-4b2c-a9b4-3ddd2ad2e377-000000@email.amazonses.com> <20170118103650.GE2349@kib.kiev.ua> Cc: freebsd-net@freebsd.org From: Colin Percival Message-ID: <01000159b390c409-5adcb488-67e8-4038-b9b0-5d4f33460205-000000@email.amazonses.com> Date: Wed, 18 Jan 2017 21:52:06 +0000 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.5.1 MIME-Version: 1.0 In-Reply-To: <20170118103650.GE2349@kib.kiev.ua> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-SES-Outgoing: 2017.01.18-54.240.8.13 Feedback-ID: 1.us-east-1.Lv9FVjaNvvR5llaqfLoOVbo2VxOELl7cjN0AOyXnPlk=:AmazonSES X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jan 2017 22:01:35 -0000 On 01/18/17 02:36, Konstantin Belousov wrote: > On Wed, Jan 18, 2017 at 04:37:40AM +0000, Colin Percival wrote: >> Thanks, looks like that was exactly it -- if the TCP send buffer was full >> we would call sbwait, and if a signal arrived it would return ERESTART. >> It looks like setting the SB_NOINTR flag will prevent this; I'm testing a >> patch right now. > > Note that passing SB_NOINTR unconditionally or even only for mounts > with nointr (default) option is wrong. You make the socket operation > uninterruptible, process terminate-ability becomes depended on the > external factor, the behaviour of the remote system. I'm not sure what you're getting at here. The fact that "NFS mounted without the intr flag" + "unresponsive NFS server" = "unkillable processes" has been a (mis)feature of NFS for decades. -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid From owner-freebsd-net@freebsd.org Wed Jan 18 22:52:05 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 13206CB6500 for ; Wed, 18 Jan 2017 22:52:05 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from NAM03-BY2-obe.outbound.protection.outlook.com (mail-by2nam03on0048.outbound.protection.outlook.com [104.47.42.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT SSL SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 810D81501 for ; Wed, 18 Jan 2017 22:52:03 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from YTXPR01MB0189.CANPRD01.PROD.OUTLOOK.COM (10.165.218.133) by YTXPR01MB0191.CANPRD01.PROD.OUTLOOK.COM (10.165.218.135) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.845.12; Wed, 18 Jan 2017 22:52:02 +0000 Received: from YTXPR01MB0189.CANPRD01.PROD.OUTLOOK.COM ([10.165.218.133]) by YTXPR01MB0189.CANPRD01.PROD.OUTLOOK.COM ([10.165.218.133]) with mapi id 15.01.0845.014; Wed, 18 Jan 2017 22:52:02 +0000 From: Rick Macklem To: Colin Percival , Konstantin Belousov CC: "freebsd-net@freebsd.org" Subject: Re: sosend returning ERESTART Thread-Topic: sosend returning ERESTART Thread-Index: AQHScXbGB6xwqeuR3ky6twdHVMq016E+x0oAgAAO/x0= Date: Wed, 18 Jan 2017 22:52:02 +0000 Message-ID: References: <01000159aac969e6-b2fc3913-d04e-42d4-befd-402ed0d830bf-000000@email.amazonses.com> <20170117100634.GS2349@kib.kiev.ua> <01000159afddb7ce-064a5d17-4b81-4b2c-a9b4-3ddd2ad2e377-000000@email.amazonses.com> <20170118103650.GE2349@kib.kiev.ua>, <01000159b390c409-5adcb488-67e8-4038-b9b0-5d4f33460205-000000@email.amazonses.com> In-Reply-To: <01000159b390c409-5adcb488-67e8-4038-b9b0-5d4f33460205-000000@email.amazonses.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=rmacklem@uoguelph.ca; x-ms-office365-filtering-correlation-id: 24ab3f7a-da3b-49f0-576c-08d43ff49e65 x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:(22001);SRVR:YTXPR01MB0191; x-microsoft-exchange-diagnostics: 1; YTXPR01MB0191; 7:tjvcnfLnOLwHfHmewaD7MpVcJH9l5lNzyqwGz0hHjUcKV4UmCJPMoLhj9ljJL5QGOvU+I3FyPgNLOvp8N2xIg/pdwE9zzKgdDygHp5ZSsiBf7wMTkPnJJsOl4LqMog783bE1mZslvrgtOe2EjLtj6QUaZ52Vu7l3NEJDroBgFPiKXlbPhoY8UQAwrEw1fxvtw7lLZZjcuzCYe9MWJCJh7F1QF9QCFpMgs1VmJAttLGzobhB2W8mkpVzSLCoWbwygBf/2xR127Zf2HwpeAz3rpk2HXPHvyYf2UErnzoU4zZZ3MdfpLCEKQvGNOTlqKPGN0F7/xXM1jvZGz88yufkyqApXz47yDFwXlIDVhC8TmMvnX8CmtECFQjdh9FKPpVMW+CU5RkfhEJ9Qk8WYDufMs+Y8xWXkWXMDP08BHPhZthJ5QLXMUoAmQpFMXNfbB6p4FTeA2nxGO1Ya3j5nuvhGLA== x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(158342451672863); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040375)(2401047)(5005006)(8121501046)(3002001)(10201501046)(6041248)(20161123555025)(20161123560025)(20161123562025)(20161123564025)(6072148); SRVR:YTXPR01MB0191; BCL:0; PCL:0; RULEID:; SRVR:YTXPR01MB0191; x-forefront-prvs: 01917B1794 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(6009001)(7916002)(39450400003)(24454002)(199003)(189002)(229853002)(33656002)(5660300001)(102836003)(77096006)(305945005)(38730400001)(74316002)(6436002)(50986999)(8676002)(7696004)(39060400001)(106356001)(97736004)(68736007)(74482002)(189998001)(101416001)(81166006)(6506006)(5001770100001)(76176999)(7116003)(54356999)(2950100002)(106116001)(3480700004)(8936002)(81156014)(105586002)(93886004)(55016002)(4326007)(9686003)(92566002)(2906002)(3660700001)(53936002)(122556002)(3280700002)(86362001)(2900100001); DIR:OUT; SFP:1101; SCL:1; SRVR:YTXPR01MB0191; H:YTXPR01MB0189.CANPRD01.PROD.OUTLOOK.COM; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; received-spf: None (protection.outlook.com: uoguelph.ca does not designate permitted sender hosts) spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-originalarrivaltime: 18 Jan 2017 22:52:02.7293 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-Transport-CrossTenantHeadersStamped: YTXPR01MB0191 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jan 2017 22:52:05 -0000 Colin Percival wrote: >On 01/18/17 02:36, Konstantin Belousov wrote: >> On Wed, Jan 18, 2017 at 04:37:40AM +0000, Colin Percival wrote: >>> Thanks, looks like that was exactly it -- if the TCP send buffer was fu= ll >>> we would call sbwait, and if a signal arrived it would return ERESTART. >>> It looks like setting the SB_NOINTR flag will prevent this; I'm testing= a >>> patch right now. >> >> Note that passing SB_NOINTR unconditionally or even only for mounts >> with nointr (default) option is wrong. You make the socket operation >> uninterruptible, process terminate-ability becomes depended on the >> external factor, the behaviour of the remote system. > >I'm not sure what you're getting at here. The fact that "NFS mounted with= out >the intr flag" + "unresponsive NFS server" =3D "unkillable processes" has = been >a (mis)feature of NFS for decades. The case I would like to see work is the forced dismount. I need to go look= at what it does and see if SB_NOINTR would break it worse than it is broken no= w. (It is currently broken when something like "umount" without -f is done, wh= ich locks up the mounted on vnode so "umount -f" never gets to the umount(2) s= yscall. I do plan on a "straight ot NFS" option for umount(8) to avoid this proble= m, but haven't gotten around to it.) The alternative to SB_NOINTR is looping and doing the sosend() again for th= e case where it returns ERESTART and "intr" wasn't set on the mount. --> For this to be ok, we must be sure that when sosend() returns ERESTART, it has not queued the data for sending so it is safe to send it all a= gain. I think this is true for this case? rick From owner-freebsd-net@freebsd.org Thu Jan 19 01:03:36 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1C55FCB606D for ; Thu, 19 Jan 2017 01:03:36 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from NAM01-BN3-obe.outbound.protection.outlook.com (mail-bn3nam01on0044.outbound.protection.outlook.com [104.47.33.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT SSL SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BCBEB1683 for ; Thu, 19 Jan 2017 01:03:35 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from YTXPR01MB0189.CANPRD01.PROD.OUTLOOK.COM (10.165.218.133) by YTXPR01MB0192.CANPRD01.PROD.OUTLOOK.COM (10.165.218.136) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.845.12; Thu, 19 Jan 2017 01:03:32 +0000 Received: from YTXPR01MB0189.CANPRD01.PROD.OUTLOOK.COM ([10.165.218.133]) by YTXPR01MB0189.CANPRD01.PROD.OUTLOOK.COM ([10.165.218.133]) with mapi id 15.01.0845.014; Thu, 19 Jan 2017 01:03:32 +0000 From: Rick Macklem To: Colin Percival , Konstantin Belousov CC: "freebsd-net@freebsd.org" Subject: Re: sosend returning ERESTART Thread-Topic: sosend returning ERESTART Thread-Index: AQHScXbGB6xwqeuR3ky6twdHVMq016E+x0oAgAAzAvQ= Date: Thu, 19 Jan 2017 01:03:31 +0000 Message-ID: References: <01000159aac969e6-b2fc3913-d04e-42d4-befd-402ed0d830bf-000000@email.amazonses.com> <20170117100634.GS2349@kib.kiev.ua> <01000159afddb7ce-064a5d17-4b81-4b2c-a9b4-3ddd2ad2e377-000000@email.amazonses.com> <20170118103650.GE2349@kib.kiev.ua>, <01000159b390c409-5adcb488-67e8-4038-b9b0-5d4f33460205-000000@email.amazonses.com> In-Reply-To: <01000159b390c409-5adcb488-67e8-4038-b9b0-5d4f33460205-000000@email.amazonses.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=rmacklem@uoguelph.ca; x-ms-office365-filtering-correlation-id: 54083a3d-0ea0-47f0-cddf-08d44006fcc4 x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:(22001);SRVR:YTXPR01MB0192; x-microsoft-exchange-diagnostics: 1; YTXPR01MB0192; 7:iNAjUezZLooMuawfxHZ+CPq95Z5OPBQTG+6+7F4qg6TUvdhhsngrzEltA4eWSaAHDhmFeewlutG33yAE3wf2aDR3nFY5swl2ubCR0Gufn93RJJ2TP76bANdgSrsE58S6+8JufJ/ZllBa1JzOKF77IsW7e/7f2gO9a9NhngwD9z0a2EqkpeoaD1XdWVuH/YqqMRwiQHJtDib9V0+p8/sXNJhCbzRlDtw1Gx2JaOTm1ImeQIQj6r9NsCtDKkK5OlsKHYgD9H5aYDSfFF1akXdyLyjQqGkuL/FZEIvwonQ1HquecT/+dOW9xBDlA8O50BOrsrm/9n+5Pwrc6thJGjeudVadGTOitAmN1mMLyL3J5qYq/oZ8dwdG2e5iGUWKrelXkKhSG09Dn9L8+DGe11xZl2nCpQa15iksecBiyoaguin9vQYjr9JJqSn4N6zz+aSkK4EjIHpLeL+KFjX01Hs9Og== x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(158342451672863); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040375)(2401047)(5005006)(8121501046)(3002001)(10201501046)(6041248)(20161123562025)(20161123555025)(20161123564025)(20161123560025)(6072148); SRVR:YTXPR01MB0192; BCL:0; PCL:0; RULEID:; SRVR:YTXPR01MB0192; x-forefront-prvs: 0192E812EC x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(979002)(6009001)(7916002)(39450400003)(199003)(24454002)(189002)(33656002)(101416001)(81156014)(97736004)(93886004)(92566002)(68736007)(54356999)(105586002)(39060400001)(38730400001)(50986999)(102836003)(76176999)(77096006)(9686003)(55016002)(2900100001)(53936002)(6436002)(6506006)(4326007)(2906002)(106116001)(3280700002)(5001770100001)(8676002)(122556002)(81166006)(2950100002)(5660300001)(106356001)(7116003)(3480700004)(74482002)(3660700001)(305945005)(229853002)(74316002)(189998001)(86362001)(8936002)(7696004)(969003)(989001)(999001)(1009001)(1019001); DIR:OUT; SFP:1101; SCL:1; SRVR:YTXPR01MB0192; H:YTXPR01MB0189.CANPRD01.PROD.OUTLOOK.COM; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; received-spf: None (protection.outlook.com: uoguelph.ca does not designate permitted sender hosts) spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-originalarrivaltime: 19 Jan 2017 01:03:31.9058 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-Transport-CrossTenantHeadersStamped: YTXPR01MB0192 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jan 2017 01:03:36 -0000 Colin Percival wrote: >On 01/18/17 02:36, Konstantin Belousov wrote: >> On Wed, Jan 18, 2017 at 04:37:40AM +0000, Colin Percival wrote: >>> Thanks, looks like that was exactly it -- if the TCP send buffer was fu= ll >>> we would call sbwait, and if a signal arrived it would return ERESTART. >>> It looks like setting the SB_NOINTR flag will prevent this; I'm testing= a >>> patch right now. >> >> Note that passing SB_NOINTR unconditionally or even only for mounts >> with nointr (default) option is wrong. You make the socket operation >> uninterruptible, process terminate-ability becomes depended on the >> external factor, the behaviour of the remote system. I looked and I think forced dismounts are broken when the thread is stuck i= n sosend(). (It assumes that the threads doing RPCs are waiting for replies.) --> I think this can be fixed by posting a signal to the threads, but only = if SB_NOINTR isn't used. --> As such, I think looping on ERESTART when PCATCH isn't set in ct_waitfl= ag is the better way to go. (At a glance, I think sosend_generic() will f= ail with ERESTART before putting any data in the send queue. For NFS, it uses t= he mbuf list "top" and is always "atomic".) Kostik, Colin has already been testing the looping case. >I'm not sure what you're getting at here. The fact that "NFS mounted with= out >the intr flag" + "unresponsive NFS server" =3D "unkillable processes" has = been >a (mis)feature of NFS for decades. As I already mentioned, I'd like to at least get forced dismounts to work, = rick From owner-freebsd-net@freebsd.org Thu Jan 19 01:52:17 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1B790CB7550 for ; Thu, 19 Jan 2017 01:52:17 +0000 (UTC) (envelope-from dbelleisle@gmail.com) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id F1DBB144E for ; Thu, 19 Jan 2017 01:52:16 +0000 (UTC) (envelope-from dbelleisle@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id F13ABCB754F; Thu, 19 Jan 2017 01:52:16 +0000 (UTC) Delivered-To: net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F0E29CB754E for ; Thu, 19 Jan 2017 01:52:16 +0000 (UTC) (envelope-from dbelleisle@gmail.com) Received: from mail-ua0-x22d.google.com (mail-ua0-x22d.google.com [IPv6:2607:f8b0:400c:c08::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C1FCA144D for ; Thu, 19 Jan 2017 01:52:16 +0000 (UTC) (envelope-from dbelleisle@gmail.com) Received: by mail-ua0-x22d.google.com with SMTP id 35so23383187uak.1 for ; Wed, 18 Jan 2017 17:52:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=RFxL4PJu7+ehL07vg9YD7dJeSnO3NGxLrWY7wW7nA6U=; b=DfBIudd6sVzDffmoxatkaTatdyfhBTBqHPrsneFiEeB64ukb5u0lDxGuE/QwbJwELZ bveItuZqFtZXuWN1lPTPG+fZQNO7gd6B0ntISYbUpaDItranoRdpaXxWFI+eBKMB+Hz0 fsWv56+0XJHd+wvZrmkVn/3jRKUZ1gtRYRF2nJHu4OaV5aKTN3T5oJWbPGQIfZwqPBbO /v6/OmTJCL1DVC/KhXDP4hnD1Grk6e6fqrG9c4C7TBUOdwgkm4Mc2d7lBob/HhwROKQZ 9VzxLLcZzJdImWSg0wZfl4xSReR4eX7fMG/hh/9PdiqQ45VhBsMSYGm7mCfcg//unJG/ 9Trw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=RFxL4PJu7+ehL07vg9YD7dJeSnO3NGxLrWY7wW7nA6U=; b=ZWoLOrIqT53i6NorhSxJgKXC3cckw3ftML9/pywRWvxjYa6x78n7YDwzQM0nhcPzGq DuCjdAbpBLpHETg6cL6imp4WACHox69d+o6Wi7A6XcZ6XMV5pda2FI2cf1mt3195cEaA Ck6fzblua4kC352UkpL7QdM2ZYH3Hu9M6mV6o5dbEGwYweL6leun736Cv5locpZGJXIT ZGArodDbg51e+d+6YXW4CPiBUQYIwtkPgoH3J4OOnfLeLeFQHAaZxEUzJ7WZMh0PMy+a IdVhrbDzfhMOxWJWfpfo0CW7PeZrUozIfkWyba+wjTrq5Zy7Ol2t/FVpxpIt8hFP3LIJ vl4w== X-Gm-Message-State: AIkVDXIEy/47pREkcA1x0Y804HlU8sxWEXKjggOstTw2SbEpip7Q5/pqhdNyt5tYBYviKNferRnQHgmUitYSIg== X-Received: by 10.176.81.51 with SMTP id e48mr3014860uaa.100.1484790735659; Wed, 18 Jan 2017 17:52:15 -0800 (PST) MIME-Version: 1.0 Received: by 10.176.74.216 with HTTP; Wed, 18 Jan 2017 17:51:55 -0800 (PST) From: David Belle-Isle Date: Wed, 18 Jan 2017 20:51:55 -0500 Message-ID: Subject: Netmap TX with no impact to host To: net@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jan 2017 01:52:17 -0000 Hi, I'm trying to open a netmap descriptor to an interface to send packets. However, I don't want to prevent the host to send and receive data (transparent). I don't think this should be hard but I can't figure out how to do this. I tried to run the bridge example in the FreeBSD distribution but even that I can't get to run without stopping the host's networking completely. I tried running ./bridge em0 em0 which, if I understand correctly should open the NIC and host rings and pass the traffic through. As soon as I start it all the networking stops. I tried testing in a VMware VM and on bare-metal with em cards and got the same results with both. Can someone help me? Thanks, David From owner-freebsd-net@freebsd.org Thu Jan 19 07:34:31 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1EC05CB7DFE for ; Thu, 19 Jan 2017 07:34:31 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BC61D198B for ; Thu, 19 Jan 2017 07:34:30 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id v0J7YPNu065423 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 19 Jan 2017 09:34:25 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua v0J7YPNu065423 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id v0J7YOEw065422; Thu, 19 Jan 2017 09:34:24 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 19 Jan 2017 09:34:24 +0200 From: Konstantin Belousov To: Rick Macklem Cc: Colin Percival , "freebsd-net@freebsd.org" Subject: Re: sosend returning ERESTART Message-ID: <20170119073424.GM2349@kib.kiev.ua> References: <01000159aac969e6-b2fc3913-d04e-42d4-befd-402ed0d830bf-000000@email.amazonses.com> <20170117100634.GS2349@kib.kiev.ua> <01000159afddb7ce-064a5d17-4b81-4b2c-a9b4-3ddd2ad2e377-000000@email.amazonses.com> <20170118103650.GE2349@kib.kiev.ua> <01000159b390c409-5adcb488-67e8-4038-b9b0-5d4f33460205-000000@email.amazonses.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.7.2 (2016-11-26) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jan 2017 07:34:31 -0000 On Wed, Jan 18, 2017 at 10:52:02PM +0000, Rick Macklem wrote: > Colin Percival wrote: > >On 01/18/17 02:36, Konstantin Belousov wrote: > >> On Wed, Jan 18, 2017 at 04:37:40AM +0000, Colin Percival wrote: > >>> Thanks, looks like that was exactly it -- if the TCP send buffer was full > >>> we would call sbwait, and if a signal arrived it would return ERESTART. > >>> It looks like setting the SB_NOINTR flag will prevent this; I'm testing a > >>> patch right now. > >> > >> Note that passing SB_NOINTR unconditionally or even only for mounts > >> with nointr (default) option is wrong. You make the socket operation > >> uninterruptible, process terminate-ability becomes depended on the > >> external factor, the behaviour of the remote system. > > > >I'm not sure what you're getting at here. The fact that "NFS mounted without > >the intr flag" + "unresponsive NFS server" = "unkillable processes" has been > >a (mis)feature of NFS for decades. > The case I would like to see work is the forced dismount. I need to go look at > what it does and see if SB_NOINTR would break it worse than it is broken now. > (It is currently broken when something like "umount" without -f is done, which > locks up the mounted on vnode so "umount -f" never gets to the umount(2) syscall. > I do plan on a "straight ot NFS" option for umount(8) to avoid this problem, but > haven't gotten around to it.) > > The alternative to SB_NOINTR is looping and doing the sosend() again for the > case where it returns ERESTART and "intr" wasn't set on the mount. Note that the condition of pending signal which triggered ERESTART is permanent until the signal is delivered or blocked. In other words, or future PCATCH sleeps will fail with ERESTART/EINTR. > --> For this to be ok, we must be sure that when sosend() returns ERESTART, > it has not queued the data for sending so it is safe to send it all again. > I think this is true for this case? > > rick From owner-freebsd-net@freebsd.org Thu Jan 19 07:56:41 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D34CFCB776A for ; Thu, 19 Jan 2017 07:56:41 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C38511503 for ; Thu, 19 Jan 2017 07:56:41 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v0J7uee5062574 for ; Thu, 19 Jan 2017 07:56:41 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 208205] re0 watchdog timeout Date: Thu, 19 Jan 2017 07:56:40 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.3-BETA2 X-Bugzilla-Keywords: needs-qa X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: ml@netfence.it X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jan 2017 07:56:41 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D208205 --- Comment #12 from ml@netfence.it --- I disabled powerd, but the problem showed up again. It only happened twice in some months, but it's still a critical problem for us. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Thu Jan 19 10:31:02 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DEB72CB8773; Thu, 19 Jan 2017 10:31:02 +0000 (UTC) (envelope-from etnapierala@gmail.com) Received: from mail-wm0-x229.google.com (mail-wm0-x229.google.com [IPv6:2a00:1450:400c:c09::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 71F80152F; Thu, 19 Jan 2017 10:31:02 +0000 (UTC) (envelope-from etnapierala@gmail.com) Received: by mail-wm0-x229.google.com with SMTP id c206so70211005wme.0; Thu, 19 Jan 2017 02:31:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:date:from:to:cc:subject:message-id:mail-followup-to :references:mime-version:content-disposition:in-reply-to:user-agent; bh=Lyl5aHThLHdR/WMskKCCB6Bjo1h4OeytD+RJYnoIS3M=; b=C8cmwVPv8Z+Pxa+d9yfsszqDqXq1zzQx5KkxQEcymP0/hrc9UvfU51ukW75uTOm7uX md7voetmgLAgAji9txYxY8VDYgMbxh/2j9FDu8tyUjbBypBP2/bydqh872lOfjQow5cH nCSGqFQh30G2/IlSUFqOq6Z/932AUJvzJdAa/TXxKTSwjFrOZWyEeNx4JUy05njEjBZ5 34S+j3kXoLDv33890tnvOMGE/rRgt1uBBpNVhqs5nDp4dFUz49KmvUVmVUIYe7UoLVnJ Ce6LTsh5DJDSEAoqex1fBAOhQf6LCU1KNxXf+boyCu/zLXa6mKCJ8TTG21VpNymnyMdW Zvag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :mail-followup-to:references:mime-version:content-disposition :in-reply-to:user-agent; bh=Lyl5aHThLHdR/WMskKCCB6Bjo1h4OeytD+RJYnoIS3M=; b=aLpb0D+Vy2sgS0+hZ7SQUCTVt4mP5xMERiausgOF8tjovZREOYixX0x0SXgNmh6FDn 3LZ1OqkMSee0J545Xwn2kEuaQ/BOmcvvwEVDgqO77WrBBpYoVPYVP2UlUvvRhH12awct 1aDpiQPHv1yLzgWiJjV5uxsr3dZgN+ohdXs8GOUpDWjifjD9iIXd+jCJ75PI6DyPzKRE SXmrJWdkS7v+zzgsWnP2DjaUxcjQsCwrhdGfdblUpcVO+c+eETrHY/7V6jFngV1qoztE rviuNof9vJki2Qs1aJLZb0HrrV+BhqLzfHuej6fwRAmwj8zGOteiU7dkMasfC5Bc2/ce Kv6A== X-Gm-Message-State: AIkVDXJccjZVPECk9idBbRxkLCSIUxz8L0OVV9rg8leXh8fb/FR+R5Ebyf1N3oq/bwgDLg== X-Received: by 10.28.45.1 with SMTP id t1mr22920678wmt.53.1484821860918; Thu, 19 Jan 2017 02:31:00 -0800 (PST) Received: from brick (global-5-141.nat-2.net.cam.ac.uk. [131.111.5.141]) by smtp.gmail.com with ESMTPSA id 63sm52305245wmg.2.2017.01.19.02.30.59 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 19 Jan 2017 02:31:00 -0800 (PST) Sender: =?UTF-8?Q?Edward_Tomasz_Napiera=C5=82a?= Date: Thu, 19 Jan 2017 10:30:58 +0000 From: Edward Tomasz =?utf-8?Q?Napiera=C5=82a?= To: "Eugene M. Zheganin" Cc: freebsd-net@FreeBSD.org, freebsd-stable Subject: Re: decent 40G network adapters Message-ID: <20170119103058.GA30694@brick> Mail-Followup-To: "Eugene M. Zheganin" , freebsd-net@FreeBSD.org, freebsd-stable References: <587F39E3.1060608@norma.perm.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <587F39E3.1060608@norma.perm.ru> User-Agent: Mutt/1.7.2 (2016-11-26) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jan 2017 10:31:03 -0000 On 0118T1448, Eugene M. Zheganin wrote: > Hi. > > Could someone recommend a decent 40Gbit adapter that are proven to be > working under FreeBSD ? The intended purpose - iSCSI traffic, not much > pps, but rates definitely above 10G. I've tried Supermicro-manufactured > Intel XL710 ones (two boards, different servers - same sad story: > packets loss, server unresponsive, spikes), seems like they have a > problem in a driver (or firmware), and though Intel support states this > is because the Supermicro tampered with the adapter, I'm still > suspicious about ixl(4). I've also seen in the ML a guy reported the > exact same problem with ixl(4) as I have found. > > So, what would you say ? Chelsio ? I'd say Chelsio; it's very well supported, it's actively involved in FreeBSD development, and there's hardware-specific iSCSI offload for it (cxgbei). From owner-freebsd-net@freebsd.org Thu Jan 19 11:00:12 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 887C4CB8F08 for ; Thu, 19 Jan 2017 11:00:12 +0000 (UTC) (envelope-from v.maffione@gmail.com) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 6635B1244 for ; Thu, 19 Jan 2017 11:00:12 +0000 (UTC) (envelope-from v.maffione@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 6595ACB8F07; Thu, 19 Jan 2017 11:00:12 +0000 (UTC) Delivered-To: net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 653A5CB8F06 for ; Thu, 19 Jan 2017 11:00:12 +0000 (UTC) (envelope-from v.maffione@gmail.com) Received: from mail-ot0-x229.google.com (mail-ot0-x229.google.com [IPv6:2607:f8b0:4003:c0f::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2A7AC1243 for ; Thu, 19 Jan 2017 11:00:12 +0000 (UTC) (envelope-from v.maffione@gmail.com) Received: by mail-ot0-x229.google.com with SMTP id f9so29287659otd.1 for ; Thu, 19 Jan 2017 03:00:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=COWIBoV+Da7v9wz4g90SWKr8tDwSiD0UFg8ZM0OC8zk=; b=EIIS1odHRvWp/Y35r4Jtv556B3WTG9llhwY2kPdIBIeoQS9G4WXTVJEsdoAXSUQhIM 5g0WvFKKNWj7h6/Mk86pS6+J28Izcd7Wu0xrlDeWb2RBt1nkm3anZEwPbDBmOmcN2XdR 4jSC2XkFgRw4oEMBmcOkgFwDFsLU3Xs0+ZvM/gqx20WU8DgQ7oxRipJir+oLZbWUe81X AjBtpPAN2p4gBogxWOW9sHwVe6NqV1o3elEftQpumMjE0q4Zw0+XTqXSbbovkszOUCue dk5GuxEbOC36xAYK9+EwHDdETGmb9I2W6Uh+OLFbfH5mGo4TvSitkGBK1rcpoc6lmber GoQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=COWIBoV+Da7v9wz4g90SWKr8tDwSiD0UFg8ZM0OC8zk=; b=TkGe8QzbQVpLiQX0sgOUt2iIQUExS/2VoOeW0F0/3WTPc1fKY/o0eSZ+qyks6yfWYc pJ+Vmpcv3NFHkW0qoxgxbDkCFZxUQWoqIv5TAdZgjQYd53ywx8K/rdupOQ7Q8RievJED cuLvqczvmBNnKddkw+uSfcmmL+hoT5n31NohL1tc/weWjfgYJuYEf6MSkXihT2fNSWOs IQefzXkCL4p/Io0DkbOo/OIjrDOejJh9DWotTT+FPTmi/7+5X5+kC1oPawjyG9pvU+0w 6ryWMinIYTPdnGTIFCXEy2VQwJK9WsF24E+48a1ZNIsHNhzycD81pF6kKkASkEfN8n/O yBIg== X-Gm-Message-State: AIkVDXJ+vXlz6PyBXGDf+N5XeJE+g9Pp4ywGxuYIXEcCHbpDvHhD1iZlEsfohTeejeM7ATMHf/qclHPnUKwVog== X-Received: by 10.157.40.46 with SMTP id m43mr4404029otb.213.1484823611306; Thu, 19 Jan 2017 03:00:11 -0800 (PST) MIME-Version: 1.0 Received: by 10.157.6.166 with HTTP; Thu, 19 Jan 2017 03:00:10 -0800 (PST) In-Reply-To: References: From: Vincenzo Maffione Date: Thu, 19 Jan 2017 12:00:10 +0100 Message-ID: Subject: Re: Netmap TX with no impact to host To: David Belle-Isle Cc: "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jan 2017 11:00:12 -0000 Hi, Before answering to the question, some important disclaimers. If you are using FreeBSD-12-current, please be aware that there are IFLIB developments in progress that affect em devices. Unfortunately, at the moment netmap does not work on em/lem/igb devices because of the switch to IFLIB at this commit https://github.com/freebsd/freebsd/commit/4bf01b136dfa554a57f25559e7a848bf32206c66. Netmap on em works correctly right before this commit. If you are using FreeBSD-11 versions, be aware that netmap in emulated mode is broken in those versions. I have the fix for that, but it is not in the released code. If you need it, I can give you a patch. Btw, can anyone point me at the procedure to push this fix to the relevant release/stable branches? If you use em devices with DEV_NETMAP enabled, emulated mode is not used, so it will work. If you use interfaces without DEV_NETMAP support, emulated netmap is used and you are affected by the broken code. Regarding your question: You can build a netmap program that does what you want: (1) transmit packets on em0 at high rates using netmap (2) let the host stack transmit packets on em0 (3) let packets received on em0 to find their way to the host stack. In netmap it is possible to open just a subset of the TX rings and/or a subset of the RX rings. In your case you need to open all the TX hw rings (to acheve (1)) and the RX host ring (a.k.a. RX sw ring) to achieve (2). You achieve (3) by simply not opening the RX hw rings. To implement this you need two calls to nm_open() (similarly to what bridge does): pa = nm_open("netmap:em0/T, NULL, 0, NULL); /* open all TX hw rings */ pb = nm_open("netmap:em0^/R, NULL, NM_OPEN_NO_MMAP, pa); /* open only RX host ring */ Then you can use "pa" to transmit packets on the hw TX rings (1). To implement (2), you have two options: (A) your application moves packets from the host RX ring (pb) to the hw TX rings (pa). In the end this is the logic of the "bridge" program does, so you could start from there. (B) you use "transparent mode" in the host->nic direction: applications marks with the NS_FORWARD flag the host RX ring slots right before they released to the kernel, and the kernel with take care of forward them to the TX ring for you. However, be aware this mode is still experimental, and not well tested yet. Regarding the use of bridge, the correct command line for what you want to do (forward all packets between host stack and NIC, in both directions) is: # bridge netmap:em0 However, to let TCP/UDP traffic go correctly, you need to disable tso and checksum offloadings on em0, since netmap will not program the NIC to perform those offloadings. This is also valid for your custom program above. Cheers, Vincenzo 2017-01-19 2:51 GMT+01:00 David Belle-Isle : > Hi, > > I'm trying to open a netmap descriptor to an interface to send packets. > However, I don't want to prevent the host to send and receive data > (transparent). I don't think this should be hard but I can't figure out how > to do this. > > I tried to run the bridge example in the FreeBSD distribution but even that > I can't get to run without stopping the host's networking completely. I > tried running ./bridge em0 em0 which, if I understand correctly should open > the NIC and host rings and pass the traffic through. As soon as I start it > all the networking stops. > > I tried testing in a VMware VM and on bare-metal with em cards and got the > same results with both. > > Can someone help me? > > Thanks, > > David > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > -- Vincenzo Maffione From owner-freebsd-net@freebsd.org Thu Jan 19 11:11:32 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 44011CB8326 for ; Thu, 19 Jan 2017 11:11:32 +0000 (UTC) (envelope-from v.maffione@gmail.com) Received: from mail-oi0-x230.google.com (mail-oi0-x230.google.com [IPv6:2607:f8b0:4003:c06::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F30DB1936; Thu, 19 Jan 2017 11:11:31 +0000 (UTC) (envelope-from v.maffione@gmail.com) Received: by mail-oi0-x230.google.com with SMTP id u143so22793612oif.3; Thu, 19 Jan 2017 03:11:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to:cc; bh=o1UeiWeAg1wTNItpbkhRwSbYNg0F+FCR4f6XeAImxns=; b=IWwZuwfkiy756TAxbCtt4kAoNVwGum66Hmqga21DI68Zq1B8Qz/MDWSEBY/+enS8RD fCLERNznSUf2UUhJcfK3WJbxLVKV+WynxassZw4j4PDheRV0XfCr8ht5z0L+MGH/wnki RRCHso05+8CiASyciBWn3FklJW7wshkqA/vG3YyXTl6mJVJw0BUc996mpNulCYVpFAuF o+qMdRe/KErxXBPHA8MsIi/ceiVvjfIzItrkkKurBm05MPT4QXXRTxijNxt4ClnNoSVZ 4eUxYJu3Z9Vgm3fRsiEPgsWGRHLbbgg6IB2gCNvZUk/4i2bvgmmGzpjZeLJE3xatvauW 3ggg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to:cc; bh=o1UeiWeAg1wTNItpbkhRwSbYNg0F+FCR4f6XeAImxns=; b=KUmpWbVw9O6Lb3ZPCrF5/j8vupj+2AvcOSc6Yx+4M/Piw/QXW3FFsjBstbKA/+wMs1 O20rGxjbqE3BcJceVunCPghli6C4FtCrHFVwt2OdiYuufkHTXuwa6vBiHrr9NFuP4ZXi N4j4QhWLA969kxpvtuqo+z7AIZU4SlLeDxySxrnnP5SGYsmCYwV5FD+pyKwW51jgToda i76Ponm7wk+OeYrQ+OpOgeeaRss0/eB/e5eXxlAx4qOzab1CzzXQ0z6CO/l8wmIPz8rB iPu2Aa822/drp9S5cSgcBLbY0M9/peu+t1jZbLeL8CmAHS8n9y0IYzdI9kYMHm9QoSqS NWrg== X-Gm-Message-State: AIkVDXI3OCaLztWLmDSq3sejCND/alPOfLctu98HdCX/FgWcPN7jBHGbplz1WekYo+/GQ74uvV2CP5LHqF07Ag== X-Received: by 10.202.212.70 with SMTP id l67mr4060048oig.153.1484824291318; Thu, 19 Jan 2017 03:11:31 -0800 (PST) MIME-Version: 1.0 Received: by 10.157.6.166 with HTTP; Thu, 19 Jan 2017 03:11:30 -0800 (PST) From: Vincenzo Maffione Date: Thu, 19 Jan 2017 12:11:30 +0100 Message-ID: Subject: Fix netmap emulated mode for FreeBSD 11 versions To: FreeBSD Net Cc: Luigi Rizzo , Giuseppe Lettieri Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jan 2017 11:11:32 -0000 Hi, A change in mb_free_ext() introduced in FreeBSD 11 versions broke the transmission support for netmap in emulated mode. This means immediate kernel crashes for netmap users in non-native (emulated) mode. This problem has been fixed in FreeBSD-12-CURRENT, which contains a recent version of Netmap. However, FreeBSD 11 versions (release, stable) are still affected, as they contain old Netmap versions. Is it possible to push the fix (provided by me) to the relevant release/stable branches? What is the procedure (I'm not familiar with the FreeBSD release engineering process)? Thanks, Vincenzo From owner-freebsd-net@freebsd.org Thu Jan 19 12:50:37 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 10D7ACB5746 for ; Thu, 19 Jan 2017 12:50:37 +0000 (UTC) (envelope-from dbelleisle@gmail.com) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id E021B1FAF for ; Thu, 19 Jan 2017 12:50:36 +0000 (UTC) (envelope-from dbelleisle@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id DC8F6CB5744; Thu, 19 Jan 2017 12:50:36 +0000 (UTC) Delivered-To: net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DC291CB5741 for ; Thu, 19 Jan 2017 12:50:36 +0000 (UTC) (envelope-from dbelleisle@gmail.com) Received: from mail-ua0-x236.google.com (mail-ua0-x236.google.com [IPv6:2607:f8b0:400c:c08::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 946291FAD for ; Thu, 19 Jan 2017 12:50:36 +0000 (UTC) (envelope-from dbelleisle@gmail.com) Received: by mail-ua0-x236.google.com with SMTP id 96so32285231uaq.3 for ; Thu, 19 Jan 2017 04:50:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=tKuNSPgtx+F6cWOgMBTgP9EKLSKB8SJy71zto75UW0A=; b=Kwzb3JMML9DcAG/7wdmi/XAtCfqa8c9+P0u+dj4TxRYWSNPiwOqyEPK7M2BRdAQg7S e/vq4iuuj9sJ/kceQwrr9ekk95QQJwqAeHRmw8KOwUGFuFlsk6Bre1XkqClTWgmoDpYU WgR3kcQ6lxAOworf+Gv7TKKW6vcFct+tn2V5sWF5e0Obszkgh2KxMCYe1LH6oI4/IzUL +mqhClZ6V7cIpr5QNC20Yshewt364M5Cee6ID5AZVBaKvdd69Tk0X/44eVAgdnO1u2u8 Ve4QwNi60qcPSd1KaHGGm4jl0iRdpn3s3OpszenDCQzOXmeCy5N9K18BtudaeHtIXmeg Gm2g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=tKuNSPgtx+F6cWOgMBTgP9EKLSKB8SJy71zto75UW0A=; b=mwUAPHhh6/MSFQcNdNdEToFuJChP7q6472DXK3AaTL+r9d9OYZhSxuycKeKPnDN4Um HwdQrlGURxkX9MBblAr0a5o8j5N3DJzwDmHS6dx6guDXcIBrFdkYhdVPQi+zDHsbVn4w J+/PgfppMCpDx0mqGeRcdGX0uGL5+YVkIzksveRuCWJ7jKMBMqYJkVEGUB1uIW3ufoXD X/7v8bxT1hj/wigSlrjCdYL7ynyW6PiW8f4zqwirBMjlIjBk1WEg4/Wv9Z5kDyYazZ/R 2XQI9h2y6zwXdGYJOi79rbQlFydiF0cR+nPHwmJF9PywhqI99Wj3BvKQTB5mMgoRHzrL REEg== X-Gm-Message-State: AIkVDXJAIbTBRTSGaNlQiKP8LFEu/v3UpgnnA2347lj+/Q9dB3+hz6HczY3B73pmy48UOvB6nf9xnfeHX2yAvQ== X-Received: by 10.176.7.2 with SMTP id h2mr4585838uah.52.1484830235714; Thu, 19 Jan 2017 04:50:35 -0800 (PST) MIME-Version: 1.0 Received: by 10.176.74.216 with HTTP; Thu, 19 Jan 2017 04:50:15 -0800 (PST) In-Reply-To: References: From: David Belle-Isle Date: Thu, 19 Jan 2017 07:50:15 -0500 Message-ID: Subject: Re: Netmap TX with no impact to host To: Vincenzo Maffione Cc: "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jan 2017 12:50:37 -0000 Oh! Thank you soooo much! I was missing the last part about the checksum offloadings and TSO. I disabled them and everything is working perfectly now! Thank you so much!! On Thu, Jan 19, 2017 at 6:00 AM, Vincenzo Maffione wrote: > Hi, > Before answering to the question, some important disclaimers. > > If you are using FreeBSD-12-current, please be aware that there are IFLIB > developments in progress that affect em devices. Unfortunately, at the > moment netmap does not work on em/lem/igb devices because of the switch to > IFLIB at this commit https://github.com/freebsd/freebsd/commit/ > 4bf01b136dfa554a57f25559e7a848bf32206c66. Netmap on em works correctly > right before this commit. > > If you are using FreeBSD-11 versions, be aware that netmap in emulated > mode is broken in those versions. I have the fix for that, but it is not in > the released code. If you need it, I can give you a patch. Btw, can anyone > point me at the procedure to push this fix to the relevant release/stable > branches? > If you use em devices with DEV_NETMAP enabled, emulated mode is not used, > so it will work. If you use interfaces without DEV_NETMAP support, emulated > netmap is used and you are affected by the broken code. > > Regarding your question: > You can build a netmap program that does what you want: > (1) transmit packets on em0 at high rates using netmap > (2) let the host stack transmit packets on em0 > (3) let packets received on em0 to find their way to the host stack. > > In netmap it is possible to open just a subset of the TX rings and/or a > subset of the RX rings. > In your case you need to open all the TX hw rings (to acheve (1)) and the > RX host ring (a.k.a. RX sw ring) to achieve (2). > You achieve (3) by simply not opening the RX hw rings. > To implement this you need two calls to nm_open() (similarly to what > bridge does): > > pa = nm_open("netmap:em0/T, NULL, 0, NULL); /* open all TX hw rings */ > pb = nm_open("netmap:em0^/R, NULL, NM_OPEN_NO_MMAP, pa); /* open only RX > host ring */ > > Then you can use "pa" to transmit packets on the hw TX rings (1). > To implement (2), you have two options: > (A) your application moves packets from the host RX ring (pb) to the hw > TX rings (pa). In the end this is the logic of the "bridge" program does, > so you could start from there. > (B) you use "transparent mode" in the host->nic direction: applications > marks with the NS_FORWARD flag the host RX ring slots right before they > released to the kernel, and the kernel with take care of forward them to > the TX ring for you. However, be aware this mode is still experimental, and > not well tested yet. > > Regarding the use of bridge, the correct command line for what you want to > do (forward all packets between host stack and NIC, in both directions) is: > > # bridge netmap:em0 > > However, to let TCP/UDP traffic go correctly, you need to disable tso and > checksum offloadings on em0, since netmap will not program the NIC to > perform those offloadings. This is also valid for your custom program above. > > Cheers, > Vincenzo > > 2017-01-19 2:51 GMT+01:00 David Belle-Isle : > >> Hi, >> >> I'm trying to open a netmap descriptor to an interface to send packets. >> However, I don't want to prevent the host to send and receive data >> (transparent). I don't think this should be hard but I can't figure out >> how >> to do this. >> >> I tried to run the bridge example in the FreeBSD distribution but even >> that >> I can't get to run without stopping the host's networking completely. I >> tried running ./bridge em0 em0 which, if I understand correctly should >> open >> the NIC and host rings and pass the traffic through. As soon as I start it >> all the networking stops. >> >> I tried testing in a VMware VM and on bare-metal with em cards and got the >> same results with both. >> >> Can someone help me? >> >> Thanks, >> >> David >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> > > > > -- > Vincenzo Maffione > From owner-freebsd-net@freebsd.org Thu Jan 19 13:24:15 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 93855CB723F for ; Thu, 19 Jan 2017 13:24:15 +0000 (UTC) (envelope-from lists.br@gmail.com) Received: from mail-wm0-x22d.google.com (mail-wm0-x22d.google.com [IPv6:2a00:1450:400c:c09::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 28F841098; Thu, 19 Jan 2017 13:24:15 +0000 (UTC) (envelope-from lists.br@gmail.com) Received: by mail-wm0-x22d.google.com with SMTP id c85so288423120wmi.1; Thu, 19 Jan 2017 05:24:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=L7b+Tn/R8gXRajWpA4aQf4VbY6/LE7zXNTUOEiSagq4=; b=pAIEbEonAxlfdDjU8e6OHRkZRLL59/NPjVQEhpaF22shsHR4hodSCtZracTNR2KfTH /fblou1wrKTEzNZourzBTgJLTj0DI3E9j6p3ce0pE/VYB5ARFUywcVMjfUTanmYYegpV WFrqY0SVg8+4J2teDIo8lQuz39DlNqgB4kP8TN0VrrurGUBNpJytaN0r4QevHbKmbKbM cLC+aOfEnEiEWlcatSEurCXyrP/4PBCtkHZ8RWUEWE11lZ0BfQgKPpyd1e87U+22aBlJ MXNOVCHEHzXqe5JlUhh+8olZehYpy7d5hiKcgrsDJnGvbihLnCuPl/1+mRgGfbRWXUyq fPcA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=L7b+Tn/R8gXRajWpA4aQf4VbY6/LE7zXNTUOEiSagq4=; b=AmjYoGa3RADmAvU+fMsfFWijri6iABPf9sIRNJzvI2DvPn1m21hwgrOZkV8S7w903+ 3MdH8LRfIALnNTl+MyFDvJ0Tir8P+8lpNOE82U2ZR9yi3glkZd21vnmZEN0WG2Hdd4td Cwq2Wvs7iOH/oRohB9g5kIAEdlxVULativJLp7hvQO6jMiErhzD0SmJNrGNYcFZiUnjF +7tD8p3BVpPMBaVONTNJ0AX248A0gTIDVTNuVq0zNQoC0Ru7Of0r7JQD9RL+qJAbIIN6 n0VU85Bg80nuBdx+KXjRnpNP86/LJt/yC1OFLq3SjmifQJcRdaxZiwbf2mF16jHgF4kD TMog== X-Gm-Message-State: AIkVDXK+RF4DAevrEi8Q/EIdSKje4BDH4jEiE+ferVjmwiBYVCJigdLl9yBm4bVMNJO5VEXkL8sEFqsdNDGhiw== X-Received: by 10.28.109.218 with SMTP id b87mr6770788wmi.52.1484832252189; Thu, 19 Jan 2017 05:24:12 -0800 (PST) MIME-Version: 1.0 Received: by 10.80.149.249 with HTTP; Thu, 19 Jan 2017 05:24:11 -0800 (PST) In-Reply-To: References: From: Luiz Otavio O Souza Date: Thu, 19 Jan 2017 11:24:11 -0200 Message-ID: Subject: Re: Fix netmap emulated mode for FreeBSD 11 versions To: Vincenzo Maffione Cc: FreeBSD Net , Giuseppe Lettieri , Luigi Rizzo Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jan 2017 13:24:15 -0000 On 19 January 2017 at 09:11, Vincenzo Maffione wrote: > Hi, > > A change in mb_free_ext() introduced in FreeBSD 11 versions broke the > transmission support for netmap in emulated mode. This means immediate > kernel crashes for netmap users in non-native (emulated) mode. > > This problem has been fixed in FreeBSD-12-CURRENT, which contains a recent > version of Netmap. However, FreeBSD 11 versions (release, stable) are still > affected, as they contain old Netmap versions. > Is it possible to push the fix (provided by me) to the relevant > release/stable branches? What is the procedure (I'm not familiar with the > FreeBSD release engineering process)? Hi Vincenzo, Yeah, I saw that trying netmap on ARM (running with emulated drivers) and I can confirm that it is fixed on -head. Please, send me the fix and I'll get it committed on stable/11 (this is a common situation and the fix can be committed directly into the affected branch). Thanks! Luiz From owner-freebsd-net@freebsd.org Thu Jan 19 13:51:28 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EEA7ECB7D04 for ; Thu, 19 Jan 2017 13:51:28 +0000 (UTC) (envelope-from v.maffione@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id CBAA2129C for ; Thu, 19 Jan 2017 13:51:28 +0000 (UTC) (envelope-from v.maffione@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id CAEB9CB7D02; Thu, 19 Jan 2017 13:51:28 +0000 (UTC) Delivered-To: net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CA820CB7D01 for ; Thu, 19 Jan 2017 13:51:28 +0000 (UTC) (envelope-from v.maffione@gmail.com) Received: from mail-ot0-x235.google.com (mail-ot0-x235.google.com [IPv6:2607:f8b0:4003:c0f::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8E3E7129B for ; Thu, 19 Jan 2017 13:51:28 +0000 (UTC) (envelope-from v.maffione@gmail.com) Received: by mail-ot0-x235.google.com with SMTP id 65so32180767otq.2 for ; Thu, 19 Jan 2017 05:51:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=p+mmWt8KKLEBzrgNlRaHmpmQk0ikpVnGM7DmzTlNrNg=; b=J2arHMFLTwl63U8E+tlIi6Cz+p1oVlvsij+W5biuq/SI/YMDYYqwNmFBA0vn91IsL4 qtFcYTDRBIWRaOtd2hR+DjmqRFG70vmDL3bMfuS3ryGmfoRkGQjgvolQ7CNbXkdgggzO 5yB32v1EyUF2BTMuucwYG/aJgE9h6H8iF4OWNqUQ3L+q1wIZ+pIelNIO1ACo+PNgEGuv Sc70P/tKCzdQx11z3Rk6rxhxdd2/kH57wqH/hrT78/bIEy0raUDIsHjkKF3qkI2h2J3X gu/UIcPbr4mit+A60PNAkEpkkeYq3gd4Bgh9JDg55A6PQz9tyN5lcR0Hy1WCmGMQSD7z HDWA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=p+mmWt8KKLEBzrgNlRaHmpmQk0ikpVnGM7DmzTlNrNg=; b=UdipR2skeANw/WBMsYFj9KTdaoKBrXk3E5nk8INafVi/ob0+by++6xjYOJCYZNICIH PypS0NB6TkFHSfj3p+FgmMSgYn2uxd0sDUjpjjMhjmRT1yjaBAIf5hRPoBQEGoiEiTDz lC0HEe9yAHlumYj6N9mtEO0CRmULlQRhXi+CYTbf4vB9bxvm7v0VL/HTS6n0VwJK7JDO jQ8gb/3ZOHAmi59HZAGNV7wAgSiZjE6NIbsd2MxcXKG7TCemXB8bql49EE+RBC+Serv/ /kvFs+s4DIQuVfVuQ+GIPP1qEL+49kRI8DUGyBfevi7B3wHXAuv2jSfEoRpDTcqcmYs9 6PGw== X-Gm-Message-State: AIkVDXIu2UN/ZR0AW2tYcEgD2wZuTKBMTJ/VZKDHjAkH5+pVzlAZP7P8PNeRo3VyVSktXHFonjZHmEjkhDH7/A== X-Received: by 10.157.24.92 with SMTP id t28mr4704563ott.238.1484833887910; Thu, 19 Jan 2017 05:51:27 -0800 (PST) MIME-Version: 1.0 Received: by 10.157.6.166 with HTTP; Thu, 19 Jan 2017 05:51:27 -0800 (PST) In-Reply-To: References: From: Vincenzo Maffione Date: Thu, 19 Jan 2017 14:51:27 +0100 Message-ID: Subject: Re: Netmap TX with no impact to host To: David Belle-Isle Cc: "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jan 2017 13:51:29 -0000 You're welcome! I know it's a bit difficult to notice, but this is written in the very last sentence of the netmap(4) man page. Cheers, Vincenzo 2017-01-19 13:50 GMT+01:00 David Belle-Isle : > Oh! Thank you soooo much! I was missing the last part about the checksum > offloadings and TSO. I disabled them and everything is working perfectly > now! Thank you so much!! > > On Thu, Jan 19, 2017 at 6:00 AM, Vincenzo Maffione > wrote: > >> Hi, >> Before answering to the question, some important disclaimers. >> >> If you are using FreeBSD-12-current, please be aware that there are IFLIB >> developments in progress that affect em devices. Unfortunately, at the >> moment netmap does not work on em/lem/igb devices because of the switch to >> IFLIB at this commit https://github.com/freebsd/fre >> ebsd/commit/4bf01b136dfa554a57f25559e7a848bf32206c66. Netmap on em works >> correctly right before this commit. >> >> If you are using FreeBSD-11 versions, be aware that netmap in emulated >> mode is broken in those versions. I have the fix for that, but it is not in >> the released code. If you need it, I can give you a patch. Btw, can anyone >> point me at the procedure to push this fix to the relevant release/stable >> branches? >> If you use em devices with DEV_NETMAP enabled, emulated mode is not used, >> so it will work. If you use interfaces without DEV_NETMAP support, emulated >> netmap is used and you are affected by the broken code. >> >> Regarding your question: >> You can build a netmap program that does what you want: >> (1) transmit packets on em0 at high rates using netmap >> (2) let the host stack transmit packets on em0 >> (3) let packets received on em0 to find their way to the host stack. >> >> In netmap it is possible to open just a subset of the TX rings and/or a >> subset of the RX rings. >> In your case you need to open all the TX hw rings (to acheve (1)) and the >> RX host ring (a.k.a. RX sw ring) to achieve (2). >> You achieve (3) by simply not opening the RX hw rings. >> To implement this you need two calls to nm_open() (similarly to what >> bridge does): >> >> pa = nm_open("netmap:em0/T, NULL, 0, NULL); /* open all TX hw rings */ >> pb = nm_open("netmap:em0^/R, NULL, NM_OPEN_NO_MMAP, pa); /* open only RX >> host ring */ >> >> Then you can use "pa" to transmit packets on the hw TX rings (1). >> To implement (2), you have two options: >> (A) your application moves packets from the host RX ring (pb) to the hw >> TX rings (pa). In the end this is the logic of the "bridge" program does, >> so you could start from there. >> (B) you use "transparent mode" in the host->nic direction: applications >> marks with the NS_FORWARD flag the host RX ring slots right before they >> released to the kernel, and the kernel with take care of forward them to >> the TX ring for you. However, be aware this mode is still experimental, and >> not well tested yet. >> >> Regarding the use of bridge, the correct command line for what you want >> to do (forward all packets between host stack and NIC, in both directions) >> is: >> >> # bridge netmap:em0 >> >> However, to let TCP/UDP traffic go correctly, you need to disable tso and >> checksum offloadings on em0, since netmap will not program the NIC to >> perform those offloadings. This is also valid for your custom program above. >> >> Cheers, >> Vincenzo >> >> 2017-01-19 2:51 GMT+01:00 David Belle-Isle : >> >>> Hi, >>> >>> I'm trying to open a netmap descriptor to an interface to send packets. >>> However, I don't want to prevent the host to send and receive data >>> (transparent). I don't think this should be hard but I can't figure out >>> how >>> to do this. >>> >>> I tried to run the bridge example in the FreeBSD distribution but even >>> that >>> I can't get to run without stopping the host's networking completely. I >>> tried running ./bridge em0 em0 which, if I understand correctly should >>> open >>> the NIC and host rings and pass the traffic through. As soon as I start >>> it >>> all the networking stops. >>> >>> I tried testing in a VMware VM and on bare-metal with em cards and got >>> the >>> same results with both. >>> >>> Can someone help me? >>> >>> Thanks, >>> >>> David >>> _______________________________________________ >>> freebsd-net@freebsd.org mailing list >>> https://lists.freebsd.org/mailman/listinfo/freebsd-net >>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >>> >> >> >> >> -- >> Vincenzo Maffione >> > > -- Vincenzo Maffione From owner-freebsd-net@freebsd.org Thu Jan 19 16:46:40 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 14C88CB857D for ; Thu, 19 Jan 2017 16:46:40 +0000 (UTC) (envelope-from v.maffione@gmail.com) Received: from mail-ot0-x242.google.com (mail-ot0-x242.google.com [IPv6:2607:f8b0:4003:c0f::242]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CAADC13CF; Thu, 19 Jan 2017 16:46:39 +0000 (UTC) (envelope-from v.maffione@gmail.com) Received: by mail-ot0-x242.google.com with SMTP id 36so4827028otx.3; Thu, 19 Jan 2017 08:46:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=QFAI4F0Q3MSr0xXAeDGb3LdEYflG/rb7VDG0iPoRwEk=; b=LQh845OXAUD/h2AM4sT9Ci2aZhZQV5mTgZhgYhl7/JNQKAsu/2eWDApSN6+LCxCKb5 L8KORTjHjbJziqIhILCMU/pIBkrFhT9NThRmzqaWPn3Q5l5oL5l9mkvXVQcE9w21W/v9 42GYbOxdAko2S8gZExWB3mYFDUkh+NjsriMteP1bKA4MpAMPmY2qmuE0QpBoTPPIo3oX F8+mrIu+B5irG8K7AzkVQvVFohh8E3t5sUox4/vB6daq6zBvY2R+ZK+MFwcwCpDBl1N+ HVqgwQG3szib6M3miK3ld+BCL4RMUEMYeufJXILiEcxoisZ8uNUFJES6tNz7OO/F5qo1 syww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=QFAI4F0Q3MSr0xXAeDGb3LdEYflG/rb7VDG0iPoRwEk=; b=g9soHnglG6LeY12P/Ex4ngaoC3tE+A999hgBxcByxK8Ifld7F3wzA3kaZw4L2HGcP/ WBfSuYSxKgMQZ8BZNiCjS46SyCs2ZRXP9JuVY/3x0R7GoJMD87AyBUj0Aqtrq2/AJl1L Yprw+MrpV4MnIPo65/eHLk/v/9qvrdEjCIeZlH3SIOQNZZRvWfdZvadZmtmsayDKt8t+ ntgXvj3Fvo88iZb61eH1PYvJNL+uDHbESD0AimXejlIeeQHRIRkAaj1aSxD+8/hPCI3g ZTBUYCAUytSdLoTEgDjhZ8/6DuSU5ADiyxQ37UTYIYIeyCCkEpebTVDXVmRbem+Wu8ar 9PKQ== X-Gm-Message-State: AIkVDXK3RogBbMq/XpsO3uhDvXVD2HmbpV/Bk9hVkZt7xdKdhevwPbN+cN0izhlnmZXRoZhinKraV3Eyc0aZcw== X-Received: by 10.157.52.66 with SMTP id v60mr4571111otb.61.1484844399112; Thu, 19 Jan 2017 08:46:39 -0800 (PST) MIME-Version: 1.0 Received: by 10.157.6.166 with HTTP; Thu, 19 Jan 2017 08:46:38 -0800 (PST) In-Reply-To: References: From: Vincenzo Maffione Date: Thu, 19 Jan 2017 17:46:38 +0100 Message-ID: Subject: Re: Fix netmap emulated mode for FreeBSD 11 versions To: Luiz Otavio O Souza Cc: FreeBSD Net , Giuseppe Lettieri , Luigi Rizzo Content-Type: multipart/mixed; boundary=001a114152c2f51c52054675462a X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jan 2017 16:46:40 -0000 --001a114152c2f51c52054675462a Content-Type: text/plain; charset=UTF-8 Hi Luiz, I attached the patch, to be applied to the source code contained in this image: ftp://ftp.freebsd.org/pub/FreeBSD/snapshots/amd64/amd64/ISO-IMAGES/11.0/FreeBSD-11.0-STABLE-amd64-20170105-r311441-disc1.iso.xz The patch is simply a backport from -head of some modifications involving the emulated netmap mode. I tried the minimize the changes to only (1) fix the crash and (2) fix another synchronization problem. I did different tests on em and vtnet interfaces, and the thing seems to work. Thanks, Vincenzo 2017-01-19 14:24 GMT+01:00 Luiz Otavio O Souza : > On 19 January 2017 at 09:11, Vincenzo Maffione wrote: > > Hi, > > > > A change in mb_free_ext() introduced in FreeBSD 11 versions broke the > > transmission support for netmap in emulated mode. This means immediate > > kernel crashes for netmap users in non-native (emulated) mode. > > > > This problem has been fixed in FreeBSD-12-CURRENT, which contains a > recent > > version of Netmap. However, FreeBSD 11 versions (release, stable) are > still > > affected, as they contain old Netmap versions. > > Is it possible to push the fix (provided by me) to the relevant > > release/stable branches? What is the procedure (I'm not familiar with the > > FreeBSD release engineering process)? > > Hi Vincenzo, > > Yeah, I saw that trying netmap on ARM (running with emulated drivers) > and I can confirm that it is fixed on -head. > > Please, send me the fix and I'll get it committed on stable/11 (this > is a common situation and the fix can be committed directly into the > affected branch). > > Thanks! > Luiz > -- Vincenzo Maffione --001a114152c2f51c52054675462a Content-Type: text/x-patch; charset=US-ASCII; name="11-stable-fix-netmap.patch" Content-Disposition: attachment; filename="11-stable-fix-netmap.patch" Content-Transfer-Encoding: base64 X-Attachment-Id: f_iy4lthgp0 ZGlmZiAtLWdpdCBhL3N5cy9kZXYvbmV0bWFwL25ldG1hcF9mcmVlYnNkLmMgYi9zeXMvZGV2L25l dG1hcC9uZXRtYXBfZnJlZWJzZC5jCmluZGV4IDI1YzljNWMuLjA0MGQxYWMgMTAwNjQ0Ci0tLSBh L3N5cy9kZXYvbmV0bWFwL25ldG1hcF9mcmVlYnNkLmMKKysrIGIvc3lzL2Rldi9uZXRtYXAvbmV0 bWFwX2ZyZWVic2QuYwpAQCAtMjE4LDMwICsyMTgsMTYgQEAgZ2VuZXJpY194bWl0X2ZyYW1lKHN0 cnVjdCBpZm5ldCAqaWZwLCBzdHJ1Y3QgbWJ1ZiAqbSwKIHsKIAlpbnQgcmV0OwogCi0JLyoKLQkg KiBUaGUgbWJ1ZiBzaG91bGQgYmUgYSBjbHVzdGVyIGZyb20gb3VyIHNwZWNpYWwgcG9vbCwKLQkg KiBzbyB3ZSBkbyBub3QgbmVlZCB0byBkbyBhbiBtX2NvcHliYWNrIGJ1dCBqdXN0IGNvcHkKLQkg KiAoYW5kIGV2ZW50dWFsbHksIGp1c3QgcmVmZXJlbmNlIHRoZSBuZXRtYXAgYnVmZmVyKQotCSAq LworCS8qIExpbmsgdGhlIGV4dGVybmFsIHN0b3JhZ2UgdG8gdGhlIG5ldG1hcCBidWZmZXIsIHNv IHRoYXQKKwkgKiBubyBjb3B5IGlzIG5lY2Vzc2FyeS4gKi8KKwltLT5tX2V4dC5leHRfYnVmID0g bS0+bV9kYXRhID0gYWRkcjsKKwltLT5tX2V4dC5leHRfc2l6ZSA9IGxlbjsKIAotCWlmIChHRVRf TUJVRl9SRUZDTlQobSkgIT0gMSkgewotCQlEKCJpbnZhbGlkIHJlZmNudCAlZCBmb3IgJXAiLAot CQkJR0VUX01CVUZfUkVGQ05UKG0pLCBtKTsKLQkJcGFuaWMoImluIGdlbmVyaWNfeG1pdF9mcmFt ZSIpOwotCX0KLQkvLyBYWFggdGhlIGV4dF9zaXplIGNoZWNrIGlzIHVubmVjZXNzYXJ5IGlmIHdl IGxpbmsgdGhlIG5ldG1hcCBidWYKLQlpZiAobS0+bV9leHQuZXh0X3NpemUgPCBsZW4pIHsKLQkJ UkQoNSwgInNpemUgJWQgPCBsZW4gJWQiLCBtLT5tX2V4dC5leHRfc2l6ZSwgbGVuKTsKLQkJbGVu ID0gbS0+bV9leHQuZXh0X3NpemU7Ci0JfQotCWlmICgwKSB7IC8qIFhYWCBzZWVtcyB0byBoYXZl IG5lZ2xpZ2libGUgYmVuZWZpdHMgKi8KLQkJbS0+bV9leHQuZXh0X2J1ZiA9IG0tPm1fZGF0YSA9 IGFkZHI7Ci0JfSBlbHNlIHsKLQkJYmNvcHkoYWRkciwgbS0+bV9kYXRhLCBsZW4pOwotCX0KIAlt LT5tX2xlbiA9IG0tPm1fcGt0aGRyLmxlbiA9IGxlbjsKLQkvLyBpbmMgcmVmY291bnQuIEFsbCBv dXJzLCB3ZSBjb3VsZCBza2lwIHRoZSBhdG9taWMKLQlhdG9taWNfZmV0Y2hhZGRfaW50KFBOVF9N QlVGX1JFRkNOVChtKSwgMSk7CisKKwkvKiBtYnVmIHJlZmNudCBpcyBub3QgY29udGVuZGVkLCBu byBuZWVkIHRvIHVzZSBhdG9taWMKKwkgKiAoYSBtZW1vcnkgYmFycmllciBpcyBlbm91Z2gpLiAq LworCVNFVF9NQlVGX1JFRkNOVChtLCAyKTsKIAlNX0hBU0hUWVBFX1NFVChtLCBNX0hBU0hUWVBF X09QQVFVRSk7CiAJbS0+bV9wa3RoZHIuZmxvd2lkID0gcmluZ19ucjsKIAltLT5tX3BrdGhkci5y Y3ZpZiA9IGlmcDsgLyogdXNlZCBmb3IgdHggbm90aWZpY2F0aW9uICovCmRpZmYgLS1naXQgYS9z eXMvZGV2L25ldG1hcC9uZXRtYXBfZ2VuZXJpYy5jIGIvc3lzL2Rldi9uZXRtYXAvbmV0bWFwX2dl bmVyaWMuYwppbmRleCA3MjU0Nzg3Li4zMjBmN2IyIDEwMDY0NAotLS0gYS9zeXMvZGV2L25ldG1h cC9uZXRtYXBfZ2VuZXJpYy5jCisrKyBiL3N5cy9kZXYvbmV0bWFwL25ldG1hcF9nZW5lcmljLmMK QEAgLTkwLDUzICs5MCw0MCBAQCBfX0ZCU0RJRCgiJEZyZWVCU0Q6IHN0YWJsZS8xMS9zeXMvZGV2 L25ldG1hcC9uZXRtYXBfZ2VuZXJpYy5jIDI5ODk1NSAyMDE2LTA1LTAzCiAvKgogICogRnJlZUJT RCBtYnVmIGFsbG9jYXRvci9kZWFsbG9jYXRvciBpbiBlbXVsYXRpb24gbW9kZToKICAqCi0gKiBX ZSBhbGxvY2F0ZSBFWFRfUEFDS0VUIG1idWYrY2x1c3RlcnMsIGJ1dCBuZWVkIHRvIHNldCBNX05P RlJFRQotICogc28gdGhhdCB0aGUgZGVzdHJ1Y3RvciwgaWYgaW52b2tlZCwgd2lsbCBub3QgZnJl ZSB0aGUgcGFja2V0LgotICogICAgSW4gcHJpbmNpcGxlIHdlIHNob3VsZCBzZXQgdGhlIGRlc3Ry dWN0b3Igb25seSBvbiBkZW1hbmQsCi0gKiBidXQgc2luY2UgdGhlcmUgbWlnaHQgYmUgYSByYWNl IHdlIGJldHRlciBkbyBpdCBvbiBhbGxvY2F0aW9uLgotICogQXMgYSBjb25zZXF1ZW5jZSwgd2Ug YWxzbyBuZWVkIHRvIHNldCB0aGUgZGVzdHJ1Y3RvciBvciB3ZQotICogd291bGQgbGVhayBidWZm ZXJzLgotICovCi0KLS8qCi0gKiBtYnVmIHdyYXBwZXJzCisgKiBXZSBhbGxvY2F0ZSBtYnVmcyB3 aXRoIG1fZ2V0aGRyKCksIHNpbmNlIHRoZSBtYnVmIGhlYWRlciBpcyBuZWVkZWQKKyAqIGJ5IHRo ZSBkcml2ZXIuIFdlIGFsc28gYXR0YWNoIGEgY3VzdG9tbHktcHJvdmlkZWQgZXh0ZXJuYWwgc3Rv cmFnZSwKKyAqIHdoaWNoIGluIHRoaXMgY2FzZSBpcyBhIG5ldG1hcCBidWZmZXIuIFdoZW4gY2Fs bGluZyBtX2V4dGFkZCgpLCBob3dldmVyCisgKiB3ZSBwYXNzIGEgTlVMTCBhZGRyZXNzLCBzaW5j ZSB0aGUgcmVhbCBhZGRyZXNzIChhbmQgbGVuZ3RoKSB3aWxsIGJlCisgKiBmaWxsZWQgaW4gYnkg bm1fb3NfZ2VuZXJpY194bWl0X2ZyYW1lKCkgcmlnaHQgYmVmb3JlIGNhbGxpbmcKKyAqIGlmX3Ry YW5zbWl0KCkuCisgKgorICogVGhlIGR0b3IgZnVuY3Rpb24gZG9lcyBub3RoaW5nLCBob3dldmVy IHdlIG5lZWQgaXQgc2luY2UgbWJfZnJlZV9leHQoKQorICogaGFzIGEgS0FTU0VSVCgpLCBjaGVj a2luZyB0aGF0IHRoZSBtYnVmIGR0b3IgZnVuY3Rpb24gaXMgbm90IE5VTEwuCiAgKi8KIAotLyog bWJ1ZiBkZXN0cnVjdG9yLCBhbHNvIG5lZWQgdG8gY2hhbmdlIHRoZSB0eXBlIHRvIEVYVF9FWFRS RUYsCi0gKiBhZGQgYW4gTV9OT0ZSRUUgZmxhZywgYW5kIHRoZW4gY2xlYXIgdGhlIGZsYWcgYW5k Ci0gKiBjaGFpbiBpbnRvIHVtYV96ZnJlZSh6b25lX3BhY2ssIG1mKQotICogKG9yIHJlaW5zdGFs bCB0aGUgYnVmZmVyID8pCi0gKi8KLSNkZWZpbmUgU0VUX01CVUZfREVTVFJVQ1RPUihtLCBmbikJ ZG8gewkJXAotCShtKS0+bV9leHQuZXh0X2ZyZWUgPSAodm9pZCAqKWZuOwlcCi0JKG0pLT5tX2V4 dC5leHRfdHlwZSA9IEVYVF9FWFRSRUY7CVwKLX0gd2hpbGUgKDApCitzdGF0aWMgdm9pZCB2b2lk X21idWZfZHRvcihzdHJ1Y3QgbWJ1ZiAqbSwgdm9pZCAqYXJnMSwgdm9pZCAqYXJnMikgeyB9CiAK LXN0YXRpYyB2b2lkCi1uZXRtYXBfZGVmYXVsdF9tYnVmX2Rlc3RydWN0b3Ioc3RydWN0IG1idWYg Km0pCitzdGF0aWMgaW5saW5lIHZvaWQKK1NFVF9NQlVGX0RFU1RSVUNUT1Ioc3RydWN0IG1idWYg Km0sIHZvaWQgKmZuKQogewotCS8qIHJlc3RvcmUgb3JpZ2luYWwgbWJ1ZiAqLwotCW0tPm1fZXh0 LmV4dF9idWYgPSBtLT5tX2RhdGEgPSBtLT5tX2V4dC5leHRfYXJnMTsKLQltLT5tX2V4dC5leHRf YXJnMSA9IE5VTEw7Ci0JbS0+bV9leHQuZXh0X3R5cGUgPSBFWFRfUEFDS0VUOwotCW0tPm1fZXh0 LmV4dF9mcmVlID0gTlVMTDsKLQlpZiAoR0VUX01CVUZfUkVGQ05UKG0pID09IDApCi0JCVNFVF9N QlVGX1JFRkNOVChtLCAxKTsKLQl1bWFfemZyZWUoem9uZV9wYWNrLCBtKTsKKwltLT5tX2V4dC5l eHRfZnJlZSA9IGZuID8gZm4gOiAodm9pZCAqKXZvaWRfbWJ1Zl9kdG9yOwogfQogCiBzdGF0aWMg aW5saW5lIHN0cnVjdCBtYnVmICoKIG5ldG1hcF9nZXRfbWJ1ZihpbnQgbGVuKQogewogCXN0cnVj dCBtYnVmICptOwotCW0gPSBtX2dldGNsKE1fTk9XQUlULCBNVF9EQVRBLCBNX1BLVEhEUik7Ci0J aWYgKG0pIHsKLQkJbS0+bV9mbGFncyB8PSBNX05PRlJFRTsJLyogWFhYTlA6IEFsbW9zdCBjZXJ0 YWlubHkgaW5jb3JyZWN0LiAqLwotCQltLT5tX2V4dC5leHRfYXJnMSA9IG0tPm1fZXh0LmV4dF9i dWY7IC8vIFhYWCBzYXZlCi0JCW0tPm1fZXh0LmV4dF9mcmVlID0gKHZvaWQgKiluZXRtYXBfZGVm YXVsdF9tYnVmX2Rlc3RydWN0b3I7Ci0JCW0tPm1fZXh0LmV4dF90eXBlID0gRVhUX0VYVFJFRjsK LQkJTkQoNSwgImNyZWF0ZSBtICVwIHJlZmNudCAlZCIsIG0sIEdFVF9NQlVGX1JFRkNOVChtKSk7 CisKKwkodm9pZClsZW47CisKKwltID0gbV9nZXRoZHIoTV9OT1dBSVQsIE1UX0RBVEEpOworCWlm IChtID09IE5VTEwpIHsKKwkJcmV0dXJuIG07CiAJfQorCisJbV9leHRhZGQobSwgTlVMTCAvKiBi dWYgKi8sIDAgLyogc2l6ZSAqLywgdm9pZF9tYnVmX2R0b3IsCisJCSBOVUxMLCBOVUxMLCAwLCBF WFRfTkVUX0RSVik7CisKIAlyZXR1cm4gbTsKIH0KIApAQCAtNDEyLDExICszOTksNiBAQCBzdGF0 aWMgdm9pZAogZ2VuZXJpY19tYnVmX2Rlc3RydWN0b3Ioc3RydWN0IG1idWYgKm0pCiB7CiAJbmV0 bWFwX2dlbmVyaWNfaXJxKE1CVUZfSUZQKG0pLCBNQlVGX1RYUShtKSwgTlVMTCk7Ci0jaWZkZWYg X19GcmVlQlNEX18KLQlpZiAobmV0bWFwX3ZlcmJvc2UpCi0JCVJEKDUsICJUeCBpcnEgKCVwKSBx dWV1ZSAlZCBpbmRleCAlZCIgLCBtLCBNQlVGX1RYUShtKSwgKGludCkodWludHB0cl90KW0tPm1f ZXh0LmV4dF9hcmcxKTsKLQluZXRtYXBfZGVmYXVsdF9tYnVmX2Rlc3RydWN0b3IobSk7Ci0jZW5k aWYgLyogX19GcmVlQlNEX18gKi8KIAlJRlJBVEUocmF0ZV9jdHgubmV3LnR4aXJxKyspOwogfQog CkBAIC00NDcsNyArNDI5LDcgQEAgZ2VuZXJpY19uZXRtYXBfdHhfY2xlYW4oc3RydWN0IG5ldG1h cF9rcmluZyAqa3JpbmcpCiAJCQkJLy8gWFhYIGhvdyBkbyB3ZSBwcm9jZWVkID8gYnJlYWsgPwog CQkJCXJldHVybiAtRU5PTUVNOwogCQkJfQotCQl9IGVsc2UgaWYgKEdFVF9NQlVGX1JFRkNOVCht KSAhPSAxKSB7CisJCX0gZWxzZSBpZiAoTUJVRl9SRUZDTlQobSkgIT0gMSkgewogCQkJYnJlYWs7 IC8qIFRoaXMgbWJ1ZiBpcyBzdGlsbCBidXN5OiBpdHMgcmVmY250IGlzIDIuICovCiAJCX0KIAkJ bisrOwpAQCAtNDc2LDYyICs0NTgsMzkgQEAgZ2VuZXJpY19uZXRtYXBfdHhfY2xlYW4oc3RydWN0 IG5ldG1hcF9rcmluZyAqa3JpbmcpCiAJcmV0dXJuIG47CiB9CiAKLQotLyoKLSAqIFdlIGhhdmUg cGVuZGluZyBwYWNrZXRzIGluIHRoZSBkcml2ZXIgYmV0d2VlbiBucl9od3RhaWwgKzEgYW5kIGh3 Y3VyLgotICogQ29tcHV0ZSBhIHBvc2l0aW9uIGluIHRoZSBtaWRkbGUsIHRvIGJlIHVzZWQgdG8g Z2VuZXJhdGUKLSAqIGEgbm90aWZpY2F0aW9uLgotICovCi1zdGF0aWMgaW5saW5lIHVfaW50Ci1n ZW5lcmljX3R4X2V2ZW50X21pZGRsZShzdHJ1Y3QgbmV0bWFwX2tyaW5nICprcmluZywgdV9pbnQg aHdjdXIpCi17Ci0JdV9pbnQgbiA9IGtyaW5nLT5ua3JfbnVtX3Nsb3RzOwotCXVfaW50IG50YyA9 IG5tX25leHQoa3JpbmctPm5yX2h3dGFpbCwgbi0xKTsKLQl1X2ludCBlOwotCi0JaWYgKGh3Y3Vy ID49IG50YykgewotCQllID0gKGh3Y3VyICsgbnRjKSAvIDI7Ci0JfSBlbHNlIHsgLyogd3JhcCBh cm91bmQgKi8KLQkJZSA9IChod2N1ciArIG4gKyBudGMpIC8gMjsKLQkJaWYgKGUgPj0gbikgewot CQkJZSAtPSBuOwotCQl9Ci0JfQotCi0JaWYgKHVubGlrZWx5KGUgPj0gbikpIHsKLQkJRCgiVGhp cyBjYW5ub3QgaGFwcGVuIik7Ci0JCWUgPSAwOwotCX0KLQotCXJldHVybiBlOwotfQotCi0vKgot ICogV2UgaGF2ZSBwZW5kaW5nIHBhY2tldHMgaW4gdGhlIGRyaXZlciBiZXR3ZWVuIG5yX2h3dGFp bCsxIGFuZCBod2N1ci4KLSAqIFNjaGVkdWxlIGEgbm90aWZpY2F0aW9uIGFwcHJveGltYXRlbHkg aW4gdGhlIG1pZGRsZSBvZiB0aGUgdHdvLgotICogVGhlcmUgaXMgYSByYWNlIGJ1dCB0aGlzIGlz IG9ubHkgY2FsbGVkIHdpdGhpbiB0eHN5bmMgd2hpY2ggZG9lcwotICogYSBkb3VibGUgY2hlY2su Ci0gKi8KIHN0YXRpYyB2b2lkCiBnZW5lcmljX3NldF90eF9ldmVudChzdHJ1Y3QgbmV0bWFwX2ty aW5nICprcmluZywgdV9pbnQgaHdjdXIpCiB7CisJdV9pbnQgbGltID0ga3JpbmctPm5rcl9udW1f c2xvdHMgLSAxOwogCXN0cnVjdCBtYnVmICptOwogCXVfaW50IGU7CisJdV9pbnQgbnRjID0gbm1f bmV4dChrcmluZy0+bnJfaHd0YWlsLCBsaW0pOyAvKiBuZXh0IHRvIGNsZWFuICovCiAKLQlpZiAo bm1fbmV4dChrcmluZy0+bnJfaHd0YWlsLCBrcmluZy0+bmtyX251bV9zbG90cyAtMSkgPT0gaHdj dXIpIHsKKwlpZiAobnRjID09IGh3Y3VyKSB7CiAJCXJldHVybjsgLyogYWxsIGJ1ZmZlcnMgYXJl IGZyZWUgKi8KIAl9Ci0JZSA9IGdlbmVyaWNfdHhfZXZlbnRfbWlkZGxlKGtyaW5nLCBod2N1cik7 CisKKwkvKgorCSAqIFdlIGhhdmUgcGVuZGluZyBwYWNrZXRzIGluIHRoZSBkcml2ZXIgYmV0d2Vl biBod3RhaWwrMQorCSAqIGFuZCBod2N1ciwgYW5kIHdlIGhhdmUgdG8gY2hvc2Ugb25lIG9mIHRo ZXNlIHNsb3QgdG8KKwkgKiBnZW5lcmF0ZSBhIG5vdGlmaWNhdGlvbi4KKwkgKiBUaGVyZSBpcyBh IHJhY2UgYnV0IHRoaXMgaXMgb25seSBjYWxsZWQgd2l0aGluIHR4c3luYyB3aGljaAorCSAqIGRv ZXMgYSBkb3VibGUgY2hlY2suCisJICovCisKKwkvKiBDaG9vc2UgdGhlIGZpcnN0IHBlbmRpbmcg c2xvdCwgdG8gYmUgc2FmZSBhZ2FpbnN0IGRyaXZlcgorCSAqIHJlb3JkZXJpbmcgbWJ1ZiB0cmFu c21pc3Npb25zLiAqLworCWUgPSBudGM7CiAKIAltID0ga3JpbmctPnR4X3Bvb2xbZV07Ci0JTkQo NSwgIlJlcXVlc3QgRXZlbnQgYXQgJWQgbWJ1ZiAlcCByZWZjbnQgJWQiLCBlLCBtLCBtID8gR0VU X01CVUZfUkVGQ05UKG0pIDogLTIgKTsKKwlORCg1LCAiUmVxdWVzdCBFdmVudCBhdCAlZCBtYnVm ICVwIHJlZmNudCAlZCIsIGUsIG0sIG0gPyBNQlVGX1JFRkNOVChtKSA6IC0yICk7CiAJaWYgKG0g PT0gTlVMTCkgewogCQkvKiBUaGlzIGNhbiBoYXBwZW4gaWYgdGhlcmUgaXMgYWxyZWFkeSBhbiBl dmVudCBvbiB0aGUgbmV0bWFwCiAJCSAgIHNsb3QgJ2UnOiBUaGVyZSBpcyBub3RoaW5nIHRvIGRv LiAqLwogCQlyZXR1cm47CiAJfQogCWtyaW5nLT50eF9wb29sW2VdID0gTlVMTDsKLQlTRVRfTUJV Rl9ERVNUUlVDVE9SKG0sIGdlbmVyaWNfbWJ1Zl9kZXN0cnVjdG9yKTsKKwlTRVRfTUJVRl9ERVNU UlVDVE9SKG0sICh2b2lkICopZ2VuZXJpY19tYnVmX2Rlc3RydWN0b3IpOwogCiAJLy8gWFhYIHdt YigpID8KIAkvKiBEZWNyZW1lbnQgdGhlIHJlZmNvdW50IGFuIGZyZWUgaXQgaWYgd2UgaGF2ZSB0 aGUgbGFzdCBvbmUuICovCmRpZmYgLS1naXQgYS9zeXMvZGV2L25ldG1hcC9uZXRtYXBfa2Vybi5o IGIvc3lzL2Rldi9uZXRtYXAvbmV0bWFwX2tlcm4uaAppbmRleCAzZDBmOWQ5Li45ODUxOGRlIDEw MDY0NAotLS0gYS9zeXMvZGV2L25ldG1hcC9uZXRtYXBfa2Vybi5oCisrKyBiL3N5cy9kZXYvbmV0 bWFwL25ldG1hcF9rZXJuLmgKQEAgLTk3LDEzICs5NywxMSBAQCBzdHJ1Y3QgbmV0bWFwX2FkYXB0 ZXIgKm5ldG1hcF9nZXRuYShpZl90IGlmcCk7CiAjZW5kaWYKIAogI2lmIF9fRnJlZUJTRF92ZXJz aW9uID49IDExMDAwMjcKLSNkZWZpbmUgR0VUX01CVUZfUkVGQ05UKG0pICAgICAgKChtKS0+bV9l eHQuZXh0X2NudCA/ICooKG0pLT5tX2V4dC5leHRfY250KSA6IC0xKQotI2RlZmluZSBTRVRfTUJV Rl9SRUZDTlQobSwgeCkgICAqKChtKS0+bV9leHQuZXh0X2NudCkgPSB4Ci0jZGVmaW5lIFBOVF9N QlVGX1JFRkNOVChtKSAgICAgICgobSktPm1fZXh0LmV4dF9jbnQpCisjZGVmaW5lIE1CVUZfUkVG Q05UKG0pCQkoKG0pLT5tX2V4dC5leHRfY291bnQpCisjZGVmaW5lIFNFVF9NQlVGX1JFRkNOVCht LCB4KSAgIChtKS0+bV9leHQuZXh0X2NvdW50ID0geAogI2Vsc2UKLSNkZWZpbmUgR0VUX01CVUZf UkVGQ05UKG0pICAgICAgKChtKS0+bV9leHQucmVmX2NudCA/ICooKG0pLT5tX2V4dC5yZWZfY250 KSA6IC0xKQorI2RlZmluZSBNQlVGX1JFRkNOVChtKQkJKChtKS0+bV9leHQucmVmX2NudCA/ICoo KG0pLT5tX2V4dC5yZWZfY250KSA6IC0xKQogI2RlZmluZSBTRVRfTUJVRl9SRUZDTlQobSwgeCkg ICAqKChtKS0+bV9leHQucmVmX2NudCkgPSB4Ci0jZGVmaW5lIFBOVF9NQlVGX1JFRkNOVChtKSAg ICAgICgobSktPm1fZXh0LnJlZl9jbnQpCiAjZW5kaWYKIAogTUFMTE9DX0RFQ0xBUkUoTV9ORVRN QVApOwo= --001a114152c2f51c52054675462a-- From owner-freebsd-net@freebsd.org Thu Jan 19 22:55:06 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2C7DACB8F04 for ; Thu, 19 Jan 2017 22:55:06 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from NAM01-SN1-obe.outbound.protection.outlook.com (mail-sn1nam01on0082.outbound.protection.outlook.com [104.47.32.82]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT SSL SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C7BD91215 for ; Thu, 19 Jan 2017 22:55:04 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from YTXPR01MB0189.CANPRD01.PROD.OUTLOOK.COM (10.165.218.133) by YTXPR01MB0191.CANPRD01.PROD.OUTLOOK.COM (10.165.218.135) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.845.12; Thu, 19 Jan 2017 22:55:01 +0000 Received: from YTXPR01MB0189.CANPRD01.PROD.OUTLOOK.COM ([10.165.218.133]) by YTXPR01MB0189.CANPRD01.PROD.OUTLOOK.COM ([10.165.218.133]) with mapi id 15.01.0845.021; Thu, 19 Jan 2017 22:55:01 +0000 From: Rick Macklem To: FreeBSD Net Subject: Re: sosend returning ERESTART Thread-Topic: sosend returning ERESTART Thread-Index: AQHScXbGB6xwqeuR3ky6twdHVMq016E+x0oAgAAO/x2AAJOyAIAA/rNAgAACVQg= Date: Thu, 19 Jan 2017 22:55:01 +0000 Message-ID: References: <01000159aac969e6-b2fc3913-d04e-42d4-befd-402ed0d830bf-000000@email.amazonses.com> <20170117100634.GS2349@kib.kiev.ua> <01000159afddb7ce-064a5d17-4b81-4b2c-a9b4-3ddd2ad2e377-000000@email.amazonses.com> <20170118103650.GE2349@kib.kiev.ua> <01000159b390c409-5adcb488-67e8-4038-b9b0-5d4f33460205-000000@email.amazonses.com> , <20170119073424.GM2349@kib.kiev.ua>, In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=rmacklem@uoguelph.ca; x-ms-office365-filtering-correlation-id: cb811a13-bd5b-44d2-95fa-08d440be333c x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:(22001);SRVR:YTXPR01MB0191; x-microsoft-exchange-diagnostics: 1; YTXPR01MB0191; 7:l3ieu1pVrPB1zfl9xRxFyP73Fv18FZaPTCx9z1RNUs69/ZkSfqpuqc9Ma6Wx1PSPkag7NsLeXba+U8xs7qm15dHshNO6PYLJ3ZoQ2ejVy8RWPCc4mmA/D1iaYEuwGN9uQ/geys3ZGV908UwXXzEGXXArAZSfAExRQf9Q/7x0k6t89BCt45V02C51Asl1GaF/uEFAb4PEF2/6u7n6ziNvhlP4lBq6HgPWlULWWixu8C0WOEcjFiueXk5KighNVNhVByyVle/ZDEFEc0iJCyKUj5HdVlHgI5g3/YXs/51qEoZ9boG7p/eGo0PfwM9lSUz1B/VCD0b/nm+AdjG/oKU7caodJ2CbgKFg5b4aS7b1VwUHs4BDMeYdnwy3JG55SPyspUWeWiCJbpj9HhCrq6w2V/gELAe2aVzp0yCamCnL56GIxmwyDwXUxC0l2+2SAYpB48nLaVyovx1D2bRCkWAYKQ== x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(158342451672863); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040375)(2401047)(5005006)(8121501046)(3002001)(10201501046)(6041248)(20161123562025)(20161123564025)(20161123560025)(20161123555025)(6072148); SRVR:YTXPR01MB0191; BCL:0; PCL:0; RULEID:; SRVR:YTXPR01MB0191; x-forefront-prvs: 0192E812EC x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(979002)(6009001)(7916002)(39450400003)(189002)(199003)(24454002)(81166006)(81156014)(106356001)(8676002)(8936002)(3480700004)(38730400001)(33656002)(50986999)(105586002)(122556002)(9686003)(6436002)(102836003)(76176999)(54356999)(74482002)(6506006)(77096006)(55016002)(101416001)(450100001)(106116001)(229853002)(2906002)(92566002)(7696004)(110136003)(2950100002)(6916009)(74316002)(3660700001)(3280700002)(305945005)(2900100001)(107886002)(97736004)(189998001)(5660300001)(93886004)(86362001)(53936002)(7116003)(68736007)(969003)(989001)(999001)(1009001)(1019001); DIR:OUT; SFP:1101; SCL:1; SRVR:YTXPR01MB0191; H:YTXPR01MB0189.CANPRD01.PROD.OUTLOOK.COM; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; received-spf: None (protection.outlook.com: uoguelph.ca does not designate permitted sender hosts) spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-originalarrivaltime: 19 Jan 2017 22:55:01.2444 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-Transport-CrossTenantHeadersStamped: YTXPR01MB0191 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jan 2017 22:55:06 -0000 Konstantin Belousov wrote: >On Wed, Jan 18, 2017 at 10:52:02PM +0000, Rick Macklem wrote: >> Colin Percival wrote: >> >On 01/18/17 02:36, Konstantin Belousov wrote: >> >> On Wed, Jan 18, 2017 at 04:37:40AM +0000, Colin Percival wrote: >> >>> Thanks, looks like that was exactly it -- if the TCP send buffer was= full >> >>> we would call sbwait, and if a signal arrived it would return ERESTA= RT. >> >>> It looks like setting the SB_NOINTR flag will prevent this; I'm test= ing a >> >>> patch right now. >> >> >> >> Note that passing SB_NOINTR unconditionally or even only for mounts >> >> with nointr (default) option is wrong. You make the socket operation >> >> uninterruptible, process terminate-ability becomes depended on the >> >> external factor, the behaviour of the remote system. >> > >> >I'm not sure what you're getting at here. The fact that "NFS mounted w= ithout >> >the intr flag" + "unresponsive NFS server" =3D "unkillable processes" h= as been >> >a (mis)feature of NFS for decades. >> The case I would like to see work is the forced dismount. I need to go l= ook at >> what it does and see if SB_NOINTR would break it worse than it is broken= now. >> (It is currently broken when something like "umount" without -f is done,= which >> locks up the mounted on vnode so "umount -f" never gets to the umount(2= ) syscall. >> I do plan on a "straight ot NFS" option for umount(8) to avoid this pro= blem, but >> haven't gotten around to it.) >> >> The alternative to SB_NOINTR is looping and doing the sosend() again for= the >> case where it returns ERESTART and "intr" wasn't set on the mount. >Note that the condition of pending signal which triggered ERESTART is >permanent until the signal is delivered or blocked. In other words, or >future PCATCH sleeps will fail with ERESTART/EINTR. Right. But presumably if the TCP connection is still working, a subsequent attempt will not have to sleep in sblock() or sbwait() in sosend() and will succeed? I think Colin was already testing this looping version before SB_NOINTR and found it worked well for his case. --> I think this does imply that it should only loop N times and then give = up and reply RPC_CANTSEND (which is what it does the first time now). - The RPC_CANTSEND is what triggers the client to create a new TCP co= nnection and this is what causes grief for his mounts against the AmazonEFS = server (which is broken because the new TCP connection often results in a NFS4ERR_BAD_SESSION which should not happen.) Colin, have you tested the "loop on ERESTART" version of the patch? And maybe you could add a loop counter to limit the number of iterations? rick [stuff snipped] From owner-freebsd-net@freebsd.org Thu Jan 19 22:55:52 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 54137CB8F80 for ; Thu, 19 Jan 2017 22:55:52 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from NAM01-SN1-obe.outbound.protection.outlook.com (mail-sn1nam01on0069.outbound.protection.outlook.com [104.47.32.69]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT SSL SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0205E12DB for ; Thu, 19 Jan 2017 22:55:51 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from YTXPR01MB0189.CANPRD01.PROD.OUTLOOK.COM (10.165.218.133) by YTXPR01MB0191.CANPRD01.PROD.OUTLOOK.COM (10.165.218.135) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.845.12; Thu, 19 Jan 2017 22:55:50 +0000 Received: from YTXPR01MB0189.CANPRD01.PROD.OUTLOOK.COM ([10.165.218.133]) by YTXPR01MB0189.CANPRD01.PROD.OUTLOOK.COM ([10.165.218.133]) with mapi id 15.01.0845.021; Thu, 19 Jan 2017 22:55:50 +0000 From: Rick Macklem To: FreeBSD Net Subject: Re: sosend returning ERESTART Thread-Topic: sosend returning ERESTART Thread-Index: AQHScXbGB6xwqeuR3ky6twdHVMq016E+x0oAgAAO/x2AAJOyAIAA/rNAgAACmls= Date: Thu, 19 Jan 2017 22:55:49 +0000 Message-ID: References: <01000159aac969e6-b2fc3913-d04e-42d4-befd-402ed0d830bf-000000@email.amazonses.com> <20170117100634.GS2349@kib.kiev.ua> <01000159afddb7ce-064a5d17-4b81-4b2c-a9b4-3ddd2ad2e377-000000@email.amazonses.com> <20170118103650.GE2349@kib.kiev.ua> <01000159b390c409-5adcb488-67e8-4038-b9b0-5d4f33460205-000000@email.amazonses.com> , <20170119073424.GM2349@kib.kiev.ua>, In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=rmacklem@uoguelph.ca; x-ms-office365-filtering-correlation-id: 32a7f463-eaa1-4016-47cc-08d440be5053 x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:(22001);SRVR:YTXPR01MB0191; x-microsoft-exchange-diagnostics: 1; YTXPR01MB0191; 7:Buo3XuLb0USOL4l9tq3sTIwf4UVvISXYtM+WkKhwlxsocQ89NSeSUwusiYyEUvc6zgQ6nJGFNmqOn6dJmHMlCIT0mg/yIF2szdU35hI4xAF5YYqyFUEl0DEsMxM+WHsg8j4cv8K0BRwNpSwgO1rTVWb4utHWNBWTCXLjWvuy0d3llyC97Lw+oKlFWZz7RQRRovH5MPMWNCBdMkMCTyUVYWvcCDTNh5t5fG15pjhlXoXercESbXYQMxK9b+I7nOAAIX+cRFTLntVY3tZZhc5HgAdh+vwpD6tCEK0vqSM5SIEqg0N7BYiMcm6ZgU+mYlKW65MS5TOCiLehRACWjciJYtSnRZYA2CHW3HLHhNxSjPM5SmcZugsIf3wOiaEXI5Tbef8L2Yf7quZH+hl/UnXTL6iu9EUCuBDdUHH7UJqH4nfzNWCZDcXIsvIVtV8Ht0QssaZoAYPDB4fkA2VEASKEtw== x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(158342451672863); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040375)(2401047)(5005006)(8121501046)(3002001)(10201501046)(6041248)(20161123562025)(20161123564025)(20161123560025)(20161123555025)(6072148); SRVR:YTXPR01MB0191; BCL:0; PCL:0; RULEID:; SRVR:YTXPR01MB0191; x-forefront-prvs: 0192E812EC x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(6009001)(7916002)(39450400003)(189002)(199003)(24454002)(81166006)(81156014)(106356001)(8676002)(8936002)(3480700004)(38730400001)(33656002)(50986999)(105586002)(122556002)(9686003)(6436002)(102836003)(76176999)(54356999)(74482002)(6506006)(77096006)(55016002)(101416001)(450100001)(106116001)(229853002)(2906002)(92566002)(7696004)(110136003)(2950100002)(6916009)(74316002)(3660700001)(3280700002)(305945005)(2900100001)(107886002)(97736004)(189998001)(5660300001)(93886004)(86362001)(53936002)(7116003)(68736007); DIR:OUT; SFP:1101; SCL:1; SRVR:YTXPR01MB0191; H:YTXPR01MB0189.CANPRD01.PROD.OUTLOOK.COM; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; received-spf: None (protection.outlook.com: uoguelph.ca does not designate permitted sender hosts) spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-originalarrivaltime: 19 Jan 2017 22:55:50.0000 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-Transport-CrossTenantHeadersStamped: YTXPR01MB0191 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jan 2017 22:55:52 -0000 Konstantin Belousov wrote: >On Wed, Jan 18, 2017 at 10:52:02PM +0000, Rick Macklem wrote: >> Colin Percival wrote: >> >On 01/18/17 02:36, Konstantin Belousov wrote: >> >> On Wed, Jan 18, 2017 at 04:37:40AM +0000, Colin Percival wrote: >> >>> Thanks, looks like that was exactly it -- if the TCP send buffer was= full >> >>> we would call sbwait, and if a signal arrived it would return ERESTA= RT. >> >>> It looks like setting the SB_NOINTR flag will prevent this; I'm test= ing a >> >>> patch right now. >> >> >> >> Note that passing SB_NOINTR unconditionally or even only for mounts >> >> with nointr (default) option is wrong. You make the socket operation >> >> uninterruptible, process terminate-ability becomes depended on the >> >> external factor, the behaviour of the remote system. >> > >> >I'm not sure what you're getting at here. The fact that "NFS mounted w= ithout >> >the intr flag" + "unresponsive NFS server" =3D "unkillable processes" h= as been >> >a (mis)feature of NFS for decades. >> The case I would like to see work is the forced dismount. I need to go l= ook at >> what it does and see if SB_NOINTR would break it worse than it is broken= now. >> (It is currently broken when something like "umount" without -f is done,= which >> locks up the mounted on vnode so "umount -f" never gets to the umount(2= ) syscall. >> I do plan on a "straight ot NFS" option for umount(8) to avoid this pro= blem, but >> haven't gotten around to it.) >> >> The alternative to SB_NOINTR is looping and doing the sosend() again for= the >> case where it returns ERESTART and "intr" wasn't set on the mount. >Note that the condition of pending signal which triggered ERESTART is >permanent until the signal is delivered or blocked. In other words, or >future PCATCH sleeps will fail with ERESTART/EINTR. Right. But presumably if the TCP connection is still working, a subsequent attempt will not have to sleep in sblock() or sbwait() in sosend() and will succeed? I think Colin was already testing this looping version before SB_NOINTR and found it worked well for his case. --> I think this does imply that it should only loop N times and then give = up and reply RPC_CANTSEND (which is what it does the first time now). - The RPC_CANTSEND is what triggers the client to create a new TCP co= nnection and this is what causes grief for his mounts against the AmazonEFS = server (which is broken because the new TCP connection often results in a NFS4ERR_BAD_SESSION which should not happen.) Colin, have you tested the "loop on ERESTART" version of the patch? And maybe you could add a loop counter to limit the number of iterations? rick [stuff snipped] From owner-freebsd-net@freebsd.org Thu Jan 19 22:58:11 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B55A7CB8047 for ; Thu, 19 Jan 2017 22:58:11 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from NAM01-SN1-obe.outbound.protection.outlook.com (mail-sn1nam01on0046.outbound.protection.outlook.com [104.47.32.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT SSL SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 69A021439 for ; Thu, 19 Jan 2017 22:58:11 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from YTXPR01MB0189.CANPRD01.PROD.OUTLOOK.COM (10.165.218.133) by YTXPR01MB0191.CANPRD01.PROD.OUTLOOK.COM (10.165.218.135) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.845.12; Thu, 19 Jan 2017 22:58:08 +0000 Received: from YTXPR01MB0189.CANPRD01.PROD.OUTLOOK.COM ([10.165.218.133]) by YTXPR01MB0189.CANPRD01.PROD.OUTLOOK.COM ([10.165.218.133]) with mapi id 15.01.0845.021; Thu, 19 Jan 2017 22:58:08 +0000 From: Rick Macklem To: FreeBSD Net Subject: re: sosend returning ERESTART Thread-Topic: sosend returning ERESTART Thread-Index: AQHScqeAB6xwqeuR3ky6twdHVMq01w== Date: Thu, 19 Jan 2017 22:58:08 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=rmacklem@uoguelph.ca; x-ms-office365-filtering-correlation-id: f2eaffc0-0791-4ff0-b408-08d440bea2e1 x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:(22001);SRVR:YTXPR01MB0191; x-microsoft-exchange-diagnostics: 1; YTXPR01MB0191; 7:ni2KRYGAb15kbgWUNugvTZkD9i5TQDr24o7NHmcrZlwxIAwm361RQ86Sqxtu7FflCg4qPx6kXmfneVvNBRdbOeVs413iRGtuDpSu4rvxYPyLNMIi3y1Sau73LVxqYVbGA9LM1IFKkFVi7pMV4wtq0olBZB5FVKb66A7t7/efozOfoCMeNgFeQ1IBCsS1cirZLlePTTFYkx9AYS/G68RitPX0QIZJM71uKWnd6sKOhsKhBmmB1nN6mzetN+nuDtzusozNUEQjPSnnPrvxdr4Dnd1uzAdI0Weu9YY0rLn8dii20kRQkJ+6SmJgq3LMgPbPMmR9t09jozUAKURjo/HlJVnUhdBY9gKg+tACZJlY5taF8PDvnIUMVz4Pjo7VQMQ1kgoVXmHSSDFc+5XT5VLjdMNWcn7fZeJZ+RHLG2qupRXPfpjtpRKJIPQRY1ZWJ41SsS+dzpn/EO4TvjHTFBzi7w== x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(158342451672863); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040375)(2401047)(5005006)(8121501046)(3002001)(10201501046)(6041248)(20161123562025)(20161123564025)(20161123560025)(20161123555025)(6072148); SRVR:YTXPR01MB0191; BCL:0; PCL:0; RULEID:; SRVR:YTXPR01MB0191; x-forefront-prvs: 0192E812EC x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(6009001)(7916002)(39450400003)(199003)(24454002)(189002)(3280700002)(74316002)(3660700001)(107886002)(305945005)(2900100001)(7696004)(110136003)(2906002)(92566002)(6916009)(68736007)(5660300001)(189998001)(97736004)(53936002)(7116003)(86362001)(105586002)(33656002)(50986999)(6506006)(102836003)(122556002)(9686003)(6436002)(54356999)(74482002)(8676002)(81166006)(106356001)(81156014)(3480700004)(38730400001)(8936002)(106116001)(450100001)(229853002)(77096006)(101416001)(55016002); DIR:OUT; SFP:1101; SCL:1; SRVR:YTXPR01MB0191; H:YTXPR01MB0189.CANPRD01.PROD.OUTLOOK.COM; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; received-spf: None (protection.outlook.com: uoguelph.ca does not designate permitted sender hosts) spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-originalarrivaltime: 19 Jan 2017 22:58:08.5215 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-Transport-CrossTenantHeadersStamped: YTXPR01MB0191 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jan 2017 22:58:11 -0000 Konstantin Belousov wrote: >On Wed, Jan 18, 2017 at 10:52:02PM +0000, Rick Macklem wrote: >> Colin Percival wrote: >> >On 01/18/17 02:36, Konstantin Belousov wrote: >> >> On Wed, Jan 18, 2017 at 04:37:40AM +0000, Colin Percival wrote: >> >>> Thanks, looks like that was exactly it -- if the TCP send buffer was= full >> >>> we would call sbwait, and if a signal arrived it would return ERESTA= RT. >> >>> It looks like setting the SB_NOINTR flag will prevent this; I'm test= ing a >> >>> patch right now. >> >> >> >> Note that passing SB_NOINTR unconditionally or even only for mounts >> >> with nointr (default) option is wrong. You make the socket operation >> >> uninterruptible, process terminate-ability becomes depended on the >> >> external factor, the behaviour of the remote system. >> > >> >I'm not sure what you're getting at here. The fact that "NFS mounted w= ithout >> >the intr flag" + "unresponsive NFS server" =3D "unkillable processes" h= as been >> >a (mis)feature of NFS for decades. >> The case I would like to see work is the forced dismount. I need to go l= ook at >> what it does and see if SB_NOINTR would break it worse than it is broken= now. >> (It is currently broken when something like "umount" without -f is done,= which >> locks up the mounted on vnode so "umount -f" never gets to the umount(2= ) syscall. >> I do plan on a "straight ot NFS" option for umount(8) to avoid this pro= blem, but >> haven't gotten around to it.) >> >> The alternative to SB_NOINTR is looping and doing the sosend() again for= the >> case where it returns ERESTART and "intr" wasn't set on the mount. >Note that the condition of pending signal which triggered ERESTART is >permanent until the signal is delivered or blocked. In other words, or >future PCATCH sleeps will fail with ERESTART/EINTR. Right. But presumably if the TCP connection is still working, a subsequent attempt will not have to sleep in sblock() or sbwait() in sosend() and will succeed? I think Colin was already testing this looping version before SB_NOINTR and found it worked well for his case. --> I think this does imply that it should only loop N times and then give = up and reply RPC_CANTSEND (which is what it does the first time now). - The RPC_CANTSEND is what triggers the client to create a new TCP co= nnection and this is what causes grief for his mounts against the AmazonEFS = server (which is broken because the new TCP connection often results in a NFS4ERR_BAD_SESSION which should not happen.) Colin, have you tested the "loop on ERESTART" version of the patch? And maybe you could add a loop counter to limit the number of iterations? rick [stuff snipped] From owner-freebsd-net@freebsd.org Fri Jan 20 03:58:07 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A019BCB8AF0 for ; Fri, 20 Jan 2017 03:58:07 +0000 (UTC) (envelope-from kevin.bowling@kev009.com) Received: from mail-ot0-x22e.google.com (mail-ot0-x22e.google.com [IPv6:2607:f8b0:4003:c0f::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4E7B31969 for ; Fri, 20 Jan 2017 03:58:07 +0000 (UTC) (envelope-from kevin.bowling@kev009.com) Received: by mail-ot0-x22e.google.com with SMTP id 73so47632532otj.0 for ; Thu, 19 Jan 2017 19:58:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kev009.com; s=google; h=mime-version:from:date:message-id:subject:to:cc; bh=OCUQlWBfDXyMnOgCl5QZuI+vUoDSet88uvTc59ttVak=; b=KjWI9n+n+iEh63Gd6dhjF91UU+x5eHc4aFqmsZZr5A/RB9VOTmCOpiAboN6UOIhgBq RUkB8NeNcXdkLfuMdII926fsIsE7WyhgJ33hz5UHJWur4xAiefgefzKvluOHEbxA7WpB birYY0Ek0D8W5ROTQWPhMlnsr4jb30iVEUp8I= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to:cc; bh=OCUQlWBfDXyMnOgCl5QZuI+vUoDSet88uvTc59ttVak=; b=CANh5lyJ5BS8DgebotQilVsVieHpniV7Q0565uBjGYa1FGTg8iUKikoanxhmOZSYDw TGqYjhWXvSkh9u8OCF6W5Kww5Rci7l7M/khB+J1bU7u34zgoUkyVUyB7/c1/1Rp+PHep VoidcJpjta1AtB1d42c5eODgHvUN7uk1QgB7+8jVwxOD8X/IXcQ8v8KaFZupIgefxN2W wmkpU66Gl5nyeQRRJmuv/G02ISbjyYoNBsFcqeBEmMPkMk25ndixr95IqNOZnS8koUBo ZefZEMl18BOBJNGKkvOGoR8haEf13JOroQO1RZZqm7Ma3FVdL5cSxFIRXkRAm+9ocT0Y svAg== X-Gm-Message-State: AIkVDXJBJuUNqG6dH/t9JlselvGT0zwxc9/9WQVQz291k6XhkMorpTPObqNRMtTANFVKgYg0cFG824t8DFtnpw== X-Received: by 10.157.54.206 with SMTP id s14mr5572574otd.59.1484884686576; Thu, 19 Jan 2017 19:58:06 -0800 (PST) MIME-Version: 1.0 Received: by 10.157.23.208 with HTTP; Thu, 19 Jan 2017 19:58:05 -0800 (PST) From: Kevin Bowling Date: Thu, 19 Jan 2017 20:58:05 -0700 Message-ID: Subject: RFC: ethctl To: freebsd-net@freebsd.org Cc: Scott Long , Drew Gallatin , Navdeep Parhar , Oded Shanoon , hps@freebsd.org, Matthew Macy , jeb.j.cramer@intel.com, eric.joyner@intel.com, arybchik@freebsd.org, shurd@freebsd.org, Sean Bruno , George Neville-Neil X-Mailman-Approved-At: Fri, 20 Jan 2017 04:07:40 +0000 Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jan 2017 03:58:07 -0000 Greetings, I'm casting a wide net in cc, try to keep the noise minimal but we need some input from a variety of HW vendors. I have heard from several vendors the need for a NIC configuration tool. Chelsio ships a cxgb/cxgbetool in FreeBSD as one example. There is precedence for some nod toward a basic unified tool in Linux ethtool. >From your perspective, 1) What are the common requirements? 2) What are specialized requirements? For instance as a full TCP offload card Chelsio needs things others wont 3) What should it _not_ do? Several of you have experience doing Ethernet driver dev on many platforms so we should attempt to avoid repeating past design mistakes. I expect we can achieve some level of inversion so the device specific code can live close to the driver and plug into the ethctl framework. It should be general enough to add completely new top level commands, so vendors can implement HW specific features. On the other hand, we should attempt to hook into common core for features every NIC provides, with a focus on iflib. I will fund Matt Macy to do the overall design and implementation. Regards, Kevin Bowling, on behalf of Limelight Networks for this effort From owner-freebsd-net@freebsd.org Fri Jan 20 08:42:50 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6ADC6CB85D8 for ; Fri, 20 Jan 2017 08:42:50 +0000 (UTC) (envelope-from bakul@bitblocks.com) Received: from mail.bitblocks.com (ns1.bitblocks.com [173.228.5.8]) by mx1.freebsd.org (Postfix) with ESMTP id 5D3D41BF7 for ; Fri, 20 Jan 2017 08:42:50 +0000 (UTC) (envelope-from bakul@bitblocks.com) Received: from bitblocks.com (localhost [127.0.0.1]) by mail.bitblocks.com (Postfix) with ESMTP id ACCF9124AEA4 for ; Fri, 20 Jan 2017 00:35:55 -0800 (PST) To: freebsd-net@freebsd.org Subject: pf & NAT issue Date: Fri, 20 Jan 2017 00:35:55 -0800 From: Bakul Shah Message-Id: <20170120083555.ACCF9124AEA4@mail.bitblocks.com> X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jan 2017 08:42:50 -0000 pf seems to drop NAT connections quite a bit. This seems to happen much more frequently if there are delays involved (slow server or interactive use). Almost seems like pf losing track of NATted connections due to an uninitialized variable.... Often a retry or two works. Connecting from outside to forwarded connections to NATTED hosts works fine. This problem started after ungrading to freebsd-10. Is there a bug fix in works or a known work around (other than using ipfw or reverting to 9, which I don't want to do)? Thanks! Bakul From owner-freebsd-net@freebsd.org Fri Jan 20 10:48:06 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0C1C6CB9756 for ; Fri, 20 Jan 2017 10:48:06 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from venus.codepro.be (venus.codepro.be [IPv6:2a01:4f8:162:1127::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.codepro.be", Issuer "Gandi Standard SSL CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CF25D1631 for ; Fri, 20 Jan 2017 10:48:05 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from [192.168.228.1] (vega.codepro.be [IPv6:2a01:4f8:162:1127::3]) (Authenticated sender: kp) by venus.codepro.be (Postfix) with ESMTPSA id C4C4535A55; Fri, 20 Jan 2017 11:48:02 +0100 (CET) From: "Kristof Provost" To: "Bakul Shah" Cc: freebsd-net@freebsd.org Subject: Re: pf & NAT issue Date: Fri, 20 Jan 2017 11:48:04 +0100 Message-ID: <7C29D00C-94C0-4550-B1B2-CE307482B544@FreeBSD.org> In-Reply-To: <20170120083555.ACCF9124AEA4@mail.bitblocks.com> References: <20170120083555.ACCF9124AEA4@mail.bitblocks.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-Mailer: MailMate (2.0BETAr6072) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jan 2017 10:48:06 -0000 On 20 Jan 2017, at 9:35, Bakul Shah wrote: > pf seems to drop NAT connections quite a bit. This seems to > happen much more frequently if there are delays involved (slow > server or interactive use). Almost seems like pf losing > track of NATted connections due to an uninitialized > variable.... Often a retry or two works. Connecting from > outside to forwarded connections to NATTED hosts works fine. > > This problem started after ungrading to freebsd-10. Is there a > bug fix in works or a known work around (other than using ipfw > or reverting to 9, which I don't want to do)? > The problem you describe doesn’t immediately ring a bell. We’ll have to gather a bit more information: * What FreeBSD version are you running exactly? * What’s your pf.conf? * Can you perform a network capture of rejected/failed connections? Ideally both on LAN and WAN on the gateway machine. Please capture full packets (so tcpdump -s0 -w lan.pcap) as pcap files). * What networking cards are you using? Regards, Kristof From owner-freebsd-net@freebsd.org Fri Jan 20 09:10:07 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9461FCB8EC3 for ; Fri, 20 Jan 2017 09:10:07 +0000 (UTC) (envelope-from shurd@sasktel.net) Received: from mail144c7.megamailservers.com (mail544c7.megamailservers.com [209.235.141.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8A61B1783; Fri, 20 Jan 2017 09:10:05 +0000 (UTC) (envelope-from shurd@sasktel.net) X-Authenticated-User: hurds.sasktel.net X-VIP: 69.49.109.87 Received: from [192.168.0.33] (ip72-194-73-141.oc.oc.cox.net [72.194.73.141]) (authenticated bits=0) by mail144c7.megamailservers.com (8.14.9/8.13.1) with ESMTP id v0K8eF76014015; Fri, 20 Jan 2017 03:40:17 -0500 Subject: Re: RFC: ethctl To: Kevin Bowling , freebsd-net@freebsd.org Cc: Scott Long , Drew Gallatin , Navdeep Parhar , Oded Shanoon , hps@freebsd.org, Matthew Macy , jeb.j.cramer@intel.com, eric.joyner@intel.com, arybchik@freebsd.org, shurd@freebsd.org, Sean Bruno , George Neville-Neil References: From: Stephen Hurd Message-ID: Date: Fri, 20 Jan 2017 00:40:14 -0800 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:49.0) Gecko/20100101 Firefox/49.0 SeaMonkey/2.46 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-CTCH-RefID: str=0001.0A020201.5881CCF2.01E4, ss=1, re=0.000, recu=0.000, reip=0.000, cl=1, cld=1, fgs=0 X-CTCH-VOD: Unknown X-CTCH-Spam: Unknown X-CTCH-Score: 0.000 X-CTCH-Rules: X-CTCH-Flags: 0 X-CTCH-ScoreCust: 0.000 X-CSC: 0 X-CHA: v=2.2 cv=J+fa1EvS c=1 sm=1 tr=0 a=l4Y+EJuLrT/8f1z5FvEQ1g==:117 a=l4Y+EJuLrT/8f1z5FvEQ1g==:17 a=IkcTkHD0fZMA:10 a=zj8lUfYbn9cKbSo8sksA:9 a=QEXdDO2ut3YA:10 X-Mailman-Approved-At: Fri, 20 Jan 2017 12:26:55 +0000 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jan 2017 09:10:07 -0000 Kevin Bowling wrote: > I have heard from several vendors the need for a NIC configuration > tool. Chelsio ships a cxgb/cxgbetool in FreeBSD as one example. > There is precedence for some nod toward a basic unified tool in Linux > ethtool. > > From your perspective, > 1) What are the common requirements? > 2) What are specialized requirements? For instance as a full TCP > offload card Chelsio needs things others wont > 3) What should it _not_ do? Several of you have experience doing > Ethernet driver dev on many platforms so we should attempt to avoid > repeating past design mistakes. Regarding #3, the current ethtool nvram access is a very poor match for how nvram is used on Broadcom devices. Treating it as a tree or at least a key/value store would make support a lot easier in the driver. Very little of the nvram contents can be addressed by offset anymore. For firmware upgrades, it's even worse. Newer Broadcom devices need to have the firmware flashed into a staging area, then have the device notified to validate the image before an upgrade is complete. The generic nvram read/write methods can't be used for firmware and a small set of critical configuration data. From owner-freebsd-net@freebsd.org Fri Jan 20 15:00:32 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7308DCB930F for ; Fri, 20 Jan 2017 15:00:32 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 51DEB1B6A; Fri, 20 Jan 2017 15:00:28 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from Julian-MBP3.local (ppp121-45-228-247.lns20.per1.internode.on.net [121.45.228.247]) (authenticated bits=0) by vps1.elischer.org (8.15.2/8.15.2) with ESMTPSA id v0KF0OpC049540 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Fri, 20 Jan 2017 07:00:27 -0800 (PST) (envelope-from julian@freebsd.org) To: "freebsd-net@freebsd.org" , Randall Stewart , =?UTF-8?Q?Ermal_Lu=c3=a7i?= From: Julian Elischer Subject: all network people please review this proposal: because someone is going to commit it soon. D5017 Message-ID: <678042cf-9d5f-2f39-6689-30eadf4214a7@freebsd.org> Date: Fri, 20 Jan 2017 23:00:18 +0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.6.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jan 2017 15:00:32 -0000 Unless eri gets to it first I will. see https://reviews.freebsd.org/D5017 If you have a server, you can put an arbitrary number of clients on the same port number because they all have different addresses. However in the case of a client accessing multiple servers we are limited to 65535 sessions because we strictly don't allow the same port to be used more than once. This is silly because TCP is a symmetrical protocol and of it works for the server it should work for the client. So this patch changes the allocation of ports to allow the client to use a port that has been used before as long as the previous port user is not talking to the same host/port. This removes the limitation of a freebsd machine being only capable of contacting 65000 hosts in a single port shutdown timeout period. With modern machines capable of initiating MILLIONS of sessions per second, having a limitation of 65000 per 2 minutes is a bit silly. Please read the patch if you suspect this will have a bad effect of any sort. Once the session is started there is no record as to who started it so any issues would have to be in the startup phase. Julian From owner-freebsd-net@freebsd.org Fri Jan 20 15:15:15 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 655FDCB989B for ; Fri, 20 Jan 2017 15:15:15 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 25F4F1674; Fri, 20 Jan 2017 15:15:15 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1cUauF-0001o6-IZ; Fri, 20 Jan 2017 18:15:11 +0300 Date: Fri, 20 Jan 2017 18:15:11 +0300 From: Slawa Olhovchenkov To: Julian Elischer Cc: "freebsd-net@freebsd.org" , Randall Stewart , Ermal =?utf-8?B?THXDp2k=?= Subject: Re: all network people please review this proposal: because someone is going to commit it soon. D5017 Message-ID: <20170120151511.GR78888@zxy.spb.ru> References: <678042cf-9d5f-2f39-6689-30eadf4214a7@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <678042cf-9d5f-2f39-6689-30eadf4214a7@freebsd.org> User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jan 2017 15:15:15 -0000 On Fri, Jan 20, 2017 at 11:00:18PM +0800, Julian Elischer wrote: > Unless eri gets to it first I will. > > see https://reviews.freebsd.org/D5017 > > If you have a server, you can put an arbitrary number of clients on > the same port number because they all have different addresses. > > However in the case of a client accessing multiple servers we are > limited to 65535 sessions because we strictly don't allow the same > port to be used more than once. This is silly because TCP is a > symmetrical protocol and of it works for the server it should work for > the client. > > So this patch changes the allocation of ports to allow the client to > use a port that has been used before as long as the previous port user > is not talking to the same host/port. This removes the limitation of a > freebsd machine being only capable of contacting 65000 hosts in a > single port shutdown timeout period. With modern machines capable of > initiating MILLIONS of sessions per second, having a limitation of > 65000 per 2 minutes is a bit silly. > > > Please read the patch if you suspect this will have a bad effect of > any sort. Once the session is started there is no record as to who > started it so any issues would have to be in the startup phase. Good. Can you use also destination port and source IP in same way as destination address? From owner-freebsd-net@freebsd.org Fri Jan 20 15:47:45 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 702AFCB83D0 for ; Fri, 20 Jan 2017 15:47:45 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-yw0-x236.google.com (mail-yw0-x236.google.com [IPv6:2607:f8b0:4002:c05::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2A5E519F6; Fri, 20 Jan 2017 15:47:45 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-yw0-x236.google.com with SMTP id v200so91487879ywc.3; Fri, 20 Jan 2017 07:47:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc:content-transfer-encoding; bh=/pzNP6rVgiaYmRurSR7P6nebKLNTcRfJWs5jAiNPtcs=; b=Ch7buXGo7MbrI8aWKGrHo6fw5pQbGQOZRfMHwG6QVIFl4FwPwtFuW+SDBQbqMmQ4pu p0906ZZwhyTnn/IjQ8Jt1nw63lQWTFSFOwwcXiHvcH+zFLPCNdK7ONPO3WqDREMbdClw CJCi5u8Qy1CLGBZjOjxbv4IbpoSAzDrG3G/0lCLNu4z8dM6wbxRWu1bQFNC7P8vKE8YJ gHsN9RDe+lwHLuZGvKxgo/PoVVsWEnpcteO/jk+IPiKUulMIlwUZzxzERgsw7Mq4Uqxa yWovtenH3Vqin0SIQ+dxjHZvYZW7vrbp8iwYb0vtjPWVxVyI26SWf5LGs8jEmzBczoN1 M95A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-transfer-encoding; bh=/pzNP6rVgiaYmRurSR7P6nebKLNTcRfJWs5jAiNPtcs=; b=NlzIcBS0QSmfhS+i6SUzWNtFgEzBmqENVtQqGg7ocKHZOUSHn8CzFR4OPlYVpdL7BJ 3K6mmRAch1oV5g/BX1WaMH9HO/65mHzUHkQy3KkIVTWom36PrFVcmXtpmU/ZNu9iEo1b pc6/L2AmnlJKj/TDgOrCbnUeOtouIY5pKcmP9B9JnmMbVZ7qcNVWNI2FUDjfT2G8MQ9A 1YbIsYGInCkHhV3u+Hjchex6jOlw9uwbYEIV03hwId1p2InZQCeFWVN1istsqhQEYJAd rhoduSj4EuHuO7WPsBvRY50sFu6cAREJjIEFiJtOVLJnDCKDskuL8qG/ZhV05fol+CRr iPqQ== X-Gm-Message-State: AIkVDXLrN7lsgkuV3jMduh6gWJd2khmhJ+ehihh4nhluRiJQhdld5eRFsyHOrt+M5zGuWUAGfLHTmdhVBi5b0A== X-Received: by 10.129.173.71 with SMTP id l7mr11801214ywk.351.1484927264179; Fri, 20 Jan 2017 07:47:44 -0800 (PST) MIME-Version: 1.0 Sender: asomers@gmail.com Received: by 10.129.38.133 with HTTP; Fri, 20 Jan 2017 07:47:43 -0800 (PST) In-Reply-To: <7C29D00C-94C0-4550-B1B2-CE307482B544@FreeBSD.org> References: <20170120083555.ACCF9124AEA4@mail.bitblocks.com> <7C29D00C-94C0-4550-B1B2-CE307482B544@FreeBSD.org> From: Alan Somers Date: Fri, 20 Jan 2017 08:47:43 -0700 X-Google-Sender-Auth: RDducjuhWXGy8FJnl-O02_7akog Message-ID: Subject: Re: pf & NAT issue To: Kristof Provost Cc: Bakul Shah , FreeBSD Net Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jan 2017 15:47:45 -0000 On Fri, Jan 20, 2017 at 3:48 AM, Kristof Provost wrote: > On 20 Jan 2017, at 9:35, Bakul Shah wrote: >> >> pf seems to drop NAT connections quite a bit. This seems to >> happen much more frequently if there are delays involved (slow >> server or interactive use). Almost seems like pf losing >> track of NATted connections due to an uninitialized >> variable.... Often a retry or two works. Connecting from >> outside to forwarded connections to NATTED hosts works fine. >> >> This problem started after ungrading to freebsd-10. Is there a >> bug fix in works or a known work around (other than using ipfw >> or reverting to 9, which I don't want to do)? >> > The problem you describe doesn=E2=80=99t immediately ring a bell. > > We=E2=80=99ll have to gather a bit more information: > > * What FreeBSD version are you running exactly? > * What=E2=80=99s your pf.conf? > * Can you perform a network capture of rejected/failed connections? Idea= lly > both on LAN and WAN on the gateway machine. Please capture full packet= s > (so > tcpdump -s0 -w lan.pcap) as pcap files). > * What networking cards are you using? > > Regards, > Kristof Under heavy load, pf can drop information from its state table. You can try increasing state table limits to see if it helps the problem. Read the "set limits" section of the pf man page. -Alan From owner-freebsd-net@freebsd.org Fri Jan 20 16:42:07 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5F77CCB96AE for ; Fri, 20 Jan 2017 16:42:07 +0000 (UTC) (envelope-from gallatin@netflix.com) Received: from mail-pg0-x232.google.com (mail-pg0-x232.google.com [IPv6:2607:f8b0:400e:c05::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 27BF91100 for ; Fri, 20 Jan 2017 16:42:07 +0000 (UTC) (envelope-from gallatin@netflix.com) Received: by mail-pg0-x232.google.com with SMTP id 204so24909892pge.0 for ; Fri, 20 Jan 2017 08:42:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netflix.com; s=google; h=subject:to:references:from:cc:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=N/XGNWZYH5gaYJJ9zFxEh902DzSPZ+QmSuHrAfSuMlM=; b=OeEAnvtfopm66WJLBLvWSTOXvTWdETm8XJpTanu1LAETjfbBkGMdNqIPAOHnLA+GnE RRCzARTfQGDzh3NjQmIrH5CKkWKdLqauzc3TZqVAryj4ZAEqK4TP4kpbGZujqBhOeyn0 wtDw7KssHxlSeQuNalzhvz4Ez2xGs43lv1qb8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:cc:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=N/XGNWZYH5gaYJJ9zFxEh902DzSPZ+QmSuHrAfSuMlM=; b=W9ZdKEKcf2j0MtT3yD/8xP33uz+q1bsbytY/Wi4D7bQiUgHsbQNrX+jV/iIP1Ko4Og JRlIPg/wVrtqP1e5bb+GXroLzp7ltluJZnj6/dLZkRSvB8aHOCupf6BW00slrKCBpkFF QZgQWDIMkc7WvXdWqAVND9DKb5RgAGKGMNlGpyD4amRctm5h4jbdFaCGTE+H6q+Sk0J1 +Bku2gzLp0AgPHUezVWgGtDbFUIqkY/HPjTOxj4vdjlushCP688caahKNdUMFAs2ZE3A uEoLxXjh0HIEzt8IcuRdtTgZaLfH9qWgJsH+8X/K8hYoePjFgJnCLx97P0p33Fqbjvad xSbg== X-Gm-Message-State: AIkVDXLQAqv6w1IvucUJakti+R4/QbJobkCri73gGrdGmwC1vG0xAwHJjNB0L8xQsO1JIXNl X-Received: by 10.84.148.134 with SMTP id k6mr23228858pla.142.1484930526692; Fri, 20 Jan 2017 08:42:06 -0800 (PST) Received: from ?IPv6:2601:5ca:c302:a71:ec4:7aff:fe71:9cc0? ([2601:5ca:c302:a71:ec4:7aff:fe71:9cc0]) by smtp.gmail.com with ESMTPSA id y23sm18027524pfi.66.2017.01.20.08.42.04 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 20 Jan 2017 08:42:06 -0800 (PST) Subject: Re: RFC: ethctl To: Kevin Bowling , freebsd-net@freebsd.org References: From: Andrew Gallatin Cc: Scott Long , Navdeep Parhar , Oded Shanoon , hps@freebsd.org, Matthew Macy , jeb.j.cramer@intel.com, eric.joyner@intel.com, arybchik@freebsd.org, shurd@freebsd.org, Sean Bruno , George Neville-Neil Message-ID: Date: Fri, 20 Jan 2017 11:42:04 -0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.5.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Fri, 20 Jan 2017 17:12:43 +0000 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jan 2017 16:42:07 -0000 On 01/19/2017 22:58, Kevin Bowling wrote: > Greetings, > > I'm casting a wide net in cc, try to keep the noise minimal but we need > some input from a variety of HW vendors. > > I have heard from several vendors the need for a NIC configuration tool. > Chelsio ships a cxgb/cxgbetool in FreeBSD as one example. There is > precedence for some nod toward a basic unified tool in Linux ethtool. > > From your perspective, > 1) What are the common requirements? > 2) What are specialized requirements? For instance as a full TCP offload > card Chelsio needs things others wont > 3) What should it _not_ do? Several of you have experience doing > Ethernet driver dev on many platforms so we should attempt to avoid > repeating past design mistakes. > > I expect we can achieve some level of inversion so the device specific > code can live close to the driver and plug into the ethctl framework. > It should be general enough to add completely new top level commands, so > vendors can implement HW specific features. On the other hand, we > should attempt to hook into common core for features every NIC provides, > with a focus on iflib. > > I will fund Matt Macy to do the overall design and implementation. > > Regards, > Kevin Bowling, on behalf of Limelight Networks for this effort In a previous job, I was the author of a few Linux drivers (as well as FreeBSD, Solaris, OSX, ESX, etc) for Myricom NICs. IMHO, the "good" thing about ethtool was the standardized kernel API to do things like change tx/rx ring size, and enable/disable offloads. That was much nicer than having to parse ioctls and/or have custom sysctls. I think Gleb had started on this in his ifnet branch, and centralizing such features in iflib is a good carrot to encourage new drivers to use iflib. However, as a user/admin, I believed that a lot of the stuff that was in Linux's ethtool should really have been in ifconfig, and it was always a bit hard to remember which tool did which thing. I think I was spoiled by the rich configuration syntax available on FreeBSD's ifconfig. Eg, this seems more natural: ifconfig mxge0 -tso than this ethtool -K eth2 tso off Eg, I don't see why we need another tool for some of this missing "ethtool" functionality; it seems like most of it would naturally fit into ifconfig. As to other features, like writing firmware images and/or reading dumping eeprom -- these were never a natural fit for us. We already had our own tools that did just what we needed and worked across *all* OSes (even Windows!). I remember trying to figure out the ethtool way, but there was no substantial customer demand for using ethtool for this that I was aware of, and the time needed to adapt our firmware image, etc, to the ethtool format was just not possible to justify. So I think asking vendors to support a FreeBSD ethtool-ish interface for this is asking a lot. Drew From owner-freebsd-net@freebsd.org Fri Jan 20 18:36:47 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BCC2CCB9206 for ; Fri, 20 Jan 2017 18:36:47 +0000 (UTC) (envelope-from wollman@hergotha.csail.mit.edu) Received: from hergotha.csail.mit.edu (wollman-1-pt.tunnel.tserv4.nyc4.ipv6.he.net [IPv6:2001:470:1f06:ccb::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 86B7E1E73 for ; Fri, 20 Jan 2017 18:36:47 +0000 (UTC) (envelope-from wollman@hergotha.csail.mit.edu) Received: from hergotha.csail.mit.edu (localhost [127.0.0.1]) by hergotha.csail.mit.edu (8.15.2/8.15.2) with ESMTP id v0KIajqp014328; Fri, 20 Jan 2017 13:36:45 -0500 (EST) (envelope-from wollman@hergotha.csail.mit.edu) Received: (from wollman@localhost) by hergotha.csail.mit.edu (8.15.2/8.14.4/Submit) id v0KIailL014327; Fri, 20 Jan 2017 13:36:44 -0500 (EST) (envelope-from wollman) Date: Fri, 20 Jan 2017 13:36:44 -0500 (EST) From: Garrett Wollman Message-Id: <201701201836.v0KIailL014327@hergotha.csail.mit.edu> To: gallatin@netflix.com Subject: Re: RFC: ethctl In-Reply-To: References: Organization: none Cc: freebsd-net@freebsd.org X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.1 (hergotha.csail.mit.edu [127.0.0.1]); Fri, 20 Jan 2017 13:36:45 -0500 (EST) X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=disabled version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on hergotha.csail.mit.edu X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jan 2017 18:36:47 -0000 In article you write: >Eg, I don't see why we need another tool for some of this missing >"ethtool" functionality; it seems like most of it would naturally fit >into ifconfig. >From the end-user perspective, I agree with Drew. Most of this stuff should just be part of ifconfig. >As to other features, like writing firmware images and/or reading >dumping eeprom -- these were never a natural fit for us. And I can't say that I've ever wanted to do this. Most of the machines we buy these days are Dells, and Dell USC takes care of all the firmware updates. Having a not-really-generic firmware programming interface that only works for network interfaces seems like very limited value. I wouldn't object to it, but I doubt I'd ever use it, and I expect most vendors will want to keep tighter control over applying firmware updates. -GAWollman From owner-freebsd-net@freebsd.org Fri Jan 20 17:56:31 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4BCA3CB8E64 for ; Fri, 20 Jan 2017 17:56:31 +0000 (UTC) (envelope-from rizzo.unipi@gmail.com) Received: from mail-io0-x242.google.com (mail-io0-x242.google.com [IPv6:2607:f8b0:4001:c06::242]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 129FF1504; Fri, 20 Jan 2017 17:56:31 +0000 (UTC) (envelope-from rizzo.unipi@gmail.com) Received: by mail-io0-x242.google.com with SMTP id c80so8769827iod.1; Fri, 20 Jan 2017 09:56:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=CwEYpwpVL5szGHzWQ1CylrB9Oxr/e5md7Dd5nJdBV84=; b=u/lZmQj872B3TxMF3phhKAh82LGin0PGtbaZ6a8XF9pfr2FmvlhpYlEW6VHOVShN5l b3Ox72KJ46ArOmPguShxBHcyZDA3oUJJlevModToRjNB+tRzYwkVeor1R0I4z2X5C4XE g4I+fTbzsoiOr3tIA/kYc1BhD5u43JrBLG18pbnBulrXum0Oaik5td26WNmNi+vbYg5t +B8JT7n9XAvztYh18GDYQTkl3yj9HMdoqtSsElt4o5+KaOV5VFIDPPKRsS3Vk8wjvUrx kaslTVBD+/9b73qbCrzvD7MoawFDqS16l9Y1EX3xPdrWZnfIwLFRURMYbKmALMqwG7wE RbbA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=CwEYpwpVL5szGHzWQ1CylrB9Oxr/e5md7Dd5nJdBV84=; b=TylQuw4zTHoonP7adjBvAa/766i7LUZWkobbqdVUG4WbzopeGNVrE+XD5LQ7eJoiKC Jjs65Ftpl2Ab0sztH6eWTAW6wtJ73mJIoDYVu4FstVSNeNDrf6sMGEJcarkpkP3Tp9tr Uu8fJpEwLkGWEsgKbPEsS/ZqLbTQ3IcKNzkPxV8D0S3pdiHcvFVavUnG4lplT33fB7XQ kImv3M7MyHeVGBxvlERS4cj2xx6dbNNkTT4ZK0s0PDya2GaLsjI1zO0FoTGQmK2urS6k 1BZhZ+LmvjhpkelmhzsKeE0mFAXjzTMYJZqCpTULMGg8NcP3K5seFQTnpDrCz2u9VLgf n5Lg== X-Gm-Message-State: AIkVDXLcO7imtSGFc21PKMxehLFE2gB4oEXDAXoCmCdCKULmcDXZOMggtfO307jYRVngr/U9jLHU10Gm/xNcjw== X-Received: by 10.107.129.222 with SMTP id l91mr13294292ioi.194.1484934990336; Fri, 20 Jan 2017 09:56:30 -0800 (PST) MIME-Version: 1.0 Sender: rizzo.unipi@gmail.com Received: by 10.107.169.213 with HTTP; Fri, 20 Jan 2017 09:56:29 -0800 (PST) In-Reply-To: References: From: Luigi Rizzo Date: Fri, 20 Jan 2017 09:56:29 -0800 X-Google-Sender-Auth: mcywOtv-BKtp4leqbEhRSG_V6Tw Message-ID: Subject: Re: RFC: ethctl To: Andrew Gallatin Cc: Kevin Bowling , "freebsd-net@freebsd.org" , Scott Long , "Joyner, Eric" , Oded Shanoon , Matthew Macy , hps@freebsd.org, "Cramer, Jeb J" , George Neville-Neil , arybchik@freebsd.org, shurd@freebsd.org, Navdeep Parhar Content-Type: text/plain; charset=UTF-8 X-Mailman-Approved-At: Fri, 20 Jan 2017 20:16:37 +0000 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jan 2017 17:56:31 -0000 To summarize Drew's comments, which I mostly agree with, and suggest a possible strategy for deployment: - Irrespective of the user-facing command (ifconfig vs ethtool), a common kernel API for the most common features is very desirable. - There are very good reasons to take inspiration from include/linux/ethtool.h to decide on this set of features: 1. it is a valid starting point and as good as any other one; 2. ethtool functions are all optional so deciding what to put in is not a hard decision 3. may possibly ease porting drivers across platforms - I think the user interface bikeshed is not solvable other than eventually implementing both "ifconfig" and "ethtool" style user commands. This does not seem a major problem, because our ifconfig already implements some of the ethtool's commands, and to tell the truth, as much as I am (was) used to ifconfig, it is such a kitchen sink that it is sometimes hard to figure out how to use it (documentation and feature mismatch, ambiguity in the syntax etc.) - for some of the high level features (e.g. flashing a device) that may be more complicated than calling an ioctl(), we could implement a fallback mechanism where, say, the ioctl returns a special message that drives the user-facing app to call an external helper program (configured through /etc/rc.d/ or whatever other mechanism) cheers luigi On Fri, Jan 20, 2017 at 8:42 AM, Andrew Gallatin via freebsd-net wrote: > On 01/19/2017 22:58, Kevin Bowling wrote: >> >> Greetings, >> >> I'm casting a wide net in cc, try to keep the noise minimal but we need >> some input from a variety of HW vendors. >> >> I have heard from several vendors the need for a NIC configuration tool. >> Chelsio ships a cxgb/cxgbetool in FreeBSD as one example. There is >> precedence for some nod toward a basic unified tool in Linux ethtool. >> >> From your perspective, >> 1) What are the common requirements? >> 2) What are specialized requirements? For instance as a full TCP offload >> card Chelsio needs things others wont >> 3) What should it _not_ do? Several of you have experience doing >> Ethernet driver dev on many platforms so we should attempt to avoid >> repeating past design mistakes. >> >> I expect we can achieve some level of inversion so the device specific >> code can live close to the driver and plug into the ethctl framework. >> It should be general enough to add completely new top level commands, so >> vendors can implement HW specific features. On the other hand, we >> should attempt to hook into common core for features every NIC provides, >> with a focus on iflib. >> >> I will fund Matt Macy to do the overall design and implementation. >> >> Regards, >> Kevin Bowling, on behalf of Limelight Networks for this effort > > > In a previous job, I was the author of a few Linux drivers (as well as > FreeBSD, Solaris, OSX, ESX, etc) for Myricom NICs. > > IMHO, the "good" thing about ethtool was the standardized kernel API > to do things like change tx/rx ring size, and enable/disable offloads. > That was much nicer than having to parse ioctls and/or have custom > sysctls. I think Gleb had started on this in his ifnet branch, and > centralizing such features in iflib is a good carrot to encourage new > drivers to use iflib. > > However, as a user/admin, I believed that a lot of the stuff that was > in Linux's ethtool should really have been in ifconfig, and it was > always a bit hard to remember which tool did which thing. I think I > was spoiled by the rich configuration syntax available on FreeBSD's > ifconfig. Eg, this seems more natural: > > ifconfig mxge0 -tso > than this > ethtool -K eth2 tso off > > Eg, I don't see why we need another tool for some of this missing > "ethtool" functionality; it seems like most of it would naturally fit > into ifconfig. > > As to other features, like writing firmware images and/or reading > dumping eeprom -- these were never a natural fit for us. We already > had our own tools that did just what we needed and worked across *all* > OSes (even Windows!). I remember trying to figure out the ethtool > way, but there was no substantial customer demand for using ethtool > for this that I was aware of, and the time needed to adapt our > firmware image, etc, to the ethtool format was just not possible to > justify. So I think asking vendors to support a FreeBSD ethtool-ish > interface for this is asking a lot. > > Drew > > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" -- -----------------------------------------+------------------------------- Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione http://www.iet.unipi.it/~luigi/ . Universita` di Pisa TEL +39-050-2217533 . via Diotisalvi 2 Mobile +39-338-6809875 . 56122 PISA (Italy) -----------------------------------------+------------------------------- From owner-freebsd-net@freebsd.org Fri Jan 20 20:31:08 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 06FDCCBAD65 for ; Fri, 20 Jan 2017 20:31:08 +0000 (UTC) (envelope-from bakul@bitblocks.com) Received: from mail.bitblocks.com (ns1.bitblocks.com [173.228.5.8]) by mx1.freebsd.org (Postfix) with ESMTP id DEDB21E59; Fri, 20 Jan 2017 20:31:07 +0000 (UTC) (envelope-from bakul@bitblocks.com) Received: from bitblocks.com (localhost [127.0.0.1]) by mail.bitblocks.com (Postfix) with ESMTP id CD2C8124AEA4; Fri, 20 Jan 2017 12:31:06 -0800 (PST) To: Alan Somers cc: Kristof Provost , FreeBSD Net Subject: Re: pf & NAT issue In-reply-to: Your message of "Fri, 20 Jan 2017 08:47:43 MST." References: <20170120083555.ACCF9124AEA4@mail.bitblocks.com> <7C29D00C-94C0-4550-B1B2-CE307482B544@FreeBSD.org> Comments: In-reply-to Alan Somers message dated "Fri, 20 Jan 2017 08:47:43 -0700." Date: Fri, 20 Jan 2017 12:31:06 -0800 From: Bakul Shah Message-Id: <20170120203106.CD2C8124AEA4@mail.bitblocks.com> X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jan 2017 20:31:08 -0000 On Fri, 20 Jan 2017 08:47:43 MST Alan Somers wrote: > On Fri, Jan 20, 2017 at 3:48 AM, Kristof Provost wrote: > > On 20 Jan 2017, at 9:35, Bakul Shah wrote: > >> > >> pf seems to drop NAT connections quite a bit. This seems to > >> happen much more frequently if there are delays involved (slow > >> server or interactive use). Almost seems like pf losing > >> track of NATted connections due to an uninitialized > >> variable.... Often a retry or two works. Connecting from > >> outside to forwarded connections to NATTED hosts works fine. > >> > >> This problem started after ungrading to freebsd-10. Is there a > >> bug fix in works or a known work around (other than using ipfw > >> or reverting to 9, which I don't want to do)? > >> > > The problem you describe doesn't immediately ring a bell. > > > > We'll have to gather a bit more information: > > > > * What FreeBSD version are you running exactly? > > * What's your pf.conf? > > * Can you perform a network capture of rejected/failed connections? Ideally > > both on LAN and WAN on the gateway machine. Please capture full packets > > (so > > tcpdump -s0 -w lan.pcap) as pcap files). > > * What networking cards are you using? > > > > Regards, > > Kristof > > Under heavy load, pf can drop information from its state table. You > can try increasing state table limits to see if it helps the problem. > Read the "set limits" section of the pf man page. > > -Alan Thanks for the suggestions. Here's some info. My inline comments are indented. $ uname -rm 10.3-RELEASE-p4 i386 $ netstat -n | grep tcp | wc -l 13 So the machine is lightly loaded. $ grep -v ^# /etc/pf.conf|uniq ext_if="rl0" int_if="em0" nat on $ext_if inet from ! ($ext_if) to any -> ($ext_if) I took out rdr entries during testing. They don't seem to affect this issue. I had changed src.track timeout to 30 seconds but that didn't seem to change anything. $ pfctl -s memory states hard limit 10000 src-nodes hard limit 10000 frags hard limit 5000 table-entries hard limit 200000 $ pfctl -s info Status: Enabled for 167 days 13:40:11 Debug: Urgent State Table Total Rate current entries 0 searches 2870986757 198.3/s # this seems high... inserts 3428240 0.2/s removals 3428240 0.2/s Counters match 1482741914 102.4/s bad-offset 0 0.0/s fragment 1 0.0/s short 0 0.0/s normalize 0 0.0/s memory 0 0.0/s bad-timestamp 0 0.0/s congestion 0 0.0/s ip-option 31 0.0/s proto-cksum 0 0.0/s state-mismatch 28931 0.0/s state-insert 1 0.0/s state-limit 0 0.0/s src-limit 0 0.0/s synproxy 0 0.0/s $ tcpdump -ni rl0 host ftp4.freebsd.org # in one window $ tcpdump -ni em0 host 192.168.125.7 # in another On an internal machine I did "telnet ftp4.freebsd.org ftp", waited for a while and then typed something. The following trace is interspersed in the correct sequence. Traffic on rl0 (external) is prefixed with < and traffic on em0 (internal )with >. > 11:56:05.743745 IP 192.168.125.7.65042 > 149.20.1.200.21: Flags [S], seq 3080825146, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val 176000 ecr 0], length 0 < 11:56:05.743776 IP 173.228.5.8.63716 > 149.20.1.200.21: Flags [S], seq 3080825146, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val 176000 ecr 0], length 0 < 11:56:05.763294 IP 149.20.1.200.21 > 173.228.5.8.63716: Flags [S.], seq 3912707359, ack 3080825147, win 65535, options [mss 1460,nop,wscale 11,sackOK,TS val 1468113699 ecr 176000], length 0 > 11:56:05.763313 IP 149.20.1.200.21 > 192.168.125.7.65042: Flags [S.], seq 3912707359, ack 3080825147, win 65535, options [mss 1460,nop,wscale 11,sackOK,TS val 1468113699 ecr 176000], length 0 > 11:56:05.764106 IP 192.168.125.7.65042 > 149.20.1.200.21: Flags [.], ack 1, win 1026, options [nop,nop,TS val 176021 ecr 1468113699], length 0 < 11:56:05.764121 IP 173.228.5.8.63716 > 149.20.1.200.21: Flags [.], ack 1, win 1026, options [nop,nop,TS val 176021 ecr 1468113699], length 0 < 11:56:05.789192 IP 149.20.1.200.21 > 173.228.5.8.63716: Flags [P.], seq 1:55, ack 1, win 32, options [nop,nop,TS val 1468113725 ecr 176021], length 54 > 11:56:05.789204 IP 149.20.1.200.21 > 192.168.125.7.65042: Flags [P.], seq 1:55, ack 1, win 32, options [nop,nop,TS val 1468113725 ecr 176021], length 54 > 11:56:05.895660 IP 192.168.125.7.65042 > 149.20.1.200.21: Flags [.], ack 55, win 1026, options [nop,nop,TS val 176152 ecr 1468113725], length 0 < 11:56:05.895675 IP 173.228.5.8.63716 > 149.20.1.200.21: Flags [.], ack 55, win 1026, options [nop,nop,TS val 176152 ecr 1468113725], length 0 > 11:56:28.168693 IP 192.168.125.7.65042 > 149.20.1.200.21: Flags [P.], seq 1:10, ack 55, win 1026, options [nop,nop,TS val 198426 ecr 1468113725], length 9 < 11:56:28.168712 IP 173.228.5.8.52015 > 149.20.1.200.21: Flags [P.], seq 3080825147:3080825156, ack 3912707414, win 1026, options [nop,nop,TS val 198426 ecr 1468113725], length 9 Right here we see the problem. NAT mapping for the port changed from 63716 to 52015. Bakul From owner-freebsd-net@freebsd.org Fri Jan 20 20:43:37 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A838FCB93D0 for ; Fri, 20 Jan 2017 20:43:37 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from venus.codepro.be (venus.codepro.be [IPv6:2a01:4f8:162:1127::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.codepro.be", Issuer "Gandi Standard SSL CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7487B19BD; Fri, 20 Jan 2017 20:43:37 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from [10.0.2.164] (ptr-8ripyyfi1726l7ds8ua.18120a2.ip6.access.telenet.be [IPv6:2a02:1811:2419:4e02:4883:fabf:4b8b:94c2]) (Authenticated sender: kp) by venus.codepro.be (Postfix) with ESMTPSA id B5CF5358B0; Fri, 20 Jan 2017 21:43:34 +0100 (CET) From: "Kristof Provost" To: "Bakul Shah" Cc: "Alan Somers" , "FreeBSD Net" Subject: Re: pf & NAT issue Date: Fri, 20 Jan 2017 21:43:33 +0100 Message-ID: In-Reply-To: <20170120203106.CD2C8124AEA4@mail.bitblocks.com> References: <20170120083555.ACCF9124AEA4@mail.bitblocks.com> <7C29D00C-94C0-4550-B1B2-CE307482B544@FreeBSD.org> <20170120203106.CD2C8124AEA4@mail.bitblocks.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-Mailer: MailMate (2.0BETAr6072) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jan 2017 20:43:37 -0000 On 20 Jan 2017, at 21:31, Bakul Shah wrote: >> 11:56:28.168693 IP 192.168.125.7.65042 > 149.20.1.200.21: Flags [P.], >> seq 1:10, ack 55, win 1026, options [nop,nop,TS val 198426 ecr >> 1468113725], length 9 > < 11:56:28.168712 IP 173.228.5.8.52015 > 149.20.1.200.21: Flags [P.], > seq 3080825147:3080825156, ack 3912707414, win 1026, options > [nop,nop,TS val 198426 ecr 1468113725], length 9 > > Right here we see the problem. NAT mapping for the > port changed from 63716 to 52015. > Changing source ports is an entirely normal NAT behaviour. The best explanation is this: imagine that you have two clients A and B, both connect to X on port 80 via the NAT gateway G. Both use port 1000 as their source port. A connects, and the gateway maps A:1000 -> X:80 to G:1000 -> X:80. B connects, and now the gateway has to map B:1000 -> X:80 onto G:1000 -> X:80, but then it wouldn’t be able to tell the two connections apart. That’s why it can remap it onto G:1001 -> X:80 instead. Regards, Kristof From owner-freebsd-net@freebsd.org Fri Jan 20 20:48:54 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 190F4CB9672 for ; Fri, 20 Jan 2017 20:48:54 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from venus.codepro.be (venus.codepro.be [IPv6:2a01:4f8:162:1127::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.codepro.be", Issuer "Gandi Standard SSL CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D90D41BF1; Fri, 20 Jan 2017 20:48:53 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from [10.0.2.164] (ptr-8ripyyfi1726l7ds8ua.18120a2.ip6.access.telenet.be [IPv6:2a02:1811:2419:4e02:4883:fabf:4b8b:94c2]) (Authenticated sender: kp) by venus.codepro.be (Postfix) with ESMTPSA id 179ED358C0; Fri, 20 Jan 2017 21:48:52 +0100 (CET) From: "Kristof Provost" To: "Bakul Shah" Cc: "Alan Somers" , "FreeBSD Net" Subject: Re: pf & NAT issue Date: Fri, 20 Jan 2017 21:48:51 +0100 Message-ID: In-Reply-To: <20170120203106.CD2C8124AEA4@mail.bitblocks.com> References: <20170120083555.ACCF9124AEA4@mail.bitblocks.com> <7C29D00C-94C0-4550-B1B2-CE307482B544@FreeBSD.org> <20170120203106.CD2C8124AEA4@mail.bitblocks.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-Mailer: MailMate (2.0BETAr6072) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jan 2017 20:48:54 -0000 On 20 Jan 2017, at 21:31, Bakul Shah wrote: > $ pfctl -s info > Status: Enabled for 167 days 13:40:11 Debug: Urgent > > State Table Total Rate > current entries 0 > searches 2870986757 198.3/s # this > seems high... > inserts 3428240 0.2/s > removals 3428240 0.2/s > Counters > match 1482741914 102.4/s > bad-offset 0 0.0/s > fragment 1 0.0/s > short 0 0.0/s > normalize 0 0.0/s > memory 0 0.0/s > bad-timestamp 0 0.0/s > congestion 0 0.0/s > ip-option 31 0.0/s > proto-cksum 0 0.0/s > state-mismatch 28931 0.0/s You gave a decent number of state-mismatch errors here. It’s worth checking if that number increments whenever you see a dropped NAT connection. Regards, Kristof From owner-freebsd-net@freebsd.org Fri Jan 20 20:59:34 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B3EBACB9885 for ; Fri, 20 Jan 2017 20:59:34 +0000 (UTC) (envelope-from bakul@bitblocks.com) Received: from mail.bitblocks.com (ns1.bitblocks.com [173.228.5.8]) by mx1.freebsd.org (Postfix) with ESMTP id 9D1FF1FA1; Fri, 20 Jan 2017 20:59:33 +0000 (UTC) (envelope-from bakul@bitblocks.com) Received: from bitblocks.com (localhost [127.0.0.1]) by mail.bitblocks.com (Postfix) with ESMTP id 8948A124AEA3; Fri, 20 Jan 2017 12:59:33 -0800 (PST) To: "Kristof Provost" cc: "Bakul Shah" , "Alan Somers" , "FreeBSD Net" Subject: Re: pf & NAT issue In-reply-to: Your message of "Fri, 20 Jan 2017 21:43:33 +0100." References: <20170120083555.ACCF9124AEA4@mail.bitblocks.com> <7C29D00C-94C0-4550-B1B2-CE307482B544@FreeBSD.org> <20170120203106.CD2C8124AEA4@mail.bitblocks.com> Comments: In-reply-to "Kristof Provost" message dated "Fri, 20 Jan 2017 21:43:33 +0100." Date: Fri, 20 Jan 2017 12:59:33 -0800 From: Bakul Shah Message-Id: <20170120205933.8948A124AEA3@mail.bitblocks.com> X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jan 2017 20:59:34 -0000 On Fri, 20 Jan 2017 21:43:33 +0100 "Kristof Provost" wrote: > On 20 Jan 2017, at 21:31, Bakul Shah wrote: > >> 11:56:28.168693 IP 192.168.125.7.65042 > 149.20.1.200.21: Flags [P.], > >> seq 1:10, ack 55, win 1026, options [nop,nop,TS val 198426 ecr > >> 1468113725], length 9 > > < 11:56:28.168712 IP 173.228.5.8.52015 > 149.20.1.200.21: Flags [P.], > > seq 3080825147:3080825156, ack 3912707414, win 1026, options > > [nop,nop,TS val 198426 ecr 1468113725], length 9 > > > > Right here we see the problem. NAT mapping for the > > port changed from 63716 to 52015. > > > Changing source ports is an entirely normal NAT behaviour. > > The best explanation is this: imagine that you have two clients A and B, > both connect to X on port 80 via the NAT gateway G. > Both use port 1000 as their source port. > A connects, and the gateway maps A:1000 -> X:80 to G:1000 -> X:80. > B connects, and now the gateway has to map B:1000 -> X:80 onto G:1000 -> > X:80, but then it wouldn't be able to tell the two connections apart. > That't can remap it onto G:1001 -> X:80 instead. It is the same connection! As a tcp connection is identified by , If the port number changes on the same connection, the remote side would see this as a separate connection. From owner-freebsd-net@freebsd.org Fri Jan 20 21:07:12 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A946DCB9B6A for ; Fri, 20 Jan 2017 21:07:12 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from venus.codepro.be (venus.codepro.be [IPv6:2a01:4f8:162:1127::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.codepro.be", Issuer "Gandi Standard SSL CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 73FFD1361; Fri, 20 Jan 2017 21:07:12 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from [10.0.2.164] (ptr-8ripyyfi1726l7ds8ua.18120a2.ip6.access.telenet.be [IPv6:2a02:1811:2419:4e02:4883:fabf:4b8b:94c2]) (Authenticated sender: kp) by venus.codepro.be (Postfix) with ESMTPSA id A6BB1358DF; Fri, 20 Jan 2017 22:07:10 +0100 (CET) From: "Kristof Provost" To: "Bakul Shah" Cc: "Alan Somers" , "FreeBSD Net" Subject: Re: pf & NAT issue Date: Fri, 20 Jan 2017 22:07:10 +0100 Message-ID: <8146C6B6-3EA6-41A2-9FC7-0304E2128AFF@FreeBSD.org> In-Reply-To: <20170120205933.8948A124AEA3@mail.bitblocks.com> References: <20170120083555.ACCF9124AEA4@mail.bitblocks.com> <7C29D00C-94C0-4550-B1B2-CE307482B544@FreeBSD.org> <20170120203106.CD2C8124AEA4@mail.bitblocks.com> <20170120205933.8948A124AEA3@mail.bitblocks.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-Mailer: MailMate (2.0BETAr6072) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jan 2017 21:07:12 -0000 On 20 Jan 2017, at 21:59, Bakul Shah wrote: > On Fri, 20 Jan 2017 21:43:33 +0100 "Kristof Provost" > wrote: >> On 20 Jan 2017, at 21:31, Bakul Shah wrote: >>>> 11:56:28.168693 IP 192.168.125.7.65042 > 149.20.1.200.21: Flags >>>> [P.], >>>> seq 1:10, ack 55, win 1026, options [nop,nop,TS val 198426 ecr >>>> 1468113725], length 9 >>> < 11:56:28.168712 IP 173.228.5.8.52015 > 149.20.1.200.21: Flags >>> [P.], >>> seq 3080825147:3080825156, ack 3912707414, win 1026, options >>> [nop,nop,TS val 198426 ecr 1468113725], length 9 >>> >>> Right here we see the problem. NAT mapping for the >>> port changed from 63716 to 52015. >>> >> Changing source ports is an entirely normal NAT behaviour. >> >> The best explanation is this: imagine that you have two clients A and >> B, >> both connect to X on port 80 via the NAT gateway G. >> Both use port 1000 as their source port. >> A connects, and the gateway maps A:1000 -> X:80 to G:1000 -> X:80. >> B connects, and now the gateway has to map B:1000 -> X:80 onto G:1000 >> -> >> X:80, but then it wouldn't be able to tell the two connections apart. >> That't can remap it onto G:1001 -> X:80 instead. > > It is the same connection! As a tcp connection is identified > by , If the port number > changes on the same connection, the remote side would see this > as a separate connection. Wait, it changes it within the same connection? I was reading it as a different connection, in which case that’d be quite ok. It’s of course not OK within one connection lifetime. I have no idea how that could be happening. Just in case, have you ever done a memtest on that box? Regards, Kristof From owner-freebsd-net@freebsd.org Fri Jan 20 21:11:57 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6D8CFCB9E81 for ; Fri, 20 Jan 2017 21:11:57 +0000 (UTC) (envelope-from bakul@bitblocks.com) Received: from mail.bitblocks.com (ns1.bitblocks.com [173.228.5.8]) by mx1.freebsd.org (Postfix) with ESMTP id 472F418C5; Fri, 20 Jan 2017 21:11:56 +0000 (UTC) (envelope-from bakul@bitblocks.com) Received: from bitblocks.com (localhost [127.0.0.1]) by mail.bitblocks.com (Postfix) with ESMTP id BF841124AEA4; Fri, 20 Jan 2017 13:11:55 -0800 (PST) To: "Kristof Provost" , FreeBSD Net , Alan Somers Subject: Re: pf & NAT issue In-reply-to: Your message of "Fri, 20 Jan 2017 12:59:33 PST." <20170120205933.8948A124AEA3@mail.bitblocks.com> References: <20170120083555.ACCF9124AEA4@mail.bitblocks.com> <7C29D00C-94C0-4550-B1B2-CE307482B544@FreeBSD.org> <20170120203106.CD2C8124AEA4@mail.bitblocks.com> <20170120205933.8948A124AEA3@mail.bitblocks.com> Comments: In-reply-to Bakul Shah message dated "Fri, 20 Jan 2017 12:59:33 -0800." Date: Fri, 20 Jan 2017 13:11:55 -0800 From: Bakul Shah Message-Id: <20170120211155.BF841124AEA4@mail.bitblocks.com> X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jan 2017 21:11:57 -0000 On Fri, 20 Jan 2017 12:59:33 PST Bakul Shah wrote: > On Fri, 20 Jan 2017 21:43:33 +0100 "Kristof Provost" wrote: > > On 20 Jan 2017, at 21:31, Bakul Shah wrote: > > >> 11:56:28.168693 IP 192.168.125.7.65042 > 149.20.1.200.21: Flags [P.], > > >> seq 1:10, ack 55, win 1026, options [nop,nop,TS val 198426 ecr > > >> 1468113725], length 9 > > > < 11:56:28.168712 IP 173.228.5.8.52015 > 149.20.1.200.21: Flags [P.], > > > seq 3080825147:3080825156, ack 3912707414, win 1026, options > > > [nop,nop,TS val 198426 ecr 1468113725], length 9 > > > > > > Right here we see the problem. NAT mapping for the > > > port changed from 63716 to 52015. > > > > > Changing source ports is an entirely normal NAT behaviour. > > > > The best explanation is this: imagine that you have two clients A and B, > > both connect to X on port 80 via the NAT gateway G. > > Both use port 1000 as their source port. > > A connects, and the gateway maps A:1000 -> X:80 to G:1000 -> X:80. > > B connects, and now the gateway has to map B:1000 -> X:80 onto G:1000 -> > > X:80, but then it wouldn't be able to tell the two connections apart. > > That't can remap it onto G:1001 -> X:80 instead. > > It is the same connection! As a tcp connection is identified > by , If the port number > changes on the same connection, the remote side would see this > as a separate connection. Let me expand on this a bit. I should've shown one more packet trace. What happens is that the new port number (52015) is associated with this connection from now on so the next packet from the local internal machine is seen by the remote side as belonging to connection <173.228.5.8.52015, 149.20.1.200.21> instead of <173.228.5.8.63716, 149.20.1.200.21> so it is of course going to throw it out. The gateway machine has behaved quite well fr months but memtest is a good idea. Thanks! From owner-freebsd-net@freebsd.org Fri Jan 20 21:12:09 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 32812CB9EAA for ; Fri, 20 Jan 2017 21:12:09 +0000 (UTC) (envelope-from ermal.luci@gmail.com) Received: from mail-io0-x242.google.com (mail-io0-x242.google.com [IPv6:2607:f8b0:4001:c06::242]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E9A0D1973; Fri, 20 Jan 2017 21:12:08 +0000 (UTC) (envelope-from ermal.luci@gmail.com) Received: by mail-io0-x242.google.com with SMTP id c80so9287774iod.1; Fri, 20 Jan 2017 13:12:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=rTk2npitw2vLrStAk+OGekhp7WzzCxJi6uSXkqTjWx4=; b=XJnOmU+st9tAdz/5Ng1Vka+cwR8D2uahu4Ach45hgW7k8zJvj/z4c1xIklCFqNf+9S 4RHbE8oYv3HLBXZNupIUEXPPHHxqjul6mrIYOvLXLZkNKEmy0qNdeMElYDPZlIjA7DSU ElqKSyl88haKdHydxi60lmDv2M5ncegazTeO7DEke2/lIj6JbDcoBh2zuM6oa0KvDQyx e5HpJWxpJoC7ut94wJxC7P5k6tzTcCwcKWwHHu2RlyNoc9xAViQFdaL2LpJHwqliMyUx KhPn/QW4GpEGGTDRWPpjvidAQ4uM2yZyh2P1Yntarr/t46PE5g+0JjmLIM3ASAvkzYQq h9wA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=rTk2npitw2vLrStAk+OGekhp7WzzCxJi6uSXkqTjWx4=; b=cHhpd0UEqvHOjMKPtigrl92f960bUEBnRd2jlnCBPAa0UisTep+8Gzj9rpd7QfpN+k zJ+o5nFAqIys988MKMvnwaAz2vsf+IeQcFmJhF6/36L04nkbaNzDpIUL9rCFNrmZM3Qd 1+9EEiAx6a5GpVAGUSELjx6S6NwHDtUb49a+lXIwEo0qRoL26l56YkZYrSQm26RJVy4Q MC/eNWP7/9KbMD/ed1KAmm3VF6RCzP+IMFLctFXHCZagPI3vH+wNzkvnTEN8hNbgth4K mBAldTuF/lg5sjyYGX97J488fLuVFJz6sAX8cD5AMLU77LhD6YzibQiTvxKFPd2LiHl5 24Nw== X-Gm-Message-State: AIkVDXJxEl1/J5tB3EJYkFd45FbEPhNf5PYyTHQFRgCRDCvWMQSyLIf0BPZ1sQTertODqDE7e7G1ZHhLkhPYsg== X-Received: by 10.107.171.135 with SMTP id u129mr14687661ioe.102.1484946728306; Fri, 20 Jan 2017 13:12:08 -0800 (PST) MIME-Version: 1.0 Sender: ermal.luci@gmail.com Received: by 10.107.129.99 with HTTP; Fri, 20 Jan 2017 13:12:07 -0800 (PST) In-Reply-To: <20170120205933.8948A124AEA3@mail.bitblocks.com> References: <20170120083555.ACCF9124AEA4@mail.bitblocks.com> <7C29D00C-94C0-4550-B1B2-CE307482B544@FreeBSD.org> <20170120203106.CD2C8124AEA4@mail.bitblocks.com> <20170120205933.8948A124AEA3@mail.bitblocks.com> From: =?UTF-8?Q?Ermal_Lu=C3=A7i?= Date: Fri, 20 Jan 2017 13:12:07 -0800 X-Google-Sender-Auth: 2N0aAYAKaJYjDf6hREFFvBqpFGY Message-ID: Subject: Re: pf & NAT issue To: Bakul Shah Cc: Kristof Provost , FreeBSD Net , Alan Somers Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jan 2017 21:12:09 -0000 On Fri, Jan 20, 2017 at 12:59 PM, Bakul Shah wrote: > On Fri, 20 Jan 2017 21:43:33 +0100 "Kristof Provost" > wrote: > > On 20 Jan 2017, at 21:31, Bakul Shah wrote: > > >> 11:56:28.168693 IP 192.168.125.7.65042 > 149.20.1.200.21: Flags [P.], > > >> seq 1:10, ack 55, win 1026, options [nop,nop,TS val 198426 ecr > > >> 1468113725], length 9 > > > < 11:56:28.168712 IP 173.228.5.8.52015 > 149.20.1.200.21: Flags [P.], > > > seq 3080825147:3080825156, ack 3912707414, win 1026, options > > > [nop,nop,TS val 198426 ecr 1468113725], length 9 > > > > > > Right here we see the problem. NAT mapping for the > > > port changed from 63716 to 52015. > > > > > Changing source ports is an entirely normal NAT behaviour. > > > > The best explanation is this: imagine that you have two clients A and B, > > both connect to X on port 80 via the NAT gateway G. > > Both use port 1000 as their source port. > > A connects, and the gateway maps A:1000 -> X:80 to G:1000 -> X:80. > > B connects, and now the gateway has to map B:1000 -> X:80 onto G:1000 -> > > X:80, but then it wouldn't be able to tell the two connections apart. > > That't can remap it onto G:1001 -> X:80 instead. > > It is the same connection! As a tcp connection is identified > by , If the port number > changes on the same connection, the remote side would see this > as a separate connection. > Most probably your timeouts are aggressive on states garbage collection. Give a look to those state limit teardown it might improve things. > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > -- > Ermal > From owner-freebsd-net@freebsd.org Fri Jan 20 21:17:22 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D8DD9CBA0E3 for ; Fri, 20 Jan 2017 21:17:22 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from venus.codepro.be (venus.codepro.be [IPv6:2a01:4f8:162:1127::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.codepro.be", Issuer "Gandi Standard SSL CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A24D21D79; Fri, 20 Jan 2017 21:17:22 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from [10.0.2.164] (ptr-8ripyyfi1726l7ds8ua.18120a2.ip6.access.telenet.be [IPv6:2a02:1811:2419:4e02:4883:fabf:4b8b:94c2]) (Authenticated sender: kp) by venus.codepro.be (Postfix) with ESMTPSA id CEC4E35939; Fri, 20 Jan 2017 22:17:20 +0100 (CET) From: "Kristof Provost" To: "Ermal =?utf-8?q?Lu=C3=A7i?=" Cc: "Bakul Shah" , "FreeBSD Net" , "Alan Somers" Subject: Re: pf & NAT issue Date: Fri, 20 Jan 2017 22:17:20 +0100 Message-ID: In-Reply-To: References: <20170120083555.ACCF9124AEA4@mail.bitblocks.com> <7C29D00C-94C0-4550-B1B2-CE307482B544@FreeBSD.org> <20170120203106.CD2C8124AEA4@mail.bitblocks.com> <20170120205933.8948A124AEA3@mail.bitblocks.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-Mailer: MailMate (2.0BETAr6072) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jan 2017 21:17:22 -0000 On 20 Jan 2017, at 22:12, Ermal Luçi wrote: > Most probably your timeouts are aggressive on states garbage > collection. > Give a look to those state limit teardown it might improve things. > Less than 30 seconds seems extremely quick to time out. I also wouldn’t expect pf to set up NAT state in the middle of a TCP connection. It’s certainly worth a try to play with the timeouts though. It might be interesting to see what they’re set to right now. `pfctl -s all` should show them. Regards, Kristof From owner-freebsd-net@freebsd.org Fri Jan 20 21:17:35 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6F713CBA114 for ; Fri, 20 Jan 2017 21:17:35 +0000 (UTC) (envelope-from bakul@bitblocks.com) Received: from mail.bitblocks.com (ns1.bitblocks.com [173.228.5.8]) by mx1.freebsd.org (Postfix) with ESMTP id 543A91E11; Fri, 20 Jan 2017 21:17:34 +0000 (UTC) (envelope-from bakul@bitblocks.com) Received: from bitblocks.com (localhost [127.0.0.1]) by mail.bitblocks.com (Postfix) with ESMTP id 488D8124AEA5; Fri, 20 Jan 2017 13:17:34 -0800 (PST) To: =?UTF-8?Q?Ermal_Lu=C3=A7i?= cc: FreeBSD Net , Alan Somers Subject: Re: pf & NAT issue In-reply-to: Your message of "Fri, 20 Jan 2017 13:12:07 PST." References: <20170120083555.ACCF9124AEA4@mail.bitblocks.com> <7C29D00C-94C0-4550-B1B2-CE307482B544@FreeBSD.org> <20170120203106.CD2C8124AEA4@mail.bitblocks.com> <20170120205933.8948A124AEA3@mail.bitblocks.com> Comments: In-reply-to =?UTF-8?Q?Ermal_Lu=C3=A7i?= message dated "Fri, 20 Jan 2017 13:12:07 -0800." Date: Fri, 20 Jan 2017 13:17:34 -0800 From: Bakul Shah Message-Id: <20170120211734.488D8124AEA5@mail.bitblocks.com> X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jan 2017 21:17:35 -0000 On Fri, 20 Jan 2017 13:12:07 PST =?UTF-8?Q?Ermal_Lu=C3=A7i?= wrote: > --001a1148cecc40685805468d1ad2 > Content-Type: text/plain; charset=UTF-8 > > On Fri, Jan 20, 2017 at 12:59 PM, Bakul Shah wrote: > > > On Fri, 20 Jan 2017 21:43:33 +0100 "Kristof Provost" > > wrote: > > > On 20 Jan 2017, at 21:31, Bakul Shah wrote: > > > >> 11:56:28.168693 IP 192.168.125.7.65042 > 149.20.1.200.21: Flags [P.], > > > >> seq 1:10, ack 55, win 1026, options [nop,nop,TS val 198426 ecr > > > >> 1468113725], length 9 > > > > < 11:56:28.168712 IP 173.228.5.8.52015 > 149.20.1.200.21: Flags [P.], > > > > seq 3080825147:3080825156, ack 3912707414, win 1026, options > > > > [nop,nop,TS val 198426 ecr 1468113725], length 9 > > > > > > > > Right here we see the problem. NAT mapping for the > > > > port changed from 63716 to 52015. > > > > > > > Changing source ports is an entirely normal NAT behaviour. > > > > > > The best explanation is this: imagine that you have two clients A and B, > > > both connect to X on port 80 via the NAT gateway G. > > > Both use port 1000 as their source port. > > > A connects, and the gateway maps A:1000 -> X:80 to G:1000 -> X:80. > > > B connects, and now the gateway has to map B:1000 -> X:80 onto G:1000 -> > > > X:80, but then it wouldn't be able to tell the two connections apart. > > > That't can remap it onto G:1001 -> X:80 instead. > > > > It is the same connection! As a tcp connection is identified > > by , If the port number > > changes on the same connection, the remote side would see this > > as a separate connection. > > > > Most probably your timeouts are aggressive on states garbage collection. > Give a look to those state limit teardown it might improve things. $ pfctl -s timeout tcp.first 120s tcp.opening 30s tcp.established 86400s tcp.closing 900s tcp.finwait 45s tcp.closed 90s tcp.tsdiff 30s udp.first 60s udp.single 30s udp.multiple 60s icmp.first 20s icmp.error 10s other.first 60s other.single 30s other.multiple 60s frag 30s interval 10s adaptive.start 6000 states adaptive.end 12000 states src.track 30s local port num changed after 23 seconds. All the tcp.* timeouts seem ok. IIRC internal is used for IP frags. Thanks Bakul From owner-freebsd-net@freebsd.org Fri Jan 20 22:22:42 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B4596CB9709 for ; Fri, 20 Jan 2017 22:22:42 +0000 (UTC) (envelope-from ermal.luci@gmail.com) Received: from mail-it0-x22b.google.com (mail-it0-x22b.google.com [IPv6:2607:f8b0:4001:c0b::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7A6731658; Fri, 20 Jan 2017 22:22:42 +0000 (UTC) (envelope-from ermal.luci@gmail.com) Received: by mail-it0-x22b.google.com with SMTP id r185so30770246ita.0; Fri, 20 Jan 2017 14:22:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=FuGKN+P+jILfDb2/Xfx5WXDi43zaje23eRTu+UHummA=; b=Lc1rr5CJ2CO2q+kQn1k4nJFr2taCSxQEya8jCSpfahplme/d0eve68SF1tzLaSZbrj wrmDN6nV9k2ZB060OmBnY8PXqaOqUMOBPiEI830dLkGTUPNc7VYq95M3SuhT9A31WaqR dcKAn1wNoKyib/WeGCPP1kafTzlX0xCe24E9DHZh57jgZl2YQnFLbAClz5YZwb5SBfCx uxSiZeOFbSC6JHDHmqOaiJGph/9ENzhGMRVaF6YmZFnP+YDl8102VKeLJYs+LAiOPN/U 1VsVx12Hq9FaUJWSB9o6OpZOpt8WbJbllfWV+LArrdSGy6BpEF9mi1LG2KX1pSCRRHH6 Z4ww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=FuGKN+P+jILfDb2/Xfx5WXDi43zaje23eRTu+UHummA=; b=fdq28a7x/9HIxzdrrVpkowcJvJR682qFrw6+cYW4tbT00/6QuhKuLKDuYEIZdKxI3T F4fh/4Saq1q5XMt52A1EMEbO1CWrIc6RIbLrNt0ayXOLwZXXIHwFZd9S4tPu5ps43hpX 7Z5zvVSBFvnAkGCNUvZddebFIDlHtqxzlawZ62EtS1t8rQOiqwqx1/vRRvR4I9C09p1W qmOwKLXo+Y3HEwyoZjuZ2MjeiuAsGXMh/SPlzvroLEtNx6VMk/wp+9RGD9FnL2jjubzW dLYS4+xI78q/c9X5Q+3P/EfdD6axfXzOt1q1yamTqPEBEjW6st1xcrlg7fMmiQMw9uk/ 7GqA== X-Gm-Message-State: AIkVDXLLFaVxJKDAK+PbaZVfjWoiibYKyKqNtHi/zLMl4bn1QIcu1cvSieEuAmLTS+8PbUR/30SgsMU5l6OuFQ== X-Received: by 10.36.65.4 with SMTP id x4mr5721328ita.69.1484950961947; Fri, 20 Jan 2017 14:22:41 -0800 (PST) MIME-Version: 1.0 Sender: ermal.luci@gmail.com Received: by 10.107.129.99 with HTTP; Fri, 20 Jan 2017 14:22:41 -0800 (PST) In-Reply-To: <20170120211734.488D8124AEA5@mail.bitblocks.com> References: <20170120083555.ACCF9124AEA4@mail.bitblocks.com> <7C29D00C-94C0-4550-B1B2-CE307482B544@FreeBSD.org> <20170120203106.CD2C8124AEA4@mail.bitblocks.com> <20170120205933.8948A124AEA3@mail.bitblocks.com> <20170120211734.488D8124AEA5@mail.bitblocks.com> From: =?UTF-8?Q?Ermal_Lu=C3=A7i?= Date: Fri, 20 Jan 2017 14:22:41 -0800 X-Google-Sender-Auth: TwWQqlXyidb1gjgCjgfD7I_oZdc Message-ID: Subject: Re: pf & NAT issue To: Bakul Shah Cc: FreeBSD Net , Alan Somers Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jan 2017 22:22:42 -0000 On Fri, Jan 20, 2017 at 1:17 PM, Bakul Shah wrote: > On Fri, 20 Jan 2017 13:12:07 PST =?UTF-8?Q?Ermal_Lu=C3=A7i?= < > eri@freebsd.org> wrote: > > --001a1148cecc40685805468d1ad2 > > Content-Type: text/plain; charset=UTF-8 > > > > On Fri, Jan 20, 2017 at 12:59 PM, Bakul Shah > wrote: > > > > > On Fri, 20 Jan 2017 21:43:33 +0100 "Kristof Provost" > > > wrote: > > > > On 20 Jan 2017, at 21:31, Bakul Shah wrote: > > > > >> 11:56:28.168693 IP 192.168.125.7.65042 > 149.20.1.200.21: Flags > [P.], > > > > >> seq 1:10, ack 55, win 1026, options [nop,nop,TS val 198426 ecr > > > > >> 1468113725], length 9 > > > > > < 11:56:28.168712 IP 173.228.5.8.52015 > 149.20.1.200.21: Flags > [P.], > > > > > seq 3080825147:3080825156, ack 3912707414, win 1026, options > > > > > [nop,nop,TS val 198426 ecr 1468113725], length 9 > > > > > > > > > > Right here we see the problem. NAT mapping for the > > > > > port changed from 63716 to 52015. > > > > > > > > > Changing source ports is an entirely normal NAT behaviour. > > > > > > > > The best explanation is this: imagine that you have two clients A > and B, > > > > both connect to X on port 80 via the NAT gateway G. > > > > Both use port 1000 as their source port. > > > > A connects, and the gateway maps A:1000 -> X:80 to G:1000 -> X:80. > > > > B connects, and now the gateway has to map B:1000 -> X:80 onto > G:1000 -> > > > > X:80, but then it wouldn't be able to tell the two connections apart. > > > > That't can remap it onto G:1001 -> X:80 instead. > > > > > > It is the same connection! As a tcp connection is identified > > > by , If the port number > > > changes on the same connection, the remote side would see this > > > as a separate connection. > > > > > > > Most probably your timeouts are aggressive on states garbage collection. > > Give a look to those state limit teardown it might improve things. > > $ pfctl -s timeout > tcp.first 120s > tcp.opening 30s > tcp.established 86400s > tcp.closing 900s > tcp.finwait 45s > tcp.closed 90s > tcp.tsdiff 30s > udp.first 60s > udp.single 30s > udp.multiple 60s > icmp.first 20s > icmp.error 10s > other.first 60s > other.single 30s > other.multiple 60s > frag 30s > interval 10s > adaptive.start 6000 states > adaptive.end 12000 states > src.track 30s > > local port num changed after 23 seconds. All the tcp.* > timeouts seem ok. IIRC internal is used for IP frags. > Well if you do not overcome those limits than it means you most probably are receiving, out of order traffic pf does not like that much. That would be my guess if the limits and GC are not triggered. One other thing is those ip-option stats you have are weird and you probably should drop ip-option traffic. > Thanks > Bakul > -- Ermal From owner-freebsd-net@freebsd.org Sat Jan 21 00:27:31 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CE455CB8644 for ; Sat, 21 Jan 2017 00:27:31 +0000 (UTC) (envelope-from eric.joyner@intel.com) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "ORSMGA101.jf.intel.com", Issuer "Intel External Issuing CA 6A" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A4C2F1ACE for ; Sat, 21 Jan 2017 00:27:31 +0000 (UTC) (envelope-from eric.joyner@intel.com) Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga101.jf.intel.com with ESMTP; 20 Jan 2017 16:26:21 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.33,260,1477983600"; d="scan'208,217";a="1115719461" Received: from fmsmsx108.amr.corp.intel.com ([10.18.124.206]) by fmsmga002.fm.intel.com with ESMTP; 20 Jan 2017 16:26:21 -0800 Received: from fmsmsx151.amr.corp.intel.com (10.18.125.4) by FMSMSX108.amr.corp.intel.com (10.18.124.206) with Microsoft SMTP Server (TLS) id 14.3.248.2; Fri, 20 Jan 2017 16:26:20 -0800 Received: from crsmsx101.amr.corp.intel.com (172.18.63.136) by FMSMSX151.amr.corp.intel.com (10.18.125.4) with Microsoft SMTP Server (TLS) id 14.3.248.2; Fri, 20 Jan 2017 16:26:13 -0800 Received: from crsmsx102.amr.corp.intel.com ([169.254.2.243]) by CRSMSX101.amr.corp.intel.com ([169.254.1.75]) with mapi id 14.03.0248.002; Fri, 20 Jan 2017 18:26:12 -0600 From: "Joyner, Eric" To: Kevin Bowling , "freebsd-net@freebsd.org" CC: "Cramer, Jeb J" Subject: RE: ethctl Thread-Topic: ethctl Thread-Index: AQHSctFu7eVtD0Gk7E2LJk6PziNGAKFB+3vggAAYr0A= Date: Sat, 21 Jan 2017 00:26:10 +0000 Message-ID: <1E98AE6AA3078D42935C9F7A3CA7C4DC36A24F1B@CRSMSX102.amr.corp.intel.com> References: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiYWQzMjA3MDAtNDU0Zi00ZGY4LTkwYjktYzk3Nzg5MmZiOWZlIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX1BVQkxJQyJ9XX1dfSwiU3ViamVjdExhYmVscyI6W10sIlRNQ1ZlcnNpb24iOiIxNS45LjYuNiIsIlRydXN0ZWRMYWJlbEhhc2giOiJQRG9aN09rWkZaalQySzFCYW92cmFNYnlvZWdMcEtSUVRlN0dDbXYzQzQ4PSJ9 x-ctpclassification: CTP_PUBLIC x-originating-ip: [172.18.205.10] MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Jan 2017 00:27:31 -0000 UmVzZW5kaW5nIHdpdGggc21hbGxlciByZWNpcGllbnRzIGxpc3TigKYNCg0KRnJvbTogSm95bmVy LCBFcmljDQpTZW50OiBGcmlkYXksIEphbnVhcnkgMjAsIDIwMTcgMzoxNyBQTQ0KVG86ICdLZXZp biBCb3dsaW5nJyA8a2V2aW4uYm93bGluZ0BrZXYwMDkuY29tPjsgZnJlZWJzZC1uZXRAZnJlZWJz ZC5vcmcNCkNjOiBTY290dCBMb25nIDxzY290dGxAbmV0ZmxpeC5jb20+OyBEcmV3IEdhbGxhdGlu IDxnYWxsYXRpbkBuZXRmbGl4LmNvbT47IE5hdmRlZXAgUGFyaGFyIDxuYXZkZWVwQGNoZWxzaW8u Y29tPjsgT2RlZCBTaGFub29uIDxvZGVkc0BtZWxsYW5veC5jb20+OyBocHNAZnJlZWJzZC5vcmc7 IE1hdHRoZXcgTWFjeSA8bW1hY3lAbmV4dGJzZC5vcmc+OyBDcmFtZXIsIEplYiBKIDxqZWIuai5j cmFtZXJAaW50ZWwuY29tPjsgYXJ5YmNoaWtAZnJlZWJzZC5vcmc7IHNodXJkQGZyZWVic2Qub3Jn OyBTZWFuIEJydW5vIDxzYnJ1bm9AZnJlZWJzZC5vcmc+OyBHZW9yZ2UgTmV2aWxsZS1OZWlsIDxn bm5AZnJlZWJzZC5vcmc+DQpTdWJqZWN0OiBSRTogZXRoY3RsDQoNCkZvciBhIHNwZWNpYWxpemVk IHJlcXVpcmVtZW50LCBJ4oCZZCBsaWtlIHRvIGJlIGFibGUgdG8gdmlldyBWRiBpbmZvcm1hdGlv biB1bmRlcm5lYXRoIHRoZSBvdXRwdXQgZm9yIGEgUEYgaW50ZXJmYWNlLiBMaWtlIHdoYXQgdGhl IOKAnGlw4oCdIHV0aWxpdHkgc2hvd3MgYW5kIGxldHMgeW91IGNoYW5nZSB3aGVuIHlvdSB1c2Ug dGhlIOKAnGxpbmvigJ0gY29tbWFuZDogbWFjIGFkZHJlc3MsIHZsYW4sIGFudGktc3Bvb2Ygc3Rh dHVzLCBhbmQgbGluayBzdGF0dXMuIElvdmN0bCB3b3VsZCBiZSBva2F5IGZvciBjb25maWd1cmlu ZyB0aGVzZSB0aGluZ3Mgb24gVkYgY3JlYXRpb24sIGJ1dCBJ4oCZZCBsaWtlIHRvIGJlIGFibGUg dG8gY29uZmlndXJlIHRoZXNlIHRoaW5ncyBxdWlja2x5IGZyb20gdGhlIGNvbW1hbmQgbGluZSwg YW5kIHdpdGhvdXQgcmUtY3JlYXRpbmcgdGhlIFZGLg0KDQpUaGF0IGluZm9ybWF0aW9uIGNvdWxk IGJlIGFkZGVkIHRvIGlmY29uZmlnLCBidXQgbGlrZSBsdWlnaSBzYWlkLCBpZmNvbmZpZyBpcyBh bHJlYWR5IGh1Z2UsIGFuZCBJIHRoaW5rIGl0IHdvdWxkIGJsb2F0IHRoZSBvdXRwdXQgbW9yZS4g V2UgY291bGQgdHJ5IGNoYW5naW5nIGlmY29uZmlnIHRvIHVzZSB0aGUgaXAgc3ludGF4LCB3aGVy ZSB0aGVyZeKAmXMgb25lIHRvcCBsZXZlbCBjb21tYW5kLCBidXQgdGhlIGFyZ3VtZW50cyBpbW1l ZGlhdGVseSBhZnRlciBjb25jZXJuIGRpZmZlcmVudCBhcmVhcyBvZiBvcGVyYXRpb24gZm9yIEV0 aGVybmV0IGRldmljZXMuIGlwIGhhcyDigJxsaW5r4oCdIChmb3Igc2V0dGluZyBMMiBzZXR0aW5n cyBsaWtlIGxpbmsgYW5kIHZsYW5zKSBhbmQg4oCcYWRkcuKAnSBmb3IgaXB2NC9pcHY2IHN0dWZm LCBmb3IgZXhhbXBsZS4NCg0KRm9yIGEgcG90ZW50aWFsIGNvbW1vbiByZXF1aXJlbWVudCwgaXQg d291bGQgYWxzbyBiZSBuaWNlIHRvIGJlIGFibGUgdG8gcG9sbCBhIGRyaXZlciB0byBzZWUgd2hp Y2ggbmV0d29yayBpbnRlcmZhY2VzIGFyZSBiZWluZyBzcGF3bmVkIGZyb20gaXQuIE5vdCBqdXN0 IGZvciBWRnMsIGJ1dCBpdOKAmWQgYmUgdXNlZnVsIGZvciB3aGVuIGEgUEbigJlzIGludGVyZmFj ZSBuYW1lIGRvZXNu4oCZdCBtYXRjaCB0aGUgL2Rldi9wY2kgbmFtZSwgb3IgaWYgYSBkcml2ZXIg ZG9lcyBxdWV1ZSBzcGxpdHRpbmcgdG8gY3JlYXRlIG11bHRpcGxlIG5ldHdvcmsgaW50ZXJmYWNl cyBmcm9tIG9uZSBwaHlzaWNhbCBwb3J0LiBUaGF0IHdvdWxkIGEgc29sdmUgYW4gaXNzdWUgZm9y IHVzIHJpZ2h0IG5vdy4NCg0KSeKAmW0gc3VyZSBtb3JlIHRoaW5ncyB3aWxsIGNvbWUgdXAgZnJv bSB0aGUgSW50ZWwgc2lkZS4NCg0KLSBFcmljDQoNCkZyb206IEtldmluIEJvd2xpbmcgW21haWx0 bzprZXZpbi5ib3dsaW5nQGtldjAwOS5jb21dDQpTZW50OiBUaHVyc2RheSwgSmFudWFyeSAxOSwg MjAxNyA3OjU4IFBNDQpUbzogZnJlZWJzZC1uZXRAZnJlZWJzZC5vcmc8bWFpbHRvOmZyZWVic2Qt bmV0QGZyZWVic2Qub3JnPg0KQ2M6IFNjb3R0IExvbmcgPHNjb3R0bEBuZXRmbGl4LmNvbTxtYWls dG86c2NvdHRsQG5ldGZsaXguY29tPj47IERyZXcgR2FsbGF0aW4gPGdhbGxhdGluQG5ldGZsaXgu Y29tPG1haWx0bzpnYWxsYXRpbkBuZXRmbGl4LmNvbT4+OyBOYXZkZWVwIFBhcmhhciA8bmF2ZGVl cEBjaGVsc2lvLmNvbTxtYWlsdG86bmF2ZGVlcEBjaGVsc2lvLmNvbT4+OyBPZGVkIFNoYW5vb24g PG9kZWRzQG1lbGxhbm94LmNvbTxtYWlsdG86b2RlZHNAbWVsbGFub3guY29tPj47IGhwc0BmcmVl YnNkLm9yZzxtYWlsdG86aHBzQGZyZWVic2Qub3JnPjsgTWF0dGhldyBNYWN5IDxtbWFjeUBuZXh0 YnNkLm9yZzxtYWlsdG86bW1hY3lAbmV4dGJzZC5vcmc+PjsgQ3JhbWVyLCBKZWIgSiA8amViLmou Y3JhbWVyQGludGVsLmNvbTxtYWlsdG86amViLmouY3JhbWVyQGludGVsLmNvbT4+OyBKb3luZXIs IEVyaWMgPGVyaWMuam95bmVyQGludGVsLmNvbTxtYWlsdG86ZXJpYy5qb3luZXJAaW50ZWwuY29t Pj47IGFyeWJjaGlrQGZyZWVic2Qub3JnPG1haWx0bzphcnliY2hpa0BmcmVlYnNkLm9yZz47IHNo dXJkQGZyZWVic2Qub3JnPG1haWx0bzpzaHVyZEBmcmVlYnNkLm9yZz47IFNlYW4gQnJ1bm8gPHNi cnVub0BmcmVlYnNkLm9yZzxtYWlsdG86c2JydW5vQGZyZWVic2Qub3JnPj47IEdlb3JnZSBOZXZp bGxlLU5laWwgPGdubkBmcmVlYnNkLm9yZzxtYWlsdG86Z25uQGZyZWVic2Qub3JnPj4NClN1Ympl Y3Q6IFJGQzogZXRoY3RsDQoNCkdyZWV0aW5ncywNCg0KSSdtIGNhc3RpbmcgYSB3aWRlIG5ldCBp biBjYywgdHJ5IHRvIGtlZXAgdGhlIG5vaXNlIG1pbmltYWwgYnV0IHdlIG5lZWQgc29tZSBpbnB1 dCBmcm9tIGEgdmFyaWV0eSBvZiBIVyB2ZW5kb3JzLg0KDQpJIGhhdmUgaGVhcmQgZnJvbSBzZXZl cmFsIHZlbmRvcnMgdGhlIG5lZWQgZm9yIGEgTklDIGNvbmZpZ3VyYXRpb24gdG9vbC4gIENoZWxz aW8gc2hpcHMgYSBjeGdiL2N4Z2JldG9vbCBpbiBGcmVlQlNEIGFzIG9uZSBleGFtcGxlLiAgVGhl cmUgaXMgcHJlY2VkZW5jZSBmb3Igc29tZSBub2QgdG93YXJkIGEgYmFzaWMgdW5pZmllZCB0b29s IGluIExpbnV4IGV0aHRvb2wuDQoNCkZyb20geW91ciBwZXJzcGVjdGl2ZSwNCjEpIFdoYXQgYXJl IHRoZSBjb21tb24gcmVxdWlyZW1lbnRzPw0KMikgV2hhdCBhcmUgc3BlY2lhbGl6ZWQgcmVxdWly ZW1lbnRzPyBGb3IgaW5zdGFuY2UgYXMgYSBmdWxsIFRDUCBvZmZsb2FkIGNhcmQgQ2hlbHNpbyBu ZWVkcyB0aGluZ3Mgb3RoZXJzIHdvbnQNCjMpIFdoYXQgc2hvdWxkIGl0IF9ub3RfIGRvPyAgU2V2 ZXJhbCBvZiB5b3UgaGF2ZSBleHBlcmllbmNlIGRvaW5nIEV0aGVybmV0IGRyaXZlciBkZXYgb24g bWFueSBwbGF0Zm9ybXMgc28gd2Ugc2hvdWxkIGF0dGVtcHQgdG8gYXZvaWQgcmVwZWF0aW5nIHBh c3QgZGVzaWduIG1pc3Rha2VzLg0KDQpJIGV4cGVjdCB3ZSBjYW4gYWNoaWV2ZSBzb21lIGxldmVs IG9mIGludmVyc2lvbiBzbyB0aGUgZGV2aWNlIHNwZWNpZmljIGNvZGUgY2FuIGxpdmUgY2xvc2Ug dG8gdGhlIGRyaXZlciBhbmQgcGx1ZyBpbnRvIHRoZSBldGhjdGwgZnJhbWV3b3JrLiAgSXQgc2hv dWxkIGJlIGdlbmVyYWwgZW5vdWdoIHRvIGFkZCBjb21wbGV0ZWx5IG5ldyB0b3AgbGV2ZWwgY29t bWFuZHMsIHNvIHZlbmRvcnMgY2FuIGltcGxlbWVudCBIVyBzcGVjaWZpYyBmZWF0dXJlcy4gIE9u IHRoZSBvdGhlciBoYW5kLCB3ZSBzaG91bGQgYXR0ZW1wdCB0byBob29rIGludG8gY29tbW9uIGNv cmUgZm9yIGZlYXR1cmVzIGV2ZXJ5IE5JQyBwcm92aWRlcywgd2l0aCBhIGZvY3VzIG9uIGlmbGli Lg0KDQpJIHdpbGwgZnVuZCBNYXR0IE1hY3kgdG8gZG8gdGhlIG92ZXJhbGwgZGVzaWduIGFuZCBp bXBsZW1lbnRhdGlvbi4NCg0KUmVnYXJkcywNCktldmluIEJvd2xpbmcsIG9uIGJlaGFsZiBvZiBM aW1lbGlnaHQgTmV0d29ya3MgZm9yIHRoaXMgZWZmb3J0DQo= From owner-freebsd-net@freebsd.org Sat Jan 21 04:21:19 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id ADE71CBABE1 for ; Sat, 21 Jan 2017 04:21:19 +0000 (UTC) (envelope-from bakul@bitblocks.com) Received: from mail.bitblocks.com (ns1.bitblocks.com [173.228.5.8]) by mx1.freebsd.org (Postfix) with ESMTP id 95E1D1C9B; Sat, 21 Jan 2017 04:21:19 +0000 (UTC) (envelope-from bakul@bitblocks.com) Received: from bitblocks.com (localhost [127.0.0.1]) by mail.bitblocks.com (Postfix) with ESMTP id 722C6124AEA4; Fri, 20 Jan 2017 20:21:18 -0800 (PST) To: =?UTF-8?Q?Ermal_Lu=C3=A7i?= cc: FreeBSD Net , Alan Somers Subject: Re: pf & NAT issue In-reply-to: Your message of "Fri, 20 Jan 2017 14:22:41 PST." References: <20170120083555.ACCF9124AEA4@mail.bitblocks.com> <7C29D00C-94C0-4550-B1B2-CE307482B544@FreeBSD.org> <20170120203106.CD2C8124AEA4@mail.bitblocks.com> <20170120205933.8948A124AEA3@mail.bitblocks.com> <20170120211734.488D8124AEA5@mail.bitblocks.com> Comments: In-reply-to =?UTF-8?Q?Ermal_Lu=C3=A7i?= message dated "Fri, 20 Jan 2017 14:22:41 -0800." Date: Fri, 20 Jan 2017 20:21:18 -0800 From: Bakul Shah Message-Id: <20170121042118.722C6124AEA4@mail.bitblocks.com> X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Jan 2017 04:21:19 -0000 I finally had some time to look at the sources & noticed /sys/netpfil/pf/pf.c:pf_purge_thread now runs 10 times a second instead of once a second, which gave me the idea of increasing "interval" timeout by a factor of 10 and this seems to have mostly fixed the problem. But I don't know where the actual problem is. The logic is too complicated to understand in a few minutes so I didn't try to find the root cause at the moment. [But I don't understand why pf times out normal connections. Long lasting idle connections are perfectly fine. And fragment GC should not be coupled with connection state expiry] Many thanks for various suggestions as that forced me think :-) Bakul From owner-freebsd-net@freebsd.org Sat Jan 21 08:12:46 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E2D35CBB745 for ; Sat, 21 Jan 2017 08:12:46 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from reviews.nyi.freebsd.org (reviews.nyi.freebsd.org [IPv6:2610:1c1:1:607c::16:b]) by mx1.freebsd.org (Postfix) with ESMTP id 9F36315B3 for ; Sat, 21 Jan 2017 08:12:46 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by reviews.nyi.freebsd.org (Postfix, from userid 1346) id 99500304C7; Sat, 21 Jan 2017 08:12:45 +0000 (UTC) Date: Sat, 21 Jan 2017 08:12:45 +0000 To: freebsd-net@freebsd.org From: "ale (Alex Dupre)" Reply-to: D9270+325+bbd470fd257eef1b@reviews.freebsd.org Subject: [Differential] D9270: Add support for user-supplied Host-Uniq tag in Netgraph PPPoE Message-ID: X-Priority: 3 X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , , , Thread-Topic: D9270: Add support for user-supplied Host-Uniq tag in Netgraph PPPoE X-Herald-Rules: <28>, <8> X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: X-Phabricator-Cc: X-Phabricator-Cc: Precedence: bulk Thread-Index: NTZkNjQzYWQxOGQ3MGJlZTIzOGZhZmQ4NGNm MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="b1_ef54bcc72775c6f9b6c4c9e9af2c7489" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Jan 2017 08:12:47 -0000 --b1_ef54bcc72775c6f9b6c4c9e9af2c7489 Content-Type: text/plain; charset = "utf-8" Content-Transfer-Encoding: base64 YWxlIGNyZWF0ZWQgdGhpcyByZXZpc2lvbi4KYWxlIGFkZGVkIHJldmlld2VyczogbmV0d29yaywg anVsaWFuLgphbGUgYWRkZWQgc3Vic2NyaWJlcnM6IGZyZWVic2QtbmV0LWxpc3QsIG1hbmRyZWUu CmFsZSBzZXQgdGhlIHJlcG9zaXRvcnkgZm9yIHRoaXMgcmV2aXNpb24gdG8gclMgRnJlZUJTRCBz cmMgcmVwb3NpdG9yeS4KSGVyYWxkIGFkZGVkIGEgc3Vic2NyaWJlcjogaW1wLgpIZXJhbGQgYWRk ZWQgYSByZXZpZXdlcjogbWFucGFnZXMuCgpSRVZJU0lPTiBTVU1NQVJZCiAgQWRkIHN1cHBvcnQg Zm9yIHVzZXItc3VwcGxpZWQgSG9zdC1VbmlxIHRhZyBpbiBOZXRncmFwaCBQUFBvRS4KICBBIGZl dyBJU1AgZmlsdGVyIFBBREkgcmVxdWVzdHMgYmFzZWQgb24gc3VjaCB0YWcsIHRvIGZvcmNlIHRo ZSB1c2Ugb2YgdGhlaXIgb3duIHJvdXRlcnMuCiAgVGhlIGN1c3RvbSBIb3N0LVVuaXEgdGFnIGlz IHBhc3NlZCBpbiB0aGUgTkdNX1BQUE9FX0NPTk5FQ1QgY29udHJvbCBtZXNzYWdlLCBzbyBpdCBj YW4gYmUgdXNlZCB3aXRoIEZyZWVCU0QgcHBwKDgpIHdpdGhvdXQgYW55IG90aGVyIGNoYW5nZS4K ClRFU1QgUExBTgogIEkndmUgdGVzdGVkIHRoZSB0aHJlZSBwb3NzaWJsZSBIb3N0LVVuaXEgdGFn cyAoYXV0by1nZW5lcmF0ZWQgYXMgYmVmb3JlLCBsb25nIHBsYWluIHN0cmluZyBhbmQgbG9uZyBi aW5hcnkgc3RyaW5nKSB3aXRoIG15IG93biBJU1AgY29ubmVjdGlvbi4KClJFUE9TSVRPUlkKICBy UyBGcmVlQlNEIHNyYyByZXBvc2l0b3J5CgpSRVZJU0lPTiBERVRBSUwKICBodHRwczovL3Jldmll d3MuZnJlZWJzZC5vcmcvRDkyNzAKCkFGRkVDVEVEIEZJTEVTCiAgc2hhcmUvbWFuL21hbjQvbmdf cHBwb2UuNAogIHN5cy9uZXRncmFwaC9uZ19wcHBvZS5jCgpFTUFJTCBQUkVGRVJFTkNFUwogIGh0 dHBzOi8vcmV2aWV3cy5mcmVlYnNkLm9yZy9zZXR0aW5ncy9wYW5lbC9lbWFpbHByZWZlcmVuY2Vz LwoKVG86IGFsZSwgI25ldHdvcmssICNtYW5wYWdlcywganVsaWFuCkNjOiBtYW5kcmVlLCBpbXAs IGZyZWVic2QtbmV0LWxpc3QK --b1_ef54bcc72775c6f9b6c4c9e9af2c7489 Content-Type: text/x-patch; charset=utf-8; name="D9270.24264.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="D9270.24264.patch" ZGlmZiAtLWdpdCBhL3N5cy9uZXRncmFwaC9uZ19wcHBvZS5jIGIvc3lzL25ldGdyYXBoL25nX3Bw cG9lLmMKLS0tIGEvc3lzL25ldGdyYXBoL25nX3BwcG9lLmMKKysrIGIvc3lzL25ldGdyYXBoL25n X3BwcG9lLmMKQEAgLTIyNiw5ICsyMjYsMTEgQEAKIAljb25zdCBzdHJ1Y3QgcHBwb2VfdGFnCSp0 YWdzW05VTVRBR1NdOwogCXVfaW50CQkJc2VydmljZV9sZW47CiAJdV9pbnQJCQlhY19uYW1lX2xl bjsKKwl1X2ludAkJCWhvc3RfdW5pcV9sZW47CiAKIAlzdHJ1Y3QgZGF0YXRhZwkJc2VydmljZTsK IAlzdHJ1Y3QgZGF0YXRhZwkJYWNfbmFtZTsKKwlzdHJ1Y3QgZGF0YXRhZwkJaG9zdF91bmlxOwog fTsKIHR5cGVkZWYgc3RydWN0IHNlc3NfbmVnICpuZWdwOwogCkBAIC01ODksMTggKzU5MSwyMCBA QAogcHBwb2VfZmluZHVuaXEobm9kZV9wIG5vZGUsIGNvbnN0IHN0cnVjdCBwcHBvZV90YWcgKnRh ZykKIHsKIAlob29rX3AJaG9vayA9IE5VTEw7Ci0JdW5pb24gdW5pcSB1bmlxOworCXNlc3NwCXNw OwogCi0JYmNvcHkodGFnICsgMSwgdW5pcS5ieXRlcywgc2l6ZW9mKHZvaWQgKikpOwogCS8qIEN5 Y2xlIHRocm91Z2ggYWxsIGtub3duIGhvb2tzLiAqLwogCUxJU1RfRk9SRUFDSChob29rLCAmbm9k ZS0+bmRfaG9va3MsIGhrX2hvb2tzKSB7CiAJCS8qIFNraXAgYW55IG5vbnNlc3Npb24gaG9vay4g Ki8KIAkJaWYgKE5HX0hPT0tfUFJJVkFURShob29rKSA9PSBOVUxMKQogCQkJY29udGludWU7Ci0J CWlmICh1bmlxLnBvaW50ZXIgPT0gTkdfSE9PS19QUklWQVRFKGhvb2spKQorCQlzcCA9IE5HX0hP T0tfUFJJVkFURShob29rKTsKKwkJaWYgKHNwLT5uZWctPmhvc3RfdW5pcV9sZW4gPT0gbnRvaHMo dGFnLT50YWdfbGVuKSAmJgorCQkgICAgYmNtcChzcC0+bmVnLT5ob3N0X3VuaXEuZGF0YSwgKGNv bnN0IGNoYXIgKikodGFnICsgMSksCisJCSAgICAgc3AtPm5lZy0+aG9zdF91bmlxX2xlbikgPT0g MCkKIAkJCWJyZWFrOwogCX0KLQlDVFIzKEtUUl9ORVQsICIlMjBzOiBtYXRjaGVkICVwIGZvciAl cCIsIF9fZnVuY19fLCBob29rLCB1bmlxLnBvaW50ZXIpOworCUNUUjMoS1RSX05FVCwgIiUyMHM6 IG1hdGNoZWQgJXAgZm9yICVwIiwgX19mdW5jX18sIGhvb2ssIHNwKTsKIAogCXJldHVybiAoaG9v ayk7CiB9CkBAIC04NTMsMjMgKzg1Nyw2NyBAQAogCQkJICogU2V0IHVwIHRoZSBzZXNzaW9uIHRv IHRoZSBjb3JyZWN0IHN0YXRlIGFuZAogCQkJICogc3RhcnQgaXQuCiAJCQkgKi8KLQkJCWludAlp LCBhY25sZW4gPSAwLCBhY25zZXAgPSAwLCBzcnZsZW47CisJCQlpbnQJYWNucG9zLCBhY25sZW4g PSAwLCBhY25zZXAgPSAwOworCQkJaW50CWh1cG9zLCBodWxlbiA9IDAsIGh1c2VwID0gMDsKKwkJ CWludAlpLCBzcnZwb3MsIHNydmxlbjsKKwkJCWFjbnBvcyA9IDA7CiAJCQlmb3IgKGkgPSAwOyBp IDwgb3VybXNnLT5kYXRhX2xlbjsgaSsrKSB7CiAJCQkJaWYgKG91cm1zZy0+ZGF0YVtpXSA9PSAn XFwnKSB7CiAJCQkJCWFjbmxlbiA9IGk7CiAJCQkJCWFjbnNlcCA9IDE7CiAJCQkJCWJyZWFrOwog CQkJCX0KIAkJCX0KLQkJCXNydmxlbiA9IG91cm1zZy0+ZGF0YV9sZW4gLSBhY25sZW4gLSBhY25z ZXA7CisJCQlodXBvcyA9IGFjbmxlbiArIGFjbnNlcDsKKwkJCWZvciAoaSA9IGh1cG9zOyBpIDwg b3VybXNnLT5kYXRhX2xlbjsgaSsrKSB7CisJCQkJaWYgKG91cm1zZy0+ZGF0YVtpXSA9PSAnfCcp IHsKKwkJCQkJaHVsZW4gPSBpIC0gaHVwb3M7CisJCQkJCWh1c2VwID0gMTsKKwkJCQkJYnJlYWs7 CisJCQkJfQorCQkJfQorCQkJc3J2cG9zID0gaHVwb3MgKyBodWxlbiArIGh1c2VwOworCQkJc3J2 bGVuID0gb3VybXNnLT5kYXRhX2xlbiAtIHNydnBvczsKIAotCQkJYmNvcHkob3VybXNnLT5kYXRh LCBuZWctPmFjX25hbWUuZGF0YSwgYWNubGVuKTsKKwkJCWJjb3B5KG91cm1zZy0+ZGF0YSArIGFj bnBvcywgbmVnLT5hY19uYW1lLmRhdGEsIGFjbmxlbik7CiAJCQluZWctPmFjX25hbWVfbGVuID0g YWNubGVuOwogCisJCQluZWctPmhvc3RfdW5pcS5oZHIudGFnX3R5cGUgPSBQVFRfSE9TVF9VTklR OworCQkJaWYgKGh1bGVuID09IDApIHsKKwkJCQkvKiBOb3QgcHJvdmlkZWQsIGdlbmVyYXRlIG9u ZSAqLworCQkJCW5lZy0+aG9zdF91bmlxLmhkci50YWdfbGVuID0gaHRvbnMoc2l6ZW9mKHNwKSk7 CisJCQkJYmNvcHkoJnNwLCBuZWctPmhvc3RfdW5pcS5kYXRhLCBzaXplb2Yoc3ApKTsKKwkJCQlu ZWctPmhvc3RfdW5pcV9sZW4gPSBzaXplb2Yoc3ApOworCQkJfSBlbHNlIGlmIChodWxlbiA+IDIg JiYgb3VybXNnLT5kYXRhW2h1cG9zXSA9PSAnMCcgJiYKKwkJCSAgb3VybXNnLT5kYXRhW2h1cG9z ICsgMV0gPT0gJ3gnICYmIGh1bGVuICUgMiA9PSAwKSB7CisJCQkJLyogSGV4IGVuY29kZWQgKi8K KwkJCQlzdGF0aWMgY29uc3QgY2hhciBoZXhkaWdbMTZdID0gIjAxMjM0NTY3ODlhYmNkZWYiOwor CQkJCWludCBqOworCisJCQkJbmVnLT5ob3N0X3VuaXEuaGRyLnRhZ19sZW4gPSBodG9ucygodWlu dDE2X3QpKGh1bGVuIC8gMiAtIDEpKTsKKwkJCQlmb3IgKGkgPSAwOyBpIDwgaHVsZW4gLSAyOyBp KyspIHsKKwkJCQkJZm9yIChqID0gMDsKKwkJCQkJICAgICBqIDwgMTYgJiYKKwkJCQkJICAgICBv dXJtc2ctPmRhdGFbaHVwb3MgKyAyICsgaV0gIT0gaGV4ZGlnW2pdOworCQkJCQkgICAgIGorKyk7 CisJCQkJCWlmIChqID09IDE2KQorCQkJCQkJTEVBVkUoRUlOVkFMKTsKKwkJCQkJaWYgKGkgJSAy ID09IDApCisJCQkJCQluZWctPmhvc3RfdW5pcS5kYXRhW2kgLyAyXSA9IGogPDwgNDsKKwkJCQkJ ZWxzZQorCQkJCQkJbmVnLT5ob3N0X3VuaXEuZGF0YVtpIC8gMl0gfD0gajsKKwkJCQl9CisJCQkJ bmVnLT5ob3N0X3VuaXFfbGVuID0gaHVsZW4gLyAyIC0gMTsKKwkJCX0gZWxzZSB7CisJCQkJLyog UGxhaW4gc3RyaW5nICovCisJCQkJbmVnLT5ob3N0X3VuaXEuaGRyLnRhZ19sZW4gPSBodG9ucygo dWludDE2X3QpaHVsZW4pOworCQkJCWJjb3B5KG91cm1zZy0+ZGF0YSArIGh1cG9zLCBuZWctPmhv c3RfdW5pcS5kYXRhLCBodWxlbik7CisJCQkJbmVnLT5ob3N0X3VuaXFfbGVuID0gaHVsZW47CisJ CQl9CisKIAkJCW5lZy0+c2VydmljZS5oZHIudGFnX3R5cGUgPSBQVFRfU1JWX05BTUU7CiAJCQlu ZWctPnNlcnZpY2UuaGRyLnRhZ19sZW4gPSBodG9ucygodWludDE2X3Qpc3J2bGVuKTsKLQkJCWJj b3B5KG91cm1zZy0+ZGF0YSArIGFjbmxlbiArIGFjbnNlcCwKLQkJCSAgICBuZWctPnNlcnZpY2Uu ZGF0YSwgc3J2bGVuKTsKKwkJCWJjb3B5KG91cm1zZy0+ZGF0YSArIHNydnBvcywgbmVnLT5zZXJ2 aWNlLmRhdGEsIHNydmxlbik7CiAJCQluZWctPnNlcnZpY2VfbGVuID0gc3J2bGVuOwogCQkJcHBw b2Vfc3RhcnQoc3ApOwogCQkJYnJlYWs7CkBAIC0xMDYxLDEwICsxMTA5LDYgQEAKIAlub2RlX3AJ bm9kZSA9IE5HX0hPT0tfTk9ERShob29rKTsKIAlwcml2X3AJcHJpdnAgPSBOR19OT0RFX1BSSVZB VEUobm9kZSk7CiAJbmVncAluZWcgPSBzcC0+bmVnOwotCXN0cnVjdCB7Ci0JCXN0cnVjdCBwcHBv ZV90YWcgaGRyOwotCQl1bmlvbgl1bmlxCWRhdGE7Ci0JfSBfX3BhY2tlZCB1bmlxdGFnOwogCXN0 cnVjdCAgbWJ1ZiAqbTA7CiAJaW50CWVycm9yOwogCkBAIC0xMDgwLDExICsxMTI0LDggQEAKIAlt ZW1jcHkoKHZvaWQgKikmbmVnLT5wa3QtPnBrdF9oZWFkZXIuZWgsICZwcml2cC0+ZWgsCiAJICAg IHNpemVvZihzdHJ1Y3QgZXRoZXJfaGVhZGVyKSk7CiAJbmVnLT5wa3QtPnBrdF9oZWFkZXIucGgu Y29kZSA9IFBBRElfQ09ERTsKLQl1bmlxdGFnLmhkci50YWdfdHlwZSA9IFBUVF9IT1NUX1VOSVE7 Ci0JdW5pcXRhZy5oZHIudGFnX2xlbiA9IGh0b25zKCh1X2ludDE2X3Qpc2l6ZW9mKHVuaXF0YWcu ZGF0YSkpOwotCXVuaXF0YWcuZGF0YS5wb2ludGVyID0gc3A7CiAJaW5pdF90YWdzKHNwKTsKLQlp bnNlcnRfdGFnKHNwLCAmdW5pcXRhZy5oZHIpOworCWluc2VydF90YWcoc3AsICZuZWctPmhvc3Rf dW5pcS5oZHIpOwogCWluc2VydF90YWcoc3AsICZuZWctPnNlcnZpY2UuaGRyKTsKIAlpZiAocHJp dnAtPm1heF9wYXlsb2FkLmRhdGEgIT0gMCkKIAkJaW5zZXJ0X3RhZyhzcCwgJnByaXZwLT5tYXhf cGF5bG9hZC5oZHIpOwpAQCAtMTQzOCw4ICsxNDc5LDcgQEAKIAkJCSAqIEZvciBub3cgc2ltcGx5 IGFjY2VwdCB0aGUgZmlyc3Qgd2UgcmVjZWl2ZS4KIAkJCSAqLwogCQkJdXRhZyA9IGdldF90YWco cGgsIFBUVF9IT1NUX1VOSVEpOwotCQkJaWYgKCh1dGFnID09IE5VTEwpIHx8Ci0JCQkgICAgKG50 b2hzKHV0YWctPnRhZ19sZW4pICE9IHNpemVvZihzcCkpKSB7CisJCQlpZiAodXRhZyA9PSBOVUxM KSB7CiAJCQkJbG9nKExPR19OT1RJQ0UsICJuZ19wcHBvZVsleF06IG5vIGhvc3QgIgogCQkJCSAg ICAidW5pcXVlIGZpZWxkXG4iLCBub2RlLT5uZF9JRCk7CiAJCQkJTEVBVkUoRU5FVFVOUkVBQ0gp OwpAQCAtMTYwNSw4ICsxNjQ1LDcgQEAKIAkJCSAqIHNldCB1cyBpbnRvIFNlc3Npb24gbW9kZS4K IAkJCSAqLwogCQkJdXRhZyA9IGdldF90YWcocGgsIFBUVF9IT1NUX1VOSVEpOwotCQkJaWYgKCh1 dGFnID09IE5VTEwpIHx8Ci0JCQkgICAgKG50b2hzKHV0YWctPnRhZ19sZW4pICE9IHNpemVvZihz cCkpKSB7CisJCQlpZiAodXRhZyA9PSBOVUxMKSB7CiAJCQkJTEVBVkUgKEVORVRVTlJFQUNIKTsK IAkJCX0KIAkJCXNlbmRob29rID0gcHBwb2VfZmluZHVuaXEobm9kZSwgdXRhZyk7CmRpZmYgLS1n aXQgYS9zaGFyZS9tYW4vbWFuNC9uZ19wcHBvZS40IGIvc2hhcmUvbWFuL21hbjQvbmdfcHBwb2Uu NAotLS0gYS9zaGFyZS9tYW4vbWFuNC9uZ19wcHBvZS40CisrKyBiL3NoYXJlL21hbi9tYW40L25n X3BwcG9lLjQKQEAgLTM1LDcgKzM1LDcgQEAKIC5cIiAkRnJlZUJTRCQKIC5cIiAkV2hpc3RsZTog bmdfcHBwb2UuOCx2IDEuMSAxOTk5LzAxLzI1IDIzOjQ2OjI3IGFyY2hpZSBFeHAgJAogLlwiCi0u RGQgU2VwdGVtYmVyIDE1LCAyMDE1CisuRGQgSmFudWFyeSAyMCwgMjAxNwogLkR0IE5HX1BQUE9F IDQKIC5PcwogLlNoIE5BTUUKQEAgLTEwNCw4ICsxMDQsMTEgQEAKIEl0IG11c3QgYmUgbmV3bHkg Y3JlYXRlZCBhbmQgYSBzZXJ2aWNlIG5hbWUgY2FuIGJlIGdpdmVuIGFzIGFuIGFyZ3VtZW50Lgog SXQgaXMgbGVnYWwgdG8gc3BlY2lmeSBhIHplcm8tbGVuZ3RoIHNlcnZpY2UgbmFtZSwgdGhpcyBp cyBjb21tb24KIG9uIHNvbWUgRFNMIHNldHVwcy4KLUl0IGlzIHBvc3NpYmxlIHRvIHJlcXVlc3Qg YSBjb25uZWN0aW9uIHRvIGEgc3BlY2lmaWMKLWFjY2VzcyBjb25jZW50cmF0b3IgYnkgaXRzIG5h bWUgdXNpbmcgdGhlICJBQy1OYW1lXFxTZXJ2aWNlLU5hbWUiIHN5bnRheC4KK0l0IGlzIHBvc3Np YmxlIHRvIHJlcXVlc3QgYSBjb25uZWN0aW9uIHRvIGEgc3BlY2lmaWMgYWNjZXNzIGNvbmNlbnRy YXRvciwKK2FuZC9vciBzZXQgYSBzcGVjaWZpYyBob3N0IHVuaXEgdGFnLCByZXF1aXJlZCBieSBz b21lIEludGVybmV0IHByb3ZpZGVycywKK3VzaW5nIHRoZSAiW0FDLU5hbWVcXF1bSG9zdC1Vbmlx fF1TZXJ2aWNlLU5hbWUiIHN5bnRheC4KK1RvIHNldCBhIGJpbmFyeSBIb3N0LVVuaXEsIGl0IG11 c3QgYmUgZW5jb2RlZCBhcyBhIGhleGFkZWNpbWFsIGxvd2VyY2FzZQorc3RyaW5nIGFuZCBwcmVm aXhlZCB3aXRoICIweCIuCiBBIHNlc3Npb24gcmVxdWVzdCBwYWNrZXQgd2lsbCBiZSBicm9hZGNh c3RlZCBvbiB0aGUgRXRoZXJuZXQuCiBUaGlzIGNvbW1hbmQgdXNlcyB0aGUKIC5EdiBuZ3BwcG9l X2luaXRfZGF0YQoK --b1_ef54bcc72775c6f9b6c4c9e9af2c7489-- From owner-freebsd-net@freebsd.org Sat Jan 21 18:17:21 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B07CBCBBB89 for ; Sat, 21 Jan 2017 18:17:21 +0000 (UTC) (envelope-from hoomanfazaeli@gmail.com) Received: from mail-wm0-x243.google.com (mail-wm0-x243.google.com [IPv6:2a00:1450:400c:c09::243]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 45690EFF for ; Sat, 21 Jan 2017 18:17:21 +0000 (UTC) (envelope-from hoomanfazaeli@gmail.com) Received: by mail-wm0-x243.google.com with SMTP id r126so15037969wmr.3 for ; Sat, 21 Jan 2017 10:17:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-transfer-encoding; bh=wWkJ0qmJrYj0vjwqg4L1LJTyyPhjZGqq0p0Rks8nF6Q=; b=t8GsoT6u6+JvYSUpX1iuL3DWl93gSIQviTi1EI71bWR4wfSKl+W97l7cKNuxNJmIZI 3ClInMqZS4tqKwPmwHmt9FW1ceYUVRL2DBR+7FD6xHb4fUsgCrIjZHKw4OkNWnGXSfhc 4DzUAWhW/oh90IPMVJGZ23dVuDaV9wYU57N6aLu/NqVdU5O3tvm96svhp6k6mCR2/ur0 zf6Ymqb7iZaw6gKoL6sB7C3WZU+OcyX9+EAMYoPbHhLKtiK6/4F3R1iUgLjagCVjWeZd 5OMVxPEypgp0uw0cxh4vz5Tu94zet7AGHUQmtNUaF+DWT0Wfy0jB3lkYdsQrDkGGRWRU qy3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :cc:subject:references:in-reply-to:content-transfer-encoding; bh=wWkJ0qmJrYj0vjwqg4L1LJTyyPhjZGqq0p0Rks8nF6Q=; b=tApGYvGjC+hOI54VTf47ijDjZplXuKDNKIksrjjw4kTw8qTmeLX4vfy/XzIEe7b7CH /it840px0EhXuLaGcobZVclsT8cXYZTwC7dsHDAAuaaPSigyn+KN1dQ8+je+t81xt0PU /ekjFgcG3WqaoYQRJf4+Gn3QT0Mpx7UIwef9ugtn+sjlQKbDLUMOaPd7ddEg5bkw2WKG nqt0FXjjYu4dFBeTvFR/6IsqIVBL+U+klr3tycYlAyzu6KrzvPzvpUngR8z5pKTQXJAW oXzwymyujiVVKgBwVbl8fNMzOkQLIPqY0eVpxXBktuz2obQD0eqlLPrIe3BVOQ7Lu1PD LG1Q== X-Gm-Message-State: AIkVDXKswzApUH/omUJWLLlOEMIColcwFiRVYjEfiPOCgxEi6vGB3g3+2tRqdT0k5ShOmw== X-Received: by 10.223.134.218 with SMTP id 26mr16317702wry.104.1485022639384; Sat, 21 Jan 2017 10:17:19 -0800 (PST) Received: from [192.168.2.30] ([2.190.191.90]) by smtp.googlemail.com with ESMTPSA id k11sm11953667wmb.18.2017.01.21.10.17.17 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 21 Jan 2017 10:17:18 -0800 (PST) Message-ID: <5883A5A7.2050503@gmail.com> Date: Sat, 21 Jan 2017 21:47:11 +0330 From: Hooman Fazaeli User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130215 Thunderbird/17.0.3 MIME-Version: 1.0 To: Kevin Bowling CC: freebsd-net@freebsd.org Subject: Re: RFC: ethctl References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Jan 2017 18:17:21 -0000 On 2017-01-20 07:28, Kevin Bowling wrote: > Greetings, > > I'm casting a wide net in cc, try to keep the noise minimal but we need > some input from a variety of HW vendors. > > I have heard from several vendors the need for a NIC configuration tool. > Chelsio ships a cxgb/cxgbetool in FreeBSD as one example. There is > precedence for some nod toward a basic unified tool in Linux ethtool. > > From your perspective, > 1) What are the common requirements? > 2) What are specialized requirements? For instance as a full TCP offload > card Chelsio needs things others wont > 3) What should it _not_ do? Several of you have experience doing Ethernet > driver dev on many platforms so we should attempt to avoid repeating past > design mistakes. > > I expect we can achieve some level of inversion so the device specific code > can live close to the driver and plug into the ethctl framework. It should > be general enough to add completely new top level commands, so vendors can > implement HW specific features. On the other hand, we should attempt to > hook into common core for features every NIC provides, with a focus on > iflib. > > I will fund Matt Macy to do the overall design and implementation. > > Regards, > Kevin Bowling, on behalf of Limelight Networks for this effort > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" I always wished FreeBSD had a device-independent tool/API which could be used to query a device's Ethernet specific statisticssuch ascollisions, symbol/sequence/CRC/alignment errors, missed packets, giants & runts,link state IRQs, mbuf allocation fails, ... which are valuable for monitoring and diagnostics (BTW, I am aware that some drivers export a few counters via sysctl interface, but what I am talking about is a standardized set of counters available for _all_ Ethernet devices). -- Best regards Hooman Fazaeli From owner-freebsd-net@freebsd.org Sat Jan 21 19:26:59 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3F099CBB598 for ; Sat, 21 Jan 2017 19:26:59 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from venus.codepro.be (venus.codepro.be [IPv6:2a01:4f8:162:1127::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.codepro.be", Issuer "Gandi Standard SSL CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 078BBB0B; Sat, 21 Jan 2017 19:26:59 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from [10.0.2.164] (ptr-8ripyyfi1726l7ds8ua.18120a2.ip6.access.telenet.be [IPv6:2a02:1811:2419:4e02:4883:fabf:4b8b:94c2]) (Authenticated sender: kp) by venus.codepro.be (Postfix) with ESMTPSA id 6C0C435876; Sat, 21 Jan 2017 20:26:56 +0100 (CET) From: "Kristof Provost" To: "Bakul Shah" Cc: "Ermal =?utf-8?q?Lu=C3=A7i?=" , "FreeBSD Net" , "Alan Somers" Subject: Re: pf & NAT issue Date: Sat, 21 Jan 2017 20:26:55 +0100 Message-ID: <8C57C982-11BB-4BB4-97B7-4A5396336DB9@FreeBSD.org> In-Reply-To: <20170121042118.722C6124AEA4@mail.bitblocks.com> References: <20170120083555.ACCF9124AEA4@mail.bitblocks.com> <7C29D00C-94C0-4550-B1B2-CE307482B544@FreeBSD.org> <20170120203106.CD2C8124AEA4@mail.bitblocks.com> <20170120205933.8948A124AEA3@mail.bitblocks.com> <20170120211734.488D8124AEA5@mail.bitblocks.com> <20170121042118.722C6124AEA4@mail.bitblocks.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-Mailer: MailMate (2.0BETAr6072) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Jan 2017 19:26:59 -0000 On 21 Jan 2017, at 5:21, Bakul Shah wrote: > I finally had some time to look at the sources & noticed > /sys/netpfil/pf/pf.c:pf_purge_thread now runs 10 times a > second instead of once a second, which gave me the idea of > increasing "interval" timeout by a factor of 10 and this seems > to have mostly fixed the problem. But I don't know where the > actual problem is. The logic is too complicated to understand > in a few minutes so I didn't try to find the root cause at the > moment. [But I don't understand why pf times out normal > connections. Long lasting idle connections are perfectly fine. Have you tried increasing the state limit? This sounds like your states are being cleaned up, which might happen because you’re running close to the limit. > And fragment GC should not be coupled with connection state > expiry] > I think that’s simply because they both need a timeout and it’s more efficient to handle both at the same time than to set two timers. Regards, Kristof From owner-freebsd-net@freebsd.org Sat Jan 21 23:15:11 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AAC32CBB1F9 for ; Sat, 21 Jan 2017 23:15:11 +0000 (UTC) (envelope-from j.moyer@student-email.org) Received: from mail-it0-x235.google.com (mail-it0-x235.google.com [IPv6:2607:f8b0:4001:c0b::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 819B3933 for ; Sat, 21 Jan 2017 23:15:11 +0000 (UTC) (envelope-from j.moyer@student-email.org) Received: by mail-it0-x235.google.com with SMTP id c7so42384031itd.1 for ; Sat, 21 Jan 2017 15:15:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=student-email.org; s=google; h=mime-version:from:in-reply-to:references:date:message-id:subject:to; bh=aKs48Z2qdVR/fyLsS6m6AluKkK2aRfG1uTprpQmzMNI=; b=EKjIoO28UP3hnhmjfkcmoRN+N6GMtPLzFR9v7BEG+o8Xp1quLRjsoT7NaGr3Hdp8HE rlc0a5UDBhk1kHlMaYohKHiQASlxMg1Or/sDVSl+yrSxsN9ZHw+J2P9SVO7NUaRiro6/ R4L6tXTY2ezK8rIyxQnUcGekg472cVnkhJwMA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:in-reply-to:references:date :message-id:subject:to; bh=aKs48Z2qdVR/fyLsS6m6AluKkK2aRfG1uTprpQmzMNI=; b=riM/QH8DF+xxwciatxFmuTIVefzYlZqHntLkNFLZF4FzEOZjLZ8P8U2acROh6yQkWS tB+Y6oogUAz8PDCFgr8FaZbLW5MtM/S6ZtKzirX1BUUOAp5cAD8e0sz6EaPm7uigkJTk 6Z4BvA+BY7sm4ce6BMvcy3BYuIeSy5Ho58pxkq0n0og3fNt+55KVnDFY9uz2693mj80l 5eJHneVL35GbxCyxtiM/+JlqqQRupDkjEQSu6C8+bsgod3FFg3rYDZYzdd0yxCZOU9Yn ZD4/xpNp9MuDtJPY4LN7WCMeNpNPu5eBeTAW7MP8SoOnAldyNuCdnuPW4Y1ECvXuSADX D7gw== X-Gm-Message-State: AIkVDXK4gym9fdiwVnZQwR7Vlv7yw39DW4HeI9B4Ax7utNfrU3QZEtw3pkK/RCJI1uTVRg5CvdUYHnW4l1pW6w== X-Received: by 10.36.69.30 with SMTP id y30mr9125840ita.119.1485040510564; Sat, 21 Jan 2017 15:15:10 -0800 (PST) Received: from 19860689092 named unknown by gmailapi.google.com with HTTPREST; Sat, 21 Jan 2017 15:15:10 -0800 MIME-Version: 1.0 From: Jennifer Moyer In-Reply-To: References: Date: Sat, 21 Jan 2017 15:15:10 -0800 Message-ID: Subject: Re: Resource Links are Broken To: freebsd-net@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Jan 2017 23:15:11 -0000 Hi again! I just wanted to follow-up on my previous email and see if you can help me with finding the right person who can fix those broken resources on your site. Would that be you? Thanks and have a great day! From owner-freebsd-net@freebsd.org Sat Jan 21 23:38:55 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7422ACBBCAE for ; Sat, 21 Jan 2017 23:38:55 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 63F11A7C for ; Sat, 21 Jan 2017 23:38:55 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v0LNcsRB096179 for ; Sat, 21 Jan 2017 23:38:55 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 216304] Adding xn0 to bridge0 causes kernel panic Date: Sat, 21 Jan 2017 23:38:55 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Jan 2017 23:38:55 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D216304 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- CC|freebsd-amd64@FreeBSD.org | Assignee|freebsd-bugs@FreeBSD.org |freebsd-net@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.=