From owner-freebsd-arch@freebsd.org Mon Sep 7 21:29:39 2015 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 15F749CCC40 for ; Mon, 7 Sep 2015 21:29:39 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id EEDD611C3 for ; Mon, 7 Sep 2015 21:29:38 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: by mailman.ysv.freebsd.org (Postfix) id EE1909CCC3F; Mon, 7 Sep 2015 21:29:38 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EDAA09CCC3E for ; Mon, 7 Sep 2015 21:29:38 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (garage.dawidek.net [91.121.88.72]) by mx1.freebsd.org (Postfix) with ESMTP id BB47C11C2 for ; Mon, 7 Sep 2015 21:29:35 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from localhost (unknown [91.206.210.19]) by mail.dawidek.net (Postfix) with ESMTPSA id C6FABCAE; Mon, 7 Sep 2015 23:29:27 +0200 (CEST) Date: Mon, 7 Sep 2015 23:29:44 +0200 From: Pawel Jakub Dawidek To: Warner Losh Cc: "freebsd-arch@freebsd.org" Subject: Re: exporting INVARIANTS easily Message-ID: <20150907212943.GB1778@garage.freebsd.pl> References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="24zk1gE8NUlDmwG9" Content-Disposition: inline In-Reply-To: X-OS: FreeBSD 11.0-CURRENT amd64 User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Sep 2015 21:29:39 -0000 --24zk1gE8NUlDmwG9 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Aug 26, 2015 at 11:20:06AM -0600, Warner Losh wrote: > Greetings, >=20 > Many of the performance eating features are exported via some kind of sys= ctl, usually > patterned after the case of witness as debug.foo. INVARIANTS isn=E2=80=99= t one of those > features. >=20 > https://reviews.freebsd.org/D3488 >=20 > implements debug.invariants. Please comment. >=20 > I=E2=80=99d thought about adding it to the kern.features sysctl, but thou= ght better of it since it > isn=E2=80=99t a facility that people can use. >=20 > If you include the kernel config in the kernel, you can get this informat= ion via > config -x | grep INVARIANTS > but not all kernels do that. This is more robust. >=20 > I also know that you can load some modules compiled INVARIANTS when the b= ase > kernel isn=E2=80=99t built that way and this won=E2=80=99t reflect that. = There=E2=80=99s no good want to include > that information and is an uncommon use case. >=20 > Our use case? We have a raft of test machines. Most run without INVARIANT= S since > we want to characterize the performance of the release under test. Some a= re running > INVARIANTS since we want to test the robustness as well, even at the expe= nse of > some performance. To ensure we don=E2=80=99t accidentally include INVARIA= NTS systems > in the performance number, we=E2=80=99ve adding a key to an internal data= base that=E2=80=99s driven > off this sysctl. >=20 > Comments? As long as the ultimate goal is to have INVARIANTS in GENERIC I'm all for it! I use to run even production machines with INVARIANTS, which was helpful to catch VirtualBox's kernel memory corruption, but we've moved to GENERIC since I wanted to use freebsd-update. Having INVARIANTS in GENERIC would be great. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://mobter.com --24zk1gE8NUlDmwG9 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCgAGBQJV7gHHAAoJEJVLhSuxKFt1JicP/RfLmso/BftpGykpO2hJPuuJ sNqQjroFMlrQrIzu/rAWrJ7PPffgDdmCtmAZ4+cpVRUodU7A15ZIXdNghGgaIhhy aplcV1uT967vpYWMTjJ1UPNi2qWLTcV2Dp0EoGCCUshUXN9M+43UJSEEd1fokywd fFRlLA5yazUnerDoZmetHanNEcumKhb7QFdAcYpBAxkBV5j3O5jEFXrE+/K7PbBy sdH1YWBf1j8VSvBrLMlQRC52GNBHYnlSWaobfOj1ng5hRrh/g+1XDoYOqPQerNSg PO+mlAsL0NgjvBY7aVc1PBAsj+ytK1HyDyf9/W3jz4J0WPJshnn7Z20S64aLfxXg IhSWkbd1ApGXDz5PunsLxmfsKPWGeaNUmZiP+pluajd5/4q799bXjBPCm+ZdnXdR K1NdTPBQ/ED8jXRmbgfqKFGDLQXelAI4lu9z8Y9iBB+oEIi9nHflN83TZSoal5Rd Rf0QDXcgnH+U9gBBClEgiLP/oqTQnto2tTMI4pDYzbOQ9Keb6wKfWRkCziyuUqMY Pi+r6MoWi5rvNEipvFMMoykrYD+bPXOD441yzrHbA3w4SJH4xuQ7uCE/9S89Y2zS uHOFzkL979RJ/RK1wkT72C5bjGzfzwXPiolWqlvwzwKd1xH15o9Dmuq+KK71FVel wVMa7nTJTxkjQoUsQ3ru =diDM -----END PGP SIGNATURE----- --24zk1gE8NUlDmwG9-- From owner-freebsd-arch@freebsd.org Mon Sep 7 22:18:32 2015 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BEFFB9CC229 for ; Mon, 7 Sep 2015 22:18:32 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 9C3D618AF for ; Mon, 7 Sep 2015 22:18:32 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mailman.ysv.freebsd.org (Postfix) id 997769CC228; Mon, 7 Sep 2015 22:18:32 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 80AB69CC227 for ; Mon, 7 Sep 2015 22:18:32 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qg0-f47.google.com (mail-qg0-f47.google.com [209.85.192.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4278F18AD for ; Mon, 7 Sep 2015 22:18:31 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by qgx61 with SMTP id 61so69580656qgx.3 for ; Mon, 07 Sep 2015 15:18:25 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=Z7Gr7udVE85S915/vis7xPpwPrmjHI5gEc9nr+8hEFs=; b=IEzj8DpnNEysDM9ssNMGX9ZdHHovmFU7tuDMxKli0gGjI6V6CT+Q50bze2zMCwBbLH vihbzhAY1u4TRtyAiTPT1huuK6ht0c+cMFm3jqyPv0xRMElGX7vPDM0sLfH79zg9d6f/ DExqnej/Ra7UuUr9GGqLMn1Kn53JrKrfyFCQv5iNjkARDmMX9igvFlCP7HXh/D5tbpaB 6PFC//2I4RriazPjbWZTG4hEcLzJ2A2YHH0RW/pUA4FjGorHrakRmx2QuzWv8+qX2w4L 9T+upYvlOoqEUbF10h0AdCQHxfLaJ5t5RovxRCu6wJ+iXI82wLa4fX/JFipDDDmBnIMO m/cQ== X-Gm-Message-State: ALoCoQk+vjo/xkEsfdofP3bXvAB+OxnvXca0TeoAizb0tnbryg9dwVfvKir9ZrbyCeRNgXrWuMYI MIME-Version: 1.0 X-Received: by 10.140.29.3 with SMTP id a3mr29714463qga.97.1441664305332; Mon, 07 Sep 2015 15:18:25 -0700 (PDT) Sender: wlosh@bsdimp.com Received: by 10.140.80.164 with HTTP; Mon, 7 Sep 2015 15:18:25 -0700 (PDT) X-Originating-IP: [69.53.245.5] In-Reply-To: <20150907212943.GB1778@garage.freebsd.pl> References: <20150907212943.GB1778@garage.freebsd.pl> Date: Mon, 7 Sep 2015 16:18:25 -0600 X-Google-Sender-Auth: OTc2PIozvZuS0XggWS_DMrW0bZ8 Message-ID: Subject: Re: exporting INVARIANTS easily From: Warner Losh To: Pawel Jakub Dawidek Cc: "freebsd-arch@freebsd.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Sep 2015 22:18:32 -0000 On Mon, Sep 7, 2015 at 3:29 PM, Pawel Jakub Dawidek wrote= : > On Wed, Aug 26, 2015 at 11:20:06AM -0600, Warner Losh wrote: > > Greetings, > > > > Many of the performance eating features are exported via some kind of > sysctl, usually > > patterned after the case of witness as debug.foo. INVARIANTS isn=E2=80= =99t one > of those > > features. > > > > https://reviews.freebsd.org/D3488 > > > > implements debug.invariants. Please comment. > > > > I=E2=80=99d thought about adding it to the kern.features sysctl, but th= ought > better of it since it > > isn=E2=80=99t a facility that people can use. > > > > If you include the kernel config in the kernel, you can get this > information via > > config -x | grep INVARIANTS > > but not all kernels do that. This is more robust. > > > > I also know that you can load some modules compiled INVARIANTS when the > base > > kernel isn=E2=80=99t built that way and this won=E2=80=99t reflect that= . There=E2=80=99s no good > want to include > > that information and is an uncommon use case. > > > > Our use case? We have a raft of test machines. Most run without > INVARIANTS since > > we want to characterize the performance of the release under test. Some > are running > > INVARIANTS since we want to test the robustness as well, even at the > expense of > > some performance. To ensure we don=E2=80=99t accidentally include INVAR= IANTS > systems > > in the performance number, we=E2=80=99ve adding a key to an internal da= tabase > that=E2=80=99s driven > > off this sysctl. > > > > Comments? > > As long as the ultimate goal is to have INVARIANTS in GENERIC I'm all > for it! I use to run even production machines with INVARIANTS, which was > helpful to catch VirtualBox's kernel memory corruption, but we've moved > to GENERIC since I wanted to use freebsd-update. Having INVARIANTS in > GENERIC would be great. That's not my goal. And I doubt my employer would run it in their kernel config because it costs too much in bandwidth when we're streaming all that video people binge watch. Warner From owner-freebsd-arch@freebsd.org Mon Sep 7 22:47:54 2015 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8F1DA9CD164 for ; Mon, 7 Sep 2015 22:47:54 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 73A5D15F0 for ; Mon, 7 Sep 2015 22:47:54 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: by mailman.ysv.freebsd.org (Postfix) id 727019CD163; Mon, 7 Sep 2015 22:47:54 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 583DC9CD162 for ; Mon, 7 Sep 2015 22:47:54 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (garage.dawidek.net [91.121.88.72]) by mx1.freebsd.org (Postfix) with ESMTP id E916115EF for ; Mon, 7 Sep 2015 22:47:53 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from localhost (unknown [91.206.210.19]) by mail.dawidek.net (Postfix) with ESMTPSA id D5F87CC5; Tue, 8 Sep 2015 00:47:51 +0200 (CEST) Date: Tue, 8 Sep 2015 00:48:08 +0200 From: Pawel Jakub Dawidek To: Warner Losh Cc: "freebsd-arch@freebsd.org" Subject: Re: exporting INVARIANTS easily Message-ID: <20150907224808.GC1778@garage.freebsd.pl> References: <20150907212943.GB1778@garage.freebsd.pl> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="uXxzq0nDebZQVNAZ" Content-Disposition: inline In-Reply-To: X-OS: FreeBSD 11.0-CURRENT amd64 User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Sep 2015 22:47:54 -0000 --uXxzq0nDebZQVNAZ Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Sep 07, 2015 at 04:18:25PM -0600, Warner Losh wrote: > On Mon, Sep 7, 2015 at 3:29 PM, Pawel Jakub Dawidek wro= te: >=20 > > On Wed, Aug 26, 2015 at 11:20:06AM -0600, Warner Losh wrote: > > > Greetings, > > > > > > Many of the performance eating features are exported via some kind of > > sysctl, usually > > > patterned after the case of witness as debug.foo. INVARIANTS isn=E2= =80=99t one > > of those > > > features. > > > > > > https://reviews.freebsd.org/D3488 > > > > > > implements debug.invariants. Please comment. > > > > > > I=E2=80=99d thought about adding it to the kern.features sysctl, but = thought > > better of it since it > > > isn=E2=80=99t a facility that people can use. > > > > > > If you include the kernel config in the kernel, you can get this > > information via > > > config -x | grep INVARIANTS > > > but not all kernels do that. This is more robust. > > > > > > I also know that you can load some modules compiled INVARIANTS when t= he > > base > > > kernel isn=E2=80=99t built that way and this won=E2=80=99t reflect th= at. There=E2=80=99s no good > > want to include > > > that information and is an uncommon use case. > > > > > > Our use case? We have a raft of test machines. Most run without > > INVARIANTS since > > > we want to characterize the performance of the release under test. So= me > > are running > > > INVARIANTS since we want to test the robustness as well, even at the > > expense of > > > some performance. To ensure we don=E2=80=99t accidentally include INV= ARIANTS > > systems > > > in the performance number, we=E2=80=99ve adding a key to an internal = database > > that=E2=80=99s driven > > > off this sysctl. > > > > > > Comments? > > > > As long as the ultimate goal is to have INVARIANTS in GENERIC I'm all > > for it! I use to run even production machines with INVARIANTS, which was > > helpful to catch VirtualBox's kernel memory corruption, but we've moved > > to GENERIC since I wanted to use freebsd-update. Having INVARIANTS in > > GENERIC would be great. >=20 >=20 > That's not my goal. And I doubt my employer would run it in their > kernel config because it costs too much in bandwidth when we're > streaming all that video people binge watch. Do you have some numbers handy? I'd be interested in seeing how much performance drop there is when you have INVARIANTS compiled in, but debug.invariants set to 0. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://mobter.com --uXxzq0nDebZQVNAZ Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCgAGBQJV7hQoAAoJEJVLhSuxKFt1vdoQAL+e03LJHYfFG9KjgJSraRuv mgslvsqdTiYobgSjgp0t679aARW3ID/VdjeqYvt2ph/phOS76hst2wAmV7ZZWKgF RJBHWK4glpACS4avQhWQyXymyeapC3q4UwRSUVmskVp03BtXyxv18tpsz4rmtts8 SnS+9RCGgXhLKrGK5TVmQjijbEqrOymudno1dNVSagGIK00LORuXGIvkO/OT+MAo YPHuseyOw9sCPpT8P2W+3n7qAWavz8YwNGeg0x6MjPe6OQ1/XO7xugoiNvq/eDg2 gy/Vxttd/CSXdQalLkWYbb5WpMP9uFl71xpKTgYpjkRCxTxTbkTzSo2LuoAxivfq 3y6zGx4ZM7Dr5P/k/kHj/rc+WtcdMvpaAOnB9uj0IhkPMjflQ6kCmU+Ey3Tz6E38 +H2LEr+mhtQ6WUpfywEjnbypHDvirqA8crn8EdP4DDl+LVV3koJ6sEwYusKDhE97 PzkBjnMRSsBdDMgxEglVA2HCP0CxyMKHE1KVEkKPSjbSCfkc99TpvI7cj7TgwXaz bFeytJq7OC649PzHH8Y005EZF2/QWiKTuuokPKlhC82NbvP3GEZy1QEDTC3Hyzbn S6kNpZghtRmOpVWIEKehHZBjR/QADw6lxScMipdTHLEJgK08GLVOzoyWJrJlYaHX 74JjUfpn4LEgBKnrXIGG =4G6i -----END PGP SIGNATURE----- --uXxzq0nDebZQVNAZ-- From owner-freebsd-arch@freebsd.org Mon Sep 7 22:52:29 2015 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2C1779CD463 for ; Mon, 7 Sep 2015 22:52:29 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 0456F1A71 for ; Mon, 7 Sep 2015 22:52:29 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: by mailman.ysv.freebsd.org (Postfix) id 0185F9CD462; Mon, 7 Sep 2015 22:52:29 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 012829CD461 for ; Mon, 7 Sep 2015 22:52:29 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mail-pa0-f48.google.com (mail-pa0-f48.google.com [209.85.220.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CA99F1A6F for ; Mon, 7 Sep 2015 22:52:28 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: by padhy16 with SMTP id hy16so103807052pad.1 for ; Mon, 07 Sep 2015 15:52:22 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:mime-version:content-type:from :in-reply-to:date:cc:message-id:references:to; bh=QpfGx8i7PSbGW+d/UrDAeHlyVJsyvQ0gM8DISOZp5hQ=; b=Vpi8/CkW5CJ8dpaIfAkwgry4bpkNdlEN1lKX65gleww3ELxcOpm3Cmzywj1Vk79Ojt e9G+qwShoK7d/divYT28dJ1CEOKqyj2JuER3ZocWbLtohLhamTPv6YrplZGgmB3fYcZ3 /7cu9L7F70HQv8Xnb5zhiOIJ08MLYCrjTJLUfKGbPcxmGywGzsXaaLshziJ8lleAjL6R 9OlWPSeQ9FIFZqidCJqy0OhtI9VyP2zq5FjderQf1DpwrjSG1Jb6Len7Cj5CiuzecqJi ZfIttPhrWOHv+fTRPbWHVBkyXV/WgfUmxnePyBqjnaqkZDSkvUdecfJ6eJqpY+hRHPoo 5jIg== X-Gm-Message-State: ALoCoQkifbf31bLVF6FT8rGffG99qKwAMYUAboLXrWg7PzTSxhLo2nAFI0YxOcEfTvICroloVIJ8 X-Received: by 10.68.68.167 with SMTP id x7mr20107092pbt.140.1441666342507; Mon, 07 Sep 2015 15:52:22 -0700 (PDT) Received: from ip-100-127-128-52.compute.internal.netflix.com ([69.53.245.5]) by smtp.gmail.com with ESMTPSA id uv5sm959303pbc.12.2015.09.07.15.52.21 (version=TLS1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 07 Sep 2015 15:52:21 -0700 (PDT) Sender: Warner Losh Subject: Re: exporting INVARIANTS easily Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2104\)) Content-Type: multipart/signed; boundary="Apple-Mail=_1810B0DF-7C29-4FD0-A635-A9AF14C71B5E"; protocol="application/pgp-signature"; micalg=pgp-sha512 X-Pgp-Agent: GPGMail 2.5 From: Warner Losh In-Reply-To: <20150907224808.GC1778@garage.freebsd.pl> Date: Mon, 7 Sep 2015 16:52:19 -0600 Cc: "freebsd-arch@freebsd.org" Message-Id: References: <20150907212943.GB1778@garage.freebsd.pl> <20150907224808.GC1778@garage.freebsd.pl> To: Pawel Jakub Dawidek X-Mailer: Apple Mail (2.2104) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Sep 2015 22:52:29 -0000 --Apple-Mail=_1810B0DF-7C29-4FD0-A635-A9AF14C71B5E Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Sep 7, 2015, at 4:48 PM, Pawel Jakub Dawidek = wrote: >=20 > On Mon, Sep 07, 2015 at 04:18:25PM -0600, Warner Losh wrote: >> On Mon, Sep 7, 2015 at 3:29 PM, Pawel Jakub Dawidek = wrote: >>=20 >>> On Wed, Aug 26, 2015 at 11:20:06AM -0600, Warner Losh wrote: >>>> Greetings, >>>>=20 >>>> Many of the performance eating features are exported via some kind = of >>> sysctl, usually >>>> patterned after the case of witness as debug.foo. INVARIANTS = isn=E2=80=99t one >>> of those >>>> features. >>>>=20 >>>> https://reviews.freebsd.org/D3488 >>>>=20 >>>> implements debug.invariants. Please comment. >>>>=20 >>>> I=E2=80=99d thought about adding it to the kern.features sysctl, = but thought >>> better of it since it >>>> isn=E2=80=99t a facility that people can use. >>>>=20 >>>> If you include the kernel config in the kernel, you can get this >>> information via >>>> config -x | grep INVARIANTS >>>> but not all kernels do that. This is more robust. >>>>=20 >>>> I also know that you can load some modules compiled INVARIANTS when = the >>> base >>>> kernel isn=E2=80=99t built that way and this won=E2=80=99t reflect = that. There=E2=80=99s no good >>> want to include >>>> that information and is an uncommon use case. >>>>=20 >>>> Our use case? We have a raft of test machines. Most run without >>> INVARIANTS since >>>> we want to characterize the performance of the release under test. = Some >>> are running >>>> INVARIANTS since we want to test the robustness as well, even at = the >>> expense of >>>> some performance. To ensure we don=E2=80=99t accidentally include = INVARIANTS >>> systems >>>> in the performance number, we=E2=80=99ve adding a key to an = internal database >>> that=E2=80=99s driven >>>> off this sysctl. >>>>=20 >>>> Comments? >>>=20 >>> As long as the ultimate goal is to have INVARIANTS in GENERIC I'm = all >>> for it! I use to run even production machines with INVARIANTS, which = was >>> helpful to catch VirtualBox's kernel memory corruption, but we've = moved >>> to GENERIC since I wanted to use freebsd-update. Having INVARIANTS = in >>> GENERIC would be great. >>=20 >>=20 >> That's not my goal. And I doubt my employer would run it in their >> kernel config because it costs too much in bandwidth when we're >> streaming all that video people binge watch. >=20 > Do you have some numbers handy? I'd be interested in seeing how much > performance drop there is when you have INVARIANTS compiled in, but > debug.invariants set to 0. I don=E2=80=99t have any direct numbers, but the belief is that we=E2=80=99= ll see between 10-20% drop in our peak throughput with an INVARIANTS kernel. Warner --Apple-Mail=_1810B0DF-7C29-4FD0-A635-A9AF14C71B5E Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJV7hUkAAoJEGwc0Sh9sBEAhTwP/jDWBBk+C0AEiTjKaqtE2AM7 v0AsyL/1dbgxihybK728U1EOE9PeQuaXqfwrlQ1nCcdH06snq62iagiHibyksGAa RZrLZRv9MYDq0nvVf42FFdRRbwAAimwLumzCEJtfyxgA3pcRd4EnLxZA6f6kwoga WzJho1gI5qK6eMwzZx+uz0K5dWgXQJSWpJzTiFkHyOQV6skmpZcxIVQDnQF1sAuA XAZlx3XqNCEi8uzjVgf5XytkCVNp6tB9Fjc82gRM249QVLSBkQIeSu8IDjk8xeEk xNtg3BM3G8TwjeRmHDGM2c27pkaaK1cWe/tBM3fyqUOZuxSLDc/lzkj11Jgntuek krdRShitMRUgDsVtUlAemroB5fg9nONzuRI91tDraArQocE1b2dZARAw0/9jWXsp OnsXH2z4NDalEY2Do6kZy65G8KpRC2dVHRpmquU/yUog6HhdOS/afglfwYvXY/zt Jff5nmk1iQvT4KyryZIPfvMmf7disJlf0cuWoUMVDVPZnx5NRHxNcekqs1i0Asbo Bz6sE5zyCSsXEXi7X32Xp3v1XVgUukiDGREbw8w6ejSz/32xaq8gpUJLCpxTvdxb jfmxB5KhOPtDAvUB6wh9MOKIZr2bbavXmwhZut3GyuunALzXaYMOtxr1jejdk+zn HiHdh1+w6ASlSZjZCnnq =xBM0 -----END PGP SIGNATURE----- --Apple-Mail=_1810B0DF-7C29-4FD0-A635-A9AF14C71B5E-- From owner-freebsd-arch@freebsd.org Tue Sep 8 19:13:16 2015 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 56E509CC89F for ; Tue, 8 Sep 2015 19:13:16 +0000 (UTC) (envelope-from etnapierala@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 370B6143C for ; Tue, 8 Sep 2015 19:13:16 +0000 (UTC) (envelope-from etnapierala@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 3608C9CC89E; Tue, 8 Sep 2015 19:13:16 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 35A2C9CC89C for ; Tue, 8 Sep 2015 19:13:16 +0000 (UTC) (envelope-from etnapierala@gmail.com) Received: from mail-wi0-x22f.google.com (mail-wi0-x22f.google.com [IPv6:2a00:1450:400c:c05::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C4A18143A for ; Tue, 8 Sep 2015 19:13:15 +0000 (UTC) (envelope-from etnapierala@gmail.com) Received: by wicgb1 with SMTP id gb1so90101856wic.1 for ; Tue, 08 Sep 2015 12:13:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:subject:message-id:mail-followup-to :mime-version:content-type:content-disposition:user-agent; bh=VxbmxBzSSmH5LPJwID8vQlcbkb1W6PPbyJSxhXjv4pc=; b=XaEaqUTqQOIXBjJUE9atzSB+6F9RhglJHjSRsU0LqRmqp3SKSdOSnjyiEi4/2hEK1O 8q4yobqm1VRm2tCrGeyvck7jLgaoc4rjV9tY+BEl5ZKA+m5bkC7lTGN2Ywc+BUH0zwYi TaupuXaM2FEteUNwmimAJPCaLv/T9p7WT7ST79A5hDNKddoBkbNch55FNCr3MGtYLC+9 RsARAXqaMZHjhPid/esZaSNactvKF7+eV4R51I4GOgGD42W6DwFkTU0J8uVn5aCMX0Lq GY1cK+1fjsyH3lpXL1m69r1NeIRHRzx+R5u5u9XuSBhqxltxOJpNsL4wA56plhhbDQty yfwg== X-Received: by 10.180.37.201 with SMTP id a9mr48639591wik.83.1441739594356; Tue, 08 Sep 2015 12:13:14 -0700 (PDT) Received: from brick.home (adit71.neoplus.adsl.tpnet.pl. [79.184.201.71]) by smtp.gmail.com with ESMTPSA id c11sm7289519wib.1.2015.09.08.12.13.12 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 08 Sep 2015 12:13:13 -0700 (PDT) Sender: =?UTF-8?Q?Edward_Tomasz_Napiera=C5=82a?= Date: Tue, 8 Sep 2015 21:13:10 +0200 From: Edward Tomasz =?utf-8?Q?Napiera=C5=82a?= To: arch@FreeBSD.org Subject: Make kern.ipc.shm_allow_removed default to 1. Message-ID: <20150908191310.GA3104@brick.home> Mail-Followup-To: arch@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Sep 2015 19:13:16 -0000 I'd like to commit - well, discuss it first - the following change: https://reviews.freebsd.org/D3603 This changes the default setting of kern.ipc.shm_allow_removed from 0 to 1. This removes the need of manually changing this flag for Google Chrome users. It also improves compatibility with Linux applications running under Linuxulator compatibility layer, and possibly also helps in porting software from Linux. Generally speaking, the flag allows applications to create the shared memory segment, attach it, remove it, and then continue to use it and to reattach it later. This means that the kernel will automatically "clean up" after the application exits. It could be argued that it's against POSIX. However, SUSv3 says this about IPC_RMID: "Remove the shared memory identifier specified by shmid from the system and destroy the shared memory segment and shmid_ds data structure associated with it." From my reading, we break it in any case by deferring removal of the segment until it's detached; we won't break it any more by also deferring removal of the identifier. This is the behaviour exhibited by Linux since... probably always, and also by OpenBSD since the following commit: revision 1.54 date: 2011/10/27 07:56:28; author: robert; state: Exp; lines: +3 -8; Allow segments to be used even after they were marked for deletion with the IPC_RMID flag. This is permitted as an extension beyond the standards and this is similar to what other operating systems like linux do. Arguments - on both sides - welcome. Thanks! From owner-freebsd-arch@freebsd.org Wed Sep 9 23:20:40 2015 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5AA7FA01DC5 for ; Wed, 9 Sep 2015 23:20:40 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 34DA91C97 for ; Wed, 9 Sep 2015 23:20:40 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from ralph.baldwin.cx (c-73-231-226-104.hsd1.ca.comcast.net [73.231.226.104]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id CCF71B976 for ; Wed, 9 Sep 2015 19:20:38 -0400 (EDT) From: John Baldwin To: 'freebsd-arch' Subject: Testing needed: truss refactor Date: Wed, 09 Sep 2015 16:20:34 -0700 Message-ID: <8360972.rDHFT4IURu@ralph.baldwin.cx> User-Agent: KMail/4.14.3 (FreeBSD/10.2-PRERELEASE; KDE/4.14.3; amd64; ; ) MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Wed, 09 Sep 2015 19:20:38 -0400 (EDT) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Sep 2015 23:20:40 -0000 I got sidetracked hacking on truss recently. There's a lot of duplicated code in the various ABI-specific backends, so I refactored it. I also tried to clean up some incomplete changes from the procfs-to-trace conversion. You can find the changes at https://reviews.freebsd.org/D3575 In particular, since I had to change the ABI-specific backends, I need testing on other architectures besides amd64 and i386. I've included the candidate commit message from the review below for more details: Several changes to truss. - Refactor the interface between the ABI-independent code and the ABI-specific backends. The backends now provide smaller hooks to fetch system call arguments and return values. The rest of the system call entry and exit handling that was previously duplicated among all the backends has been moved to one place. - Merge the loop when waiting for an event with the loop for handling stops. This also means not emulating a procfs-like interface on top of ptrace(). Instead, use a single event loop that fetches process events via waitid(). Among other things this allows us to report the full 32-bit exit value. - Use PT_FOLLOW_FORK to follow new child processes instead of forking a new truss process for each new child. This allows one truss process to monitor a tree of processes and truss -c should now display one total for the entire tree instead of separate summaries per process. - Use the recently added fields to ptrace_lwpinfo to determine the current system call number and argument count. The latter is especially useful and fixes a regression since the conversion from procfs. truss now generally prints the correct number of arguments for most system calls rather than printing extra arguments for any call not listed in the table in syscalls.c. - Actually check the new ABI when processes call exec. The comments claimed that this happened but it was not being done (perhaps this was another regression in the conversion to ptrace()). If the new ABI after exec is not supported, truss detaches from the process. If truss does not support the ABI for a newly executed process the process is killed before it returns from exec. - Along with the refactor, teach the various ABI-specific backends to fetch both return values, not just the first. Use this to properly report the full 64-bit return value from lseek(). In addition, the handler for "pipe" now pulls the pair of descriptors out of the return values (which is the true kernel system call interface) but displays them as an argument (which matches the interface exported by libc). - Each ABI handler adds entries to a linker set rather than requiring a statically defined table of handlers in main.c. - The arm and mips system call fetching code was changed to follow the same pattern as amd64 (and the in-kernel handler) of fetching register arguments first and then reading any remaining arguments from the stack. This should fix indirect system call arguments on at least arm. - The mipsn32 and n64 ABIs will now look for arguments in A4 through A7. - Use register %ebp for the 6th system call argument for Linux/i386 ABIs to match the in-kernel argument fetch code. - For powerpc binaries on a powerpc64 system, fetch the extra arguments on the stack as 32-bit values that are then copied into the 64-bit argument array instead of reading the 32-bit values directly into the 64-bit array. -- John Baldwin From owner-freebsd-arch@freebsd.org Fri Sep 11 16:29:30 2015 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E74D4A02A63 for ; Fri, 11 Sep 2015 16:29:29 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C07341E48; Fri, 11 Sep 2015 16:29:29 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from ralph.baldwin.cx (c-73-231-226-104.hsd1.ca.comcast.net [73.231.226.104]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 09531B926; Fri, 11 Sep 2015 12:29:28 -0400 (EDT) From: John Baldwin To: freebsd-arch@freebsd.org Cc: Edward Tomasz =?utf-8?B?TmFwaWVyYcWCYQ==?= Subject: Re: Make kern.ipc.shm_allow_removed default to 1. Date: Fri, 11 Sep 2015 09:24:12 -0700 Message-ID: <5328060.4avXYiGHFa@ralph.baldwin.cx> User-Agent: KMail/4.14.3 (FreeBSD/10.2-PRERELEASE; KDE/4.14.3; amd64; ; ) In-Reply-To: <20150908191310.GA3104@brick.home> References: <20150908191310.GA3104@brick.home> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Fri, 11 Sep 2015 12:29:28 -0400 (EDT) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Sep 2015 16:29:30 -0000 On Tuesday, September 08, 2015 09:13:10 PM Edward Tomasz Napiera=C5=82a= wrote: > I'd like to commit - well, discuss it first - the following change: >=20 > https://reviews.freebsd.org/D3603 >=20 > This changes the default setting of kern.ipc.shm_allow_removed from 0= to 1. > This removes the need of manually changing this flag for Google Chrom= e > users. It also improves compatibility with Linux applications running= under > Linuxulator compatibility layer, and possibly also helps in porting s= oftware > from Linux. >=20 > Generally speaking, the flag allows applications to create the shared= memory > segment, attach it, remove it, and then continue to use it and to rea= ttach it > later. This means that the kernel will automatically "clean up" after= the > application exits. >=20 > It could be argued that it's against POSIX. However, SUSv3 says this > about IPC_RMID: "Remove the shared memory identifier specified by shm= id from > the system and destroy the shared memory segment and shmid_ds data st= ructure > associated with it." From my reading, we break it in any case by defe= rring > removal of the segment until it's detached; we won't break it any mor= e > by also deferring removal of the identifier. It is against POSIX. When I last raised this with kib@ and alc@, Alan = noted that our current behavior is not really against POSIX as he felt there = was enough wriggle room to permit existing mappings to remain valid. That said, in the thread kib@ felt it was ok to change the default. I = think it's probably fine. It's a hack for the fact that shmget() isn't retur= ning a real fd the way shm_open() does, so in that sense it is ok. That is,= in shm_open() you can do this: =09fd =3D shm_open("/some/path"); =09shm_unlink("/some/path"); =09 =09mmap(..., fd); =09/* later */ =09mmap(..., fd); =09/* or pass fd over a UNIX domain socket and map it in another proces= s */ =09close(fd); =09/* now mmap won't work */ With FreeBSD's SHM_ANON you can collapse the first two steps down to: =09fd =3D shm_open(SHM_ANON); If you think of the return value from shmget() as an fd, then setting t= his to 1 does make sense in a way. That is you can have a sequence of: =09id =3D shmget(...); =09hold =3D shmat(id, ...); /* "hold" mapping */ =09shmctl(IPC_RMID, id); =09/* later */ =09shmat(id, ...); =09/* or pass 'id' as a value across a socket and shmat in another proc= ess */ shmdt(hold); /* now shmat may or may not work */ However, note that this does require you to do some things carefully: 1) You have to keep an existing mapping around via shmat() that serve= s as the equivalent of the open file descriptor to keep the 'id' valid.= This also means you have to defer the IPC_RMID until after you hav= e created this "hold" mapping. 2) You always have to invoke IPC_RMID. In particular, even if you us= e shmget() with IPC_PRIVATE you have to explicitly IPC_RMID. So, I think it is fine to change this (and have asked about doing it my= self in the past). However, I think shm_open() is definitely a superior API= for this sort of thing with semantics that are clearer and more inline with= UNIX. --=20 John Baldwin From owner-freebsd-arch@freebsd.org Fri Sep 11 17:06:19 2015 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5E393A01EA3 for ; Fri, 11 Sep 2015 17:06:19 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DE2871B09; Fri, 11 Sep 2015 17:06:18 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id t8BH6CCx021022 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Fri, 11 Sep 2015 20:06:12 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua t8BH6CCx021022 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id t8BH6CHP021021; Fri, 11 Sep 2015 20:06:12 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 11 Sep 2015 20:06:12 +0300 From: Konstantin Belousov To: John Baldwin Cc: freebsd-arch@freebsd.org, Edward Tomasz Napiera??a Subject: Re: Make kern.ipc.shm_allow_removed default to 1. Message-ID: <20150911170612.GK2072@kib.kiev.ua> References: <20150908191310.GA3104@brick.home> <5328060.4avXYiGHFa@ralph.baldwin.cx> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5328060.4avXYiGHFa@ralph.baldwin.cx> User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Sep 2015 17:06:19 -0000 On Fri, Sep 11, 2015 at 09:24:12AM -0700, John Baldwin wrote: > On Tuesday, September 08, 2015 09:13:10 PM Edward Tomasz Napiera??a wrote: > > I'd like to commit - well, discuss it first - the following change: > > > > https://reviews.freebsd.org/D3603 > > > > This changes the default setting of kern.ipc.shm_allow_removed from 0 to 1. > > This removes the need of manually changing this flag for Google Chrome > > users. It also improves compatibility with Linux applications running under > > Linuxulator compatibility layer, and possibly also helps in porting software > > from Linux. > > > > Generally speaking, the flag allows applications to create the shared memory > > segment, attach it, remove it, and then continue to use it and to reattach it > > later. This means that the kernel will automatically "clean up" after the > > application exits. > > > > It could be argued that it's against POSIX. However, SUSv3 says this > > about IPC_RMID: "Remove the shared memory identifier specified by shmid from > > the system and destroy the shared memory segment and shmid_ds data structure > > associated with it." From my reading, we break it in any case by deferring > > removal of the segment until it's detached; we won't break it any more > > by also deferring removal of the identifier. > > It is against POSIX. When I last raised this with kib@ and alc@, Alan noted > that our current behavior is not really against POSIX as he felt there was > enough wriggle room to permit existing mappings to remain valid. > > That said, in the thread kib@ felt it was ok to change the default. I think > it's probably fine. It's a hack for the fact that shmget() isn't returning > a real fd the way shm_open() does, so in that sense it is ok. That is, > in shm_open() you can do this: > > fd = shm_open("/some/path"); > shm_unlink("/some/path"); > > mmap(..., fd); > > /* later */ > > mmap(..., fd); > > /* or pass fd over a UNIX domain socket and map it in another process */ > > close(fd); > > /* now mmap won't work */ > > With FreeBSD's SHM_ANON you can collapse the first two steps down to: > > fd = shm_open(SHM_ANON); > > If you think of the return value from shmget() as an fd, then setting this to 1 > does make sense in a way. That is you can have a sequence of: > > id = shmget(...); > > hold = shmat(id, ...); /* "hold" mapping */ > > shmctl(IPC_RMID, id); > > /* later */ > > shmat(id, ...); > > /* or pass 'id' as a value across a socket and shmat in another process */ > > shmdt(hold); > > /* now shmat may or may not work */ > > However, note that this does require you to do some things carefully: > > 1) You have to keep an existing mapping around via shmat() that serves as > the equivalent of the open file descriptor to keep the 'id' valid. > This also means you have to defer the IPC_RMID until after you have > created this "hold" mapping. > > 2) You always have to invoke IPC_RMID. In particular, even if you use > shmget() with IPC_PRIVATE you have to explicitly IPC_RMID. > > So, I think it is fine to change this (and have asked about doing it myself > in the past). However, I think shm_open() is definitely a superior API for > this sort of thing with semantics that are clearer and more inline with UNIX. The request to change the defaults written after somewhat long discussion elsewere, which was started by the fact that chrome requires the shm_allow_removed = 1 behaviour. My opinion is that yes, =1 behaviour contradicts the SUSv4tc1 specification: >From shmat() description: [EINVAL] The value of shmid is not a valid shared memory identifier >From shmctl(): IPC_RMID Remove the shared memory identifier specified by shmid from the system and destroy the shared memory segment and shmid_ds data structure associated with it. Note the part about IPC_RMID that required it to not only delete the identifier, but also to 'destroy the shared memory segment'. This was interpreted as being non-compliant with the setting of =0. Solaris man page describes out =0 behaviour, not the POSIX destruction of the segment on IPC_RMID. IMO we can switch the default, but then we would become even more non-compliant. From owner-freebsd-arch@freebsd.org Fri Sep 11 17:36:08 2015 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 73301A02DCF for ; Fri, 11 Sep 2015 17:36:08 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 338BC117C; Fri, 11 Sep 2015 17:36:08 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from ralph.baldwin.cx (c-73-231-226-104.hsd1.ca.comcast.net [73.231.226.104]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 9065EB91E; Fri, 11 Sep 2015 13:36:06 -0400 (EDT) From: John Baldwin To: Konstantin Belousov Cc: freebsd-arch@freebsd.org, Edward Tomasz Napiera??a Subject: Re: Make kern.ipc.shm_allow_removed default to 1. Date: Fri, 11 Sep 2015 10:35:37 -0700 Message-ID: <2286809.Mpxn5Xj9ya@ralph.baldwin.cx> User-Agent: KMail/4.14.3 (FreeBSD/10.2-PRERELEASE; KDE/4.14.3; amd64; ; ) In-Reply-To: <20150911170612.GK2072@kib.kiev.ua> References: <20150908191310.GA3104@brick.home> <5328060.4avXYiGHFa@ralph.baldwin.cx> <20150911170612.GK2072@kib.kiev.ua> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Fri, 11 Sep 2015 13:36:06 -0400 (EDT) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Sep 2015 17:36:08 -0000 On Friday, September 11, 2015 08:06:12 PM Konstantin Belousov wrote: > On Fri, Sep 11, 2015 at 09:24:12AM -0700, John Baldwin wrote: > > On Tuesday, September 08, 2015 09:13:10 PM Edward Tomasz Napiera??a wrote: > > > I'd like to commit - well, discuss it first - the following change: > > > > > > https://reviews.freebsd.org/D3603 > > > > > > This changes the default setting of kern.ipc.shm_allow_removed from 0 to 1. > > > This removes the need of manually changing this flag for Google Chrome > > > users. It also improves compatibility with Linux applications running under > > > Linuxulator compatibility layer, and possibly also helps in porting software > > > from Linux. > > > > > > Generally speaking, the flag allows applications to create the shared memory > > > segment, attach it, remove it, and then continue to use it and to reattach it > > > later. This means that the kernel will automatically "clean up" after the > > > application exits. > > > > > > It could be argued that it's against POSIX. However, SUSv3 says this > > > about IPC_RMID: "Remove the shared memory identifier specified by shmid from > > > the system and destroy the shared memory segment and shmid_ds data structure > > > associated with it." From my reading, we break it in any case by deferring > > > removal of the segment until it's detached; we won't break it any more > > > by also deferring removal of the identifier. > > > > It is against POSIX. When I last raised this with kib@ and alc@, Alan noted > > that our current behavior is not really against POSIX as he felt there was > > enough wriggle room to permit existing mappings to remain valid. > > > > That said, in the thread kib@ felt it was ok to change the default. I think > > it's probably fine. It's a hack for the fact that shmget() isn't returning > > a real fd the way shm_open() does, so in that sense it is ok. That is, > > in shm_open() you can do this: > > > > fd = shm_open("/some/path"); > > shm_unlink("/some/path"); > > > > mmap(..., fd); > > > > /* later */ > > > > mmap(..., fd); > > > > /* or pass fd over a UNIX domain socket and map it in another process */ > > > > close(fd); > > > > /* now mmap won't work */ > > > > With FreeBSD's SHM_ANON you can collapse the first two steps down to: > > > > fd = shm_open(SHM_ANON); > > > > If you think of the return value from shmget() as an fd, then setting this to 1 > > does make sense in a way. That is you can have a sequence of: > > > > id = shmget(...); > > > > hold = shmat(id, ...); /* "hold" mapping */ > > > > shmctl(IPC_RMID, id); > > > > /* later */ > > > > shmat(id, ...); > > > > /* or pass 'id' as a value across a socket and shmat in another process */ > > > > shmdt(hold); > > > > /* now shmat may or may not work */ > > > > However, note that this does require you to do some things carefully: > > > > 1) You have to keep an existing mapping around via shmat() that serves as > > the equivalent of the open file descriptor to keep the 'id' valid. > > This also means you have to defer the IPC_RMID until after you have > > created this "hold" mapping. > > > > 2) You always have to invoke IPC_RMID. In particular, even if you use > > shmget() with IPC_PRIVATE you have to explicitly IPC_RMID. > > > > So, I think it is fine to change this (and have asked about doing it myself > > in the past). However, I think shm_open() is definitely a superior API for > > this sort of thing with semantics that are clearer and more inline with UNIX. > > The request to change the defaults written after somewhat long > discussion elsewere, which was started by the fact that chrome requires > the shm_allow_removed = 1 behaviour. > > My opinion is that yes, =1 behaviour contradicts the SUSv4tc1 specification: > From shmat() description: > [EINVAL] The value of shmid is not a valid shared memory identifier > From shmctl(): > IPC_RMID > Remove the shared memory identifier specified by shmid from the > system and destroy the shared memory segment and shmid_ds data structure > associated with it. > > Note the part about IPC_RMID that required it to not only delete the > identifier, but also to 'destroy the shared memory segment'. This was > interpreted as being non-compliant with the setting of =0. Solaris > man page describes out =0 behaviour, not the POSIX destruction of the > segment on IPC_RMID. > > IMO we can switch the default, but then we would become even more > non-compliant. Regardless of the switch I think we should add an IMPLEMENTATION NOTE or the like to shmctl(2) to describe this sysctl and explain how it affects the behavior. We should also probably have a STANDARDS section in that manpage that details our deviations from the spec. In an earlier thread, Alan stated that he did not know of any UNIX that invalidated the memory segment when IPC_RMID was issued. All of them leave existing mappings in tact. -- John Baldwin From owner-freebsd-arch@freebsd.org Fri Sep 11 20:50:37 2015 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D2EC1A01CDC for ; Fri, 11 Sep 2015 20:50:37 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qg0-f49.google.com (mail-qg0-f49.google.com [209.85.192.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 93E2012D3 for ; Fri, 11 Sep 2015 20:50:37 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by qgev79 with SMTP id v79so72946344qge.0 for ; Fri, 11 Sep 2015 13:50:36 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=QfyvhagEUz6sokijOVhVbaPpjy29FPUxUhPdWqN5+Ls=; b=GnjPLQ0fbnfYTNsHIIL0/bKgjWAGB7WWXa+Qg0nOYUX0yzSvap7P/RdyVEezgC8G5/ vd28nXY2OEksRxeh0QnceXhZRsGMwMzA8NvC0vGvp99w7fZVnoYYtfXvLbEr9FG4gUBq iIHKofsf6OKuQSoohB9+6iquLkepuSkYNvDuTzaGqBiHX94Bi4+qGTRSfAcOoa2acVHr yJE4elPCBJ/xHma9IlRfCpyLGP2swnwZLdaDV8JsUcV49bRQlkIC+ocJSAYkPkPM2KUr TeFxdPAxq3qz5o031mGvNxDENX80QucWY90NRBqPZvCMdBoAoRZHVeD+obrMUbvuyXm8 5cGQ== X-Gm-Message-State: ALoCoQl1Xuo2Gu8nD0m02MjrPVhMalvwhbyQa7HnMeSpFQRvfwx9BpYxUdIbIbnowA6wyCJpsvqY MIME-Version: 1.0 X-Received: by 10.140.129.22 with SMTP id 22mr1379073qhb.74.1442004636013; Fri, 11 Sep 2015 13:50:36 -0700 (PDT) Sender: wlosh@bsdimp.com Received: by 10.140.80.164 with HTTP; Fri, 11 Sep 2015 13:50:35 -0700 (PDT) X-Originating-IP: [50.253.99.174] In-Reply-To: <2286809.Mpxn5Xj9ya@ralph.baldwin.cx> References: <20150908191310.GA3104@brick.home> <5328060.4avXYiGHFa@ralph.baldwin.cx> <20150911170612.GK2072@kib.kiev.ua> <2286809.Mpxn5Xj9ya@ralph.baldwin.cx> Date: Fri, 11 Sep 2015 14:50:35 -0600 X-Google-Sender-Auth: 5pPVRU73iYwJ2arLE7_w5tx5Zdg Message-ID: Subject: Re: Make kern.ipc.shm_allow_removed default to 1. From: Warner Losh To: John Baldwin Cc: Konstantin Belousov , "Edward Tomasz Napiera??a" , "freebsd-arch@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Sep 2015 20:50:37 -0000 I'll note in the early days of the project we'd often make the decision to not be POSIX by default when POSIX was stupid. I think the same precedent applies here. Be more useful by default, but document how to get the pedantic standard's conforming behavior for those that need it. Warner On Fri, Sep 11, 2015 at 11:35 AM, John Baldwin wrote: > On Friday, September 11, 2015 08:06:12 PM Konstantin Belousov wrote: > > On Fri, Sep 11, 2015 at 09:24:12AM -0700, John Baldwin wrote: > > > On Tuesday, September 08, 2015 09:13:10 PM Edward Tomasz Napiera??a > wrote: > > > > I'd like to commit - well, discuss it first - the following change: > > > > > > > > https://reviews.freebsd.org/D3603 > > > > > > > > This changes the default setting of kern.ipc.shm_allow_removed from > 0 to 1. > > > > This removes the need of manually changing this flag for Google > Chrome > > > > users. It also improves compatibility with Linux applications > running under > > > > Linuxulator compatibility layer, and possibly also helps in porting > software > > > > from Linux. > > > > > > > > Generally speaking, the flag allows applications to create the > shared memory > > > > segment, attach it, remove it, and then continue to use it and to > reattach it > > > > later. This means that the kernel will automatically "clean up" > after the > > > > application exits. > > > > > > > > It could be argued that it's against POSIX. However, SUSv3 says this > > > > about IPC_RMID: "Remove the shared memory identifier specified by > shmid from > > > > the system and destroy the shared memory segment and shmid_ds data > structure > > > > associated with it." From my reading, we break it in any case by > deferring > > > > removal of the segment until it's detached; we won't break it any > more > > > > by also deferring removal of the identifier. > > > > > > It is against POSIX. When I last raised this with kib@ and alc@, > Alan noted > > > that our current behavior is not really against POSIX as he felt there > was > > > enough wriggle room to permit existing mappings to remain valid. > > > > > > That said, in the thread kib@ felt it was ok to change the default. > I think > > > it's probably fine. It's a hack for the fact that shmget() isn't > returning > > > a real fd the way shm_open() does, so in that sense it is ok. That is, > > > in shm_open() you can do this: > > > > > > fd = shm_open("/some/path"); > > > shm_unlink("/some/path"); > > > > > > mmap(..., fd); > > > > > > /* later */ > > > > > > mmap(..., fd); > > > > > > /* or pass fd over a UNIX domain socket and map it in another > process */ > > > > > > close(fd); > > > > > > /* now mmap won't work */ > > > > > > With FreeBSD's SHM_ANON you can collapse the first two steps down to: > > > > > > fd = shm_open(SHM_ANON); > > > > > > If you think of the return value from shmget() as an fd, then setting > this to 1 > > > does make sense in a way. That is you can have a sequence of: > > > > > > id = shmget(...); > > > > > > hold = shmat(id, ...); /* "hold" mapping */ > > > > > > shmctl(IPC_RMID, id); > > > > > > /* later */ > > > > > > shmat(id, ...); > > > > > > /* or pass 'id' as a value across a socket and shmat in another > process */ > > > > > > shmdt(hold); > > > > > > /* now shmat may or may not work */ > > > > > > However, note that this does require you to do some things carefully: > > > > > > 1) You have to keep an existing mapping around via shmat() that > serves as > > > the equivalent of the open file descriptor to keep the 'id' valid. > > > This also means you have to defer the IPC_RMID until after you > have > > > created this "hold" mapping. > > > > > > 2) You always have to invoke IPC_RMID. In particular, even if you > use > > > shmget() with IPC_PRIVATE you have to explicitly IPC_RMID. > > > > > > So, I think it is fine to change this (and have asked about doing it > myself > > > in the past). However, I think shm_open() is definitely a superior > API for > > > this sort of thing with semantics that are clearer and more inline > with UNIX. > > > > The request to change the defaults written after somewhat long > > discussion elsewere, which was started by the fact that chrome requires > > the shm_allow_removed = 1 behaviour. > > > > My opinion is that yes, =1 behaviour contradicts the SUSv4tc1 > specification: > > From shmat() description: > > [EINVAL] The value of shmid is not a valid shared memory identifier > > From shmctl(): > > IPC_RMID > > Remove the shared memory identifier specified by shmid from the > > system and destroy the shared memory segment and shmid_ds data > structure > > associated with it. > > > > Note the part about IPC_RMID that required it to not only delete the > > identifier, but also to 'destroy the shared memory segment'. This was > > interpreted as being non-compliant with the setting of =0. Solaris > > man page describes out =0 behaviour, not the POSIX destruction of the > > segment on IPC_RMID. > > > > IMO we can switch the default, but then we would become even more > > non-compliant. > > Regardless of the switch I think we should add an IMPLEMENTATION NOTE or > the > like to shmctl(2) to describe this sysctl and explain how it affects the > behavior. We should also probably have a STANDARDS section in that manpage > that details our deviations from the spec. In an earlier thread, Alan > stated > that he did not know of any UNIX that invalidated the memory segment when > IPC_RMID was issued. All of them leave existing mappings in tact. > > -- > John Baldwin > _______________________________________________ > freebsd-arch@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" >