From owner-freebsd-arch@freebsd.org Mon Sep 7 22:52:29 2015 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2C1779CD463 for ; Mon, 7 Sep 2015 22:52:29 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 0456F1A71 for ; Mon, 7 Sep 2015 22:52:29 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: by mailman.ysv.freebsd.org (Postfix) id 0185F9CD462; Mon, 7 Sep 2015 22:52:29 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 012829CD461 for ; Mon, 7 Sep 2015 22:52:29 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mail-pa0-f48.google.com (mail-pa0-f48.google.com [209.85.220.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CA99F1A6F for ; Mon, 7 Sep 2015 22:52:28 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: by padhy16 with SMTP id hy16so103807052pad.1 for ; Mon, 07 Sep 2015 15:52:22 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:mime-version:content-type:from :in-reply-to:date:cc:message-id:references:to; bh=QpfGx8i7PSbGW+d/UrDAeHlyVJsyvQ0gM8DISOZp5hQ=; b=Vpi8/CkW5CJ8dpaIfAkwgry4bpkNdlEN1lKX65gleww3ELxcOpm3Cmzywj1Vk79Ojt e9G+qwShoK7d/divYT28dJ1CEOKqyj2JuER3ZocWbLtohLhamTPv6YrplZGgmB3fYcZ3 /7cu9L7F70HQv8Xnb5zhiOIJ08MLYCrjTJLUfKGbPcxmGywGzsXaaLshziJ8lleAjL6R 9OlWPSeQ9FIFZqidCJqy0OhtI9VyP2zq5FjderQf1DpwrjSG1Jb6Len7Cj5CiuzecqJi ZfIttPhrWOHv+fTRPbWHVBkyXV/WgfUmxnePyBqjnaqkZDSkvUdecfJ6eJqpY+hRHPoo 5jIg== X-Gm-Message-State: ALoCoQkifbf31bLVF6FT8rGffG99qKwAMYUAboLXrWg7PzTSxhLo2nAFI0YxOcEfTvICroloVIJ8 X-Received: by 10.68.68.167 with SMTP id x7mr20107092pbt.140.1441666342507; Mon, 07 Sep 2015 15:52:22 -0700 (PDT) Received: from ip-100-127-128-52.compute.internal.netflix.com ([69.53.245.5]) by smtp.gmail.com with ESMTPSA id uv5sm959303pbc.12.2015.09.07.15.52.21 (version=TLS1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 07 Sep 2015 15:52:21 -0700 (PDT) Sender: Warner Losh Subject: Re: exporting INVARIANTS easily Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2104\)) Content-Type: multipart/signed; boundary="Apple-Mail=_1810B0DF-7C29-4FD0-A635-A9AF14C71B5E"; protocol="application/pgp-signature"; micalg=pgp-sha512 X-Pgp-Agent: GPGMail 2.5 From: Warner Losh In-Reply-To: <20150907224808.GC1778@garage.freebsd.pl> Date: Mon, 7 Sep 2015 16:52:19 -0600 Cc: "freebsd-arch@freebsd.org" Message-Id: References: <20150907212943.GB1778@garage.freebsd.pl> <20150907224808.GC1778@garage.freebsd.pl> To: Pawel Jakub Dawidek X-Mailer: Apple Mail (2.2104) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Sep 2015 22:52:29 -0000 --Apple-Mail=_1810B0DF-7C29-4FD0-A635-A9AF14C71B5E Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Sep 7, 2015, at 4:48 PM, Pawel Jakub Dawidek = wrote: >=20 > On Mon, Sep 07, 2015 at 04:18:25PM -0600, Warner Losh wrote: >> On Mon, Sep 7, 2015 at 3:29 PM, Pawel Jakub Dawidek = wrote: >>=20 >>> On Wed, Aug 26, 2015 at 11:20:06AM -0600, Warner Losh wrote: >>>> Greetings, >>>>=20 >>>> Many of the performance eating features are exported via some kind = of >>> sysctl, usually >>>> patterned after the case of witness as debug.foo. INVARIANTS = isn=E2=80=99t one >>> of those >>>> features. >>>>=20 >>>> https://reviews.freebsd.org/D3488 >>>>=20 >>>> implements debug.invariants. Please comment. >>>>=20 >>>> I=E2=80=99d thought about adding it to the kern.features sysctl, = but thought >>> better of it since it >>>> isn=E2=80=99t a facility that people can use. >>>>=20 >>>> If you include the kernel config in the kernel, you can get this >>> information via >>>> config -x | grep INVARIANTS >>>> but not all kernels do that. This is more robust. >>>>=20 >>>> I also know that you can load some modules compiled INVARIANTS when = the >>> base >>>> kernel isn=E2=80=99t built that way and this won=E2=80=99t reflect = that. There=E2=80=99s no good >>> want to include >>>> that information and is an uncommon use case. >>>>=20 >>>> Our use case? We have a raft of test machines. Most run without >>> INVARIANTS since >>>> we want to characterize the performance of the release under test. = Some >>> are running >>>> INVARIANTS since we want to test the robustness as well, even at = the >>> expense of >>>> some performance. To ensure we don=E2=80=99t accidentally include = INVARIANTS >>> systems >>>> in the performance number, we=E2=80=99ve adding a key to an = internal database >>> that=E2=80=99s driven >>>> off this sysctl. >>>>=20 >>>> Comments? >>>=20 >>> As long as the ultimate goal is to have INVARIANTS in GENERIC I'm = all >>> for it! I use to run even production machines with INVARIANTS, which = was >>> helpful to catch VirtualBox's kernel memory corruption, but we've = moved >>> to GENERIC since I wanted to use freebsd-update. Having INVARIANTS = in >>> GENERIC would be great. >>=20 >>=20 >> That's not my goal. And I doubt my employer would run it in their >> kernel config because it costs too much in bandwidth when we're >> streaming all that video people binge watch. >=20 > Do you have some numbers handy? I'd be interested in seeing how much > performance drop there is when you have INVARIANTS compiled in, but > debug.invariants set to 0. I don=E2=80=99t have any direct numbers, but the belief is that we=E2=80=99= ll see between 10-20% drop in our peak throughput with an INVARIANTS kernel. Warner --Apple-Mail=_1810B0DF-7C29-4FD0-A635-A9AF14C71B5E Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJV7hUkAAoJEGwc0Sh9sBEAhTwP/jDWBBk+C0AEiTjKaqtE2AM7 v0AsyL/1dbgxihybK728U1EOE9PeQuaXqfwrlQ1nCcdH06snq62iagiHibyksGAa RZrLZRv9MYDq0nvVf42FFdRRbwAAimwLumzCEJtfyxgA3pcRd4EnLxZA6f6kwoga WzJho1gI5qK6eMwzZx+uz0K5dWgXQJSWpJzTiFkHyOQV6skmpZcxIVQDnQF1sAuA XAZlx3XqNCEi8uzjVgf5XytkCVNp6tB9Fjc82gRM249QVLSBkQIeSu8IDjk8xeEk xNtg3BM3G8TwjeRmHDGM2c27pkaaK1cWe/tBM3fyqUOZuxSLDc/lzkj11Jgntuek krdRShitMRUgDsVtUlAemroB5fg9nONzuRI91tDraArQocE1b2dZARAw0/9jWXsp OnsXH2z4NDalEY2Do6kZy65G8KpRC2dVHRpmquU/yUog6HhdOS/afglfwYvXY/zt Jff5nmk1iQvT4KyryZIPfvMmf7disJlf0cuWoUMVDVPZnx5NRHxNcekqs1i0Asbo Bz6sE5zyCSsXEXi7X32Xp3v1XVgUukiDGREbw8w6ejSz/32xaq8gpUJLCpxTvdxb jfmxB5KhOPtDAvUB6wh9MOKIZr2bbavXmwhZut3GyuunALzXaYMOtxr1jejdk+zn HiHdh1+w6ASlSZjZCnnq =xBM0 -----END PGP SIGNATURE----- --Apple-Mail=_1810B0DF-7C29-4FD0-A635-A9AF14C71B5E--