From owner-freebsd-hackers@freebsd.org Sat Dec 30 20:35:35 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D4012EAB3A3 for ; Sat, 30 Dec 2017 20:35:35 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id 6B63B6BDA5; Sat, 30 Dec 2017 20:35:34 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (unknown [192.168.55.3]) by phk.freebsd.dk (Postfix) with ESMTP id ED4A627365; Sat, 30 Dec 2017 20:35:32 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.15.2/8.15.2) with ESMTPS id vBUKZHSl063078 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sat, 30 Dec 2017 20:35:17 GMT (envelope-from phk@critter.freebsd.dk) Received: (from phk@localhost) by critter.freebsd.dk (8.15.2/8.15.2/Submit) id vBUKZDWv063077; Sat, 30 Dec 2017 20:35:13 GMT (envelope-from phk) To: Ian Lepore cc: "Rodney W. Grimes" , Eugene Grosbein , Yuri , Freebsd hackers list Subject: Re: Is it considered to be ok to not check the return code of close(2) in base? In-reply-to: <1514654174.12000.15.camel@freebsd.org> From: "Poul-Henning Kamp" References: <201712301544.vBUFivES076428@pdx.rh.CN85.dnsmgr.net> <1514654174.12000.15.camel@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <63075.1514666113.1@critter.freebsd.dk> Date: Sat, 30 Dec 2017 20:35:13 +0000 Message-ID: <63076.1514666113@critter.freebsd.dk> X-Mailman-Approved-At: Sun, 31 Dec 2017 00:07:03 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Dec 2017 20:35:35 -0000 -------- In message <1514654174.12000.15.camel@freebsd.org>, Ian Lepore writes: >No, you're trying to impose a ridiculously simple rule onto what is >inherently a complex situation. The idea that every close() should be >checked is just ludicrous. I used to think so too, but I changed my mind: It is a lot simpler and much more robust to assert all calls to close succeed, than to determine which of them are important enough that it needs to be checked. Needless to say, if we are talking about /tmp/hack.c which is run once and thrown away, the rules are different than if you check the program into FreeBSD. But for code meant to *live*, I propose that the correct policy at this time and age is to assert *all* assumptions, so the compiler and the subsequent maintainers can see what you thought you knew. And I have hard evidence to back up this position because this policy has been in effect from day one of the Varnish Cache project, where as a result close to 10% of source lines contain some kind of assert. It took 11 years for before Varnish hit the first major security issue. But as the cherry on top, that issue was "only" a DoS, because the crash was caused by a well placed assert which prevented it from becoming a buffer overflow and remote execution problem. I've also found along the way, that all the asserts greatly speed up both development and debugging, because they document the usually tacit assumptions, and in difference from comments, which are almost always out of date and imprecise, you can trust asserts, because they are there on every single run through the code. >What about output to stdout? Does a program under your rules explicitly >need to close() stdout and check the return code? This is an interesting thing to bring up. UNIX provides some hackish conveniences for std{in,out,err} when they are pipes or terminals, but not when they are regular files. This means that programs which happily stop when your terminal session dies does not if the filesystem you redirected std* to fills. (Consistent behaviour would be to also send SIGPIPE for failed std* writes to regular files.) Most programs dont explicityly close std* fd's and explictly do *not* check all writes, because these workarounds would give a lot of false positives. Python does check, and that is why running a python-script into head(1) results in an annoying error. For all these reasons I don't think we should use std* as a strawman in the discussion about close. >Even the earlier simplistic notion that all close() calls should be >asserted still overlooks the fact that virtually every application has >multiple files open, and if an assert triggers on one close(), none of >the other closes (perhaps equally or more important ones) will ever get >checked or reported. This is an utterly bogus argument. If you want to explicitly ignore the return value of close(2) because (for whatever reason) it is not important, you should explicitly tell us so in your source code: (void)close(fd); If you want to make sure it worked, but have important stuff to do in case it fails, you should obviously not use assert(), but write the proper code to test & handle the issue. But if you just close a file, and you're 100% sure that will work, you should write it as: assert(close(fd) == 0); To tell the rest of us about your assumption and your confidence in it. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-hackers@freebsd.org Sat Dec 30 20:37:06 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 81624EAB45E for ; Sat, 30 Dec 2017 20:37:06 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 685056BDCE; Sat, 30 Dec 2017 20:37:05 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (localhost [127.0.0.1]) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3) with ESMTP id vBUKb2rZ077531; Sat, 30 Dec 2017 12:37:02 -0800 (PST) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: (from freebsd-rwg@localhost) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3/Submit) id vBUKb1Pb077530; Sat, 30 Dec 2017 12:37:01 -0800 (PST) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201712302037.vBUKb1Pb077530@pdx.rh.CN85.dnsmgr.net> Subject: Re: Is it considered to be ok to not check the return code of close(2) in base? In-Reply-To: <20171230172047.GK22177@mcvoy.com> To: Larry McVoy Date: Sat, 30 Dec 2017 12:37:01 -0800 (PST) CC: Ian Lepore , Eugene Grosbein , Yuri , Freebsd hackers list X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-Mailman-Approved-At: Sun, 31 Dec 2017 00:07:22 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Dec 2017 20:37:06 -0000 > On Sat, Dec 30, 2017 at 10:16:14AM -0700, Ian Lepore wrote: > > On Sat, 2017-12-30 at 07:44 -0800, Rodney W. Grimes wrote: > > > [ Charset windows-1252 unsupported, converting... ] > > > > > > > > > > > 30.12.2017 21:06, Rodney W. Grimes wrote: > > > > > > > > > > > > > > > > > > > > > Of course, there are cases when that's irrelevant, f.e. closing temporary file > > > > > > that is no more needed and being unlinked anyway. > > > > > No error on close should be treated as irreleveant, they signify something > > > > > has gone wrong and itis best to inform the user and let the user decide > > > > > if this is irrelevant or not. > > > > > > > > > > Code that does not test return codes from EVERY single thing that can > > > > > return an error should be taken out back shown the danish axe, clean it > > > > > up or get it out of our tree! > > > > An application checks for error code after writing to stderr to report an error > > > > and finds it failed too; and what should be done then? :-) > > > One should do the right thing, you know that this is a case of the > > > output to stderr failed, and that it would not be right to try to > > > output another error, but probably just call exit with a non zero > > > value so that the caller may be able to do something with that > > > data. > > > > > > Your trying to over complicate what is a very simple rule, if a status > > > can be returned, one should check it and try to do the best thing possible. > > > Or maybe we should just turn off the compiler warning that flags this? > > > > No, you're trying to impose a ridiculously simple rule onto what is > > inherently a complex situation. ?The idea that every close() should be > > checked is just ludicrous. ?What about output to stdout? ?Does a > > program under your rules explicitly need to close() stdout and check > > the return code? ?If not, justify how a failure to write to stdout in a > > typical unix pipeline-oriented tool is less important than writes > > directly to a disk file. ?How about stderr? ?If that fails, what do you > > do? > > > > Even the earlier simplistic notion that all close() calls should be > > asserted still overlooks the fact that virtually every application has > > multiple files open, and if an assert triggers on one close(), none of > > the other closes (perhaps equally or more important ones) will ever get > > checked or reported. ?Writing code into every simple tool to handle > > failures of close() without short-cutting the checking of following > > closes and cleanup work would just be an absurd waste of time. > > Gonna side with Ian on this one. I took my queues from Rochkind (I think > that's who it was) that wrote the Advanced Unix Programming - when I read > that I was amazed at how many error returns he didn't check. I slowly > learned that he checked the ones that could throw errors where he could > do something about it and ignored the other ones. Mindlessly checking > everything leads to dumb programming. > > I've seen the same thing in POSIX compliance testing. Zillions and > zillions of mindless tests that somehow manage to not test the stuff > that needs testing. As a person that has had to chace the failure to exit with a failure code when a program had a failure that was ignored I am going to disagree. Even if it is just simply for the fact that you should exit with an error if any close failed so that the calling program has the oppertunity to do something. I'll agree that the assert of a failed close may not be the best solution, as you could simply store the errno and continue to close the other files then exit with errno to indicate you did infact have a problem. You really really do want to exit with error and not just ignore a failed close! -- Rod Grimes rgrimes@freebsd.org From owner-freebsd-hackers@freebsd.org Sun Dec 31 19:47:54 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B807BEA414E for ; Sun, 31 Dec 2017 19:47:54 +0000 (UTC) (envelope-from lists@eitanadler.com) Received: from mail-yb0-x233.google.com (mail-yb0-x233.google.com [IPv6:2607:f8b0:4002:c09::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 78EDB80A05 for ; Sun, 31 Dec 2017 19:47:54 +0000 (UTC) (envelope-from lists@eitanadler.com) Received: by mail-yb0-x233.google.com with SMTP id 129so16103145ybe.5 for ; Sun, 31 Dec 2017 11:47:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=eitanadler.com; s=0xdeadbeef; h=mime-version:from:date:message-id:subject:to; bh=kewqP13PUJHv91GzmoOcmmCEYzlxbWtrHli/PT0USBA=; b=LZFiGqh1EYMc9/v4SN0urqpi+U9rWigDtFu8/IGILuQab3mD43g4yqMDQbrGe9nAjm xpKkUpMutrEneMbFHzjTpHqNiswFF8pg1Tgds4BK8+4VsFhXljSWz5t3Akw4+G/tLM8/ TRZnVWThUNWAKLJz+bxDH0DZiWHEyyAlkrMRc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=kewqP13PUJHv91GzmoOcmmCEYzlxbWtrHli/PT0USBA=; b=CcQgp1xOG34JjjcfGuinGlEe6kbj3ZdFkXf9fL1kjwhWRglqp/skIkak3J8Fnt/yGp 5yIQMh/imufjVPSP/2t9Y/0q+xBForopP9PMA2Ab2h2m6s8Yrvfa4DLcpWgxE7WMeSuX HoazfOT/5aXVnAdQemiKL5xpMTDkip2AyfcggsKhEd6TXhws0tbw3948QW43RdQeobJe nCkwdqrAIAxQuyzwsLoT9F7szrHXK/3phe6CG+AufeRFgm1tB1F8R+SfRaPKVqznotcx /KRT0L+6xwoH9cse5C8CFHHGWvQNZ4QjooTQRWUEnzF9euGwZbDGFVRC9mdiscknLpob ZaoQ== X-Gm-Message-State: AKGB3mLrNVSt0kpOfou0pllQ5lhoetOd5qi70ujZ3Y1D8T7/hWfZtgQ7 Q1vgUHgiUImpUN604O0CzTrJU4zxRnqP5dNMQ+7A/cjA X-Google-Smtp-Source: ACJfBotdPmc7aW7qIey0yU+bKxACq1ZuOrTeqXxPyOgLxsHSgVaME2aqRhTNlbAm2OA3k2gOmHdOVGNewC2ecA1q2dA= X-Received: by 10.37.210.216 with SMTP id j207mr21247143ybg.517.1514749673429; Sun, 31 Dec 2017 11:47:53 -0800 (PST) MIME-Version: 1.0 Received: by 10.37.217.21 with HTTP; Sun, 31 Dec 2017 11:47:22 -0800 (PST) From: Eitan Adler Date: Sun, 31 Dec 2017 11:47:22 -0800 Message-ID: Subject: Heads Up: bug status changed for untouched bugs since 2014 To: FreeBSD Hackers , FBSD Doc project Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 31 Dec 2017 19:47:54 -0000 Hi all, I recently made the following mass change: For bugs matching the following criteria: Status: In Progress Product: Base System, Community, Documentation, Other Changed: (is less than) 2014-06-01 Reset to default assignee and clear in-progress status. Note that mail was *skipped* for this change due to the large volume of changes. A list of affected bugs is available at https://people.freebsd.org/~eadler/files/2017-12-31-bug-change/list.txt -- Eitan Adler From owner-freebsd-hackers@freebsd.org Sun Dec 31 22:11:46 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 68892EA9E9E for ; Sun, 31 Dec 2017 22:11:46 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: from asp.reflexion.net (outbound-mail-210-137.reflexion.net [208.70.210.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id F2EF965826 for ; Sun, 31 Dec 2017 22:11:45 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: (qmail 22259 invoked from network); 31 Dec 2017 22:11:38 -0000 Received: from unknown (HELO mail-cs-01.app.dca.reflexion.local) (10.81.19.1) by 0 (rfx-qmail) with SMTP; 31 Dec 2017 22:11:38 -0000 Received: by mail-cs-01.app.dca.reflexion.local (Reflexion email security v8.40.3) with SMTP; Sun, 31 Dec 2017 17:11:38 -0500 (EST) Received: (qmail 2613 invoked from network); 31 Dec 2017 22:11:38 -0000 Received: from unknown (HELO iron2.pdx.net) (69.64.224.71) by 0 (rfx-qmail) with (AES256-SHA encrypted) SMTP; 31 Dec 2017 22:11:38 -0000 Received: from [192.168.1.25] (c-76-115-7-162.hsd1.or.comcast.net [76.115.7.162]) by iron2.pdx.net (Postfix) with ESMTPSA id 66346EC8171; Sun, 31 Dec 2017 14:11:37 -0800 (PST) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: Re: Is it considered to be ok to not check the return code of close(2) in base? Message-Id: <79FFA3EB-5994-4B7B-BE35-F1AB9D4AE3CD@dsl-only.net> Date: Sun, 31 Dec 2017 14:11:36 -0800 To: phk@phk.freebsd.dk, FreeBSD Hackers X-Mailer: Apple Mail (2.3273) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 31 Dec 2017 22:11:46 -0000 Poul-Henning Kamp phk at phk.freebsd.dk wrote on Sat Dec 30 20:35:35 UTC 2017 : > But if you just close a file, and you're 100% sure that will work, > you should write it as: >=20 > assert(close(fd) =3D=3D 0); >=20 > To tell the rest of us about your assumption and your confidence in = it. Quoting the FreeBSD assert man page: QUOTE The assert() macro may be removed at compile time by defining = NDEBUG as a macro (e.g., by using the cc(1) option -DNDEBUG). ENDQUOTE This makes required-actions inside asserts dangerous, at least without guarantees that NDEBUG will be undefined. Trying to guarantee that NDEBUG will be undefined would generally be a bad idea. So, assert(close(fd) =3D=3D 0); is a bad coding practice in my view. =3D=3D=3D Mark Millard markmi at dsl-only.net From owner-freebsd-hackers@freebsd.org Sun Dec 31 22:39:07 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1F54FEAB23C for ; Sun, 31 Dec 2017 22:39:07 +0000 (UTC) (envelope-from lists@eitanadler.com) Received: from mail-yw0-x234.google.com (mail-yw0-x234.google.com [IPv6:2607:f8b0:4002:c05::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D2E29676F0 for ; Sun, 31 Dec 2017 22:39:06 +0000 (UTC) (envelope-from lists@eitanadler.com) Received: by mail-yw0-x234.google.com with SMTP id k80so10891110ywe.0 for ; Sun, 31 Dec 2017 14:39:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=eitanadler.com; s=0xdeadbeef; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=vtlXKK+z2hEqTjEmOQVlrhrVA/lWkKAlq14Otp0G8mM=; b=lBxG10znXWPCJjUzoZvk7FOI4PBK7zuDgSeeO4an0Gxx9FH1A5HAHVr/Uo8muNg4YG 1ksJ1m15yRc26iQfidUuge2dftu4q3Fmx+wIRDNNnpn3QJxJl+UKMQxXUtVIx4qm6mYx YXUz9IfX1yBXydjHZKkUYYB7kq7HQqVm+7YVs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=vtlXKK+z2hEqTjEmOQVlrhrVA/lWkKAlq14Otp0G8mM=; b=lXo3+fN1XFhhJCukPIuOtja2RmTRIC3H5ivhq1utiY0PQ75iNWb/GDHIcAQwooA6YO rd7mZ8vBDZIz4fzn33DQ6NFdSn7O45N1r8hHLDomHQ5sK5rMrTt/KtEX6yPZWGfKWkRM CMdIAkW0kxwfOymywHSLZyMOxDmXq1kInz7eLR4qEkMewZ+KmdmCi8z9RA3r5Kqr44lH SJ6AisgEhgHQ9TNmRWKS8sPDMSZVHZAxAsimhHM3zmADV2OPl0dxJUaYzoVKiLKTraI7 4he/PwP+LCJk7imfWkznwZ9uh1F8u7nngtFW03fNxT/5F2cTtGxm+Pq1ngHhcP0zn/Xq PVjw== X-Gm-Message-State: AKGB3mK+00a3XOZvkKjTs/Rdxh3AI6Rd7pdZP16B3KLdjSTm6n4YEhFS voWYgykIZb93u7yQHdUrVX1wCohxxfo+hOfKziHuFh1W X-Google-Smtp-Source: ACJfBovCccWssJY4CxmEc1G2s657TeLws9OZrTtTszyatAAjrvZZN+HWCL+i8uPtPPQNxY/Q9yYMbVy4p13jiRma4Us= X-Received: by 10.129.212.65 with SMTP id g1mr30600690ywl.425.1514759945766; Sun, 31 Dec 2017 14:39:05 -0800 (PST) MIME-Version: 1.0 Received: by 10.37.217.21 with HTTP; Sun, 31 Dec 2017 14:38:35 -0800 (PST) In-Reply-To: References: From: Eitan Adler Date: Sun, 31 Dec 2017 14:38:35 -0800 Message-ID: Subject: Re: Heads Up: bug status changed for untouched bugs since 2014 To: FreeBSD Hackers , FBSD Doc project Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 31 Dec 2017 22:39:07 -0000 On 31 December 2017 at 11:47, Eitan Adler wrote: > Hi all, > > I recently made the following mass change: > > For bugs matching the following criteria: > Status: In Progress > Product: Base System, Community, Documentation, Other > Changed: (is less than) 2014-06-01 > Reset to default assignee and clear in-progress status. > > Note that mail was *skipped* for this change due to the large volume of changes. > > A list of affected bugs is available at > https://people.freebsd.org/~eadler/files/2017-12-31-bug-change/list.txt Sorry for the spam! Due to the vagaries of bugzilla, in the previous iteration a comment was left, but the actual change did not occur. I just redid this now. -- Eitan Adler From owner-freebsd-hackers@freebsd.org Mon Jan 1 03:05:52 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 17DEAEB484C for ; Mon, 1 Jan 2018 03:05:52 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id EB9646F180 for ; Mon, 1 Jan 2018 03:05:51 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (localhost [127.0.0.1]) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3) with ESMTP id w0135mG4084159; Sun, 31 Dec 2017 19:05:48 -0800 (PST) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: (from freebsd-rwg@localhost) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3/Submit) id w0135luG084158; Sun, 31 Dec 2017 19:05:47 -0800 (PST) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201801010305.w0135luG084158@pdx.rh.CN85.dnsmgr.net> Subject: Re: Is it considered to be ok to not check the return code of close(2) in base? In-Reply-To: <79FFA3EB-5994-4B7B-BE35-F1AB9D4AE3CD@dsl-only.net> To: Mark Millard Date: Sun, 31 Dec 2017 19:05:47 -0800 (PST) CC: phk@phk.freebsd.dk, FreeBSD Hackers X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-Mailman-Approved-At: Mon, 01 Jan 2018 03:43:58 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Jan 2018 03:05:52 -0000 > Poul-Henning Kamp phk at phk.freebsd.dk wrote on > Sat Dec 30 20:35:35 UTC 2017 : > > > But if you just close a file, and you're 100% sure that will work, > > you should write it as: > > > > assert(close(fd) == 0); > > > > To tell the rest of us about your assumption and your confidence in it. > > Quoting the FreeBSD assert man page: > > QUOTE > The assert() macro may be removed at compile time by defining NDEBUG as a > macro (e.g., by using the cc(1) option -DNDEBUG). > ENDQUOTE > > This makes required-actions inside asserts dangerous, > at least without guarantees that NDEBUG will be > undefined. Trying to guarantee that NDEBUG will be > undefined would generally be a bad idea. > > So, > > assert(close(fd) == 0); > > is a bad coding practice in my view. Less bad than either of (void)close(fd)); close(fd); -- Rod Grimes rgrimes@freebsd.org From owner-freebsd-hackers@freebsd.org Mon Jan 1 05:04:19 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 388A4EB7117 for ; Mon, 1 Jan 2018 05:04:19 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: from asp.reflexion.net (outbound-mail-210-149.reflexion.net [208.70.210.149]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D55E5761F3 for ; Mon, 1 Jan 2018 05:04:18 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: (qmail 31920 invoked from network); 1 Jan 2018 03:17:37 -0000 Received: from unknown (HELO mail-cs-02.app.dca.reflexion.local) (10.81.19.2) by 0 (rfx-qmail) with SMTP; 1 Jan 2018 03:17:37 -0000 Received: by mail-cs-02.app.dca.reflexion.local (Reflexion email security v8.40.3) with SMTP; Sun, 31 Dec 2017 22:17:37 -0500 (EST) Received: (qmail 6044 invoked from network); 1 Jan 2018 03:17:37 -0000 Received: from unknown (HELO iron2.pdx.net) (69.64.224.71) by 0 (rfx-qmail) with (AES256-SHA encrypted) SMTP; 1 Jan 2018 03:17:37 -0000 Received: from [192.168.1.25] (c-76-115-7-162.hsd1.or.comcast.net [76.115.7.162]) by iron2.pdx.net (Postfix) with ESMTPSA id 90DAFEC8171; Sun, 31 Dec 2017 19:17:36 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: Re: Is it considered to be ok to not check the return code of close(2) in base? From: Mark Millard In-Reply-To: <201801010305.w0135luG084158@pdx.rh.CN85.dnsmgr.net> Date: Sun, 31 Dec 2017 19:17:35 -0800 Cc: phk@phk.freebsd.dk, FreeBSD Hackers Content-Transfer-Encoding: quoted-printable Message-Id: <559541DD-3287-4473-B7DE-B4DDC6860DF7@dsl-only.net> References: <201801010305.w0135luG084158@pdx.rh.CN85.dnsmgr.net> To: "Rodney W. Grimes" X-Mailer: Apple Mail (2.3273) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Jan 2018 05:04:19 -0000 On 2017-Dec-31, at 7:05 PM, Rodney W. Grimes = wrote: >> Poul-Henning Kamp phk at phk.freebsd.dk wrote on >> Sat Dec 30 20:35:35 UTC 2017 : >>=20 >>> But if you just close a file, and you're 100% sure that will work, >>> you should write it as: >>>=20 >>> assert(close(fd) =3D=3D 0); >>>=20 >>> To tell the rest of us about your assumption and your confidence in = it. >>=20 >> Quoting the FreeBSD assert man page: >>=20 >> QUOTE >> The assert() macro may be removed at compile time by = defining NDEBUG as a >> macro (e.g., by using the cc(1) option -DNDEBUG). >> ENDQUOTE >>=20 >> This makes required-actions inside asserts dangerous, >> at least without guarantees that NDEBUG will be >> undefined. Trying to guarantee that NDEBUG will be >> undefined would generally be a bad idea. >>=20 >> So, >>=20 >> assert(close(fd) =3D=3D 0); >>=20 >> is a bad coding practice in my view. >=20 > Less bad than either of > (void)close(fd)); > close(fd); Hmm. If one is not to check the status code, then just do not close the file, at least for NDEBUG style compiles. Seems odd to me. "assert" indicates optional code, not required code. (This is despite its name.) In my view: Both errors need to be avoided and my point still stands: the example is not a reasonable way to code. One could invent an alternate to assert under a related name that does not make code disappear (not even for NDEBUG). "assert" is just not the right facility to express just: tell the rest of us about your assumption and your confidence in it without also indicating: optional code. =3D=3D=3D Mark Millard markmi at dsl-only.net From owner-freebsd-hackers@freebsd.org Mon Jan 1 05:03:18 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 771FFEB3C0D for ; Mon, 1 Jan 2018 05:03:18 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: from asp.reflexion.net (outbound-mail-210-136.reflexion.net [208.70.210.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1BE25752D1 for ; Mon, 1 Jan 2018 05:03:17 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: (qmail 18493 invoked from network); 1 Jan 2018 05:03:11 -0000 Received: from unknown (HELO mail-cs-01.app.dca.reflexion.local) (10.81.19.1) by 0 (rfx-qmail) with SMTP; 1 Jan 2018 05:03:11 -0000 Received: by mail-cs-01.app.dca.reflexion.local (Reflexion email security v8.40.3) with SMTP; Mon, 01 Jan 2018 00:03:11 -0500 (EST) Received: (qmail 24297 invoked from network); 1 Jan 2018 05:03:11 -0000 Received: from unknown (HELO iron2.pdx.net) (69.64.224.71) by 0 (rfx-qmail) with (AES256-SHA encrypted) SMTP; 1 Jan 2018 05:03:11 -0000 Received: from [192.168.1.25] (c-76-115-7-162.hsd1.or.comcast.net [76.115.7.162]) by iron2.pdx.net (Postfix) with ESMTPSA id 2BAFBEC8B38; Sun, 31 Dec 2017 21:03:10 -0800 (PST) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: Re: Is it considered to be ok to not check the return code of close(2) in base? Date: Sun, 31 Dec 2017 21:03:09 -0800 References: <201801010305.w0135luG084158@pdx.rh.CN85.dnsmgr.net> <559541DD-3287-4473-B7DE-B4DDC6860DF7@dsl-only.net> To: "Rodney W. Grimes" , phk@phk.freebsd.dk, FreeBSD Hackers In-Reply-To: <559541DD-3287-4473-B7DE-B4DDC6860DF7@dsl-only.net> Message-Id: <5AD2D86A-2515-4D4D-91B2-1919531F7CC3@dsl-only.net> X-Mailer: Apple Mail (2.3273) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Jan 2018 05:03:18 -0000 On 2017-Dec-31, at 7:17 PM, Mark Millard wrote: > On 2017-Dec-31, at 7:05 PM, Rodney W. Grimes = wrote: >=20 >>> Poul-Henning Kamp phk at phk.freebsd.dk wrote on >>> Sat Dec 30 20:35:35 UTC 2017 : >>>=20 >>>> But if you just close a file, and you're 100% sure that will work, >>>> you should write it as: >>>>=20 >>>> assert(close(fd) =3D=3D 0); >>>>=20 >>>> To tell the rest of us about your assumption and your confidence in = it. >>>=20 >>> Quoting the FreeBSD assert man page: >>>=20 >>> QUOTE >>> The assert() macro may be removed at compile time by = defining NDEBUG as a >>> macro (e.g., by using the cc(1) option -DNDEBUG). >>> ENDQUOTE >>>=20 >>> This makes required-actions inside asserts dangerous, >>> at least without guarantees that NDEBUG will be >>> undefined. Trying to guarantee that NDEBUG will be >>> undefined would generally be a bad idea. >>>=20 >>> So, >>>=20 >>> assert(close(fd) =3D=3D 0); >>>=20 >>> is a bad coding practice in my view. >>=20 >> Less bad than either of >> (void)close(fd)); >> close(fd); >=20 > Hmm. If one is not to check the status code, > then just do not close the file, at least for > NDEBUG style compiles. Seems odd to me. >=20 > "assert" indicates optional code, not required > code. (This is despite its name.) >=20 > In my view: Both errors need to be avoided and > my point still stands: the example is not a > reasonable way to code. >=20 > One could invent an alternate to assert under > a related name that does not make code > disappear (not even for NDEBUG). "assert" > is just not the right facility to express > just: >=20 > tell the rest of us about your assumption and your confidence in it >=20 > without also indicating: optional code. Going in another direction for when NDEBUG is undefined: assert(expression_evaluating_to_false) is, in part, defined to call abort(). abort() in turn is defined such that: "functions passed to atexit() are not called" also, "[w]hether open resources such as files are closed is implementation defined". (Quoted from http://en.cppreference.com/w/c/program/abort . I can get the standard's wording if needed.) Open Group Base Specifications Issue 7 states in its description of abort: "The functionality described on this reference page is aligned with the ISO C standard. Any conflict between the requirements described here and the ISO C standard is unintentional. This volume of POSIX.1-2008 defers to the ISO C standard". Before Issue 6 the wording apparently required an attempt to fclose all streams --but that made abort necessarily not async-signal-safe. C99 requires the async-signal-safe status. ( http://pubs.opengroup.org/onlinepubs/9699919799/functions/abort.html ) As for C++: "Destructors of variables with automatic, thread local (since C++11) and static storage durations are not called. Functions registered with std::atexit() and std::at_quick_exit (since C++11) are also not called. Whether open resources such as files are closed is implementation defined." ( http://en.cppreference.com/w/cpp/utility/program/abort ) asserts that call abort are difficult to guarantee specific program-exit behavior for, based on just the standards anyway. =3D=3D=3D Mark Millard markmi at dsl-only.net From owner-freebsd-hackers@freebsd.org Mon Jan 1 09:31:13 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 763DFE89B7E for ; Mon, 1 Jan 2018 09:31:13 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id 3EE3765DF6 for ; Mon, 1 Jan 2018 09:31:12 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (unknown [192.168.55.3]) by phk.freebsd.dk (Postfix) with ESMTP id 8D61E273A3; Mon, 1 Jan 2018 09:31:09 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.15.2/8.15.2) with ESMTPS id w019UrUD069604 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 1 Jan 2018 09:30:53 GMT (envelope-from phk@critter.freebsd.dk) Received: (from phk@localhost) by critter.freebsd.dk (8.15.2/8.15.2/Submit) id w019UrS6069603; Mon, 1 Jan 2018 09:30:53 GMT (envelope-from phk) To: "Rodney W. Grimes" cc: Mark Millard , FreeBSD Hackers Subject: Re: Is it considered to be ok to not check the return code of close(2) in base? In-reply-to: <201801010305.w0135luG084158@pdx.rh.CN85.dnsmgr.net> From: "Poul-Henning Kamp" References: <201801010305.w0135luG084158@pdx.rh.CN85.dnsmgr.net> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <69601.1514799053.1@critter.freebsd.dk> Content-Transfer-Encoding: quoted-printable Date: Mon, 01 Jan 2018 09:30:53 +0000 Message-ID: <69602.1514799053@critter.freebsd.dk> X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Jan 2018 09:31:13 -0000 -------- In message <201801010305.w0135luG084158@pdx.rh.CN85.dnsmgr.net>, "Rodney W= . Gri mes" writes: >> Quoting the FreeBSD assert man page: >> = >> QUOTE >> The assert() macro may be removed at compile time by defining NDEB= UG as a >> macro (e.g., by using the cc(1) option -DNDEBUG). >> ENDQUOTE >> = >> This makes required-actions inside asserts dangerous, >> at least without guarantees that NDEBUG will be >> undefined. Trying to guarantee that NDEBUG will be >> undefined would generally be a bad idea. There is a trivial workaround: Define your own assert() macro. -- = Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe = Never attribute to malice what can adequately be explained by incompetence= . From owner-freebsd-hackers@freebsd.org Mon Jan 1 10:03:29 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E1532EA166C for ; Mon, 1 Jan 2018 10:03:29 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id A5FAD66F5B for ; Mon, 1 Jan 2018 10:03:29 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (unknown [192.168.55.3]) by phk.freebsd.dk (Postfix) with ESMTP id E91C327395; Mon, 1 Jan 2018 10:03:27 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.15.2/8.15.2) with ESMTPS id w01A3C5H069783 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 1 Jan 2018 10:03:12 GMT (envelope-from phk@critter.freebsd.dk) Received: (from phk@localhost) by critter.freebsd.dk (8.15.2/8.15.2/Submit) id w01A3C8h069782; Mon, 1 Jan 2018 10:03:12 GMT (envelope-from phk) To: Mark Millard cc: "Rodney W. Grimes" , FreeBSD Hackers Subject: Re: Is it considered to be ok to not check the return code of close(2) in base? In-reply-to: <559541DD-3287-4473-B7DE-B4DDC6860DF7@dsl-only.net> From: "Poul-Henning Kamp" References: <201801010305.w0135luG084158@pdx.rh.CN85.dnsmgr.net> <559541DD-3287-4473-B7DE-B4DDC6860DF7@dsl-only.net> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <69780.1514800992.1@critter.freebsd.dk> Content-Transfer-Encoding: quoted-printable Date: Mon, 01 Jan 2018 10:03:12 +0000 Message-ID: <69781.1514800992@critter.freebsd.dk> X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Jan 2018 10:03:30 -0000 -------- In message <559541DD-3287-4473-B7DE-B4DDC6860DF7@dsl-only.net>, Mark Milla= rd wr ites: >"assert" indicates optional code, not required >code. (This is despite its name.) Assert statements are not debugging, although they greatly help debugging, they are an integral part of the program, which documents for the maintainers and the running system what assumptions are being made. Who ever added "#ifndef NDEBUG" not only failed Sensible Naming 101, they also totally misunderstood the nature of assert() as a programming construct. -- = Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe = Never attribute to malice what can adequately be explained by incompetence= . From owner-freebsd-hackers@freebsd.org Mon Jan 1 10:07:38 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B97D1EA1995 for ; Mon, 1 Jan 2018 10:07:38 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id 811676713C for ; Mon, 1 Jan 2018 10:07:37 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (unknown [192.168.55.3]) by phk.freebsd.dk (Postfix) with ESMTP id AC88E27395; Mon, 1 Jan 2018 10:07:36 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.15.2/8.15.2) with ESMTPS id w01A7KY0069803 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 1 Jan 2018 10:07:21 GMT (envelope-from phk@critter.freebsd.dk) Received: (from phk@localhost) by critter.freebsd.dk (8.15.2/8.15.2/Submit) id w01A7KJU069802; Mon, 1 Jan 2018 10:07:20 GMT (envelope-from phk) To: Mark Millard cc: "Rodney W. Grimes" , FreeBSD Hackers Subject: Re: Is it considered to be ok to not check the return code of close(2) in base? In-reply-to: <5AD2D86A-2515-4D4D-91B2-1919531F7CC3@dsl-only.net> From: "Poul-Henning Kamp" References: <201801010305.w0135luG084158@pdx.rh.CN85.dnsmgr.net> <559541DD-3287-4473-B7DE-B4DDC6860DF7@dsl-only.net> <5AD2D86A-2515-4D4D-91B2-1919531F7CC3@dsl-only.net> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <69800.1514801240.1@critter.freebsd.dk> Content-Transfer-Encoding: quoted-printable Date: Mon, 01 Jan 2018 10:07:20 +0000 Message-ID: <69801.1514801240@critter.freebsd.dk> X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Jan 2018 10:07:38 -0000 -------- In message <5AD2D86A-2515-4D4D-91B2-1919531F7CC3@dsl-only.net>, Mark Milla= rd wr ites: >asserts that call abort are difficult to >guarantee specific program-exit behavior >for, based on just the standards anyway. One should read "assert" in a source code as a curse along the lines of "Strike me by lightning if this is not true!" If you want more gentle behaviour you should implement proper errorhandlin= g. But for all the places where you think "Nahh ... that's never going to happen", you should document your decision with assert(). -- = Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe = Never attribute to malice what can adequately be explained by incompetence= . From owner-freebsd-hackers@freebsd.org Mon Jan 1 15:54:53 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5FADAEAFE20 for ; Mon, 1 Jan 2018 15:54:53 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: from asp.reflexion.net (outbound-mail-210-135.reflexion.net [208.70.210.135]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0B5BC715B5 for ; Mon, 1 Jan 2018 15:54:52 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: (qmail 25503 invoked from network); 1 Jan 2018 15:48:05 -0000 Received: from unknown (HELO mail-cs-01.app.dca.reflexion.local) (10.81.19.1) by 0 (rfx-qmail) with SMTP; 1 Jan 2018 15:48:05 -0000 Received: by mail-cs-01.app.dca.reflexion.local (Reflexion email security v8.40.3) with SMTP; Mon, 01 Jan 2018 10:48:05 -0500 (EST) Received: (qmail 3495 invoked from network); 1 Jan 2018 15:48:05 -0000 Received: from unknown (HELO iron2.pdx.net) (69.64.224.71) by 0 (rfx-qmail) with (AES256-SHA encrypted) SMTP; 1 Jan 2018 15:48:05 -0000 Received: from [192.168.1.25] (c-76-115-7-162.hsd1.or.comcast.net [76.115.7.162]) by iron2.pdx.net (Postfix) with ESMTPSA id 814D1EC8FD9; Mon, 1 Jan 2018 07:48:04 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: Re: Is it considered to be ok to not check the return code of close(2) in base? From: Mark Millard In-Reply-To: <69781.1514800992@critter.freebsd.dk> Date: Mon, 1 Jan 2018 07:48:03 -0800 Cc: "Rodney W. Grimes" , FreeBSD Hackers Content-Transfer-Encoding: quoted-printable Message-Id: <510305A9-460C-407F-B2FC-3521A6E1D78B@dsl-only.net> References: <201801010305.w0135luG084158@pdx.rh.CN85.dnsmgr.net> <559541DD-3287-4473-B7DE-B4DDC6860DF7@dsl-only.net> <69781.1514800992@critter.freebsd.dk> To: Poul-Henning Kamp X-Mailer: Apple Mail (2.3273) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Jan 2018 15:54:53 -0000 On 2018-Jan-1, at 2:03 AM, Poul-Henning Kamp = wrote: > -------- > In message <559541DD-3287-4473-B7DE-B4DDC6860DF7@dsl-only.net>, Mark = Millard wr > ites: >=20 >> "assert" indicates optional code, not required >> code. (This is despite its name.) >=20 > Assert statements are not debugging, although they greatly help > debugging, they are an integral part of the program, which documents > for the maintainers and the running system what assumptions are > being made. >=20 > Who ever added "#ifndef NDEBUG" not only failed Sensible Naming > 101, they also totally misunderstood the nature of assert() as > a programming construct. None of us invented assert as it was first historically created or as it is in the standards. It possibly predates pre-conditions, invariants, and the like (e.g., predicate transformers) as a programming technique. You are arguing with a definition we are not in control of if the standard's header is used (in C or in C++). It clearly was invented to allow avoiding the performance consequences of the contained expression. That suggests that it was invented for debugging, like it or not. If one wants to use assert, then instead of: assert(close(fd) =3D=3D 0); use code like: close_status=3D close(fd); assert(close_stats=3D=3D0); to avoid the close disappearing if NDEBUG is defined. Just a different coding organization for that specific point to be addressed. (This does not deal with the potential consequences of use of abort() for assert failure, especially code targeting multiple environments.) I wrote earlier: "One could invent an alternate to assert under a related name". You wrote: "Define your own assert() macro." Other than possible confusions over if a long standing definition is in use or not, these can address the issues with assert. =3D=3D=3D Mark Millard markmi at dsl-only.net From owner-freebsd-hackers@freebsd.org Mon Jan 1 16:15:01 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 27842EB09E9 for ; Mon, 1 Jan 2018 16:15:01 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id DDD8C71E9A for ; Mon, 1 Jan 2018 16:15:00 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (unknown [192.168.55.3]) by phk.freebsd.dk (Postfix) with ESMTP id 6F4A6273A3; Mon, 1 Jan 2018 16:14:50 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.15.2/8.15.2) with ESMTPS id w01GEYmV078984 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 1 Jan 2018 16:14:34 GMT (envelope-from phk@critter.freebsd.dk) Received: (from phk@localhost) by critter.freebsd.dk (8.15.2/8.15.2/Submit) id w01GEXfv078983; Mon, 1 Jan 2018 16:14:33 GMT (envelope-from phk) To: Mark Millard cc: FreeBSD Hackers , "Rodney W. Grimes" Subject: Re: Is it considered to be ok to not check the return code of close(2) in base? In-reply-to: <510305A9-460C-407F-B2FC-3521A6E1D78B@dsl-only.net> From: "Poul-Henning Kamp" References: <201801010305.w0135luG084158@pdx.rh.CN85.dnsmgr.net> <559541DD-3287-4473-B7DE-B4DDC6860DF7@dsl-only.net> <69781.1514800992@critter.freebsd.dk> <510305A9-460C-407F-B2FC-3521A6E1D78B@dsl-only.net> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <78981.1514823273.1@critter.freebsd.dk> Content-Transfer-Encoding: quoted-printable Date: Mon, 01 Jan 2018 16:14:33 +0000 Message-ID: <78982.1514823273@critter.freebsd.dk> X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Jan 2018 16:15:01 -0000 -------- In message <510305A9-460C-407F-B2FC-3521A6E1D78B@dsl-only.net>, Mark Milla= rd wr ites: >None of us invented assert as it was >first historically created or as it is >in the standards. Asserts are way older than UNIX. >If one wants to use assert, then >instead of: Just do: #undef NDEBUG #include But this is bikeshedding at this point anyway. The important thing is this: Yes, you should check the return value of close(2) (except possibly for the special cases of stdin/-out/-err) and if you are sure they will never fail, doing so with an assert makes sense. Over&Out -- = Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe = Never attribute to malice what can adequately be explained by incompetence= . From owner-freebsd-hackers@freebsd.org Mon Jan 1 16:18:24 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 30551EB0CC1 for ; Mon, 1 Jan 2018 16:18:24 +0000 (UTC) (envelope-from lm@mcvoy.com) Received: from mcvoy.com (mcvoy.com [192.169.23.250]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 02B8E72140 for ; Mon, 1 Jan 2018 16:18:23 +0000 (UTC) (envelope-from lm@mcvoy.com) Received: by mcvoy.com (Postfix, from userid 3546) id 86C9535E139; Mon, 1 Jan 2018 08:18:17 -0800 (PST) Date: Mon, 1 Jan 2018 08:18:17 -0800 From: Larry McVoy To: Poul-Henning Kamp Cc: Mark Millard , FreeBSD Hackers , "Rodney W. Grimes" Subject: Re: Is it considered to be ok to not check the return code of close(2) in base? Message-ID: <20180101161817.GF4678@mcvoy.com> References: <201801010305.w0135luG084158@pdx.rh.CN85.dnsmgr.net> <559541DD-3287-4473-B7DE-B4DDC6860DF7@dsl-only.net> <69781.1514800992@critter.freebsd.dk> <510305A9-460C-407F-B2FC-3521A6E1D78B@dsl-only.net> <78982.1514823273@critter.freebsd.dk> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <78982.1514823273@critter.freebsd.dk> User-Agent: Mutt/1.5.24 (2015-08-30) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Jan 2018 16:18:24 -0000 On Mon, Jan 01, 2018 at 04:14:33PM +0000, Poul-Henning Kamp wrote: > But this is bikeshedding at this point anyway. +1 From owner-freebsd-hackers@freebsd.org Mon Jan 1 16:20:54 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BF043EB100F for ; Mon, 1 Jan 2018 16:20:54 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: from asp.reflexion.net (outbound-mail-210-112.reflexion.net [208.70.210.112]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1439E723D9 for ; Mon, 1 Jan 2018 16:20:53 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: (qmail 721 invoked from network); 1 Jan 2018 15:54:12 -0000 Received: from unknown (HELO rtc-sm-01.app.dca.reflexion.local) (10.81.150.1) by 0 (rfx-qmail) with SMTP; 1 Jan 2018 15:54:12 -0000 Received: by rtc-sm-01.app.dca.reflexion.local (Reflexion email security v8.40.3) with SMTP; Mon, 01 Jan 2018 10:54:12 -0500 (EST) Received: (qmail 10890 invoked from network); 1 Jan 2018 15:54:11 -0000 Received: from unknown (HELO iron2.pdx.net) (69.64.224.71) by 0 (rfx-qmail) with (AES256-SHA encrypted) SMTP; 1 Jan 2018 15:54:11 -0000 Received: from [192.168.1.25] (c-76-115-7-162.hsd1.or.comcast.net [76.115.7.162]) by iron2.pdx.net (Postfix) with ESMTPSA id 3154CEC8E10; Mon, 1 Jan 2018 07:54:11 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: Re: Is it considered to be ok to not check the return code of close(2) in base? From: Mark Millard In-Reply-To: <69801.1514801240@critter.freebsd.dk> Date: Mon, 1 Jan 2018 07:54:10 -0800 Cc: "Rodney W. Grimes" , FreeBSD Hackers Content-Transfer-Encoding: quoted-printable Message-Id: <6658FD9E-3170-4249-8AEC-002E112473F4@dsl-only.net> References: <201801010305.w0135luG084158@pdx.rh.CN85.dnsmgr.net> <559541DD-3287-4473-B7DE-B4DDC6860DF7@dsl-only.net> <5AD2D86A-2515-4D4D-91B2-1919531F7CC3@dsl-only.net> <69801.1514801240@critter.freebsd.dk> To: Poul-Henning Kamp X-Mailer: Apple Mail (2.3273) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Jan 2018 16:20:54 -0000 On 2018-Jan-1, at 2:07 AM, Poul-Henning Kamp = wrote: > -------- > In message <5AD2D86A-2515-4D4D-91B2-1919531F7CC3@dsl-only.net>, Mark = Millard wr > ites: >=20 >> asserts that call abort are difficult to >> guarantee specific program-exit behavior >> for, based on just the standards anyway. >=20 > One should read "assert" in a source code as a curse along the lines = of > "Strike me by lightning if this is not true!" >=20 > If you want more gentle behaviour you should implement proper = errorhandling. >=20 > But for all the places where you think "Nahh ... that's never going to > happen", you should document your decision with assert(). (I'm ignoring the NDEBUG issue here.) Just be sure that for your context of use that the consequences of failure can be tolerated. Even the vintage of linux glibc matters: http://man7.org/linux/man-pages/man3/abort.3.html says: (note the reference to deadlocks and data corruption) Up until glibc 2.26, if the abort() function caused process termination, all open streams were closed and flushed (as with fclose(3)). However, in some cases this could result in = deadlocks and data corruption. Therefore, starting with glibc 2.27, = abort() terminates the process without flushing streams. POSIX.1 permits either possible behavior, saying that abort() "may include an = attempt to effect fclose() on all open streams". =3D=3D=3D Mark Millard markmi at dsl-only.net From owner-freebsd-hackers@freebsd.org Mon Jan 1 16:57:20 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C85D4EB263C for ; Mon, 1 Jan 2018 16:57:20 +0000 (UTC) (envelope-from lm@mcvoy.com) Received: from mcvoy.com (mcvoy.com [192.169.23.250]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A562673AFF for ; Mon, 1 Jan 2018 16:57:20 +0000 (UTC) (envelope-from lm@mcvoy.com) Received: by mcvoy.com (Postfix, from userid 3546) id A321E35E139; Mon, 1 Jan 2018 08:57:18 -0800 (PST) Date: Mon, 1 Jan 2018 08:57:18 -0800 From: Larry McVoy To: "Rodney W. Grimes" Cc: Larry McVoy , Poul-Henning Kamp , FreeBSD Hackers Subject: Re: Is it considered to be ok to not check the return code of close(2) in base? Message-ID: <20180101165718.GI4678@mcvoy.com> References: <20180101161817.GF4678@mcvoy.com> <201801011652.w01GqvCx087076@pdx.rh.CN85.dnsmgr.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201801011652.w01GqvCx087076@pdx.rh.CN85.dnsmgr.net> User-Agent: Mutt/1.5.24 (2015-08-30) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Jan 2018 16:57:20 -0000 On Mon, Jan 01, 2018 at 08:52:57AM -0800, Rodney W. Grimes wrote: > > On Mon, Jan 01, 2018 at 04:14:33PM +0000, Poul-Henning Kamp wrote: > > > But this is bikeshedding at this point anyway. > > > > +1 > > Bike shedding is good, people learn things from it. I never knew that > assert was altered by NDEBUG for example, thanks for that enlightenment Um, does the FreeBSD man page not start like the Linux man page with If the macro NDEBUG was defined at the moment was last included, the macro assert() generates no code, and hence does nothing at all. ? And bikeshedding has the effect of making people hit the delete key. I've deleted without reading about 80% of this thread. So if there was signal in that 80%, I for one, did not get it. And the amount of back and forth on something that is this basic is sort of mind numbing. As a new person on FreeBSD it doesn't show the project in a good light. Just sayin. From owner-freebsd-hackers@freebsd.org Mon Jan 1 16:53:02 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9B161EB23A0 for ; Mon, 1 Jan 2018 16:53:02 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7D99E7395B for ; Mon, 1 Jan 2018 16:53:02 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (localhost [127.0.0.1]) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3) with ESMTP id w01GqwgL087077; Mon, 1 Jan 2018 08:52:58 -0800 (PST) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: (from freebsd-rwg@localhost) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3/Submit) id w01GqvCx087076; Mon, 1 Jan 2018 08:52:57 -0800 (PST) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201801011652.w01GqvCx087076@pdx.rh.CN85.dnsmgr.net> Subject: Re: Is it considered to be ok to not check the return code of close(2) in base? In-Reply-To: <20180101161817.GF4678@mcvoy.com> To: Larry McVoy Date: Mon, 1 Jan 2018 08:52:57 -0800 (PST) CC: Poul-Henning Kamp , FreeBSD Hackers X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-Mailman-Approved-At: Mon, 01 Jan 2018 17:00:03 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Jan 2018 16:53:02 -0000 > On Mon, Jan 01, 2018 at 04:14:33PM +0000, Poul-Henning Kamp wrote: > > But this is bikeshedding at this point anyway. > > +1 Bike shedding is good, people learn things from it. I never knew that assert was altered by NDEBUG for example, thanks for that enlightenment Mark. Didnt even realize that assert had been bastardized by standards, and phk is right, it predates all those things, probably by a decade or more. In summary, given the original question, it would be in the interest of the project to evaluate those close() calls that do not check for errors and do something about it. Either document the intent to intentionally ignore them, or handle the error in some fasion. -- Rod Grimes rgrimes@freebsd.org From owner-freebsd-hackers@freebsd.org Mon Jan 1 17:07:23 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4587FEB2FBE for ; Mon, 1 Jan 2018 17:07:23 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from outbound1b.ore.mailhop.org (outbound1b.ore.mailhop.org [54.200.247.200]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 283F6741D2 for ; Mon, 1 Jan 2018 17:07:22 +0000 (UTC) (envelope-from ian@freebsd.org) X-MHO-User: 31d70ed2-ef16-11e7-8486-0934409070aa X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 73.78.92.27 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [73.78.92.27]) by outbound1.ore.mailhop.org (Halon) with ESMTPSA id 31d70ed2-ef16-11e7-8486-0934409070aa; Mon, 01 Jan 2018 17:07:06 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id w01H7Ki3010640; Mon, 1 Jan 2018 10:07:20 -0700 (MST) (envelope-from ian@freebsd.org) Message-ID: <1514826440.12000.42.camel@freebsd.org> Subject: Re: Is it considered to be ok to not check the return code of close(2) in base? From: Ian Lepore To: Larry McVoy , "Rodney W. Grimes" Cc: FreeBSD Hackers , Poul-Henning Kamp Date: Mon, 01 Jan 2018 10:07:20 -0700 In-Reply-To: <20180101165718.GI4678@mcvoy.com> References: <20180101161817.GF4678@mcvoy.com> <201801011652.w01GqvCx087076@pdx.rh.CN85.dnsmgr.net> <20180101165718.GI4678@mcvoy.com> Content-Type: text/plain; charset="ISO-8859-1" X-Mailer: Evolution 3.18.5.1 FreeBSD GNOME Team Port Mime-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Jan 2018 17:07:23 -0000 On Mon, 2018-01-01 at 08:57 -0800, Larry McVoy wrote: > And the amount of back and forth on something that is this basic is sort > of mind numbing.  As a new person on FreeBSD it doesn't show the project > in a good light.  Just sayin. So when you're unable to make a point using technical merits, you instead attempt to shame people into thinking the way you do?  And then, like so many people these days, you seem to think that adding "Just sayin" the end of something inappropriate you've said somehow makes it okay?  The whole problem is that you're "just sayin" that. -- Ian From owner-freebsd-hackers@freebsd.org Mon Jan 1 17:13:58 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CEF67EB36A6 for ; Mon, 1 Jan 2018 17:13:58 +0000 (UTC) (envelope-from lm@mcvoy.com) Received: from mcvoy.com (mcvoy.com [192.169.23.250]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A4F7274803; Mon, 1 Jan 2018 17:13:57 +0000 (UTC) (envelope-from lm@mcvoy.com) Received: by mcvoy.com (Postfix, from userid 3546) id 14C8035E139; Mon, 1 Jan 2018 09:13:56 -0800 (PST) Date: Mon, 1 Jan 2018 09:13:56 -0800 From: Larry McVoy To: Ian Lepore Cc: Larry McVoy , "Rodney W. Grimes" , FreeBSD Hackers , Poul-Henning Kamp Subject: Re: Is it considered to be ok to not check the return code of close(2) in base? Message-ID: <20180101171356.GK4678@mcvoy.com> References: <20180101161817.GF4678@mcvoy.com> <201801011652.w01GqvCx087076@pdx.rh.CN85.dnsmgr.net> <20180101165718.GI4678@mcvoy.com> <1514826440.12000.42.camel@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1514826440.12000.42.camel@freebsd.org> User-Agent: Mutt/1.5.24 (2015-08-30) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Jan 2018 17:13:58 -0000 On Mon, Jan 01, 2018 at 10:07:20AM -0700, Ian Lepore wrote: > On Mon, 2018-01-01 at 08:57 -0800, Larry McVoy wrote: > > And the amount of back and forth on something that is this basic is sort > > of mind numbing.? As a new person on FreeBSD it doesn't show the project > > in a good light.? Just sayin. > > So when you're unable to make a point using technical merits, you > instead attempt to shame people into thinking the way you do? ?And > then, like so many people these days, you seem to think that adding > "Just sayin" the end of something inappropriate you've said somehow > makes it okay? ?The whole problem is that you're "just sayin" that. OK, take out the "just sayin" and I stand by what I wrote. I spent a long time on the Linux Kernel mailing list and while they can get lost, I don't remember this amount of bike shedding on such a simple topic. When people consider the fact that -DNDEBUG is new information I sort of wonder how you get so far in life without knowing the basics. We're not talking about anything that is subtle, hard, or obscure. What's next? A long discussion about whether to use parens around args in cpp macros? Aren't you supposed to just know this stuff? I'm not trying to shame anyone into anything, I'm simply stating that this thread is not something you'd want to advertise to get more people interested in FreeBSD. Quite the opposite. I'm new here, and it's perfectly clear to me that new people's opinion don't count, so feel free to ignore my comments. But I stand by them. From owner-freebsd-hackers@freebsd.org Mon Jan 1 17:16:54 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DEE97EB38D2 for ; Mon, 1 Jan 2018 17:16:54 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-it0-x22d.google.com (mail-it0-x22d.google.com [IPv6:2607:f8b0:4001:c0b::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A5D4B749B2 for ; Mon, 1 Jan 2018 17:16:54 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-it0-x22d.google.com with SMTP id x28so36781935ita.0 for ; Mon, 01 Jan 2018 09:16:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=FNrvj8dgSRQaPSuM4WUVuO5lJyi/k7okRt/5o9h+z7E=; b=AjRLPY2xc2fjs/7zdQ+GXBxu2xtCt2BrmeA9PuJjejKbt8+rDF1x9XFGcD/Job3s0o gdSqpPFlefhxz0qPPmkxBQqMtRaL0Uvb/Dua0jQeGVEXC/dh8lPt1MhVice1qe9pcLgR FWU3XmI5FjBmwhfZ/QRwUr0ap+EYJpuJjKTtw04V30aRHUF1c+PYMryYNSDVowziiJ8/ Smq0kTHAjL2LNh/PjxKjefJfr0tTjQx1jhBIJ7TAJ7og+awHd3Yg4SnYw5II9soAYdI8 w+5tWyJk93+QULomKRmvsCTIYTTZ2GJH4DaukL2Nh5u/wCc0+b6myX7ftcVSwEaGzBsN uHJg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=FNrvj8dgSRQaPSuM4WUVuO5lJyi/k7okRt/5o9h+z7E=; b=MJNUuFc9dXR4aiMmN3v/M0rJ/IA56V3GcakpXs9n8sT9E9mGNlZmZnhEjcxgRTMQRr gusX+KBv4Dr+qXS2m6JAY0XW+jwIxDHxSypv4Zb19J04x6Hof6oEx/Kg9j7pBt5TP9yO xBbUe2WRmkpVX3/S56ZVskTCYfnkPn5yoL3QCmYBwD50mHg453VxwPk/yZdPE4dwZuHe pti4HpQTEUT/JrkkGaCB/6nQjPcYyr6R6ND2bFOHW95xv0gEWlB6HVLWeqxRfCqbosVp RAHlGcRcNE/uQqwVfcgtazhpBfh10MbJ5Yrv+ycX06OKWP3urijAX2DwphvpNREd4pDZ i0ig== X-Gm-Message-State: AKGB3mKdCIzSHBsL7Sl8zdjNjfToXsCtGBuR9MScl+Pl5xbBNVj1d7Cl 31wShFF1qAZg2p76xF2H9NBVG+2UwMQD6cZfeOTd3UnI X-Google-Smtp-Source: ACJfBosZ0ModxorIfoaN9qmhGNm/og9fs64uvBiz+mS8+plXyo0UyguIPBIRAFfCpeCVYilwA6h9w+3iwdUKS29Ve0g= X-Received: by 10.36.77.143 with SMTP id l137mr56730599itb.50.1514827013861; Mon, 01 Jan 2018 09:16:53 -0800 (PST) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 10.79.108.204 with HTTP; Mon, 1 Jan 2018 09:16:53 -0800 (PST) X-Originating-IP: [2603:300b:6:5100:1052:acc7:f9de:2b6d] In-Reply-To: <20180101165718.GI4678@mcvoy.com> References: <20180101161817.GF4678@mcvoy.com> <201801011652.w01GqvCx087076@pdx.rh.CN85.dnsmgr.net> <20180101165718.GI4678@mcvoy.com> From: Warner Losh Date: Mon, 1 Jan 2018 10:16:53 -0700 X-Google-Sender-Auth: 04pyRIJ_RGJomDzrr_4GQJO-XD8 Message-ID: Subject: Re: Is it considered to be ok to not check the return code of close(2) in base? To: Larry McVoy Cc: "Rodney W. Grimes" , FreeBSD Hackers , Poul-Henning Kamp Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Jan 2018 17:16:55 -0000 On Mon, Jan 1, 2018 at 9:57 AM, Larry McVoy wrote: > On Mon, Jan 01, 2018 at 08:52:57AM -0800, Rodney W. Grimes wrote: > > > On Mon, Jan 01, 2018 at 04:14:33PM +0000, Poul-Henning Kamp wrote: > > > > But this is bikeshedding at this point anyway. > > > > > > +1 > > > > Bike shedding is good, people learn things from it. I never knew that > > assert was altered by NDEBUG for example, thanks for that enlightenment > > Um, does the FreeBSD man page not start like the Linux man page with > > If the macro NDEBUG was defined at the moment was > last > included, the macro assert() generates no code, and hence does > nothing > at all. > > ? > It does: The assert() macro may be removed at compile time by defining NDEBUG as a macro (e.g., by using the cc(1) option -DNDEBUG). and the man page is short enough that you can't really miss it.Let' > And bikeshedding has the effect of making people hit the delete key. I've > deleted without reading about 80% of this thread. So if there was signal > in that 80%, I for one, did not get it. > > And the amount of back and forth on something that is this basic is sort > of mind numbing. As a new person on FreeBSD it doesn't show the project > in a good light. Just sayin. As long as we're Just Sayin' things, I point you to the level of discourse on LKM. Just Sayin', ya know? Bikesheds happen in all projects. This one was just a bit nutty and happened to happen now. Warner From owner-freebsd-hackers@freebsd.org Mon Jan 1 17:55:29 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 60A29EB4F9D for ; Mon, 1 Jan 2018 17:55:29 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2324E75C80 for ; Mon, 1 Jan 2018 17:55:28 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (localhost [127.0.0.1]) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3) with ESMTP id w01HtP4K087354; Mon, 1 Jan 2018 09:55:25 -0800 (PST) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: (from freebsd-rwg@localhost) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3/Submit) id w01HtOtD087353; Mon, 1 Jan 2018 09:55:24 -0800 (PST) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201801011755.w01HtOtD087353@pdx.rh.CN85.dnsmgr.net> Subject: Re: Is it considered to be ok to not check the return code of close(2) in base? In-Reply-To: <20180101165718.GI4678@mcvoy.com> To: Larry McVoy Date: Mon, 1 Jan 2018 09:55:24 -0800 (PST) CC: Poul-Henning Kamp , FreeBSD Hackers X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-Mailman-Approved-At: Mon, 01 Jan 2018 18:05:25 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Jan 2018 17:55:29 -0000 > On Mon, Jan 01, 2018 at 08:52:57AM -0800, Rodney W. Grimes wrote: > > > On Mon, Jan 01, 2018 at 04:14:33PM +0000, Poul-Henning Kamp wrote: > > > > But this is bikeshedding at this point anyway. > > > > > > +1 > > > > Bike shedding is good, people learn things from it. I never knew that > > assert was altered by NDEBUG for example, thanks for that enlightenment > > Um, does the FreeBSD man page not start like the Linux man page with > > If the macro NDEBUG was defined at the moment was last > included, the macro assert() generates no code, and hence does nothing > at all. > > ? It further says that an assert() appeared in V6, and I bet that NDEBUG didnt exist at that time. I dont read man pages for something I have been using for 20+ years. Further investage leads me to learn that Turing and von Neumann both wrote of assertions, so maybe my old finger memory has been broken by standards that arrose long after I learned how to do "check error codes" when calling functions, even if they seem impossible or improbable. > And bikeshedding has the effect of making people hit the delete key. I've > deleted without reading about 80% of this thread. So if there was signal > in that 80%, I for one, did not get it. :-( And people wont even engage in disucssions sighting "oh that well just be a bike shed." So rather than having discussions we have anarchy. So lets have a bikeshed on bikesheds? > And the amount of back and forth on something that is this basic is sort > of mind numbing. As a new person on FreeBSD it doesn't show the project > in a good light. Just sayin. Basics are as important as fine details, and given the disparity that occured during this "bikeshead" it shows that there are even wide spread opionions on the very basics of if you should check error returns from function calls, and what ways there are of doing that. Mind numbing, perhaps, pointless, no I do not believe so. -- Rod Grimes rgrimes@freebsd.org From owner-freebsd-hackers@freebsd.org Mon Jan 1 18:26:48 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1B6E3EB63E9 for ; Mon, 1 Jan 2018 18:26:48 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id D755876CF6 for ; Mon, 1 Jan 2018 18:26:47 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (unknown [192.168.55.3]) by phk.freebsd.dk (Postfix) with ESMTP id 3ADBD2738A; Mon, 1 Jan 2018 18:26:45 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.15.2/8.15.2) with ESMTPS id w01IQTDq079513 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 1 Jan 2018 18:26:29 GMT (envelope-from phk@critter.freebsd.dk) Received: (from phk@localhost) by critter.freebsd.dk (8.15.2/8.15.2/Submit) id w01IQSZs079512; Mon, 1 Jan 2018 18:26:28 GMT (envelope-from phk) To: "Rodney W. Grimes" cc: Larry McVoy , FreeBSD Hackers Subject: Re: Is it considered to be ok to not check the return code of close(2) in base? In-reply-to: <201801011755.w01HtOtD087353@pdx.rh.CN85.dnsmgr.net> From: "Poul-Henning Kamp" References: <201801011755.w01HtOtD087353@pdx.rh.CN85.dnsmgr.net> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <79510.1514831188.1@critter.freebsd.dk> Content-Transfer-Encoding: quoted-printable Date: Mon, 01 Jan 2018 18:26:28 +0000 Message-ID: <79511.1514831188@critter.freebsd.dk> X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Jan 2018 18:26:48 -0000 -------- In message <201801011755.w01HtOtD087353@pdx.rh.CN85.dnsmgr.net>, "Rodney W= . Gri mes" writes: >It further says that an assert() appeared in V6, and I bet that NDEBUG >didnt exist at that time. Well, I think you'll loose that bet, because the earliest copy of assert.h which has been uncovered does have NDEBUG: https://github.com/dspinellis/unix-history-repo/blob/Research-V7-Snapshot= -Development/usr/include/assert.h -- = Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe = Never attribute to malice what can adequately be explained by incompetence= . From owner-freebsd-hackers@freebsd.org Mon Jan 1 18:44:02 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A334BEB6DA2 for ; Mon, 1 Jan 2018 18:44:02 +0000 (UTC) (envelope-from hiren@strugglingcoder.info) Received: from mail.strugglingcoder.info (strugglingcoder.info [104.236.146.68]) by mx1.freebsd.org (Postfix) with ESMTP id 8C4E577577; Mon, 1 Jan 2018 18:44:02 +0000 (UTC) (envelope-from hiren@strugglingcoder.info) Received: from localhost (unknown [10.1.1.3]) (Authenticated sender: hiren@strugglingcoder.info) by mail.strugglingcoder.info (Postfix) with ESMTPA id 1F9A817FEF; Mon, 1 Jan 2018 10:43:59 -0800 (PST) Date: Mon, 1 Jan 2018 10:43:59 -0800 From: hiren panchasara To: Larry McVoy Cc: Ian Lepore , FreeBSD Hackers Subject: Re: Is it considered to be ok to not check the return code of close(2) in base? Message-ID: <20180101184359.GA50547@strugglingcoder.info> References: <20180101161817.GF4678@mcvoy.com> <201801011652.w01GqvCx087076@pdx.rh.CN85.dnsmgr.net> <20180101165718.GI4678@mcvoy.com> <1514826440.12000.42.camel@freebsd.org> <20180101171356.GK4678@mcvoy.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="zYM0uCDKw75PZbzx" Content-Disposition: inline In-Reply-To: <20180101171356.GK4678@mcvoy.com> User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Jan 2018 18:44:02 -0000 --zYM0uCDKw75PZbzx Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 01/01/18 at 09:13P, Larry McVoy wrote: >=20 > When people consider the fact that -DNDEBUG is new information I sort > of wonder how you get so far in life without knowing the basics. We're > not talking about anything that is subtle, hard, or obscure. ^ This is what is the most demotivating thing to me if I look at entire thread. And mind you, I am new compared to a lot of people discussing here. We want to have a space where newcomers can come and ask questions without being judged like this. So, no, discussions don't push people away, at least not the newcomers, imo. Problem is we all have a different definition on 'new'. So, let's not ask people to stop discussing something just because we don't like it. Use the 'delete' key as needed. :-) (I do it on most of the emails on mailing lists.) Also, happy new year! Cheers, Hiren --zYM0uCDKw75PZbzx Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQF8BAABCgBmBQJaSoFsXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRBNEUyMEZBMUQ4Nzg4RjNGMTdFNjZGMDI4 QjkyNTBFMTU2M0VERkU1AAoJEIuSUOFWPt/l/CcH/iIONGLIVJLQH3vlMGI508+X nhnAwV1J5BUmlZe62VO9PKiuD81If9Fdbi81ehUaKVHVggX8bKkI+SzPWBCpHQ1B d2ZJx0bqVc/aeTJLuuiZ4Spx5lPS0dR00L+EzBTkMbhmLHHZSnZuOFXbYvjuOmb4 N01l2CZ6Yu5kKn1EDeX61RcSIsJ2R9S7hgNIHj8PGcVk5qNhrt3pDA4eTQbOhe/g v6e8JhRl0PKGX5ViWJi55vQ/j54WBWzuQZYt/Y3FdXbJtYS2M8v6EataHokUUNnF znQv90xf72MdB7V22HMbbqo7tLKgOp+odvWd78LFlRPsdgMUx0abnw143xq+FJc= =nGMx -----END PGP SIGNATURE----- --zYM0uCDKw75PZbzx-- From owner-freebsd-hackers@freebsd.org Mon Jan 1 18:52:12 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3AFFDEB734E for ; Mon, 1 Jan 2018 18:52:12 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-io0-x22f.google.com (mail-io0-x22f.google.com [IPv6:2607:f8b0:4001:c06::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F273F77B00 for ; Mon, 1 Jan 2018 18:52:11 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-io0-x22f.google.com with SMTP id 87so42917010ior.5 for ; Mon, 01 Jan 2018 10:52:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=sK0jdou5+NSh/Yuzl/9+qJjS5HgQ1S2qVgxOmdZJxho=; b=ns47/lu2ZHd1PmAMD3ZvW8e0JRPKBmkd0baC+zmB6VqCr2Ds6OLYuZTixNK4nQY++C sMOKb//jYmv2WwHKuoUtqbGBy/FvQ90fkZVzq6IQUZi1HwC5YE5u5iCrH9R/qLIiBlXn tGS7z8yTXk8ub9B3CNNyh7QVHDvL6fd3I1OUF61ljtglR5rLvqIr4Lkf07nOMLkD5sCC u7yV/u15f9+Tqm55q9nZPllqpYnV/fVTzNKePn2RWDiHV45ApSVJvaLN71LxjE24HF2g lNHW4AV84q586Qu1lGGGW4RR3tyd8g4zyid86JEvXllaQ9bEZIc6xjsifsJKf1bx9IXY Mu4A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=sK0jdou5+NSh/Yuzl/9+qJjS5HgQ1S2qVgxOmdZJxho=; b=VwR+Y6hxF95l8tRal4dwhxqyUOyOVGbhrMdMj44OlxxZA41UfU6YCgd135mXaZu5Fy WhgvxPcv2vdmZGl/tbRwl4vbXLmtgUdB5/Nf6N0Gz0ZTEQyLOEaNE7ODkgqeamhhtKe/ b5nqRlRCEi1IEF9zjKuYYXvoNJ/5PEylj378QZp41M38mPMutIJ+zhTVnySCfuOlttg4 pQoa5VOpuVPpVs36021F4SVLtVPs2OxhgMiCsTdOtzGj76Lu5NQs3gu/LtZdazme6LjX iAQDTS/6O5yz0pQtjVmFGDaeFZFzlBOC7cRVZmeevl1DFbP/Vb6ioWmXqsDf/2Fd7TVS VPaQ== X-Gm-Message-State: AKGB3mL89Ss0xPniChCaccOUgcIrFau+YCOkhEtnuOSNL3xMjF/8S2T+ 4dyuq4yF7HA+Yqc2uBrPLebRvHrrjWpNbANOI50McznY X-Google-Smtp-Source: ACJfBovuUASyAeXL3fskcz2IndNcBgFfcnlmwlCFeXVOo5ejY+TIzeaTb80oRYhVUkztE/SNhB9SmGCMPq6AXOpIT/M= X-Received: by 10.107.13.67 with SMTP id 64mr12467200ion.301.1514832730841; Mon, 01 Jan 2018 10:52:10 -0800 (PST) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 10.79.108.204 with HTTP; Mon, 1 Jan 2018 10:52:10 -0800 (PST) X-Originating-IP: [2603:300b:6:5100:1052:acc7:f9de:2b6d] In-Reply-To: <201801011755.w01HtOtD087353@pdx.rh.CN85.dnsmgr.net> References: <20180101165718.GI4678@mcvoy.com> <201801011755.w01HtOtD087353@pdx.rh.CN85.dnsmgr.net> From: Warner Losh Date: Mon, 1 Jan 2018 11:52:10 -0700 X-Google-Sender-Auth: tYnTc1dGMdPuyC4KpOdsHJ3Iigc Message-ID: Subject: Re: Is it considered to be ok to not check the return code of close(2) in base? To: "Rodney W. Grimes" Cc: Larry McVoy , FreeBSD Hackers , Poul-Henning Kamp Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Jan 2018 18:52:12 -0000 On Mon, Jan 1, 2018 at 10:55 AM, Rodney W. Grimes < freebsd-rwg@pdx.rh.cn85.dnsmgr.net> wrote: > > On Mon, Jan 01, 2018 at 08:52:57AM -0800, Rodney W. Grimes wrote: > > > > On Mon, Jan 01, 2018 at 04:14:33PM +0000, Poul-Henning Kamp wrote: > > > > > But this is bikeshedding at this point anyway. > > > > > > > > +1 > > > > > > Bike shedding is good, people learn things from it. I never knew that > > > assert was altered by NDEBUG for example, thanks for that enlightenment > > > > Um, does the FreeBSD man page not start like the Linux man page with > > > > If the macro NDEBUG was defined at the moment > was last > > included, the macro assert() generates no code, and hence does > nothing > > at all. > > > > ? > > It further says that an assert() appeared in V6, and I bet that NDEBUG > didnt exist at that time. http://www.tuhs.org/cgi-bin/utree.pl?file=V7/usr/include/assert.h says by at least v7 it was there. The man page appears to be incorrect that it appeared in V6 unix, as I can find no file named *assert* in the two distributions available at TUHS, nor could grep turn up anything on assert apart from: doc/c/c3:an assertion that when a construction of So it was definitely in V7 (I'll fix the man page), and had NDEBUG from the start. Warner From owner-freebsd-hackers@freebsd.org Mon Jan 1 18:35:35 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 06B07EB68EF for ; Mon, 1 Jan 2018 18:35:35 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A485977214 for ; Mon, 1 Jan 2018 18:35:34 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (localhost [127.0.0.1]) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3) with ESMTP id w01IZWQR087573; Mon, 1 Jan 2018 10:35:32 -0800 (PST) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: (from freebsd-rwg@localhost) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3/Submit) id w01IZWtG087572; Mon, 1 Jan 2018 10:35:32 -0800 (PST) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201801011835.w01IZWtG087572@pdx.rh.CN85.dnsmgr.net> Subject: Re: Is it considered to be ok to not check the return code of close(2) in base? In-Reply-To: <79511.1514831188@critter.freebsd.dk> To: Poul-Henning Kamp Date: Mon, 1 Jan 2018 10:35:32 -0800 (PST) CC: Larry McVoy , FreeBSD Hackers X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-Mailman-Approved-At: Mon, 01 Jan 2018 20:22:11 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Jan 2018 18:35:35 -0000 > -------- > In message <201801011755.w01HtOtD087353@pdx.rh.CN85.dnsmgr.net>, "Rodney W. Gri > mes" writes: > > >It further says that an assert() appeared in V6, and I bet that NDEBUG > >didnt exist at that time. > > Well, I think you'll loose that bet, because the earliest copy of assert.h > which has been uncovered does have NDEBUG: > > https://github.com/dspinellis/unix-history-repo/blob/Research-V7-Snapshot-Development/usr/include/assert.h Um V7 was after V6, and is probably when asssert.h was introduced, so I would win the bit, assert.h didnt even exist at V6. I can go pull the hardcopy manuals and check.... be later today. -- Rod Grimes rgrimes@freebsd.org From owner-freebsd-hackers@freebsd.org Mon Jan 1 20:46:13 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 949A2EBBE9C for ; Mon, 1 Jan 2018 20:46:13 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-it0-x22a.google.com (mail-it0-x22a.google.com [IPv6:2607:f8b0:4001:c0b::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 582B97CEE4 for ; Mon, 1 Jan 2018 20:46:13 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-it0-x22a.google.com with SMTP id x28so37064592ita.0 for ; Mon, 01 Jan 2018 12:46:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=fcYVQZuHsBVX4/BzyPwRAoFMVnn/0mYLaq2cC/+BKnc=; b=VfRhxFYXV3CVbW8hz3KfUG23i6kn8IaIKlZZSyJxFlkZvJ+W0ljI3KpzzHDawXt51+ XOuurLlLbJsGK49FuMwyrtlGRK8WqBdbH8VFdHqfi8Z+6mH0v0AI2rF8EpMZBSibg8yy 6zWvGINmDTHGFC01O63pj7h2mt9VLzzDI5bXzNEaTNPaCSjn1PxXzvM8tcmfru/zW/Yr l/CtSv3Mudd6ljs085viCyrwJ5/Vr1AJocYLidLv1PgLqa14I4ko2Emun/XtXak55ine F3f0l7HcZKCjJa0uZbgCRKUw7dAdHGzNAuVGcByqggre+WGs8VqoObSu9Ua2svVrZLO9 LlOA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=fcYVQZuHsBVX4/BzyPwRAoFMVnn/0mYLaq2cC/+BKnc=; b=BJieM1+VrXQvanJUiLFgYZIjBtl3AN3bZGNAQx1PsN1myVBKlTnFaNX1U+KEg2Uub8 83v/v+6oA/bowTEbEiypAb+wfUhnZcXatITpWv63+aUzl6ARUIdV8MBhvMaih8QxQi91 QDSAshSgA5zKcHm/MeqdGm4S7bhXC0Nt1lABctuOND6kLECVemSFOdi6OVNqF0/w4SIu ZZPi52i3W2aKZ0FrNEfNnKf5RLXrXpntoUtXaXAODGkX5FY91z6SM3ajwRGTd09Q8bqA Z81XdSqlFu60H+LiFHVY9MqGp/gpd6AzUblwubn/U+KPCAsR4dL8UYrZaJqe8mFGKtSx VSAA== X-Gm-Message-State: AKGB3mK3e+fNTrn94KTeTdK4u6+278Gb0AHzm7I3D8UWXrVcpJTiuVs3 NmDafHDiIaxnuvGROZuQAzY9+ohh/MPxUUu34adoNg== X-Google-Smtp-Source: ACJfBosuhIHFyCCQfrKqDMnYsu3yOwj20MXXUicxplH7EhAMbn2EBpeSCpLrMyFMKiQmtQ2dVOhRZ9owoYjSD+6tvZM= X-Received: by 10.36.164.13 with SMTP id z13mr59000674ite.115.1514839572430; Mon, 01 Jan 2018 12:46:12 -0800 (PST) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 10.79.108.204 with HTTP; Mon, 1 Jan 2018 12:46:11 -0800 (PST) X-Originating-IP: [2603:300b:6:5100:1052:acc7:f9de:2b6d] In-Reply-To: <201801011835.w01IZWtG087572@pdx.rh.CN85.dnsmgr.net> References: <79511.1514831188@critter.freebsd.dk> <201801011835.w01IZWtG087572@pdx.rh.CN85.dnsmgr.net> From: Warner Losh Date: Mon, 1 Jan 2018 13:46:11 -0700 X-Google-Sender-Auth: LWt8oBABwsr2yGHncdkeIOgq3PU Message-ID: Subject: Re: Is it considered to be ok to not check the return code of close(2) in base? To: "Rodney W. Grimes" Cc: Poul-Henning Kamp , FreeBSD Hackers , Larry McVoy Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Jan 2018 20:46:13 -0000 On Mon, Jan 1, 2018 at 11:35 AM, Rodney W. Grimes < freebsd-rwg@pdx.rh.cn85.dnsmgr.net> wrote: > > -------- > > In message <201801011755.w01HtOtD087353@pdx.rh.CN85.dnsmgr.net>, > "Rodney W. Gri > > mes" writes: > > > > >It further says that an assert() appeared in V6, and I bet that NDEBUG > > >didnt exist at that time. > > > > Well, I think you'll loose that bet, because the earliest copy of > assert.h > > which has been uncovered does have NDEBUG: > > > > https://github.com/dspinellis/unix-history-repo/blob/ > Research-V7-Snapshot-Development/usr/include/assert.h > > Um V7 was after V6, and is probably when asssert.h was introduced, so I > would win the bit, assert.h didnt even exist at V6. > I can go pull the hardcopy manuals and check.... be later today. > I can state with 100% certainty that neither assert nor ASSERT were defined in the existing v6 distributions available from TUHS. In pwb, such a macro exists in the malloc.c source (ASSERT), but isn't available to anything but the source/s4/malloc.c, which doesn't count as being available. I stand by my analysis that says v7 was the first time it was available. And even so, if debug wasn't defined, ASSERT was defined to be nothing. Warner From owner-freebsd-hackers@freebsd.org Mon Jan 1 19:15:27 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 58E9FEB832D for ; Mon, 1 Jan 2018 19:15:27 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 107B478C9B for ; Mon, 1 Jan 2018 19:15:26 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (localhost [127.0.0.1]) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3) with ESMTP id w01JFL4k087716; Mon, 1 Jan 2018 11:15:21 -0800 (PST) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: (from freebsd-rwg@localhost) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3/Submit) id w01JFLS5087715; Mon, 1 Jan 2018 11:15:21 -0800 (PST) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201801011915.w01JFLS5087715@pdx.rh.CN85.dnsmgr.net> Subject: Re: Is it considered to be ok to not check the return code of close(2) in base? In-Reply-To: To: Warner Losh Date: Mon, 1 Jan 2018 11:15:21 -0800 (PST) CC: Larry McVoy , FreeBSD Hackers , Poul-Henning Kamp X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-Mailman-Approved-At: Mon, 01 Jan 2018 21:31:47 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Jan 2018 19:15:27 -0000 [ Charset UTF-8 unsupported, converting... ] > On Mon, Jan 1, 2018 at 10:55 AM, Rodney W. Grimes < > freebsd-rwg@pdx.rh.cn85.dnsmgr.net> wrote: > > > > On Mon, Jan 01, 2018 at 08:52:57AM -0800, Rodney W. Grimes wrote: > > > > > On Mon, Jan 01, 2018 at 04:14:33PM +0000, Poul-Henning Kamp wrote: > > > > > > But this is bikeshedding at this point anyway. > > > > > > > > > > +1 > > > > > > > > Bike shedding is good, people learn things from it. I never knew that > > > > assert was altered by NDEBUG for example, thanks for that enlightenment > > > > > > Um, does the FreeBSD man page not start like the Linux man page with > > > > > > If the macro NDEBUG was defined at the moment > > was last > > > included, the macro assert() generates no code, and hence does > > nothing > > > at all. > > > > > > ? > > > > It further says that an assert() appeared in V6, and I bet that NDEBUG > > didnt exist at that time. > > > http://www.tuhs.org/cgi-bin/utree.pl?file=V7/usr/include/assert.h says by > at least v7 it was there. The man page appears to be incorrect that it > appeared in V6 unix, as I can find no file named *assert* in the two > distributions available at TUHS, nor could grep turn up anything on assert > apart from: > > doc/c/c3:an assertion that when a construction of > > So it was definitely in V7 (I'll fix the man page), and had NDEBUG from the > start. Please do not do that, so far I have found in a v6 pwb: #define ASSERT(p) #ifdef debug #define ASSERT(p) if(!(p))botch("p");else botch(s) char *s; { printf("assertion botched: %s\n",s); abort(); } #endif Which is more of how I rember it being, not the presence of NDEBUG cleaning the code of asserts, but the lack of DEBUG. The online manual pages for v6 are abismal compared to the printed hard copies. I have a set in storage, I'll retrieve them and see what is there. -- Rod Grimes rgrimes@freebsd.org From owner-freebsd-hackers@freebsd.org Tue Jan 2 19:48:51 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4F25CEB5944 for ; Tue, 2 Jan 2018 19:48:51 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: from asp.reflexion.net (outbound-mail-210-141.reflexion.net [208.70.210.141]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 125B56F7CA for ; Tue, 2 Jan 2018 19:48:49 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: (qmail 20361 invoked from network); 2 Jan 2018 19:48:43 -0000 Received: from unknown (HELO mail-cs-02.app.dca.reflexion.local) (10.81.19.2) by 0 (rfx-qmail) with SMTP; 2 Jan 2018 19:48:43 -0000 Received: by mail-cs-02.app.dca.reflexion.local (Reflexion email security v8.40.3) with SMTP; Tue, 02 Jan 2018 14:48:43 -0500 (EST) Received: (qmail 18158 invoked from network); 2 Jan 2018 19:48:43 -0000 Received: from unknown (HELO iron2.pdx.net) (69.64.224.71) by 0 (rfx-qmail) with (AES256-SHA encrypted) SMTP; 2 Jan 2018 19:48:43 -0000 Received: from [192.168.1.25] (c-76-115-7-162.hsd1.or.comcast.net [76.115.7.162]) by iron2.pdx.net (Postfix) with ESMTPSA id C8A6CEC7B31; Tue, 2 Jan 2018 11:48:42 -0800 (PST) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: rpi2 head -r327485 (e.g.): rpi2 leaves one "CPU n" always idle for some boots Message-Id: <3258F809-F179-4EB7-857C-74667BECD360@dsl-only.net> Date: Tue, 2 Jan 2018 11:48:42 -0800 To: Freebsd-arm , FreeBSD Hackers , FreeBSD Current X-Mailer: Apple Mail (2.3273) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Jan 2018 19:48:51 -0000 I've seen this over many versions of head for months but have never managed to find a way to force it to happen. It just shows up once and a while. Thus, I'm just dumping out some top and kernel information here for reference. I've used: openssl speed 1>/dev/null 2>&1 & openssl speed 1>/dev/null 2>&1 & openssl speed 1>/dev/null 2>&1 & openssl speed 1>/dev/null 2>&1 & to give the rpi2 4 active processes. Various outputs are from different times without a reboot between. top -CaePores shows the likes of: PID USERNAME THR PRI NICE SIZE RES SWAP STATE C TIME = CPU COMMAND 614 root 1 20 0 10452K 10480K 0K select 1 0:00 = 0.03% /usr/sbin/ntpd -g -c /etc/ntp.conf -p /var/run/ntpd.pid -f = /var/db/ntpd.drift 661 root 1 52 0 9984K 6132K 0K select 1 0:00 = 0.00% /usr/sbin/sshd 751 root 1 101 0 7256K 4276K 0K RUN 1 0:28 = 99.57% openssl speed 750 root 1 100 0 7256K 4276K 0K CPU0 0 0:32 = 94.83% openssl speed 753 root 1 86 0 7256K 4276K 0K RUN 3 0:13 = 52.36% openssl speed 752 root 1 86 0 7256K 4276K 0K CPU3 3 0:14 = 46.54% openssl speed 363 root 1 20 0 6428K 3840K 0K select 3 0:00 = 0.00% /sbin/devd . . . and: last pid: 754; load averages: 3.70, 2.38, 1.58 = = up 0+00:16:50 01:59:37 21 processes: 5 running, 16 sleeping CPU 0: 94.9% user, 0.0% nice, 0.0% system, 5.1% interrupt, 0.0% idle CPU 1: 99.6% user, 0.0% nice, 0.0% system, 0.4% interrupt, 0.0% idle CPU 2: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle CPU 3: 100% user, 0.0% nice, 0.0% system, 0.0% interrupt, 0.0% idle Mem: 12M Active, 1136K Inact, 56M Wired, 30M Buf, 722M Free Swap: 1536M Total, 6M Free =46rom problem boot to problem boot, the CPU that stays idle has varied but usually has been CPU 2. I've never seen 2 or more stuck in idle. show allpcpu shows the likes of: db> show allpcpu Current CPU: 0 cpuid =3D 0 dynamic pcpu =3D 0x3d2540 curthread =3D 0xd8478ae0: pid 2032 tid 100150 "openssl" curpcb =3D 0xd852ae98 fpcurthread =3D 0xd8478ae0: pid 2032 "openssl" idlethread =3D 0xc376fae0: tid 100002 "idle: cpu0" curpmap =3D 0xd8e43bf4 curvnet =3D 0 cpuid =3D 1 dynamic pcpu =3D 0x3998540 curthread =3D 0xd7e5b3a0: pid 2031 tid 100173 "openssl" curpcb =3D 0xda7e0e98 fpcurthread =3D 0xd7e5b3a0: pid 2031 "openssl" idlethread =3D 0xc376f740: tid 100003 "idle: cpu1" curpmap =3D 0xd8e43ec4 curvnet =3D 0 cpuid =3D 2 dynamic pcpu =3D 0x3999540 curthread =3D 0xc376f3a0: pid 10 tid 100004 "idle: cpu2" curpcb =3D 0xc378ae98 fpcurthread =3D none idlethread =3D 0xc376f3a0: tid 100004 "idle: cpu2" curpmap =3D 0 curvnet =3D 0 cpuid =3D 3 dynamic pcpu =3D 0x399a540 curthread =3D 0xd8477000: pid 2034 tid 100167 "openssl" curpcb =3D 0xd876de98 fpcurthread =3D 0xd8477000: pid 2034 "openssl" idlethread =3D 0xc376f000: tid 100005 "idle: cpu3" curpmap =3D 0xc377ab04 curvnet =3D 0 In other words: it appears that the cpuN (here cpu2) is left with idle scheduled all the time for some reason. ps from db> shows things like: db> ps pid ppid pgrp uid state wmesg wchan cmd 2034 714 2034 0 R+ openssl 2033 714 2033 0 R+ CPU 3 openssl 2032 714 2032 0 R+ CPU 0 openssl 2031 714 2031 0 R+ CPU 1 openssl (then later:) db> ps pid ppid pgrp uid state wmesg wchan cmd 2034 714 2034 0 R+ CPU 3 openssl 2033 714 2033 0 R+ openssl 2032 714 2032 0 R+ CPU 0 openssl 2031 714 2031 0 R+ CPU 1 openssl There is also: 10 0 0 0 RL (threaded) [idle] 100002 CanRun [idle: cpu0] 100003 CanRun [idle: cpu1] 100004 CanRun [idle: cpu2] 100005 CanRun [idle: cpu3] =20 These are from: # uname -apKU FreeBSD rpi2 12.0-CURRENT FreeBSD 12.0-CURRENT r327485M arm armv7 = 1200054 1200054 =3D=3D=3D Mark Millard markmi at dsl-only.net From owner-freebsd-hackers@freebsd.org Tue Jan 2 19:59:20 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6CC51EB6436 for ; Tue, 2 Jan 2018 19:59:20 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: from asp.reflexion.net (outbound-mail-210-138.reflexion.net [208.70.210.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 31C19701D8 for ; Tue, 2 Jan 2018 19:59:19 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: (qmail 10069 invoked from network); 2 Jan 2018 19:59:13 -0000 Received: from unknown (HELO rtc-sm-01.app.dca.reflexion.local) (10.81.150.1) by 0 (rfx-qmail) with SMTP; 2 Jan 2018 19:59:13 -0000 Received: by rtc-sm-01.app.dca.reflexion.local (Reflexion email security v8.40.3) with SMTP; Tue, 02 Jan 2018 14:59:13 -0500 (EST) Received: (qmail 13481 invoked from network); 2 Jan 2018 19:59:13 -0000 Received: from unknown (HELO iron2.pdx.net) (69.64.224.71) by 0 (rfx-qmail) with (AES256-SHA encrypted) SMTP; 2 Jan 2018 19:59:13 -0000 Received: from [192.168.1.25] (c-76-115-7-162.hsd1.or.comcast.net [76.115.7.162]) by iron2.pdx.net (Postfix) with ESMTPSA id 4E89DEC94C7; Tue, 2 Jan 2018 11:59:12 -0800 (PST) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: Re: rpi2 head -r327485 (e.g.): rpi2 leaves one "CPU n" always idle for some boots Date: Tue, 2 Jan 2018 11:59:11 -0800 References: <3258F809-F179-4EB7-857C-74667BECD360@dsl-only.net> To: Freebsd-arm , FreeBSD Hackers , FreeBSD Current In-Reply-To: <3258F809-F179-4EB7-857C-74667BECD360@dsl-only.net> Message-Id: <422E2742-7170-4D1A-894F-F310EE819E3A@dsl-only.net> X-Mailer: Apple Mail (2.3273) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Jan 2018 19:59:20 -0000 On 2018-Jan-2, at 11:48 AM, Mark Millard wrote: > I've seen this over many versions of head for months > but have never managed to find a way to force it to > happen. It just shows up once and a while. >=20 > Thus, I'm just dumping out some top and kernel information > here for reference. I've used: >=20 > openssl speed 1>/dev/null 2>&1 & > openssl speed 1>/dev/null 2>&1 & > openssl speed 1>/dev/null 2>&1 & > openssl speed 1>/dev/null 2>&1 & >=20 > to give the rpi2 4 active processes. Various outputs > are from different times without a reboot between. >=20 > top -CaePores shows the likes of: >=20 > PID USERNAME THR PRI NICE SIZE RES SWAP STATE C TIME = CPU COMMAND > 614 root 1 20 0 10452K 10480K 0K select 1 0:00 = 0.03% /usr/sbin/ntpd -g -c /etc/ntp.conf -p /var/run/ntpd.pid -f = /var/db/ntpd.drift > 661 root 1 52 0 9984K 6132K 0K select 1 0:00 = 0.00% /usr/sbin/sshd > 751 root 1 101 0 7256K 4276K 0K RUN 1 0:28 = 99.57% openssl speed > 750 root 1 100 0 7256K 4276K 0K CPU0 0 0:32 = 94.83% openssl speed > 753 root 1 86 0 7256K 4276K 0K RUN 3 0:13 = 52.36% openssl speed > 752 root 1 86 0 7256K 4276K 0K CPU3 3 0:14 = 46.54% openssl speed > 363 root 1 20 0 6428K 3840K 0K select 3 0:00 = 0.00% /sbin/devd > . . . >=20 > and: >=20 > last pid: 754; load averages: 3.70, 2.38, 1.58 = = up 0+00:16:50 01:59:37 > 21 processes: 5 running, 16 sleeping > CPU 0: 94.9% user, 0.0% nice, 0.0% system, 5.1% interrupt, 0.0% = idle > CPU 1: 99.6% user, 0.0% nice, 0.0% system, 0.4% interrupt, 0.0% = idle > CPU 2: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% = idle > CPU 3: 100% user, 0.0% nice, 0.0% system, 0.0% interrupt, 0.0% = idle > Mem: 12M Active, 1136K Inact, 56M Wired, 30M Buf, 722M Free > Swap: 1536M Total, 6M Free >=20 > =46rom problem boot to problem boot, the CPU that stays > idle has varied but usually has been CPU 2. I've never > seen 2 or more stuck in idle. >=20 > show allpcpu shows the likes of: >=20 > db> show allpcpu > Current CPU: 0 >=20 > cpuid =3D 0 > dynamic pcpu =3D 0x3d2540 > curthread =3D 0xd8478ae0: pid 2032 tid 100150 "openssl" > curpcb =3D 0xd852ae98 > fpcurthread =3D 0xd8478ae0: pid 2032 "openssl" > idlethread =3D 0xc376fae0: tid 100002 "idle: cpu0" > curpmap =3D 0xd8e43bf4 > curvnet =3D 0 >=20 > cpuid =3D 1 > dynamic pcpu =3D 0x3998540 > curthread =3D 0xd7e5b3a0: pid 2031 tid 100173 "openssl" > curpcb =3D 0xda7e0e98 > fpcurthread =3D 0xd7e5b3a0: pid 2031 "openssl" > idlethread =3D 0xc376f740: tid 100003 "idle: cpu1" > curpmap =3D 0xd8e43ec4 > curvnet =3D 0 >=20 > cpuid =3D 2 > dynamic pcpu =3D 0x3999540 > curthread =3D 0xc376f3a0: pid 10 tid 100004 "idle: cpu2" > curpcb =3D 0xc378ae98 > fpcurthread =3D none > idlethread =3D 0xc376f3a0: tid 100004 "idle: cpu2" > curpmap =3D 0 > curvnet =3D 0 >=20 > cpuid =3D 3 > dynamic pcpu =3D 0x399a540 > curthread =3D 0xd8477000: pid 2034 tid 100167 "openssl" > curpcb =3D 0xd876de98 > fpcurthread =3D 0xd8477000: pid 2034 "openssl" > idlethread =3D 0xc376f000: tid 100005 "idle: cpu3" > curpmap =3D 0xc377ab04 > curvnet =3D 0 >=20 > In other words: it appears that the cpuN (here cpu2) is > left with idle scheduled all the time for some reason. >=20 > ps from db> shows things like: >=20 >=20 > db> ps > pid ppid pgrp uid state wmesg wchan cmd > 2034 714 2034 0 R+ openssl > 2033 714 2033 0 R+ CPU 3 openssl > 2032 714 2032 0 R+ CPU 0 openssl > 2031 714 2031 0 R+ CPU 1 openssl >=20 > (then later:) >=20 > db> ps > pid ppid pgrp uid state wmesg wchan cmd > 2034 714 2034 0 R+ CPU 3 openssl > 2033 714 2033 0 R+ openssl > 2032 714 2032 0 R+ CPU 0 openssl > 2031 714 2031 0 R+ CPU 1 openssl >=20 > There is also: >=20 > 10 0 0 0 RL (threaded) [idle] > 100002 CanRun [idle: cpu0] > 100003 CanRun [idle: cpu1] > 100004 CanRun [idle: cpu2] > 100005 CanRun [idle: cpu3] >=20 >=20 > These are from: >=20 > # uname -apKU > FreeBSD rpi2 12.0-CURRENT FreeBSD 12.0-CURRENT r327485M arm armv7 = 1200054 1200054 I probably should have reported that as I remember the following sort of thing is true for the problem cpuN (here cpu2): 100074 D - 0xd6f5be80 [softirq_0] 100075 D - 0xd6f5be00 [softirq_1] 100076 RunQ [softirq_2] 100077 D - 0xd6f5bd00 [softirq_3] 100078 D - 0xd6f5bc80 [if_io_tqg_0] 100079 D - 0xd6f5bc00 [if_io_tqg_1] 100080 RunQ [if_io_tqg_2] 100081 D - 0xd6f5bb00 [if_io_tqg_3] =3D=3D=3D Mark Millard markmi at dsl-only.net From owner-freebsd-hackers@freebsd.org Tue Jan 2 20:01:24 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BD409EB6715 for ; Tue, 2 Jan 2018 20:01:24 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from mx1.scaleengine.net (mx1.scaleengine.net [209.51.186.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9EF3B705C9 for ; Tue, 2 Jan 2018 20:01:24 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from [10.1.1.2] (Seawolf.HML3.ScaleEngine.net [209.51.186.28]) (Authenticated sender: allanjude.freebsd@scaleengine.com) by mx1.scaleengine.net (Postfix) with ESMTPSA id 5EEFC13547 for ; Tue, 2 Jan 2018 20:01:21 +0000 (UTC) Subject: Re: rpi2 head -r327485 (e.g.): rpi2 leaves one "CPU n" always idle for some boots To: freebsd-hackers@freebsd.org References: <3258F809-F179-4EB7-857C-74667BECD360@dsl-only.net> From: Allan Jude Message-ID: <9da67685-d1c2-b6e0-cc32-efb49e67f0b1@freebsd.org> Date: Tue, 2 Jan 2018 15:01:20 -0500 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.5.0 MIME-Version: 1.0 In-Reply-To: <3258F809-F179-4EB7-857C-74667BECD360@dsl-only.net> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Jan 2018 20:01:24 -0000 On 2018-01-02 14:48, Mark Millard wrote: > I've seen this over many versions of head for months > but have never managed to find a way to force it to > happen. It just shows up once and a while. > > Thus, I'm just dumping out some top and kernel information > here for reference. I've used: > > openssl speed 1>/dev/null 2>&1 & > openssl speed 1>/dev/null 2>&1 & > openssl speed 1>/dev/null 2>&1 & > openssl speed 1>/dev/null 2>&1 & > > to give the rpi2 4 active processes. Various outputs > are from different times without a reboot between. > > top -CaePores shows the likes of: > > PID USERNAME THR PRI NICE SIZE RES SWAP STATE C TIME CPU COMMAND > 614 root 1 20 0 10452K 10480K 0K select 1 0:00 0.03% /usr/sbin/ntpd -g -c /etc/ntp.conf -p /var/run/ntpd.pid -f /var/db/ntpd.drift > 661 root 1 52 0 9984K 6132K 0K select 1 0:00 0.00% /usr/sbin/sshd > 751 root 1 101 0 7256K 4276K 0K RUN 1 0:28 99.57% openssl speed > 750 root 1 100 0 7256K 4276K 0K CPU0 0 0:32 94.83% openssl speed > 753 root 1 86 0 7256K 4276K 0K RUN 3 0:13 52.36% openssl speed > 752 root 1 86 0 7256K 4276K 0K CPU3 3 0:14 46.54% openssl speed > 363 root 1 20 0 6428K 3840K 0K select 3 0:00 0.00% /sbin/devd > . . . > > and: > > last pid: 754; load averages: 3.70, 2.38, 1.58 up 0+00:16:50 01:59:37 > 21 processes: 5 running, 16 sleeping > CPU 0: 94.9% user, 0.0% nice, 0.0% system, 5.1% interrupt, 0.0% idle > CPU 1: 99.6% user, 0.0% nice, 0.0% system, 0.4% interrupt, 0.0% idle > CPU 2: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle > CPU 3: 100% user, 0.0% nice, 0.0% system, 0.0% interrupt, 0.0% idle > Mem: 12M Active, 1136K Inact, 56M Wired, 30M Buf, 722M Free > Swap: 1536M Total, 6M Free > > From problem boot to problem boot, the CPU that stays > idle has varied but usually has been CPU 2. I've never > seen 2 or more stuck in idle. > > show allpcpu shows the likes of: > > db> show allpcpu > Current CPU: 0 > > cpuid = 0 > dynamic pcpu = 0x3d2540 > curthread = 0xd8478ae0: pid 2032 tid 100150 "openssl" > curpcb = 0xd852ae98 > fpcurthread = 0xd8478ae0: pid 2032 "openssl" > idlethread = 0xc376fae0: tid 100002 "idle: cpu0" > curpmap = 0xd8e43bf4 > curvnet = 0 > > cpuid = 1 > dynamic pcpu = 0x3998540 > curthread = 0xd7e5b3a0: pid 2031 tid 100173 "openssl" > curpcb = 0xda7e0e98 > fpcurthread = 0xd7e5b3a0: pid 2031 "openssl" > idlethread = 0xc376f740: tid 100003 "idle: cpu1" > curpmap = 0xd8e43ec4 > curvnet = 0 > > cpuid = 2 > dynamic pcpu = 0x3999540 > curthread = 0xc376f3a0: pid 10 tid 100004 "idle: cpu2" > curpcb = 0xc378ae98 > fpcurthread = none > idlethread = 0xc376f3a0: tid 100004 "idle: cpu2" > curpmap = 0 > curvnet = 0 > > cpuid = 3 > dynamic pcpu = 0x399a540 > curthread = 0xd8477000: pid 2034 tid 100167 "openssl" > curpcb = 0xd876de98 > fpcurthread = 0xd8477000: pid 2034 "openssl" > idlethread = 0xc376f000: tid 100005 "idle: cpu3" > curpmap = 0xc377ab04 > curvnet = 0 > > In other words: it appears that the cpuN (here cpu2) is > left with idle scheduled all the time for some reason. > > ps from db> shows things like: > > > db> ps > pid ppid pgrp uid state wmesg wchan cmd > 2034 714 2034 0 R+ openssl > 2033 714 2033 0 R+ CPU 3 openssl > 2032 714 2032 0 R+ CPU 0 openssl > 2031 714 2031 0 R+ CPU 1 openssl > > (then later:) > > db> ps > pid ppid pgrp uid state wmesg wchan cmd > 2034 714 2034 0 R+ CPU 3 openssl > 2033 714 2033 0 R+ openssl > 2032 714 2032 0 R+ CPU 0 openssl > 2031 714 2031 0 R+ CPU 1 openssl > > There is also: > > 10 0 0 0 RL (threaded) [idle] > 100002 CanRun [idle: cpu0] > 100003 CanRun [idle: cpu1] > 100004 CanRun [idle: cpu2] > 100005 CanRun [idle: cpu3] > > > These are from: > > # uname -apKU > FreeBSD rpi2 12.0-CURRENT FreeBSD 12.0-CURRENT r327485M arm armv7 1200054 1200054 > > > === > Mark Millard > markmi at dsl-only.net > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > Have you tried using 'cpuset' to force each of the 4 to run on a specific CPU? What happens if you force it to run on cpu #2? -- Allan Jude From owner-freebsd-hackers@freebsd.org Tue Jan 2 20:13:53 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C8D90EB6FD4 for ; Tue, 2 Jan 2018 20:13:53 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: from asp.reflexion.net (outbound-mail-210-137.reflexion.net [208.70.210.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8B70770E2F for ; Tue, 2 Jan 2018 20:13:52 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: (qmail 21489 invoked from network); 2 Jan 2018 20:13:46 -0000 Received: from unknown (HELO rtc-sm-01.app.dca.reflexion.local) (10.81.150.1) by 0 (rfx-qmail) with SMTP; 2 Jan 2018 20:13:46 -0000 Received: by rtc-sm-01.app.dca.reflexion.local (Reflexion email security v8.40.3) with SMTP; Tue, 02 Jan 2018 15:13:46 -0500 (EST) Received: (qmail 27208 invoked from network); 2 Jan 2018 20:13:45 -0000 Received: from unknown (HELO iron2.pdx.net) (69.64.224.71) by 0 (rfx-qmail) with (AES256-SHA encrypted) SMTP; 2 Jan 2018 20:13:45 -0000 Received: from [192.168.1.25] (c-76-115-7-162.hsd1.or.comcast.net [76.115.7.162]) by iron2.pdx.net (Postfix) with ESMTPSA id 2F6EDEC7B1F; Tue, 2 Jan 2018 12:13:45 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: Re: rpi2 head -r327485 (e.g.): rpi2 leaves one "CPU n" always idle for some boots From: Mark Millard In-Reply-To: <9da67685-d1c2-b6e0-cc32-efb49e67f0b1@freebsd.org> Date: Tue, 2 Jan 2018 12:13:44 -0800 Cc: FreeBSD Hackers Content-Transfer-Encoding: quoted-printable Message-Id: <85268695-3440-4B94-BF3D-68A38E204F50@dsl-only.net> References: <3258F809-F179-4EB7-857C-74667BECD360@dsl-only.net> <9da67685-d1c2-b6e0-cc32-efb49e67f0b1@freebsd.org> To: Allan Jude X-Mailer: Apple Mail (2.3273) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Jan 2018 20:13:53 -0000 On 2018-Jan-2, at 12:01 PM, Allan Jude wrote: > On 2018-01-02 14:48, Mark Millard wrote: >> I've seen this over many versions of head for months >> but have never managed to find a way to force it to >> happen. It just shows up once and a while. >>=20 >> Thus, I'm just dumping out some top and kernel information >> here for reference. I've used: >>=20 >> openssl speed 1>/dev/null 2>&1 & >> openssl speed 1>/dev/null 2>&1 & >> openssl speed 1>/dev/null 2>&1 & >> openssl speed 1>/dev/null 2>&1 & >>=20 >> to give the rpi2 4 active processes. Various outputs >> are from different times without a reboot between. >>=20 >> top -CaePores shows the likes of: >>=20 >> PID USERNAME THR PRI NICE SIZE RES SWAP STATE C = TIME CPU COMMAND >> 614 root 1 20 0 10452K 10480K 0K select 1 = 0:00 0.03% /usr/sbin/ntpd -g -c /etc/ntp.conf -p /var/run/ntpd.pid -f = /var/db/ntpd.drift >> 661 root 1 52 0 9984K 6132K 0K select 1 = 0:00 0.00% /usr/sbin/sshd >> 751 root 1 101 0 7256K 4276K 0K RUN 1 = 0:28 99.57% openssl speed >> 750 root 1 100 0 7256K 4276K 0K CPU0 0 = 0:32 94.83% openssl speed >> 753 root 1 86 0 7256K 4276K 0K RUN 3 = 0:13 52.36% openssl speed >> 752 root 1 86 0 7256K 4276K 0K CPU3 3 = 0:14 46.54% openssl speed >> 363 root 1 20 0 6428K 3840K 0K select 3 = 0:00 0.00% /sbin/devd >> . . . >>=20 >> and: >>=20 >> last pid: 754; load averages: 3.70, 2.38, 1.58 = = up 0+00:16:50 01:59:37 >> 21 processes: 5 running, 16 sleeping >> CPU 0: 94.9% user, 0.0% nice, 0.0% system, 5.1% interrupt, 0.0% = idle >> CPU 1: 99.6% user, 0.0% nice, 0.0% system, 0.4% interrupt, 0.0% = idle >> CPU 2: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% = idle >> CPU 3: 100% user, 0.0% nice, 0.0% system, 0.0% interrupt, 0.0% = idle >> Mem: 12M Active, 1136K Inact, 56M Wired, 30M Buf, 722M Free >> Swap: 1536M Total, 6M Free >>=20 >> =46rom problem boot to problem boot, the CPU that stays >> idle has varied but usually has been CPU 2. I've never >> seen 2 or more stuck in idle. >>=20 >> show allpcpu shows the likes of: >>=20 >> db> show allpcpu >> Current CPU: 0 >>=20 >> cpuid =3D 0 >> dynamic pcpu =3D 0x3d2540 >> curthread =3D 0xd8478ae0: pid 2032 tid 100150 "openssl" >> curpcb =3D 0xd852ae98 >> fpcurthread =3D 0xd8478ae0: pid 2032 "openssl" >> idlethread =3D 0xc376fae0: tid 100002 "idle: cpu0" >> curpmap =3D 0xd8e43bf4 >> curvnet =3D 0 >>=20 >> cpuid =3D 1 >> dynamic pcpu =3D 0x3998540 >> curthread =3D 0xd7e5b3a0: pid 2031 tid 100173 "openssl" >> curpcb =3D 0xda7e0e98 >> fpcurthread =3D 0xd7e5b3a0: pid 2031 "openssl" >> idlethread =3D 0xc376f740: tid 100003 "idle: cpu1" >> curpmap =3D 0xd8e43ec4 >> curvnet =3D 0 >>=20 >> cpuid =3D 2 >> dynamic pcpu =3D 0x3999540 >> curthread =3D 0xc376f3a0: pid 10 tid 100004 "idle: cpu2" >> curpcb =3D 0xc378ae98 >> fpcurthread =3D none >> idlethread =3D 0xc376f3a0: tid 100004 "idle: cpu2" >> curpmap =3D 0 >> curvnet =3D 0 >>=20 >> cpuid =3D 3 >> dynamic pcpu =3D 0x399a540 >> curthread =3D 0xd8477000: pid 2034 tid 100167 "openssl" >> curpcb =3D 0xd876de98 >> fpcurthread =3D 0xd8477000: pid 2034 "openssl" >> idlethread =3D 0xc376f000: tid 100005 "idle: cpu3" >> curpmap =3D 0xc377ab04 >> curvnet =3D 0 >>=20 >> In other words: it appears that the cpuN (here cpu2) is >> left with idle scheduled all the time for some reason. >>=20 >> ps from db> shows things like: >>=20 >>=20 >> db> ps >> pid ppid pgrp uid state wmesg wchan cmd >> 2034 714 2034 0 R+ openssl >> 2033 714 2033 0 R+ CPU 3 openssl >> 2032 714 2032 0 R+ CPU 0 openssl >> 2031 714 2031 0 R+ CPU 1 openssl >>=20 >> (then later:) >>=20 >> db> ps >> pid ppid pgrp uid state wmesg wchan cmd >> 2034 714 2034 0 R+ CPU 3 openssl >> 2033 714 2033 0 R+ openssl >> 2032 714 2032 0 R+ CPU 0 openssl >> 2031 714 2031 0 R+ CPU 1 openssl >>=20 >> There is also: >>=20 >> 10 0 0 0 RL (threaded) [idle] >> 100002 CanRun [idle: cpu0] >> 100003 CanRun [idle: cpu1] >> 100004 CanRun [idle: cpu2] >> 100005 CanRun [idle: cpu3] >>=20 >>=20 >> These are from: >>=20 >> # uname -apKU >> FreeBSD rpi2 12.0-CURRENT FreeBSD 12.0-CURRENT r327485M arm armv7 = 1200054 1200054 >>=20 >>=20 >> =3D=3D=3D >> Mark Millard >> markmi at dsl-only.net >>=20 >> _______________________________________________ >> freebsd-hackers@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> To unsubscribe, send any mail to = "freebsd-hackers-unsubscribe@freebsd.org" >>=20 >=20 > Have you tried using 'cpuset' to force each of the 4 to run on a > specific CPU? What happens if you force it to run on cpu #2? # cpuset -c -l 2 openssl speed 1>/dev/null 2>&1 & results in: last pid: 2120; load averages: 2.95, 2.44, 2.25 = = up 0+10:26:07 12:08:54 18 processes: 2 running, 16 sleeping CPU 0: 0.0% user, 0.0% nice, 0.0% system, 4.7% interrupt, 95.3% idle CPU 1: 0.4% user, 0.0% nice, 0.0% system, 0.0% interrupt, 99.6% idle CPU 2: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle CPU 3: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle Mem: 1852K Active, 587M Inact, 5368K Laundry, 145M Wired, 86M Buf, 53M = Free Swap: 1536M Total, 1536M Free PID USERNAME THR PRI NICE SIZE RES SWAP STATE C TIME = CPU COMMAND 2120 root 1 20 0 6432K 2872K 0K CPU1 1 0:00 = 0.22% top -CawPopid 2118 root 1 29 0 7256K 4432K 0K RUN 1 0:00 = 0.00% openssl speed . . . With the other 3 also assigned: last pid: 2138; load averages: 4.45, 3.04, 2.51 = = up 0+10:28:48 12:11:35 21 processes: 5 running, 16 sleeping CPU 0: 93.7% user, 0.0% nice, 1.6% system, 4.7% interrupt, 0.0% idle CPU 1: 100% user, 0.0% nice, 0.0% system, 0.0% interrupt, 0.0% idle CPU 2: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle CPU 3: 100% user, 0.0% nice, 0.0% system, 0.0% interrupt, 0.0% idle Mem: 3632K Active, 587M Inact, 5368K Laundry, 145M Wired, 86M Buf, 51M = Free Swap: 1536M Total, 1536M Free PID USERNAME THR PRI NICE SIZE RES SWAP STATE C TIME = CPU COMMAND 2138 root 1 20 0 6472K 2888K 0K CPU1 1 0:00 = 0.24% top -CawPopid 2137 root 1 100 0 7256K 4436K 0K CPU3 3 0:25 = 96.79% openssl speed 2124 root 1 102 0 7256K 4436K 0K RUN 1 0:37 = 101.66% openssl speed 2123 root 1 101 0 7256K 4436K 0K CPU0 0 0:44 = 95.07% openssl speed 2118 root 1 29 0 7256K 4432K 0K RUN 1 0:00 = 0.00% openssl speed . . . Thus the difference is that the one instance of openssl speed does not run at all. =3D=3D=3D Mark Millard markmi at dsl-only.net From owner-freebsd-hackers@freebsd.org Tue Jan 2 20:27:55 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AF2A5EB7BD4 for ; Tue, 2 Jan 2018 20:27:55 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: from asp.reflexion.net (outbound-mail-210-104.reflexion.net [208.70.210.104]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6D64271A0E for ; Tue, 2 Jan 2018 20:27:54 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: (qmail 19500 invoked from network); 2 Jan 2018 20:27:48 -0000 Received: from unknown (HELO rtc-sm-01.app.dca.reflexion.local) (10.81.150.1) by 0 (rfx-qmail) with SMTP; 2 Jan 2018 20:27:48 -0000 Received: by rtc-sm-01.app.dca.reflexion.local (Reflexion email security v8.40.3) with SMTP; Tue, 02 Jan 2018 15:27:48 -0500 (EST) Received: (qmail 19525 invoked from network); 2 Jan 2018 20:27:48 -0000 Received: from unknown (HELO iron2.pdx.net) (69.64.224.71) by 0 (rfx-qmail) with (AES256-SHA encrypted) SMTP; 2 Jan 2018 20:27:48 -0000 Received: from [192.168.1.25] (c-76-115-7-162.hsd1.or.comcast.net [76.115.7.162]) by iron2.pdx.net (Postfix) with ESMTPSA id B8351EC88A2; Tue, 2 Jan 2018 12:27:47 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: Re: rpi2 head -r327485 (e.g.): rpi2 leaves one "CPU n" always idle for some boots From: Mark Millard In-Reply-To: <85268695-3440-4B94-BF3D-68A38E204F50@dsl-only.net> Date: Tue, 2 Jan 2018 12:27:47 -0800 Cc: FreeBSD Hackers Content-Transfer-Encoding: quoted-printable Message-Id: References: <3258F809-F179-4EB7-857C-74667BECD360@dsl-only.net> <9da67685-d1c2-b6e0-cc32-efb49e67f0b1@freebsd.org> <85268695-3440-4B94-BF3D-68A38E204F50@dsl-only.net> To: Allan Jude X-Mailer: Apple Mail (2.3273) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Jan 2018 20:27:55 -0000 [It looks like the opensll speed on problem cpuN (2) can not get ^C and the like. On 2018-Jan-2, at 12:13 PM, Mark Millard wrote: > On 2018-Jan-2, at 12:01 PM, Allan Jude = wrote: >=20 >> On 2018-01-02 14:48, Mark Millard wrote: >>> I've seen this over many versions of head for months >>> but have never managed to find a way to force it to >>> happen. It just shows up once and a while. >>>=20 >>> Thus, I'm just dumping out some top and kernel information >>> here for reference. I've used: >>>=20 >>> openssl speed 1>/dev/null 2>&1 & >>> openssl speed 1>/dev/null 2>&1 & >>> openssl speed 1>/dev/null 2>&1 & >>> openssl speed 1>/dev/null 2>&1 & >>>=20 >>> to give the rpi2 4 active processes. Various outputs >>> are from different times without a reboot between. >>>=20 >>> top -CaePores shows the likes of: >>>=20 >>> PID USERNAME THR PRI NICE SIZE RES SWAP STATE C = TIME CPU COMMAND >>> 614 root 1 20 0 10452K 10480K 0K select 1 = 0:00 0.03% /usr/sbin/ntpd -g -c /etc/ntp.conf -p /var/run/ntpd.pid -f = /var/db/ntpd.drift >>> 661 root 1 52 0 9984K 6132K 0K select 1 = 0:00 0.00% /usr/sbin/sshd >>> 751 root 1 101 0 7256K 4276K 0K RUN 1 = 0:28 99.57% openssl speed >>> 750 root 1 100 0 7256K 4276K 0K CPU0 0 = 0:32 94.83% openssl speed >>> 753 root 1 86 0 7256K 4276K 0K RUN 3 = 0:13 52.36% openssl speed >>> 752 root 1 86 0 7256K 4276K 0K CPU3 3 = 0:14 46.54% openssl speed >>> 363 root 1 20 0 6428K 3840K 0K select 3 = 0:00 0.00% /sbin/devd >>> . . . >>>=20 >>> and: >>>=20 >>> last pid: 754; load averages: 3.70, 2.38, 1.58 = = up 0+00:16:50 01:59:37 >>> 21 processes: 5 running, 16 sleeping >>> CPU 0: 94.9% user, 0.0% nice, 0.0% system, 5.1% interrupt, 0.0% = idle >>> CPU 1: 99.6% user, 0.0% nice, 0.0% system, 0.4% interrupt, 0.0% = idle >>> CPU 2: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% = idle >>> CPU 3: 100% user, 0.0% nice, 0.0% system, 0.0% interrupt, 0.0% = idle >>> Mem: 12M Active, 1136K Inact, 56M Wired, 30M Buf, 722M Free >>> Swap: 1536M Total, 6M Free >>>=20 >>> =46rom problem boot to problem boot, the CPU that stays >>> idle has varied but usually has been CPU 2. I've never >>> seen 2 or more stuck in idle. >>>=20 >>> show allpcpu shows the likes of: >>>=20 >>> db> show allpcpu >>> Current CPU: 0 >>>=20 >>> cpuid =3D 0 >>> dynamic pcpu =3D 0x3d2540 >>> curthread =3D 0xd8478ae0: pid 2032 tid 100150 "openssl" >>> curpcb =3D 0xd852ae98 >>> fpcurthread =3D 0xd8478ae0: pid 2032 "openssl" >>> idlethread =3D 0xc376fae0: tid 100002 "idle: cpu0" >>> curpmap =3D 0xd8e43bf4 >>> curvnet =3D 0 >>>=20 >>> cpuid =3D 1 >>> dynamic pcpu =3D 0x3998540 >>> curthread =3D 0xd7e5b3a0: pid 2031 tid 100173 "openssl" >>> curpcb =3D 0xda7e0e98 >>> fpcurthread =3D 0xd7e5b3a0: pid 2031 "openssl" >>> idlethread =3D 0xc376f740: tid 100003 "idle: cpu1" >>> curpmap =3D 0xd8e43ec4 >>> curvnet =3D 0 >>>=20 >>> cpuid =3D 2 >>> dynamic pcpu =3D 0x3999540 >>> curthread =3D 0xc376f3a0: pid 10 tid 100004 "idle: cpu2" >>> curpcb =3D 0xc378ae98 >>> fpcurthread =3D none >>> idlethread =3D 0xc376f3a0: tid 100004 "idle: cpu2" >>> curpmap =3D 0 >>> curvnet =3D 0 >>>=20 >>> cpuid =3D 3 >>> dynamic pcpu =3D 0x399a540 >>> curthread =3D 0xd8477000: pid 2034 tid 100167 "openssl" >>> curpcb =3D 0xd876de98 >>> fpcurthread =3D 0xd8477000: pid 2034 "openssl" >>> idlethread =3D 0xc376f000: tid 100005 "idle: cpu3" >>> curpmap =3D 0xc377ab04 >>> curvnet =3D 0 >>>=20 >>> In other words: it appears that the cpuN (here cpu2) is >>> left with idle scheduled all the time for some reason. >>>=20 >>> ps from db> shows things like: >>>=20 >>>=20 >>> db> ps >>> pid ppid pgrp uid state wmesg wchan cmd >>> 2034 714 2034 0 R+ openssl >>> 2033 714 2033 0 R+ CPU 3 openssl >>> 2032 714 2032 0 R+ CPU 0 openssl >>> 2031 714 2031 0 R+ CPU 1 openssl >>>=20 >>> (then later:) >>>=20 >>> db> ps >>> pid ppid pgrp uid state wmesg wchan cmd >>> 2034 714 2034 0 R+ CPU 3 openssl >>> 2033 714 2033 0 R+ openssl >>> 2032 714 2032 0 R+ CPU 0 openssl >>> 2031 714 2031 0 R+ CPU 1 openssl >>>=20 >>> There is also: >>>=20 >>> 10 0 0 0 RL (threaded) [idle] >>> 100002 CanRun [idle: cpu0] >>> 100003 CanRun [idle: cpu1] >>> 100004 CanRun [idle: cpu2] >>> 100005 CanRun [idle: cpu3] >>>=20 >>>=20 >>> These are from: >>>=20 >>> # uname -apKU >>> FreeBSD rpi2 12.0-CURRENT FreeBSD 12.0-CURRENT r327485M arm armv7 = 1200054 1200054 >>>=20 >>>=20 >>> =3D=3D=3D >>> Mark Millard >>> markmi at dsl-only.net >>>=20 >>> _______________________________________________ >>> freebsd-hackers@freebsd.org mailing list >>> https://lists.freebsd.org/mailman/listinfo/freebsd-hackers >>> To unsubscribe, send any mail to = "freebsd-hackers-unsubscribe@freebsd.org" >>>=20 >>=20 >> Have you tried using 'cpuset' to force each of the 4 to run on a >> specific CPU? What happens if you force it to run on cpu #2? >=20 > # cpuset -c -l 2 openssl speed 1>/dev/null 2>&1 & >=20 > results in: >=20 > last pid: 2120; load averages: 2.95, 2.44, 2.25 = = up 0+10:26:07 12:08:54 > 18 processes: 2 running, 16 sleeping > CPU 0: 0.0% user, 0.0% nice, 0.0% system, 4.7% interrupt, 95.3% = idle > CPU 1: 0.4% user, 0.0% nice, 0.0% system, 0.0% interrupt, 99.6% = idle > CPU 2: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% = idle > CPU 3: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% = idle > Mem: 1852K Active, 587M Inact, 5368K Laundry, 145M Wired, 86M Buf, 53M = Free > Swap: 1536M Total, 1536M Free >=20 > PID USERNAME THR PRI NICE SIZE RES SWAP STATE C TIME = CPU COMMAND > 2120 root 1 20 0 6432K 2872K 0K CPU1 1 0:00 = 0.22% top -CawPopid > 2118 root 1 29 0 7256K 4432K 0K RUN 1 0:00 = 0.00% openssl speed > . . . >=20 >=20 > With the other 3 also assigned: >=20 > last pid: 2138; load averages: 4.45, 3.04, 2.51 = = up 0+10:28:48 12:11:35 > 21 processes: 5 running, 16 sleeping > CPU 0: 93.7% user, 0.0% nice, 1.6% system, 4.7% interrupt, 0.0% = idle > CPU 1: 100% user, 0.0% nice, 0.0% system, 0.0% interrupt, 0.0% = idle > CPU 2: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% = idle > CPU 3: 100% user, 0.0% nice, 0.0% system, 0.0% interrupt, 0.0% = idle > Mem: 3632K Active, 587M Inact, 5368K Laundry, 145M Wired, 86M Buf, 51M = Free > Swap: 1536M Total, 1536M Free >=20 > PID USERNAME THR PRI NICE SIZE RES SWAP STATE C TIME = CPU COMMAND > 2138 root 1 20 0 6472K 2888K 0K CPU1 1 0:00 = 0.24% top -CawPopid > 2137 root 1 100 0 7256K 4436K 0K CPU3 3 0:25 = 96.79% openssl speed > 2124 root 1 102 0 7256K 4436K 0K RUN 1 0:37 = 101.66% openssl speed > 2123 root 1 101 0 7256K 4436K 0K CPU0 0 0:44 = 95.07% openssl speed > 2118 root 1 29 0 7256K 4432K 0K RUN 1 0:00 = 0.00% openssl speed > . . . >=20 > Thus the difference is that the one instance > of openssl speed does not run at all. I tried to fg and ^C each of the 4 openssl's, with cpu2's instance being last. All worked but for cpu2's. ^C and ^Z do not work after the fg. But ^T reports: # fg cpuset -c -l 3 openssl speed >/dev/null 2>&1 ^C # fg cpuset -c -l 1 openssl speed >/dev/null 2>&1 ^C # fg cpuset -c -l 0 openssl speed >/dev/null 2>&1 ^C# fg cpuset -c -l 2 openssl speed >/dev/null 2>&1 ^C ^Z load: 3.03 cmd: openssl 2118 [runnable] 768.91r 0.06u 0.02s 0% 4424k load: 3.07 cmd: openssl 2118 [runnable] 825.03r 0.06u 0.02s 0% 4424k load: 3.06 cmd: openssl 2118 [runnable] 1177.78r 0.06u 0.02s 0% 4424k Using an ssh sessions as the console is stuck in the above state, it appears that the load average is now stuck at around 3 despite the basically idle state: last pid: 2172; load averages: 3.03, 3.29, 3.08 = = up 0+10:39:36 12:22:23 23 processes: 2 running, 21 sleeping CPU 0: 0.0% user, 0.0% nice, 0.0% system, 3.1% interrupt, 96.9% idle CPU 1: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle CPU 2: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle CPU 3: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle Mem: 4480K Active, 590M Inact, 5364K Laundry, 145M Wired, 86M Buf, 46M = Free Swap: 1536M Total, 1536M Free PID USERNAME THR PRI NICE SIZE RES SWAP STATE C TIME = CPU COMMAND 2172 root 1 20 0 6344K 2888K 0K CPU3 3 0:00 = 0.24% top -CawPopid 2155 root 1 22 0 5484K 2660K 0K wait 3 0:00 = 0.00% su (sh) 2154 markmi 1 23 0 5260K 2648K 0K wait 0 0:00 = 0.00% su 2148 markmi 1 28 0 5524K 2724K 0K wait 3 0:00 = 0.00% -sh (sh) 2147 markmi 1 20 0 11024K 6208K 0K select 0 0:00 = 0.03% sshd: markmi@pts/0 (sshd) 2143 root 1 26 0 10644K 6128K 0K select 3 0:00 = 0.00% sshd: markmi [priv] (sshd) 2118 root 1 20 0 7256K 4432K 0K RUN 1 0:00 = 0.00% openssl speed 714 root 1 20 0 5568K 2828K 0K wait 3 0:00 = 0.00% -sh (sh) 713 root 1 22 0 5300K 2600K 0K wait 1 0:00 = 0.00% login [pam] (login) 712 root 1 52 0 4356K 2004K 0K ttyin 0 0:00 = 0.00% /usr/libexec/getty Pc ttyv0 665 root 1 52 0 4816K 2332K 0K nanslp 3 0:01 = 0.00% /usr/sbin/cron -s 661 root 1 20 0 9984K 4832K 0K select 0 0:00 = 0.00% /usr/sbin/sshd 617 root 1 20 0 4308K 1860K 0K select 0 0:06 = 0.01% /usr/sbin/powerd 614 root 1 20 0 10452K 10480K 0K select 3 0:04 = 0.01% /usr/sbin/ntpd -g -c /etc/ntp.conf -p /var/run/ntpd.pid -f = /var/db/ntpd.drift 576 root 32 52 0 4388K 2160K 0K rpcsvc 0 0:00 = 0.00% nfsd: server (nfsd) 575 root 1 52 0 4720K 2044K 0K select 3 0:00 = 0.00% nfsd: master (nfsd) 573 root 1 52 0 5052K 2216K 0K select 3 0:00 = 0.00% /usr/sbin/mountd -r 443 root 1 20 0 4724K 2088K 0K select 0 0:00 = 0.00% /usr/sbin/rpcbind 434 root 1 20 0 4772K 2280K 0K select 1 0:01 = 0.00% /usr/sbin/syslogd -s 363 root 1 20 0 6428K 3576K 0K select 0 0:00 = 0.00% /sbin/devd 362 _dhcp 1 42 0 5128K 2388K 0K select 3 0:00 = 0.00% dhclient: ue0 (dhclient) 316 root 1 52 0 5032K 2292K 0K select 0 0:00 = 0.00% dhclient: ue0 [priv] (dhclient) 313 root 1 52 0 4800K 2148K 0K select 3 0:00 = 0.00% dhclient: system.syslog (dhclient) Any more experiments before a reboot (and possibly not seeing the problem at all)? =3D=3D=3D Mark Millard markmi at dsl-only.net From owner-freebsd-hackers@freebsd.org Tue Jan 2 23:06:00 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AB03DEBE0FC for ; Tue, 2 Jan 2018 23:06:00 +0000 (UTC) (envelope-from zbeeble@gmail.com) Received: from mail-wm0-x236.google.com (mail-wm0-x236.google.com [IPv6:2a00:1450:400c:c09::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4219C779AE for ; Tue, 2 Jan 2018 23:06:00 +0000 (UTC) (envelope-from zbeeble@gmail.com) Received: by mail-wm0-x236.google.com with SMTP id g75so496636wme.0 for ; Tue, 02 Jan 2018 15:06:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=xThPWWuiTEAgYFejqMLen/eOBaYx3FaK+7VwIcl35Gs=; b=XKmWC0RUlV0cTFD0dIz16YsC/fhaRKeGaq0OwiiBYjQ6uc9vMGDa2kCuZQBCOjOzMm lBueMUtjnaZY8M0wd2L8OKf+y7PYIJ2N2sAf5itZpfifYRFhjyH3aC9ECvuxoY5vnQx4 DRQHgyeTLjG0mosmZjzr6HoT3y3g5ESnbzedCIqkheVORrKgbe7myKwaQXFP66f2sF/D agQRh8DJWjr18cHEUIfBAuaD0rxp0gaGbXTOZwZ9AvLkTatpmXwCWLk0bgF72UEIaY8h ht3+td01b5OVtretdMF6LAe/DcfZt2RV0LRv0D2XQMBaWqTRdkpe9QJYpKWDaXdtvM3I Fdyw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=xThPWWuiTEAgYFejqMLen/eOBaYx3FaK+7VwIcl35Gs=; b=GG4tpcgq5qiIsAf7VCE0BYQlDjnISLOWTdxAsgcVOKPVVmpo7PJWgGtvUnpfXbvcNG 74e5dF2ChP95SZMo4sdWf63KdWDkw6M3N9r9IlYZSDX0IgyJbDGVsYRtNpe5DWqYfZJY d+w6UUEW9Eg+7EHYR8I0HQWD1/hRptK+OpGmEwrkpPTN+ws8LHAcErcMGtvWDvCwsj/K 1D1ma/q0yk1jetL511Oat/SNRC8bVZ19Cw8xzPLuVdtYD6YJcjx7oHHt/bUykwzy5/8n ha05doZc65rF4lySpbiyvfOqOJtQupK1VEddarH/AamnvNLArj/p4Gv1GEhlke/Ymg5b l+Ug== X-Gm-Message-State: AKGB3mLjN4+5OjDZsOq43WCO8TYOFRUbDQeHuzWBZUNEkCTDetr5FB27 09ElSbnGyLafiDCb+i7AS1vn/Y07WfjS0coIxA== X-Google-Smtp-Source: ACJfBouSrjPcvHZDoe9FSnTJz+oUFEnOciyzRAqPGAVdT6kbLWxmfbcVbO7uILLhg8GqpeCCfiUJ+Wvj9N4x2YMVxGU= X-Received: by 10.80.176.70 with SMTP id i64mr64646886edd.187.1514934357629; Tue, 02 Jan 2018 15:05:57 -0800 (PST) MIME-Version: 1.0 Received: by 10.80.181.59 with HTTP; Tue, 2 Jan 2018 15:05:57 -0800 (PST) In-Reply-To: References: From: Zaphod Beeblebrox Date: Tue, 2 Jan 2018 18:05:57 -0500 Message-ID: Subject: Re: We do serial differently. To: Kyle Evans Cc: FreeBSD Hackers Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Jan 2018 23:06:00 -0000 To summarize, the app "arduino" that controls arduinos works, but the app "pronterface" that controls an arduino with a specific application loaded, do not. Further this seems to be something to do with the handling of the DTR and/or the actual handling of the DTR signal... and what this all means to the arduino in question. To confound things, this all works (of course) on linux. If this were a purely work thing, I'd eat my bile and just run linux. Even now, I have a R-pi attached to it running linux to "get things done" ... but I want FreeBSD to work. So... if I'm charged to fix this, what are my tools? Is ktrace sufficient to indicate all the calls to the serial? How do I interpret it so that I can make sense of this. Do I potentially need DTrace or is that overkill? Some help and pointers would be appreciated. On Sat, Oct 28, 2017 at 6:01 PM, Zaphod Beeblebrox wrote: > I've tried hacking at directly. I can't send it a file (as in that > script) because the printer isn't setup yet. I'm still working on it. > > ... I have poked on the .send() method (with M105) ... and that doesn't > seem to have effect. Also ... if I ctrl-C the python process, it doesn't > exit. I'm not patient... but so far I've only been able to exit with > CTRL-\ ... (core dump). > > On 27/10/2017 07:59, Kyle Evans wrote: > > What happens if you just invoke printcore directly, like the example > script in the printrun readme? > > Pronterface has got to be doing something not-entirely-sane here. The > reset should always coincide with the port being open()d, and it should > behave like the Arduino IDE and cu(1) do. The mega doesn't really have any > crazy requirements just to function properly. > > > On Oct 26, 2017 11:46 PM, "Zaphod Beeblebrox" wrote: > > OK. I played with this all again ... a bit. > > Firstly, when I run "arduino" (the IDE) ... the arduino board resets > immediately (I can tell this because it has an LCD screen attached). > > But when I run pronterface, it doesn't reset until 5 seconds (roughly) > after pronterface exits. > > I tried adding a hardcoded setDTR(0) or setDTR(1) near this code ... but > it doesn't seem to make any palpable difference. > > On Wed, Oct 25, 2017 at 1:11 PM, Kyle Evans wrote: > >> On Wed, Oct 25, 2017 at 11:43 AM, Kyle Evans wrote: >> >>> On Wed, Oct 25, 2017 at 11:34 AM, Zaphod Beeblebrox >>> wrote: >>> >>>> On Mon, Oct 23, 2017 at 9:45 AM, Kyle Evans wrote: >>>> >>>>> Hi, >>>>> >>>>> Are you able to connect to it otherwise (w/ cu or friends) and issue, >>>>> say, an M105 manually? >>>>> >>>> >>>> yes. With CU I can connect, it resets, then I can issue an "M105" >>>> and it parrots back some status. >>>> >>> >>> Ok, cool, that's expected and sounds like Pronterface is doing something >>> it shouldn't be. >>> >>> I'll poke at it a little bit more- last I checked, it didn't look like >>> it was doing anything too crazy with pyserial and I've got a working >>> OctoPrint (w/ pyserial) setup, so I know that works to some extent. >>> >>> >> For the sake of argument, can you try applying the following patch [1] to >> printrun? I don't see a need to be toggling DTR here, and that might narrow >> things down a little bit. >> >> [1] >> diff --git a/printrun/printcore.py b/printrun/printcore.py >> index b54e750..fd531c3 100644 >> --- a/printrun/printcore.py >> +++ b/printrun/printcore.py >> @@ -218,11 +218,6 @@ class printcore(): >> parity = PARITY_ODD) >> self.printer.close() >> self.printer.parity = PARITY_NONE >> - try: #this appears not to work on many platforms, >> so we're going to call it but not care if it fails >> - self.printer.setDTR(dtr); >> - except: >> - #self.logError(_("Could not set DTR on this >> platform")) #not sure whether to output an error message >> - pass >> self.printer.open() >> except SerialException as e: >> self.logError(_("Could not connect to %s at baudrate >> %s:") % (self.port, self.baud) + >> >> >> > > > From owner-freebsd-hackers@freebsd.org Thu Jan 4 01:31:31 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 38568EBB033 for ; Thu, 4 Jan 2018 01:31:31 +0000 (UTC) (envelope-from lists@eitanadler.com) Received: from mail-yw0-x22c.google.com (mail-yw0-x22c.google.com [IPv6:2607:f8b0:4002:c05::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EE5BC70A5B for ; Thu, 4 Jan 2018 01:31:30 +0000 (UTC) (envelope-from lists@eitanadler.com) Received: by mail-yw0-x22c.google.com with SMTP id g191so78627ywe.7 for ; Wed, 03 Jan 2018 17:31:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=eitanadler.com; s=0xdeadbeef; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to; bh=Yf1CuOAMy1s8JfAcXYQIqe1nfYSzrTJWSKJjkBa5ZG4=; b=sZwjYFkwQzrVn0p66RGsNZoqzztDnpIrcpWD1M41MO6wd57TnKkhWZ7LWorJr5DH8+ YJQuFTqea17Xhy4r51bYDXBSuvmcIJN4+F/Lr11gXzFWuYFVzjinET5BqHqW1m668f4/ BlMdemQ7HiqUvMeJFEmLFW/AkdkLPnrtJkqc0= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd-org.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to; bh=Yf1CuOAMy1s8JfAcXYQIqe1nfYSzrTJWSKJjkBa5ZG4=; b=SNK7CGVQ+u2Xc7csfB+5tR6wIHitZAK5hCbMkIKpOzB+SPFymYyBh0aGf8kKo9DUOe TT1PAQDEVXJcPYd7qQl5l86qOqGR7Nva/uQxN3dIqRunbS0p/p6EzCUaRImj6Bmg47l3 bP5cc8453mmHFd6CJq8C3sYMw5q+7x8XKYtF2FLBkmEuDELb1OtNYfF9Ive/Rr6OGLE0 g077DwkRhNZvm0rvR+WtHiE1Pyqjhwp8gZCm+5f0PSg1VDrmI75kI9Ut1XIJt5dR9KyN qeN12bLWmhxTvhpGT3DYc6VLEmLk7woMlrV7WX772Ox/Gj9tX1nRPQaZn+x66kMTeNJe QWzA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to; bh=Yf1CuOAMy1s8JfAcXYQIqe1nfYSzrTJWSKJjkBa5ZG4=; b=W3UNGWjIlrmJDfbmj7c5ZC4Hv88gm2IjTUKFAqInQ1GvfDmFccKGIdqzu8faBc4EnQ E5BcwjquTVvpXmz+HoaXfKc9FTQ/NpRofkliGOItqlJr9W9G0UONCO2hpi10mhZSwodm DZm8i0nPMt0YnjyyfivR5L95RfsbSQl7c4TzHCYlvCWDrqmyAsjws9WMreg1dunkGsVL BtNmLwobJFGUGr/vOVUuI4D/vrG3wnGhXy4kndLQ4LMz3j0HMhnmeFSKpgumC0+vJ31L j399+GnZw/gOIdaXUQ5pvmYUQW5pVLll296Ai2+2H8yIVMvbnEpqWT9f8G4UPe7Zob8Z zj7g== X-Gm-Message-State: AKGB3mJI9L6EHyX2SqFEIMi3ndkKgZtUjIOHm8rbchHVIoafUgbFP71f Q5nU4bfXr8wI8rgrt57HVDCd1jsjZtBTWVG0uCoi4A== X-Google-Smtp-Source: ACJfBovqQ7e9U01mR8ZtO/fDE2pBq5e8Js9Ba3diWfkoOXzfiUdKBEI49MEPIVIWxjTTFmuQmj9cWalymWr4QUUQhJw= X-Received: by 10.129.212.65 with SMTP id g1mr3053831ywl.425.1515029489551; Wed, 03 Jan 2018 17:31:29 -0800 (PST) MIME-Version: 1.0 Sender: lists@eitanadler.com Received: by 10.37.217.21 with HTTP; Wed, 3 Jan 2018 17:30:58 -0800 (PST) In-Reply-To: References: From: Eitan Adler Date: Wed, 3 Jan 2018 17:30:58 -0800 X-Google-Sender-Auth: rB7e7IRn5TE5U1eOlkLnYtyiVok Message-ID: Subject: Fwd: [Differential] [Request, 113 lines] D13757: morse: implement support for decoding morse code To: FreeBSD Hackers Content-Type: multipart/mixed; boundary="089e082242dc8f5cea0561e94ada" X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Jan 2018 01:31:31 -0000 --089e082242dc8f5cea0561e94ada Content-Type: text/plain; charset="UTF-8" Review of this is appreciated. Mostly stolen from a bug report, but modified a bit. ---------- Forwarded message ---------- From: eadler (Eitan Adler) Date: 3 January 2018 at 17:30 Subject: [Differential] [Request, 113 lines] D13757: morse: implement support for decoding morse code To: eadler@freebsd.org eadler created this revision. Herald added a subscriber: imp. Herald added a reviewer: manpages. View Revision *REPOSITORY* rS FreeBSD src repository *REVISION DETAIL* https://reviews.freebsd.org/D13757 *AFFECTED FILES* usr.bin/morse/morse.6 usr.bin/morse/morse.c *EMAIL PREFERENCES* https://reviews.freebsd.org/settings/panel/emailpreferences/ *To: *eadler, manpages *Cc: *imp -- Eitan Adler Source, Ports, Doc committer Bugmeister, Ports Security teams --089e082242dc8f5cea0561e94ada Content-Type: text/x-patch; charset="utf-8"; name="D13757.37489.patch" Content-Disposition: attachment; filename="D13757.37489.patch" Content-Transfer-Encoding: base64 X-Attachment-Id: 58e200bd961652ec_0.1 ZGlmZiAtLWdpdCBhL3Vzci5iaW4vbW9yc2UvbW9yc2UuYyBiL3Vzci5iaW4vbW9yc2UvbW9yc2Uu YwotLS0gYS91c3IuYmluL21vcnNlL21vcnNlLmMKKysrIGIvdXNyLmJpbi9tb3JzZS9tb3JzZS5j CkBAIC01OSw2ICs1OSw4IEBACiAvKiBBbHdheXMgdXNlIHRoZSBzcGVha2VyLCBsZXQgdGhlIG9w ZW4gZmFpbCBpZiAtcCBpcyBzZWxlY3RlZCAqLwogI2RlZmluZSBTUEVBS0VSICIvZGV2L3NwZWFr ZXIiCiAKKyNkZWZpbmUgV0hJVEVTUEFDRSAiIFx0XG4iCisKICNpZmRlZiBTUEVBS0VSCiAjaW5j bHVkZSA8ZGV2L3NwZWFrZXIvc3BlYWtlci5oPgogI2VuZGlmCkBAIC0yNjcsMTQgKzI2OSwxMSBA QAogfTsKIAogc3RhdGljIHZvaWQJc2hvdyhjb25zdCBjaGFyICopLCBwbGF5KGNvbnN0IGNoYXIg KiksIG1vcnNlKGNoYXIpOworc3RhdGljIHZvaWQJZGVjb2RlIChjaGFyICopLCBmZGVjb2RlKEZJ TEUgKik7CiBzdGF0aWMgdm9pZAl0dHlvdXQoY29uc3QgY2hhciAqKTsKIHN0YXRpYyB2b2lkCXNp Z2hhbmRsZXIoaW50KTsKIAotI2RlZmluZSBHRVRPUFRPUFRTICJjOmQ6ZWY6bHN3OiIKLSNkZWZp bmUgVVNBR0UgXAotInVzYWdlOiBtb3JzZSBbLWVsc10gWy1kIGRldmljZV0gWy13IHNwZWVkXSBb LWMgc3BlZWRdIFstZiBmcmVxdWVuY3ldIFtzdHJpbmcgLi4uXVxuIgotCi1zdGF0aWMgaW50CXBm bGFnLCBsZmxhZywgc2ZsYWcsIGVmbGFnOworc3RhdGljIGludAlwZmxhZywgbGZsYWcsIHNmbGFn LCBkZmxhZywgZWZsYWc7CiBzdGF0aWMgaW50CXdwbSA9IDIwOwkvKiBlZmZlY3RpdmUgd29yZHMg cGVyIG1pbnV0ZSAqLwogc3RhdGljIGludAljcG07CQkvKiBlZmZlY3RpdmUgd29yZHMgcGVyIG1p bnV0ZSBiZXR3ZWVuCiAJCQkJICogY2hhcmFjdGVycyAqLwpAQCAtMjkzLDExICsyOTIsMTQgQEAK IAogI2lmZGVmIFNQRUFLRVIKIHN0YXRpYyB0b25lX3QJc291bmQ7Ci0jdW5kZWYgR0VUT1BUT1BU UwotI2RlZmluZSBHRVRPUFRPUFRTICJjOmQ6ZWY6bHBzdzoiCi0jdW5kZWYgVVNBR0UKKyNkZWZp bmUgR0VUT1BUT1BUUyAiYzpkOmVmOmxwc3c6RCIKKyNkZWZpbmUgVVNBR0UgXAorInVzYWdlOiBt b3JzZSBbLWVscHNEXSBbLWQgZGV2aWNlXSBbLXcgc3BlZWRdIFstYyBzcGVlZF0gWy1mIGZyZXF1 ZW5jeV0gW3N0cmluZyAuLi5dXG4iCisjZWxzZQorI2RlZmluZSBHRVRPUFRPUFRTICJjOmQ6ZWY6 bHN3OkQiCiAjZGVmaW5lIFVTQUdFIFwKLSJ1c2FnZTogbW9yc2UgWy1lbHBzXSBbLWQgZGV2aWNl XSBbLXcgc3BlZWRdIFstYyBzcGVlZF0gWy1mIGZyZXF1ZW5jeV0gW3N0cmluZyAuLi5dXG4iCisi dXNhZ2U6IG1vcnNlIFstZWxzRF0gWy1kIGRldmljZV0gWy13IHNwZWVkXSBbLWMgc3BlZWRdIFst ZiBmcmVxdWVuY3ldIFtzdHJpbmcgLi4uXVxuIgorCiAjZW5kaWYKIAogc3RhdGljIGNvbnN0IHN0 cnVjdCBtb3JzZXRhYiAqaGlnaHRhYjsKQEAgLTMzNywxMyArMzM5LDE2IEBACiAJCWNhc2UgJ3cn OgogCQkJd3BtID0gYXRvaShvcHRhcmcpOwogCQkJYnJlYWs7CisJCWNhc2UgJ0QnOgorCQkJZGZs YWcgPSAxOworCQkJYnJlYWs7CiAJCWNhc2UgJz8nOgogCQlkZWZhdWx0OgogCQkJZnB1dHMoVVNB R0UsIHN0ZGVycik7CiAJCQlleGl0KDEpOwogCQl9Ci0JaWYgKHNmbGFnICYmIGxmbGFnKSB7Ci0J CWZwdXRzKCJtb3JzZTogb25seSBvbmUgb2YgLWwgYW5kIC1zIGFsbG93ZWRcbiIsIHN0ZGVycik7 CisJaWYgKChzZmxhZyAmJiBsZmxhZykgfHwgKHNmbGFnICYmIGRmbGFnKSB8fCAobGZsYWcgJiYg ZGZsYWcpKSB7CisJCWZwdXRzKCJtb3JzZTogb25seSBvbmUgb2YgLWwgYW5kIC1zIGFuZCAtRCBh bGxvd2VkXG4iLCBzdGRlcnIpOwogCQlleGl0KDEpOwogCX0KIAlpZiAoKHBmbGFnIHx8IGRldmlj ZSkgJiYgKHNmbGFnIHx8IGxmbGFnKSkgewpAQCAtNDIxLDcgKzQyNiwyNSBAQAogCiAJaWYgKGxm bGFnKQogCQlwcmludGYoIm0iKTsKLQlpZiAoKmFyZ3YpIHsKKwlpZiAoZGZsYWcpIHsKKwkJaWYg KCphcmd2KSB7CisJCQlkbyB7CisJCQkJcD1zdHJ0b2soKmFyZ3YsICIgXHQiKTsKKwkJCQlpZiAo cCA9PSBOVUxMKQorCQkJCQlkZWNvZGUoKmFyZ3YpOworCQkJCWVsc2UgeworCQkJCQl3aGlsZSAo cCkgeworCQkJCQkJZGVjb2RlKHApOworCQkJCQkJcD1zdHJ0b2soTlVMTCwgIiBcdCIpOworCQkJ CQl9CisJCQkJfQorCQkJfSB3aGlsZSAoKisrYXJndik7CisJCQlwdXRjaGFyKCdcbicpOworCQl9 IGVsc2UgeworCQkJZmRlY29kZShzdGRpbik7CisJCX0KKwl9CisJZWxzZSBpZiAoKmFyZ3YpIHsK IAkJZG8gewogCQkJZm9yIChwID0gKmFyZ3Y7ICpwOyArK3ApIHsKIAkJCQlpZiAoZWZsYWcpCkBA IC01NzcsNiArNjAwLDY3IEBACiAJdXNsZWVwKGR1cmF0aW9uKTsKIH0KIAordm9pZAorZmRlY29k ZShGSUxFICpzdHJlYW0pCit7CisJY2hhciAqbiwgKnAsICpzOworCWNoYXIgYnVmW0JVRlNJWl07 CisKKwlzPWJ1ZjsKKwl3aGlsZSAoZmdldHMocywgQlVGU0laIC0gKHMgLSBidWYpLCBzdGRpbikp IHsKKwkJcD1idWY7CisKKwkJd2hpbGUgKCpwICYmIGlzYmxhbmsoKnApKQorCQkJcCsrOworCQl3 aGlsZSAoKnAgJiYgaXNzcGFjZSgqcCkpIHsKKwkJCXArKzsKKwkJCXB1dGNoYXIgKCcgJyk7CisJ CX0KKwkJd2hpbGUgKCpwKSB7CisJCQluPXN0cnBicmsocCwgV0hJVEVTUEFDRSk7CisJCQlpZiAo biA9PSBOVUxMKSB7CisJCQkJLyogVGhlIHRva2VuIHdhcyBpbnRlcnJ1cHRlZCBhdCB0aGUgZW5k CisJCQkJICogb2YgdGhlIGJ1ZmZlci4gU2hpZnQgaXQgdG8gdGhlIGJlZ2luCisJCQkJICogb2Yg dGhlIGJ1ZmZlci4KKwkJCQkgKi8KKwkJCQlmb3IgKHM9YnVmOyAqcDsgKnMrKyA9ICpwKyspCisJ CQkJCTsKKwkJCX0gZWxzZSB7CisJCQkJKm49J1wwJzsKKwkJCQluKys7CisJCQkJZGVjb2RlKHAp OworCQkJCXA9bjsKKwkJCX0KKwkJfQorCX0KKwlwdXRjaGFyKCdcbicpOworfQorCit2b2lkCitk ZWNvZGUoY2hhciAqcCkKK3sKKwljaGFyIGM7CisJY29uc3Qgc3RydWN0IG1vcnNldGFiICptOwor CisJYyA9ICcgJzsKKwlmb3IgKG0gPSBtdGFiOyBtICE9IE5VTEwgJiYgbS0+aW5jaGFyICE9ICdc MCc7IG0rKykgeworCQlpZiAoc3RyY21wKG0tPm1vcnNlLCBwKSA9PSAwKSB7CisJCQljID0gbS0+ aW5jaGFyOworCQkJYnJlYWs7CisJCX0KKwl9CisKKwlpZiAoYyA9PSAnICcpCisJCWZvciAobSA9 IGhpZ2h0YWI7IG0gIT0gTlVMTCAmJiBtLT5pbmNoYXIgIT0gJ1wwJzsgbSsrKSB7CisJCQlpZiAo c3RyY21wKG0tPm1vcnNlLCBwKSA9PSAwKSB7CisJCQkJYyA9IG0tPmluY2hhcjsKKwkJCQlicmVh azsKKwkJCX0KKwkJfQorCisJcHV0Y2hhcihjKTsKK30KKwogc3RhdGljIHZvaWQKIHNpZ2hhbmRs ZXIoaW50IHNpZ25vKQogewpkaWZmIC0tZ2l0IGEvdXNyLmJpbi9tb3JzZS9tb3JzZS42IGIvdXNy LmJpbi9tb3JzZS9tb3JzZS42Ci0tLSBhL3Vzci5iaW4vbW9yc2UvbW9yc2UuNgorKysgYi91c3Iu YmluL21vcnNlL21vcnNlLjYKQEAgLTkyLDYgKzkyLDExIEBACiAuRmwgcAogb3IKIC5GbCBkIC4K Ky5JdCBGbCBECitEZWNvZGUgbW9yc2Ugb3V0cHV0IGNvbnNpc3Rpbmcgb2YgZG90cyBhbmQgZGFz aGVzIChhcyBnZW5lcmF0ZWQgYnkgdXNpbmcKK3RoZQorLkZsIHMKK29wdGlvbikuCiAuRWwKIC5Q cAogVGhlCgo= --089e082242dc8f5cea0561e94ada-- From owner-freebsd-hackers@freebsd.org Thu Jan 4 05:09:31 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 46CE9E8AA3F; Thu, 4 Jan 2018 05:09:31 +0000 (UTC) (envelope-from kaduk@mit.edu) Received: from dmz-mailsec-scanner-7.mit.edu (dmz-mailsec-scanner-7.mit.edu [18.7.68.36]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id ADA7E79E56; Thu, 4 Jan 2018 05:09:30 +0000 (UTC) (envelope-from kaduk@mit.edu) X-AuditID: 12074424-339ff700000004d1-c5-5a4db590d190 Received: from mailhub-auth-4.mit.edu ( [18.7.62.39]) (using TLS with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by dmz-mailsec-scanner-7.mit.edu (Symantec Messaging Gateway) with SMTP id AE.FC.01233.195BD4A5; Thu, 4 Jan 2018 00:03:14 -0500 (EST) Received: from outgoing.mit.edu (OUTGOING-AUTH-1.MIT.EDU [18.9.28.11]) by mailhub-auth-4.mit.edu (8.13.8/8.9.2) with ESMTP id w04539wO021612; Thu, 4 Jan 2018 00:03:10 -0500 Received: from kduck.kaduk.org (24-107-191-124.dhcp.stls.mo.charter.com [24.107.191.124]) (authenticated bits=56) (User authenticated as kaduk@ATHENA.MIT.EDU) by outgoing.mit.edu (8.13.8/8.12.4) with ESMTP id w04535SA013744 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Thu, 4 Jan 2018 00:03:08 -0500 Date: Wed, 3 Jan 2018 23:03:05 -0600 From: Benjamin Kaduk To: freebsd-hackers@FreeBSD.org Cc: freebsd-current@FreeBSD.org Subject: Second Call for 2017Q4 quarterly status reports Message-ID: <20180104050305.GL50827@kduck.kaduk.org> References: <20171226002716.GG59505@kduck.kaduk.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="82I3+IH0IqGh5yIs" Content-Disposition: inline In-Reply-To: <20171226002716.GG59505@kduck.kaduk.org> User-Agent: Mutt/1.9.1 (2017-09-22) X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFtrKKsWRmVeSWpSXmKPExsUixG6nrjtpq2+UQcNFFYs5bz4wWWzf/I/R gcljxqf5LAGMUVw2Kak5mWWpRfp2CVwZv/+uYinYK1ZxddtllgbGacJdjBwcEgImEu97KroY uTiEBBYzSezo2c7excgJ5GxglJh6IhoicYVJYteUTawgCRYBFYmebd/YQGw2ATWJ9SuuMYPY IgLyEvua3oM1MwPZv7Y2gdnCAhYS1x88YASxeYGWbW/4xAyxwERiyak1UHFBiZMzn7CAHMQs UCYx55UDhCktsfwfB0gFp4CpxKONe8G2igooS+ztO8Q+gVFgFpLmWQjNsxCaZ4GdoyVx499L JgxhbYllC18zQ9i2EuvWvWdZwMi+ilE2JbdKNzcxM6c4NVm3ODkxLy+1SNdcLzezRC81pXQT IzjkXVR2MHb3eB9iFOBgVOLhbbjhEyXEmlhWXJl7iFGSg0lJlDcn1TtKiC8pP6UyI7E4I76o NCe1+BCjCtCuRxtWX2CUYsnLz0tVEuF1ywJq5U1JrKxKLcqHKZPmYFES5/Uw0Y4SEkhPLEnN Tk0tSC2CyWpwcAi0XDx5kAlqiARvxhbfKCHBotT01Iq0zJwShFImDs5DjBIcPECL7EFqeIsL EnOLM9Mh8qcYdTmezXzdwCwENkhKnNcWpEgApCijNA9uDii1SWTvr3nFKA70rjCvM0gVDzAt wk16BbSECWjJn/OeIEtKEhFSUg2MYqvmrZv6L/2SXp3NtPP9MQu9d3bP5XThSm3d/nORpNbW 5ZkzvE0Nrn8yYd9udEdMNXTZxwnzbZ9lvNixvz/ZoNnjd7XZqpDQeF2LBHZ/CaGtH3IXBqso n8r3eL57EbPhAfHDLsrLTYU6rjPkZBkeXWtcsev0+c/ffz4XMX3h5hh//X/gsScrlFiKMxIN tZiLihMBhpYdc0gDAAA= X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Jan 2018 05:09:31 -0000 --82I3+IH0IqGh5yIs Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Happy new year, and I hope that meltdown and spectre are not taking too much of everyone's time. That said, the submission deadline is still January 14th, so please do send in your status report entries to us! Thanks, Ben On Mon, Dec 25, 2017 at 06:27:16PM -0600, Benjamin Kaduk wrote: > Dear FreeBSD Community, >=20 > The deadline for the next FreeBSD Quarterly Status update is January 14, > 2018, for work done in October through December. >=20 > Status report submissions do not need to be very long. They may be > about anything happening in the FreeBSD project and community, and > provide a great way to inform FreeBSD users and developers about > work that is underway and completed. Submission of reports is not > restricted to committers; anyone doing anything interesting and > FreeBSD related can -- and should -- write one! >=20 > The preferred and easiest submission method is to use the XML > generator [1] with the results emailed to the status report team at > monthly@FreeBSD.org . (Do be sure, though, to save the form output > and not the form itself! In particular, the Google Chrome "save as" > function does not save the generated output for some reason.) There > is also an XML template [2] that can be filled out manually and > attached if preferred. For the expected content and style, please > study our guidelines on how to write a good status report [3]. You > can also review previous issues [4][5] for ideas on the style and > format. >=20 > We look forward to seeing your 2017Q4 reports! >=20 > Thanks, >=20 > Ben (on behalf of monthly@) >=20 > [1] https://www.FreeBSD.org/cgi/monthly.cgi > [2] https://www.FreeBSD.org/news/status/report-sample.xml > [3] https://www.FreeBSD.org/news/status/howto.html > [4] https://www.FreeBSD.org/news/status/report-2017-07-2017-09.html > [5] https://www.FreeBSD.org/news/status/report-2017-04-2017-06.html >=20 >=20 --82I3+IH0IqGh5yIs Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQG3BAABCgAdFiEE2WGV4E2ARf9BYP0XKNmm82TrdRIFAlpNtYMACgkQKNmm82Tr dRIlmgweLJpJVe7eOt7L3M7R0C50IcufXsJy3NvVxymHExOgeh6Izxn6HWmaL2+H ByrJLZkp8k2jQpvSICyakoIA/PPlo6troZUBAzNJqDeN+VEH35TB8b64SdJFBRtr /rQ5c9pJNDL6/KfPWajPIYc80lPUyhIYZgQKuc7UkW4d/He0JBgAuOTOQP6drxsw 71P0qpnm8vzN8AH6jQWhwDT9yINPn2s0JSprVJTW7w5don7EtXuLLhtnZk/T2pWd 0o5pot0KqyeqU/5DFvciZHEvcK1M/BXSVvN4wfu+Hopibc6quHEwosTW45qA19WL fRGUbApiTlT8AxnuB0Z197Z4QVE9CBaMdj22jWLViez93snBvnEoHAAyioScHmyQ W4/c6bxzku2o3Ql9szOyH/rVlYjnBJUQGsbBQUyMzGSw+nejdwo6WQ7nXtza4BIl 8iYnszR02ThEwRylYLpc3fnN0S69gqGuhu1mIhE6Svlf4sPW1nY8yJABf97nQ5I/ YMcMmXiJ3X0gew== =flZ4 -----END PGP SIGNATURE----- --82I3+IH0IqGh5yIs-- From owner-freebsd-hackers@freebsd.org Thu Jan 4 14:50:53 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1F2B7EB9AFC; Thu, 4 Jan 2018 14:50:53 +0000 (UTC) (envelope-from srs0=fypj=d7=freebsd.org=kp@codepro.be) Received: from venus.codepro.be (venus.codepro.be [IPv6:2a01:4f8:162:1127::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.codepro.be", Issuer "Gandi Standard SSL CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E07526C91F; Thu, 4 Jan 2018 14:50:52 +0000 (UTC) (envelope-from srs0=fypj=d7=freebsd.org=kp@codepro.be) Received: from [172.16.2.10] (vega.codepro.be [IPv6:2a01:4f8:162:1127::3]) (Authenticated sender: kp) by venus.codepro.be (Postfix) with ESMTPSA id D24884506C; Thu, 4 Jan 2018 15:50:49 +0100 (CET) From: "Kristof Provost" To: freebsd-arch@freebsd.org Cc: freebsd-hackers , "FreeBSD Current" Subject: RFC: mallocarray() Date: Thu, 04 Jan 2018 15:51:09 +0100 X-Mailer: MailMate (2.0BETAr6102) Message-ID: <070C0DEB-E70C-42D9-B734-E0735A6C9B8E@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed; markup=markdown Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Jan 2018 14:50:53 -0000 Hi, I’d like to make it easier to avoid integer overflow issues in the kernel. It’d be a lot nicer to have a malloc function figure this out for us, so I’d like mallocarray(). https://reviews.freebsd.org/D13766 Are there any objections to this? Regards, Kristof From owner-freebsd-hackers@freebsd.org Thu Jan 4 19:04:54 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2C722EAEDD0 for ; Thu, 4 Jan 2018 19:04:54 +0000 (UTC) (envelope-from kevans91@ksu.edu) Received: from NAM03-BY2-obe.outbound.protection.outlook.com (mail-by2nam03on0053.outbound.protection.outlook.com [104.47.42.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT SSL SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A4BA579FC0 for ; Thu, 4 Jan 2018 19:04:52 +0000 (UTC) (envelope-from kevans91@ksu.edu) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ksu.edu; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=xUgaJTiJR/uboG5/lb0B1/K63W9m28iV27F6tp2tzho=; b=Ls+YI4IqUFl4DoCr4lroTFw+v24qOjauSfZij+C5c98CQDaAyfExshurYw1ICkEzrYdInWqr6kNnvB+5YBbMvUpQd8KGm7wcM18CXWSgUClctttvQ+RI2tJ4b7XjnzI70wDIeGYp8KE2ncMeedobTFefByFFZ33bClp1K+5o138= Received: from SN4PR0501CA0039.namprd05.prod.outlook.com (10.167.112.144) by BLUPR05MB1955.namprd05.prod.outlook.com (10.162.224.21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.407.1; Thu, 4 Jan 2018 19:04:51 +0000 Received: from SN1NAM02FT009.eop-nam02.prod.protection.outlook.com (2a01:111:f400:7e44::203) by SN4PR0501CA0039.outlook.office365.com (2603:10b6:803:41::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.407.1 via Frontend Transport; Thu, 4 Jan 2018 19:04:50 +0000 Authentication-Results: spf=pass (sender IP is 129.130.18.151) smtp.mailfrom=ksu.edu; freebsd.org; dkim=none (message not signed) header.d=none;freebsd.org; dmarc=bestguesspass action=none header.from=ksu.edu; Received-SPF: Pass (protection.outlook.com: domain of ksu.edu designates 129.130.18.151 as permitted sender) receiver=protection.outlook.com; client-ip=129.130.18.151; helo=ome-vm-smtp1.campus.ksu.edu; Received: from ome-vm-smtp1.campus.ksu.edu (129.130.18.151) by SN1NAM02FT009.mail.protection.outlook.com (10.152.73.32) with Microsoft SMTP Server id 15.20.345.12 via Frontend Transport; Thu, 4 Jan 2018 19:04:50 +0000 Received: from calypso.engg.ksu.edu (calypso.engg.ksu.edu [129.130.43.181]) by ome-vm-smtp1.campus.ksu.edu (8.14.4/8.14.4/Debian-2ubuntu2.1) with ESMTP id w04J4oxH003210 for ; Thu, 4 Jan 2018 13:04:50 -0600 Received: by calypso.engg.ksu.edu (Postfix, from userid 110) id 0A236248305; Thu, 4 Jan 2018 13:04:50 -0600 (CST) Received: from mail-it0-f46.google.com (mail-it0-f46.google.com [209.85.214.46]) by calypso.engg.ksu.edu (Postfix) with ESMTPA id 9D61C24800B for ; Thu, 4 Jan 2018 13:04:47 -0600 (CST) Received: by mail-it0-f46.google.com with SMTP id f190so3480257ita.5 for ; Thu, 04 Jan 2018 11:04:47 -0800 (PST) X-Gm-Message-State: AKGB3mIvEmnlVZLkX6G9+CdA7ogXC7z+jJKVIqPmRj45nSSdboTBArFH ksoj4cyHuNLHQcGGvyzHwVrQnU4gU7kTlSOv4BI= X-Google-Smtp-Source: ACJfBouUsoy6S7UK2Wo78KBmT/PdsEnzwkr70ARjoE+8RLXKEB+B52yEiSFGArNXU/ieuZt90/dIOatll+YXEJSVDwo= X-Received: by 10.36.51.202 with SMTP id k193mr520895itk.130.1515092686819; Thu, 04 Jan 2018 11:04:46 -0800 (PST) MIME-Version: 1.0 Received: by 10.107.149.147 with HTTP; Thu, 4 Jan 2018 11:04:26 -0800 (PST) In-Reply-To: References: From: Kyle Evans Date: Thu, 4 Jan 2018 13:04:26 -0600 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: We do serial differently. To: Zaphod Beeblebrox CC: FreeBSD Hackers Content-Type: text/plain; charset="UTF-8" X-EOPAttributedMessage: 0 X-Forefront-Antispam-Report: CIP:129.130.18.151; IPV:NLI; CTRY:US; EFV:NLI; SFV:NSPM; SFS:(10009020)(376002)(39860400002)(39380400002)(346002)(396003)(2980300002)(438002)(24454002)(199004)(189003)(305945005)(33896004)(16586007)(42186006)(786003)(316002)(61726006)(5890100001)(59536001)(55446002)(3480700004)(75432002)(498394004)(86362001)(9686003)(95326003)(53546011)(5660300001)(93886005)(8936002)(5820100001)(4326008)(106002)(478600001)(23676004)(76176011)(6862004)(93516011)(6246003)(98316002)(2950100002)(1411001)(90966002)(106466001)(2906002)(8676002)(246002)(229853002)(9896002)(61266001)(47776003)(50466002)(88552002)(356003)(55456009); DIR:OUT; SFP:1101; SCL:1; SRVR:BLUPR05MB1955; H:ome-vm-smtp1.campus.ksu.edu; FPR:; SPF:Pass; PTR:ip-18-151.net.ksu.edu; A:1; MX:1; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; SN1NAM02FT009; 1:7VkOxpV+0oDdLDUp54seUaLo/vEtDXvn22v5RQaJTpJEfp5sKhM7UCqV2tr6+6g6PPLzvGiuAo2lxxfPF2boxfIC5on0AwDAxDMUmpP1G/xrmvBPXdKIb/8+VPJjbFxE X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: d89f081a-9123-4c01-2e17-08d553a60813 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(4534020)(4602075)(4627115)(8989060)(201703031133081)(201702281549075)(8990040)(5600026)(4604075)(4608076)(2017052603307)(7153060); SRVR:BLUPR05MB1955; X-Microsoft-Exchange-Diagnostics: 1; BLUPR05MB1955; 3:JQnOyU+MSp9l1L0WSpqRDsOnF2pHzErLuKpcx90RX2VhUuPQRSg7+QwrgEAwnNFJwpyvam0m5xDEOdqJ06NvbQ9qkw6UK9OANz97nAhLcjQzGHz4gj9Hbfa7jqE03zb+uVrpJ/U8ym/KIrZjl4XWZyBhO5jlAo2p6Cmf4eghilnug+Ox7lyn/EyezvI3APwF2UCvcw81KrOHg79/C/fzLiSmCbeNTjMrBjrhY3NNVKlJz5JHYlibYlm3Rn9LJ/Am5dELuEMilMWITFKWGYqukruJMLOi9H2W589Y7qpjQtgfiZIioHbHWafQ03ijyehqjOyOkRO5gAXCxDe8IVeG3SRpgjZ3PUnUj4ZzVGtHKfU=; 25:urGQGUZ12MBd5Slpq2UBONDa8eylnTGwrC+z26DQa2cW0bPm2L2RRHVCxxPQMphH6jlhEx1P6hyxFo/Fecw1gFi9XW8KdFwkNXq46qHEfhhWN70W3XmgYvhdKcu8zrFhsyRzkiY9nx2Y5JRLtuoNH1HVvj3C7C5vzOZYXzwaBqunrI9VFaLUzDW+aziZUzOvKMOyleeO/I+tSt+1ZIeM/NgYJBH39YtuiPd+aczwntD9ulMKutwNdyzEloEQIOlzl7H2VYSVzEXeU6qND89x2qwkyCUyNH85k612W1rEHYhDjL7wdRRDZ13ylPj1ctJ+TYNiVryfEIhKXvFbRZ1SUg== X-MS-TrafficTypeDiagnostic: BLUPR05MB1955: X-Microsoft-Exchange-Diagnostics: 1; BLUPR05MB1955; 31:2qQEkmyjqvLmv926TC5vuS1QernbZoSTvLjoGDwMJ44kiAyFB8fHnbL6cPb5M06EM3KgQKwy+KObaxWFQcJ7g7jUDodBmTIrXiZvFlCVtuyPt+ongVlphevwB6ykBUMzMikWIWqDRUrXlXnOvrrnGJvBN5BWze9+ZYU5f+6tFbAsa7psclmzUUXBdoHGx1P2Xd6a2Gepy+0tDquHLnFxi3Qt8ydp1Q5bIiGI8wtVBV8=; 20:NpO780FflldT5FDNRMabnpKNztIctulrK13WYIJBbceamqn9qXg0hGBFuiwbzJJWfuexduoRr7xSlTw3CHqU59r99hSt5VjzRGe+J7QdiAjZESYMoZosrR4s6tyz3OITWieMBNpH43K9aCnibl/V0pfVMdwjMDN/NDbUfaAdPlGI76X+XSKsdadzJYKthJYDIw9Obo9jMM87qZonMCnNWGzhNOjMjzwUwyNulfO3dGesNa/tM+BrCs/pAG2LwioVWT6C2qFqIItY4OKnVwhiyJrrV2Xt2mpTxvrenqByThYq/DF8i7dWLq3pwwV7TtjGxtPXS1jt4Hzc6XjrBxuPGenj8IHdzHxhBScHTmRvNRHHkeqcgy6lK7U6fbnfq4mCCScf7kYM/76WSfBQJF9CF64k7rH4lQ1cDL5mASmJTdKJbjefvgL9vD+tMz18XESbeMlYCYIsrdyAq2R/mHpBxx2xmM1PVnIBNzljln0496n1KStfo+iRLDhW2fDdSt8y X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040470)(2401047)(5005006)(8121501046)(10201501046)(3002001)(3231023)(944501075)(93006095)(93004095)(6041268)(20161123560045)(201703131423095)(201702281529075)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123562045)(20161123564045)(20161123558120)(6072148)(201708071742011); SRVR:BLUPR05MB1955; BCL:0; PCL:0; RULEID:(100000803101)(100110400095); SRVR:BLUPR05MB1955; X-Microsoft-Exchange-Diagnostics: 1; BLUPR05MB1955; 4:4PLoXjANkTZ6JuqK5Pn/+aVkvm9bAxRiUJqvy/a8m9+We660+q+6YKQlP5v09J5PqrYWEZdmxyo4uVtX1gWTB7jts7U/Wkitmw+3iFvxUFGNRT3zzu0yHp4+3HWyMwzeBEigWvRdpBcL0EQuIoammK8P2x3Vbyv8FPxRS4Mgvo2XP/of0JeC6qfGwOMVtBAy/cbf69UCn64KWAJV1RH5kQAsaGFeEKcWI6z0GBPUzMYXEeu3EU/1dycMsnrUVaJKh5QLlj3j823EIuzTttYs/Q== X-Forefront-PRVS: 054231DC40 X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtCTFVQUjA1TUIxOTU1OzIzOm40ZEJZcDRzbjBKbk03MEFIYXBIcU1LVWZl?= =?utf-8?B?bGFHaE51YnJVaWpJWjBQMVJHVVN6VDlBemJvOTFqNzM3aU9ZUmRab0s0Ky9Q?= =?utf-8?B?WjBqMW9WU0RsSk52UkVlWFg1bHg0akN0RlMzM0xKdDliSkdvTUxuZzd1dENF?= =?utf-8?B?dGFWVXE3QmpHTDloakJJTWthbTA2anFHZ0xOeWxzcWVIZ2VJSUpQOHBIL0hv?= =?utf-8?B?MWgyb0oycStmd0NoNTFob3krcGNQVFVpTFZ5K1pGb3AxaE4waFVJVkpEZnIr?= =?utf-8?B?STJpL0tZNlZjWUVkS0drT3J6b2d4ZVJhOGFRZHpXd3JzOUl5ektUenR2TjRB?= =?utf-8?B?Q3prQ0NBd29yS2txMjFGRFFhcTQ0TE16ay9YNmVscUllT29veFNiOFQ4SUhu?= =?utf-8?B?aVF1SS93NnByVitKZ0ZuaXhqQ2NSZUlNQi9zY1hhNUF6bXM2a1ZWaW5odjgy?= =?utf-8?B?SWdIelpwUnIvRXBhbWJod3FVM0pqdExURE4zMW9UUWRrbVBCQVlUc1ZySS9W?= =?utf-8?B?TS9TUEU1bU5BVFFVTGlOVlc0UWtXeVh5Y241b1pnczRXUGptY0kwSkNxeGlq?= =?utf-8?B?YTZtVDIyMmdGVlBHRUo1VU9sZ3diRGhraWdTVC9vdUVGVTdpd0VkR0FxZ3dw?= =?utf-8?B?Z2YzaGVJVE1FeS9aVGdtZyswU3FpTUZSaXdOc09zbWpYaVJhSlNPdTl2Y0oz?= =?utf-8?B?Q3NrR3o4dHNKVFRpWEN0L21oSzVLbUxEeUhyay9JaHN3ekc4OHVubU5OSnQr?= =?utf-8?B?YThRcjRUVXJib1dXdzJ3eElvY0haSjRPZDNWQldrTkF4SDAxRHg1ZVRDamkr?= =?utf-8?B?aHh4UXA3MlFIV3Nub0hqS3lwQzVaUHI5Nk9zdnJYbm94UktBS1dBYmZPb1Vm?= =?utf-8?B?OTZRdHNqN0U0ZCtQajdQVjZ5V3NBczhNYnVTTnMwSUwyN2dZTEZWdlNIUTBy?= =?utf-8?B?c243NlJqblVoUWhiNm9vSWJjekd4WkNUZ0tXWmY1ZXFnUWt5aVljcm9EOUNJ?= =?utf-8?B?MEFVTkRWeGlyTERKZnRHT1RRanpteDU2azA1RzB6UWZYVmhTa3BJWGU0QTJt?= =?utf-8?B?dmJBdndxcTl0d0lMRUQ3MGhIdDRVRk5GOFlEcHFqNFpOK1lYditham4yeWFE?= =?utf-8?B?L0g3WnZwTWtGOFpNNWV2WjJ3a2xHUjFEendYL2NMVHlWWER5NU1MRU1rTTBT?= =?utf-8?B?WWVBTk5PWUErWkJhZ0hJbjJFSHBpVzFqQTVsbDVFcCt6WWdYeWtmczAwR3lo?= =?utf-8?B?M29xdVVsV3pZQ21qK3FWNXNSTUlWQ1V3bGNwK0YrMUpndDdJYzd4MUU3YzNY?= =?utf-8?B?OWtCbDR2eFBIRS9SUmRPcHRGWEJyZ3VZY1hLS2E1NFRHd0wvK1ZXT0ZxeXFq?= =?utf-8?B?UnVIM2szRkJadEpoT0xNSnROSlgyRVZSMDVXM3RpTnB6QldkYU1ObDVpeFkx?= =?utf-8?B?dGxYeHdZK2VoaTNPUU9CMk9ZYmdncUtRVTNkbVRka0FXSGRwVGozakVHcHlC?= =?utf-8?B?WTVUdURWRVJyNnJ6UHFONHNQMjM0UEkrOUJmL2NVZUh2dkx2d3pPcXpmek9E?= =?utf-8?B?ckFQdlBCQ2NMZzZtMVc5UG5WT3N4MklQdGU4OWRYRU5laXZvQ0dDYVBRMzNN?= =?utf-8?B?NUpORHMwSVBWM0VHdUg0a2dGWnNxVENJZ3A1b0tzNCsySytWeHZpR2VGYWU2?= =?utf-8?Q?Mn2RAOGlzowr76hKjhR+8sC7PZVz6s3id8tP57s?= X-Microsoft-Exchange-Diagnostics: 1; BLUPR05MB1955; 6:ZnbkB9PEV3z7U513+7O3A7rjmBkh6CVMnRCgUxnqHGGXENINEC61BnLxDHSmyW04cAc9JqYWRcxZ8h338+XSvizqEAHSzljyYF/bhxXWneAicgXRzYk1aHubUr3dg4+J4FyXXi2PesG4e0tEiYIK/P15yBQ625I417fqo0bGGxG7p8YPvY81dMaE2F52jr8RQnFo1MU0jWbcG/hI8HOr25PaBASp6Da6oSn34tBf8hE4YosD7xlxD/RPU1tGF+oWPk4ezo3i7SThasl/LrhyKz5p4qIk+Eqal3CCzmoOTRF3qCpd58kY/M8DBBC8E2afNG+X5raI9+eZ37WnNEwqytt3EAN/LNkDPQPd3UgQS30=; 5:qy7L8mOMiLGR19UQo023hckUeOMwku/BJCEMzhq2wwow7GnK1htz9yp0zxX6u6VyknnO0e5nLvEB1wVyvv5GAhy1IR/WDNt3JdvHOJTzI+Pf/9f/r4QI/cZtffw+E/R9gAthNOWE759BjY97EiinTd27fhYCwUBh1zRxT8xacOc=; 24:mwLzFFLwAhooc+QK8F27sxCIIiHBDqdfrO+1ExMIS/joNx0aYrmWOFc28HrwqAseEil9SueqYeGn6V7RSGbhC+OWo68J6vugN/jQd9AuIYI=; 7:N55bMd5EzQwlrxmIvUnS+5m0Hs6L2m8zRHcGgWud3/aWh9U2g+2Hd8OINP3hKs+wv2J/O2jaUVbfDNFjRbQFRnmdXH2X7QGZQvK6fIy+zXqmo23hmCFWPB2QUDTXHAy0sHxrSwsAp4aMIoYvg0yK1hYJlJcaXnfnjV6DdZna6j1WRflaTGK2PPYTWrp6L79yA9Kk+G3mPd8lPplJzKRDN5AFamXlxAcYAAuDaqsENeHGegESFychRrHpUrsYjpY2 SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; BLUPR05MB1955; 20:d/PfZRkpgVHhA9MnvDafCPlr26f1CmnqG4GNuFb9Yyd/5SpHy8Rz5nNJNNcupbO/7XKIfmKjpCC6H9IjnUtTvmZ/oNTt5oUPMTZjxu2sv0x5EEtUNMop7Z+YcG1xHrUIlLVtpvSgH7bs/yT5u40aKV7DU8dbB0Bwf/ojV/NgDGI= X-OriginatorOrg: ksu.edu X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Jan 2018 19:04:50.6069 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: d89f081a-9123-4c01-2e17-08d553a60813 X-MS-Exchange-CrossTenant-Id: d9a2fa71-d67d-4cb6-b541-06ccaa8013fb X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=d9a2fa71-d67d-4cb6-b541-06ccaa8013fb; Ip=[129.130.18.151]; Helo=[ome-vm-smtp1.campus.ksu.edu] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BLUPR05MB1955 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Jan 2018 19:04:54 -0000 On Tue, Jan 2, 2018 at 5:05 PM, Zaphod Beeblebrox wrote: > To summarize, the app "arduino" that controls arduinos works, but the app > "pronterface" that controls an arduino with a specific application loaded, > do not. Further this seems to be something to do with the handling of the > DTR and/or the actual handling of the DTR signal... and what this all means > to the arduino in question. To confound things, this all works (of course) > on linux. > > If this were a purely work thing, I'd eat my bile and just run linux. Even > now, I have a R-pi attached to it running linux to "get things done" ... but > I want FreeBSD to work. > > So... if I'm charged to fix this, what are my tools? Is ktrace sufficient > to indicate all the calls to the serial? How do I interpret it so that I > can make sense of this. Do I potentially need DTrace or is that overkill? Unfortunately, I'm not much help on the tool side- all of my problems thus far have been diagnosed just by observing behavior. I think this previous statement of yours says much: >> Firstly, when I run "arduino" (the IDE) ... the arduino board resets immediately (I can tell this because it has an LCD screen attached). >> But when I run pronterface, it doesn't reset until 5 seconds (roughly) after pronterface exits. IIRC: If the port gets opened, there will be a reset, full-stop. If there is no reset, the odds that the port actually got successfully opened are slim to none. This should be easily verified with truss or ktrace- you should try and perhaps post the results back here. From owner-freebsd-hackers@freebsd.org Fri Jan 5 07:37:13 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C6B6EEB9E42 for ; Fri, 5 Jan 2018 07:37:13 +0000 (UTC) (envelope-from yuri@rawbw.com) Received: from shell1.rawbw.com (shell1.rawbw.com [198.144.192.42]) by mx1.freebsd.org (Postfix) with ESMTP id 9C11E7B803 for ; Fri, 5 Jan 2018 07:37:13 +0000 (UTC) (envelope-from yuri@rawbw.com) Received: from yv.noip.me (c-24-6-186-56.hsd1.ca.comcast.net [24.6.186.56]) (authenticated bits=0) by shell1.rawbw.com (8.15.1/8.15.1) with ESMTPSA id w057b7v2073137 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Thu, 4 Jan 2018 23:37:07 -0800 (PST) (envelope-from yuri@rawbw.com) X-Authentication-Warning: shell1.rawbw.com: Host c-24-6-186-56.hsd1.ca.comcast.net [24.6.186.56] claimed to be yv.noip.me Subject: Re: Fwd: [Differential] [Request, 113 lines] D13757: morse: implement support for decoding morse code To: FreeBSD Hackers References: From: Yuri Message-ID: <99d5f2ab-bd99-d831-a376-19c7c4713090@rawbw.com> Date: Thu, 4 Jan 2018 23:37:05 -0800 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.5.2 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jan 2018 07:37:13 -0000 On 01/03/18 17:30, Eitan Adler wrote: > Review of this is appreciated. Mostly stolen from a bug report, but > modified a bit. Why 'morse' is in the base and not in ports? Yuri From owner-freebsd-hackers@freebsd.org Fri Jan 5 10:08:19 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DE3ECEC1C0B for ; Fri, 5 Jan 2018 10:08:19 +0000 (UTC) (envelope-from repeatable_compression@yahoo.com) Received: from sonic301-31.consmr.mail.ne1.yahoo.com (sonic301-31.consmr.mail.ne1.yahoo.com [66.163.184.200]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id ACA5A1744 for ; Fri, 5 Jan 2018 10:08:19 +0000 (UTC) (envelope-from repeatable_compression@yahoo.com) X-YMail-OSG: tPYEH6cVM1l5zIptmFMmFqUhIFFALqD8GkaWzpDhwxio1eTwycVVAuvDaQws9vP T4FAG3Z72GpY5tsIACENF9XWn7CyNMsO5o7oCssBN2b1VQddpq5B5wNuc9czB_5uiCBXCXnwJFcB z0o39Y2Y7zeyqB4Zy_Bx7DlA8.Mi2C2ZX5hVux87tRiKciJ.7GMC97yb.U4rr40EcZSEe2bUNmxu cyS4aFQFI_U0TntrAKi5tpmQE6FL5xv71QULySgUkX1A_WqZk4jOSQsDV71pLw_nV.LjNol9Ty7C MR6I5ZFNwCX4utWVPsLme3moztVNWxzPq1jTOTPG.bpLahDnonhYqwzTSyCrx0MhXGwZSVoXDBgJ 6bX8gEn7FjeJvGqrxo5V9yuJ72hU.90AS6kD4sJ0zyy5ZiNdEPair8fqLBRs2iHVb4er3FMzQa.E Mw4tQgasMi.tB3lHeSZRrq8PD44MUp0jsgvOvaw5KBzpb2escrDY56ScyLmwEi_mNPLzJnpEPfPe UqvsqDklmsz1Irp9ICob3dA-- Received: from sonic.gate.mail.ne1.yahoo.com by sonic301.consmr.mail.ne1.yahoo.com with HTTP; Fri, 5 Jan 2018 10:08:18 +0000 Date: Fri, 5 Jan 2018 10:07:01 +0000 (UTC) From: Jules Gilbert To: "Ronald F. Guilmette" , Eric McCorkle , Freebsd Security , Brett Glass , =?UTF-8?Q?Dag-Erling_Sm=C3=B8rgrav?= , Poul-Henning Kamp , "freebsd-arch@freebsd.org" , FreeBSD Hackers , Shawn Webb , Nathan Whitehorn Message-ID: <809675000.867372.1515146821354@mail.yahoo.com> In-Reply-To: <2594.1515141192@segfault.tristatelogic.com> References: <736a2b77-d4a0-b03f-8a6b-6a717f5744d4@metricspace.net> <2594.1515141192@segfault.tristatelogic.com> Subject: Re: Intel hardware bug MIME-Version: 1.0 X-Mailer: WebService/1.1.11150 YMailNorrin Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:57.0) Gecko/20100101 Firefox/57.0 X-Mailman-Approved-At: Fri, 05 Jan 2018 11:38:31 +0000 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jan 2018 10:08:20 -0000 Sorry guys, you just convinced me that no one, not the NSA, not the FSB, no= one!, has in the past, or will in the future be able to exploit this to ac= tually do something not nice. I'm not saying that the hardware shouldn't be fixed, I am saying that we do= n't need to worry about this. In the early days of DOS their was a hardware bug in nearly all floppy cont= rollers, it wasn't even discovered until (I think,) 1985 or so.=C2=A0 The t= hing is..., no one reported unusual problems. So what is this, really?, it's a market exploit opportunity for AMD. =20 On Friday, January 5, 2018, 3:33:31 AM EST, Ronald F. Guilmette wrote: =20 =20 =20 In message <736a2b77-d4a0-b03f-8a6b-6a717f5744d4@metricspace.net>,=20 Eric McCorkle wrote: >The attack looks like this: > >1) Fetch kernel/other process memory, which eventually faults >2) Do a bit-shift/mask operation to pluck out one bit of the fetched >value.=C2=A0 This gets executed speculatively on the fetched value in (1). >3) Execute fetches of two different addresses depending on some bit in >the fetched value in (1) (say, 0x100000 for 0 vs 0x200000 for 1).=C2=A0 Th= is >also gets executed speculatively despite the fact that (1) ends up faultin= g. >4) Recover from fault in (1) >5) Measure performance of accesses to the two addresses to determine >which one is cached. I must say, that's one hell of a round-about way to read just one bit that you wern't supposed to have access to.=C2=A0 But of course, that doesn't re= ally matter if you are an attacker. If the above steps can be repeated, programatically, ad infinitum, to read bits from "protected" memory... and I see no reason why they can't be... then yea, this bug is every bit as bad as the media is making it out to be, and maybe even worse. All your secrets are belong to us! Time to invest in abacuses... or is that abacai? Regards, rfg _______________________________________________ freebsd-security@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-security To unsubscribe, send any mail to "freebsd-security-unsubscribe@freebsd.org" =20 From owner-freebsd-hackers@freebsd.org Fri Jan 5 12:42:55 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F0D1AEA6BA7; Fri, 5 Jan 2018 12:42:55 +0000 (UTC) (envelope-from eric@metricspace.net) Received: from mail.metricspace.net (mail.metricspace.net [IPv6:2001:470:1f11:617::107]) by mx1.freebsd.org (Postfix) with ESMTP id C44D8670C3; Fri, 5 Jan 2018 12:42:55 +0000 (UTC) (envelope-from eric@metricspace.net) Received: from [172.16.0.82] (unknown [172.16.0.82]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: eric) by mail.metricspace.net (Postfix) with ESMTPSA id 3ED298850; Fri, 5 Jan 2018 12:42:54 +0000 (UTC) Subject: Re: Intel hardware bug To: Jules Gilbert , "Ronald F. Guilmette" , Freebsd Security , Brett Glass , =?UTF-8?Q?Dag-Erling_Sm=c3=b8rgrav?= , Poul-Henning Kamp , "freebsd-arch@freebsd.org" , FreeBSD Hackers , Shawn Webb , Nathan Whitehorn References: <736a2b77-d4a0-b03f-8a6b-6a717f5744d4@metricspace.net> <2594.1515141192@segfault.tristatelogic.com> <809675000.867372.1515146821354@mail.yahoo.com> From: Eric McCorkle Message-ID: <250f3a77-822b-fba5-dcd7-758dfec94554@metricspace.net> Date: Fri, 5 Jan 2018 07:42:53 -0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.5.0 MIME-Version: 1.0 In-Reply-To: <809675000.867372.1515146821354@mail.yahoo.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Fri, 05 Jan 2018 12:48:51 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jan 2018 12:42:56 -0000 On 01/05/2018 05:07, Jules Gilbert wrote: > Sorry guys, you just convinced me that no one, not the NSA, not the FSB, > no one!, has in the past, or will in the future be able to exploit this > to actually do something not nice. Attacks have already been demonstrated, pulling secrets out of kernel space with meltdown and http headers/passwords out of a browser with spectre. Javascript PoCs are already in existence, and we can expect them to find their way into adware-based malware within a week or two. Also, I'd be willing to bet you a year's rent that certain three-letter organizations have known about and used this for some time. > So what is this, really?, it's a market exploit opportunity for AMD. Don't bet on it. There's reports of AMD vulnerabilities, also for ARM. I doubt any major architecture is going to make it out unscathed. (But if one does, my money's on Power) From owner-freebsd-hackers@freebsd.org Fri Jan 5 14:48:50 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 27A19EAD345; Fri, 5 Jan 2018 14:48:50 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id DD7BC6BE94; Fri, 5 Jan 2018 14:48:49 +0000 (UTC) (envelope-from des@des.no) Received: from desk.des.no (smtp.des.no [194.63.250.102]) by smtp.des.no (Postfix) with ESMTP id BC691107C6; Fri, 5 Jan 2018 14:48:47 +0000 (UTC) Received: by desk.des.no (Postfix, from userid 1001) id 7E0425CEB6; Fri, 5 Jan 2018 14:47:15 +0000 (UTC) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Jules Gilbert Cc: "Ronald F. Guilmette" , Eric McCorkle , Freebsd Security , Poul-Henning Kamp , "freebsd-arch\@freebsd.org" , FreeBSD Hackers , Shawn Webb , Nathan Whitehorn Subject: Re: Intel hardware bug References: <736a2b77-d4a0-b03f-8a6b-6a717f5744d4@metricspace.net> <2594.1515141192@segfault.tristatelogic.com> <809675000.867372.1515146821354@mail.yahoo.com> Date: Fri, 05 Jan 2018 15:47:15 +0100 In-Reply-To: <809675000.867372.1515146821354@mail.yahoo.com> (Jules Gilbert's message of "Fri, 5 Jan 2018 10:07:01 +0000 (UTC)") Message-ID: <861sj4tlak.fsf@desk.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jan 2018 14:48:50 -0000 Jules Gilbert writes: > Sorry guys, you just convinced me that no one, not the NSA, not the > FSB, no one!, has in the past, or will in the future be able to > exploit this to actually do something not nice. The technique has already been proven by multiple independent parties to work quite well, allowing an attacker to read kernel memory at speeds of up to 500 kB/s. But I guess you know better... DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-hackers@freebsd.org Fri Jan 5 13:30:32 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 370F1EA923D; Fri, 5 Jan 2018 13:30:32 +0000 (UTC) (envelope-from aduane@juniper.net) Received: from mx0a-00273201.pphosted.com (mx0a-00273201.pphosted.com [208.84.65.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.pphosted.com", Issuer "thawte SHA256 SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E414668C62; Fri, 5 Jan 2018 13:30:31 +0000 (UTC) (envelope-from aduane@juniper.net) Received: from pps.filterd (m0108156.ppops.net [127.0.0.1]) by mx0a-00273201.pphosted.com (8.16.0.21/8.16.0.21) with SMTP id w05DTiQ3014492; Fri, 5 Jan 2018 05:30:28 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=juniper.net; h=from : to : subject : date : message-id : references : in-reply-to : content-type : content-transfer-encoding : mime-version; s=PPS1017; bh=Rq10H2AKnhJa6OQt+JES3iPtcqyMMxVpJ8rjOFIiCR0=; b=nA2Jv9BZbdDRqYZr9CEPdyO298kV445+NaO05/NX7bi6/PeaDmNFl0EehV6xiLVSikff sC3fZdfLmVURWW3wg7TMvLD4uxmOq/iGYHBfrpbOtAmH1GsNGTXmTyYC4iRXoKBzcGTc iU2m2GslE3owIGc7XhMfOAhJdusb6LEYjvd+o5kfPYYL7Foqp7zTUSD1+0S52S0KD+cU Ek5cUnBVym/SOL5l68st5elNoylP+PLqU58dCuSltZJYXpdegI8GUO/yAUb9QdSMg4jE owJv7vz269JIY6iGmL3rUXMzMj5nppzORhwBBjnxWHhw0ZHZhDLOoCZdm/jXsdQjjH7X oA== Received: from nam02-sn1-obe.outbound.protection.outlook.com (mail-sn1nam02lp0022.outbound.protection.outlook.com [216.32.180.22]) by mx0a-00273201.pphosted.com with ESMTP id 2fa9ymr13j-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Fri, 05 Jan 2018 05:30:27 -0800 Received: from SN1PR0501MB2125.namprd05.prod.outlook.com (10.163.228.152) by SN1PR0501MB1693.namprd05.prod.outlook.com (10.163.130.151) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.407.1; Fri, 5 Jan 2018 13:30:26 +0000 Received: from SN1PR0501MB2125.namprd05.prod.outlook.com ([10.163.228.152]) by SN1PR0501MB2125.namprd05.prod.outlook.com ([10.163.228.152]) with mapi id 15.20.0407.000; Fri, 5 Jan 2018 13:30:26 +0000 From: Andrew Duane To: Eric McCorkle , Jules Gilbert , "Ronald F. Guilmette" , Freebsd Security , Brett Glass , =?iso-8859-1?Q?Dag-Erling_Sm=F8rgrav?= , Poul-Henning Kamp , "freebsd-arch@freebsd.org" , FreeBSD Hackers , Shawn Webb , Nathan Whitehorn Subject: RE: Intel hardware bug Thread-Topic: Intel hardware bug Thread-Index: AQHThhm6gtRbndOyekeN4M7Qcuy2NqNlOTSAgAAMhiA= Date: Fri, 5 Jan 2018 13:30:26 +0000 Message-ID: References: <736a2b77-d4a0-b03f-8a6b-6a717f5744d4@metricspace.net> <2594.1515141192@segfault.tristatelogic.com> <809675000.867372.1515146821354@mail.yahoo.com> <250f3a77-822b-fba5-dcd7-758dfec94554@metricspace.net> In-Reply-To: <250f3a77-822b-fba5-dcd7-758dfec94554@metricspace.net> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [66.129.241.11] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; SN1PR0501MB1693; 7:5aYxUZZcdWX6ZDNcffZq8FqMPnCx+mx3MXNLc2/udTNDnhkzQzmumE+DdiNTjTR1BtKeMihQNc15xux2UI0tTjMTONlkHptUb77yHz7uV9DfHPnI7jhfP/C/qZWFuK2aGIWzrBcBrxPI6IO/Y0n79rb1d4L0bk5yqD2P3uv3jyTtd6NxBhe1P2eXDjQZFBmFFiv8sQsPcVC0c1AjVVOUjlfkSxsO6xUheIX4e1FrUWVVZH7KxIlUPyXSdzVR9yrl x-ms-exchange-antispam-srfa-diagnostics: SSOS; x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: 54d83e8a-b5b8-4844-b342-08d554407afc x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020020)(48565401081)(5600026)(4604075)(3008032)(4534040)(4602075)(4627136)(201703031133081)(201702281549075)(2017052603307)(7153060); SRVR:SN1PR0501MB1693; x-ms-traffictypediagnostic: SN1PR0501MB1693: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(20558992708506)(192374486261705)(138986009662008)(201166117486090); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040470)(2401047)(8121501046)(5005006)(3231023)(944501075)(3002001)(10201501046)(93006095)(93001095)(6055026)(6041268)(20161123564045)(20161123558120)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123562045)(20161123560045)(6072148)(201708071742011); SRVR:SN1PR0501MB1693; BCL:0; PCL:0; RULEID:(100000803101)(100110400095); SRVR:SN1PR0501MB1693; x-forefront-prvs: 05437568AA x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(346002)(39860400002)(376002)(396003)(39380400002)(366004)(199004)(189003)(13464003)(24454002)(551544002)(86362001)(68736007)(6246003)(7116003)(229853002)(77096006)(5660300001)(7736002)(3660700001)(33656002)(106356001)(6436002)(6116002)(7416002)(8936002)(3846002)(97736004)(55016002)(2501003)(39060400002)(305945005)(110136005)(316002)(2950100002)(53936002)(81166006)(81156014)(2906002)(53546011)(2900100001)(3280700002)(9686003)(93886005)(8676002)(6506007)(105586002)(99286004)(3480700004)(2521001)(478600001)(74316002)(14454004)(102836004)(59450400001)(25786009)(76176011)(7696005)(66066001)(921003)(1121003); DIR:OUT; SFP:1102; SCL:1; SRVR:SN1PR0501MB1693; H:SN1PR0501MB2125.namprd05.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; received-spf: None (protection.outlook.com: juniper.net does not designate permitted sender hosts) x-microsoft-antispam-message-info: crjRIwDX+1fBxiiPleRIx7Ldfgx6Ycl8CntGyXBX33M43E4qXDlbW9Z1yDgZ4Nh2iTbqKC68AG8ZwWePouqWyA== spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: juniper.net X-MS-Exchange-CrossTenant-Network-Message-Id: 54d83e8a-b5b8-4844-b342-08d554407afc X-MS-Exchange-CrossTenant-originalarrivaltime: 05 Jan 2018 13:30:26.0345 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: bea78b3c-4cdb-4130-854a-1d193232e5f4 X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN1PR0501MB1693 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2018-01-05_06:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_spam_notspam policy=outbound_spam score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1711220000 definitions=main-1801050191 X-Mailman-Approved-At: Fri, 05 Jan 2018 15:28:25 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jan 2018 13:30:32 -0000 I wouldn't think Javascript would have the accurate timing required to leve= rage this attack, but I don't really know enough about the language. Regardless, is there someone within FreeBSD that is working on patches for = this set of problems, at least for Intel? Linux already has at least some, = and I believe NetBSD does too. Of course Windows has already pushed out a W= indows10 fix, 7 and 8 are coming. .................................... Andrew L. Duane - Principal Resident Engineer AT&T Advanced Services Technical Lead Juniper Quality Ambassador m=A0=A0=A0+1 603.770.7088 o +1 408.933.6944 (2-6944) skype: andrewlduane aduane@juniper.net -----Original Message----- From: owner-freebsd-hackers@freebsd.org [mailto:owner-freebsd-hackers@freeb= sd.org] On Behalf Of Eric McCorkle Sent: Friday, January 5, 2018 7:43 AM To: Jules Gilbert ; Ronald F. Guilmette <= rfg@tristatelogic.com>; Freebsd Security ; Br= ett Glass ; Dag-Erling Sm=F8rgrav ; Poul-Henn= ing Kamp ; freebsd-arch@freebsd.org; FreeBSD Hackers ; Shawn Webb ; Natha= n Whitehorn Subject: Re: Intel hardware bug On 01/05/2018 05:07, Jules Gilbert wrote: > Sorry guys, you just convinced me that no one, not the NSA, not the=20 > FSB, no one!, has in the past, or will in the future be able to=20 > exploit this to actually do something not nice. Attacks have already been demonstrated, pulling secrets out of kernel space= with meltdown and http headers/passwords out of a browser with spectre. J= avascript PoCs are already in existence, and we can expect them to find the= ir way into adware-based malware within a week or two. Also, I'd be willing to bet you a year's rent that certain three-letter org= anizations have known about and used this for some time. > So what is this, really?, it's a market exploit opportunity for AMD. Don't bet on it. There's reports of AMD vulnerabilities, also for ARM. I doubt any major architecture is going to make it out unscathed. (But if = one does, my money's on Power) From owner-freebsd-hackers@freebsd.org Fri Jan 5 15:53:01 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BABB9EB0A16; Fri, 5 Jan 2018 15:53:01 +0000 (UTC) (envelope-from eric@metricspace.net) Received: from mail.metricspace.net (mail.metricspace.net [IPv6:2001:470:1f11:617::107]) by mx1.freebsd.org (Postfix) with ESMTP id 921806F00F; Fri, 5 Jan 2018 15:53:01 +0000 (UTC) (envelope-from eric@metricspace.net) Received: from [192.168.43.57] (mobile-166-171-187-244.mycingular.net [166.171.187.244]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: eric) by mail.metricspace.net (Postfix) with ESMTPSA id 1FE5D8900; Fri, 5 Jan 2018 15:35:14 +0000 (UTC) Subject: Re: Intel hardware bug To: =?UTF-8?Q?C_Bergstr=c3=b6m?= Cc: Freebsd Security , "freebsd-arch@freebsd.org" , FreeBSD Hackers References: <736a2b77-d4a0-b03f-8a6b-6a717f5744d4@metricspace.net> <2594.1515141192@segfault.tristatelogic.com> <809675000.867372.1515146821354@mail.yahoo.com> <250f3a77-822b-fba5-dcd7-758dfec94554@metricspace.net> From: Eric McCorkle Message-ID: <755a65eb-b02e-05c5-e1a2-701cfd8bc837@metricspace.net> Date: Fri, 5 Jan 2018 10:35:13 -0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.5.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jan 2018 15:53:01 -0000 On 01/05/2018 09:55, C Bergström wrote: > Don't bet on it.  There's reports of AMD vulnerabilities, also for ARM. > I doubt any major architecture is going to make it out unscathed.  (But > if one does, my money's on Power) > > > Nope, the only arch that I'm aware of that gets past this is SPARC(hah!) > due to the seperate userland and kernel memory virtualization. Alas, poor Sparc. I knew them, Horatio... It looks like Red Hat is indeed reporting Power9 to be vulnerable: https://access.redhat.com/security/vulnerabilities/speculativeexecution Unfortunate. I hope they get fixed silicon out in time for the Talos II workstation. From owner-freebsd-hackers@freebsd.org Fri Jan 5 14:55:36 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5DCF1EADBCC for ; Fri, 5 Jan 2018 14:55:36 +0000 (UTC) (envelope-from cbergstrom@pathscale.com) Received: from mail-qt0-x22a.google.com (mail-qt0-x22a.google.com [IPv6:2607:f8b0:400d:c0d::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 15F956C89B for ; Fri, 5 Jan 2018 14:55:36 +0000 (UTC) (envelope-from cbergstrom@pathscale.com) Received: by mail-qt0-x22a.google.com with SMTP id r39so5944841qtr.13 for ; Fri, 05 Jan 2018 06:55:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pathscale-com.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=4PD0Zfi+So7k9EVdvPlqubBSbdOt7U6U7UmHZ4hrJJQ=; b=Xokg4wOEZfSx4nOCr2bW8OeUDcNsk4cP12sEOsQAMNhZt1HkGLLCMO4PYiRIoxAqjH +xGX4uTdMHxAcrR14EBws2oSYKjXKLDbtTZxVk4psFepbUO+RMf9Sc/LAoMqAlU2qSsy OGxeD4DHh1X+BqNW7My7kk08sMfVwA5IBPI80CEPaJ/LSQAra1/m03feM12mpuFKLPLs NmxWmf4ylUUQVXGqN279GMFhc3EztWR6CP/Ir8xEO4tql/TRWgupmOgYmJbwIWkcBSBl +iFOPm+2CziRbPDQalaTxnDcbmvTbuzktUpCymeK7pgFdvrIeiQRJsV3rnKjXiasaFwp QdQQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=4PD0Zfi+So7k9EVdvPlqubBSbdOt7U6U7UmHZ4hrJJQ=; b=s9w1x8vlfwx0/kgSTw+VrpNE89GTL5St/o84p9b/KW7omZTEDC2b6db0MW8/tZA/jS nYML5VgT2MqFptoQBR8OqcKVnsUXFg7/K1nxd9RfAPNVYwOmOyb8g7zyuWzPGoNVv+sW egR7Y52cUyW0ohuiTaEaFR6UPG2Gl7UXFGvGtXv5jTtPo0oXwOqXrQK/GdMcOQv86ESI uLkn4eBOZq1HxC2q2pm8HtUBKHVkkvhw8PvHUvs7JWlJoY2X0o0jXU0IrPMJICMCbnaE FE3Q4TiPQ0cLSvTrOLs36FjkTxhI3fcIVqeKQ6bzZIpvrotRYs/+S3a4nLYM5XXzizTa hTbA== X-Gm-Message-State: AKwxyteK9/YoXHPWxy7zLoiQ9VxIo6viNW/k90+5GpXSspHav471Rtsn d7i5BZUiq8poBh4pA/qbztUaOQFmibFcpqAFF7MnNw== X-Google-Smtp-Source: ACJfBouxeRPh4iDo+U5erDZNZHzWjCcih60x2pDe8UTfSk98drVvsV41tDpZAWQkpTHCPThKblkQWFpVHO6ZMuR8MSQ= X-Received: by 10.237.33.154 with SMTP id l26mr4553016qtc.100.1515164135206; Fri, 05 Jan 2018 06:55:35 -0800 (PST) MIME-Version: 1.0 Received: by 10.12.210.3 with HTTP; Fri, 5 Jan 2018 06:55:14 -0800 (PST) X-Originating-IP: [202.83.99.25] In-Reply-To: <250f3a77-822b-fba5-dcd7-758dfec94554@metricspace.net> References: <736a2b77-d4a0-b03f-8a6b-6a717f5744d4@metricspace.net> <2594.1515141192@segfault.tristatelogic.com> <809675000.867372.1515146821354@mail.yahoo.com> <250f3a77-822b-fba5-dcd7-758dfec94554@metricspace.net> From: =?UTF-8?B?QyBCZXJnc3Ryw7Zt?= Date: Fri, 5 Jan 2018 22:55:14 +0800 Message-ID: Subject: Re: Intel hardware bug To: Eric McCorkle Cc: Jules Gilbert , "Ronald F. Guilmette" , Freebsd Security , Brett Glass , =?UTF-8?Q?Dag=2DErling_Sm=C3=B8rgrav?= , Poul-Henning Kamp , "freebsd-arch@freebsd.org" , FreeBSD Hackers , Shawn Webb , Nathan Whitehorn X-Mailman-Approved-At: Fri, 05 Jan 2018 15:54:28 +0000 Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jan 2018 14:55:36 -0000 On Fri, Jan 5, 2018 at 8:42 PM, Eric McCorkle wrote: > On 01/05/2018 05:07, Jules Gilbert wrote: > > Sorry guys, you just convinced me that no one, not the NSA, not the FSB, > > no one!, has in the past, or will in the future be able to exploit this > > to actually do something not nice. > > Attacks have already been demonstrated, pulling secrets out of kernel > space with meltdown and http headers/passwords out of a browser with > spectre. Javascript PoCs are already in existence, and we can expect > them to find their way into adware-based malware within a week or two. > > Also, I'd be willing to bet you a year's rent that certain three-letter > organizations have known about and used this for some time. > > > So what is this, really?, it's a market exploit opportunity for AMD. > > Don't bet on it. There's reports of AMD vulnerabilities, also for ARM. > I doubt any major architecture is going to make it out unscathed. (But > if one does, my money's on Power) > Nope, the only arch that I'm aware of that gets past this is SPARC(hah!) due to the seperate userland and kernel memory virtualization. From owner-freebsd-hackers@freebsd.org Fri Jan 5 15:26:04 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2CFE9EAF1FD for ; Fri, 5 Jan 2018 15:26:04 +0000 (UTC) (envelope-from repeatable_compression@yahoo.com) Received: from sonic306-21.consmr.mail.ne1.yahoo.com (sonic306-21.consmr.mail.ne1.yahoo.com [66.163.189.83]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DB42E6DB10 for ; Fri, 5 Jan 2018 15:26:03 +0000 (UTC) (envelope-from repeatable_compression@yahoo.com) X-YMail-OSG: _SddRmcVM1nJxZCtrZgFURVxnQKAGb2iASLhSLJnIiLQbCS6oOI8qaV3dM1hi4w SuR4XEcrNnbotGWzz2cB1RcWpSB1xsID_7tekZTsYHGjwmwhBZUX3Uxfer74oK8ZZpq8DHeO5LEY eeuAb2eFA0_m8B3xyJ0U9s7AytItHsCZ2yzBpQ0SVkTfx.35GjTd0xjGnxD7fy5Qy78qAvRFRsfQ ZNhBUrCcDhnNBIutlsAj5myteoKJ6ntp2NWJeF_A2g2fE84MDuDpIkG3d2Jc_3QRHuWg2.avm5ZM 4da9fZS0GqBoVA8_ekbGtdIOen3sOKRQsarEddWTozNERLkluYwivu3wNG1Gra.4Dbh7zxMjNU5H Ygk40Y9ORP7d7EB11heasRrxS71RutuocaLB4dtCiPv.moiQvwyFRwC8SHS2NcWahwFfIvURAPJt hplC7FHgei1GlSYZHPAvV6Zp7zHb5Rzw8XdNxdkRVR85EoNzCwHRicSwc4O9q4oRvn7_tNhaVrNR FFCBnZyqu3qz5DNmNLgU9Hw-- Received: from sonic.gate.mail.ne1.yahoo.com by sonic306.consmr.mail.ne1.yahoo.com with HTTP; Fri, 5 Jan 2018 15:25:57 +0000 Date: Fri, 5 Jan 2018 15:25:34 +0000 (UTC) From: Jules Gilbert To: =?UTF-8?Q?Dag-Erling_Sm=C3=B8rgrav?= Cc: "Ronald F. Guilmette" , Eric McCorkle , Freebsd Security , Poul-Henning Kamp , "freebsd-arch@freebsd.org" , FreeBSD Hackers , Shawn Webb , Nathan Whitehorn Message-ID: <302406914.1010662.1515165934929@mail.yahoo.com> In-Reply-To: <861sj4tlak.fsf@desk.des.no> References: <736a2b77-d4a0-b03f-8a6b-6a717f5744d4@metricspace.net> <2594.1515141192@segfault.tristatelogic.com> <809675000.867372.1515146821354@mail.yahoo.com> <861sj4tlak.fsf@desk.des.no> Subject: Re: Intel hardware bug MIME-Version: 1.0 X-Mailer: WebService/1.1.11150 YMailNorrin Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:57.0) Gecko/20100101 Firefox/57.0 X-Mailman-Approved-At: Fri, 05 Jan 2018 15:54:45 +0000 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jan 2018 15:26:04 -0000 Ah, sorry I'm wrong.=C2=A0 I apologize.=C2=A0 I won't intrude further.=C2= =A0 I spoke up because selectively choosing to read sections of kernel memo= ry is one thing, obtaining useful information from an arbitrary block of ke= rnel memory you don't get to choose is quite another. But their are several people here I respect very much and if they say I'm w= rong about an area they focus on,... me bad. On Friday, January 5, 2018, 9:48:50 AM EST, Dag-Erling Sm=C3=B8rgrav wrote: =20 =20 Jules Gilbert writes: > Sorry guys, you just convinced me that no one, not the NSA, not the > FSB, no one!, has in the past, or will in the future be able to > exploit this to actually do something not nice. The technique has already been proven by multiple independent parties to work quite well, allowing an attacker to read kernel memory at speeds of up to 500 kB/s.=C2=A0 But I guess you know better... DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no =20 From owner-freebsd-hackers@freebsd.org Fri Jan 5 16:40:53 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6C6FEEB30CB; Fri, 5 Jan 2018 16:40:53 +0000 (UTC) (envelope-from nwhitehorn@freebsd.org) Received: from c.mail.sonic.net (c.mail.sonic.net [64.142.111.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5067171044; Fri, 5 Jan 2018 16:40:52 +0000 (UTC) (envelope-from nwhitehorn@freebsd.org) Received: from comporellon.tachypleus.net (cpe-75-82-218-62.socal.res.rr.com [75.82.218.62]) (authenticated bits=0) by c.mail.sonic.net (8.15.1/8.15.1) with ESMTPSA id w05GeeDm023309 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Fri, 5 Jan 2018 08:40:40 -0800 Subject: Re: Intel hardware bug To: =?UTF-8?Q?C_Bergstr=c3=b6m?= , Eric McCorkle Cc: FreeBSD Hackers , "freebsd-arch@freebsd.org" , Shawn Webb , Freebsd Security , Poul-Henning Kamp , "Ronald F. Guilmette" , =?UTF-8?Q?Dag-Erling_Sm=c3=b8rgrav?= , Brett Glass , Jules Gilbert References: <736a2b77-d4a0-b03f-8a6b-6a717f5744d4@metricspace.net> <2594.1515141192@segfault.tristatelogic.com> <809675000.867372.1515146821354@mail.yahoo.com> <250f3a77-822b-fba5-dcd7-758dfec94554@metricspace.net> From: Nathan Whitehorn Message-ID: Date: Fri, 5 Jan 2018 08:40:39 -0800 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.5.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-Sonic-CAuth: UmFuZG9tSVZILE4do3EwanuIbHwb+oXQqQMbU4e5/RAi1Rs7bbWb8We8TNMbM+6jFRuna3/GZxymxTMek3deOLiPVdx5Re4Gm/lJED6aZkw= X-Sonic-ID: C;anAGKjfy5xGXhCeh2dYaJA== M;xAySKjfy5xGXhCeh2dYaJA== X-Spam-Flag: No X-Sonic-Spam-Details: 0.0/5.0 by cerberusd X-Mailman-Approved-At: Fri, 05 Jan 2018 16:47:30 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jan 2018 16:40:53 -0000 On 01/05/18 06:55, C Bergström wrote: > On Fri, Jan 5, 2018 at 8:42 PM, Eric McCorkle wrote: > >> On 01/05/2018 05:07, Jules Gilbert wrote: >>> Sorry guys, you just convinced me that no one, not the NSA, not the FSB, >>> no one!, has in the past, or will in the future be able to exploit this >>> to actually do something not nice. >> Attacks have already been demonstrated, pulling secrets out of kernel >> space with meltdown and http headers/passwords out of a browser with >> spectre. Javascript PoCs are already in existence, and we can expect >> them to find their way into adware-based malware within a week or two. >> >> Also, I'd be willing to bet you a year's rent that certain three-letter >> organizations have known about and used this for some time. >> >>> So what is this, really?, it's a market exploit opportunity for AMD. >> Don't bet on it. There's reports of AMD vulnerabilities, also for ARM. >> I doubt any major architecture is going to make it out unscathed. (But >> if one does, my money's on Power) >> > Nope, the only arch that I'm aware of that gets past this is SPARC(hah!) > due to the seperate userland and kernel memory virtualization. > _______________________________________________ POWER has the same thing. It's actually stronger separation, since user processes don't share addresses either -- all processes, including the kernel, have windowed access to an 80-bit address space, so no process can even describe an address in another process's address space. There are ways, of course, in which IBM could have messed up the implementation, so the fact that it *should* be secure does not mean it *is*. SPARC avoids the issue because almost all implementations are in-order. -Nathan From owner-freebsd-hackers@freebsd.org Fri Jan 5 18:24:23 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 35CB0EB83A4; Fri, 5 Jan 2018 18:24:23 +0000 (UTC) (envelope-from bsd-lists@BSDforge.com) Received: from udns.ultimatedns.net (static-24-113-41-81.wavecable.com [24.113.41.81]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D01AF76945; Fri, 5 Jan 2018 18:24:16 +0000 (UTC) (envelope-from bsd-lists@BSDforge.com) Received: from udns.ultimatedns.net (localhost [127.0.0.1]) by udns.ultimatedns.net (8.14.9/8.14.9) with ESMTP id w05IO4bV079748; Fri, 5 Jan 2018 10:24:10 -0800 (PST) (envelope-from bsd-lists@BSDforge.com) X-Mailer: UDNSMS MIME-Version: 1.0 Cc: "=?UTF-8?B?RnJlZWJzZCBTZWN1cml0eSIgPGZyZWVic2Qtc2VjdXJpdHlAZnJlZWJzZC5vcmc+LCAiRnJlZUJTRCBIYWNrZXJzIiA8ZnJlZWJzZC1oYWNrZXJzQGZyZWVic2Qub3JnPiwgIiBmcmVlYnNkLWFyY2hAZnJlZWJzZC5vcmc+IiA8ZnJlZWJzZC1hcmNoQGZyZWVic2Qub3JnPiwgIkMgQmVyZ3N0csO2bSIgPGNiZXJnc3Ryb21AcGF0aHNjYWxlLmNvbT4=?= In-Reply-To: <755a65eb-b02e-05c5-e1a2-701cfd8bc837@metricspace.net> From: "Chris H" Reply-To: bsd-lists@BSDforge.com To: "Eric McCorkle" Subject: Re: Intel hardware bug Date: Fri, 05 Jan 2018 10:24:10 -0800 Message-Id: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable X-Mailman-Approved-At: Fri, 05 Jan 2018 18:40:23 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jan 2018 18:24:23 -0000 On Fri, 5 Jan 2018 10:35:13 -0500 "Eric McCorkle" sa= id > On 01/05/2018 09:55, C Bergstr=C3=B6m wrote: >=20 > > Don't bet on it=2E=C2=A0 There's reports of AMD vulnerabilities, also= for ARM=2E > > I doubt any major architecture is going to make it out unscathed=2E= =C2=A0 (But > > if one does, my money's on Power) > >=20 > >=20 > > Nope, the only arch that I'm aware of that gets past this is SPARC(hah!= ) > > due to the seperate userland and kernel memory virtualization=2E >=20 > Alas, poor Sparc=2E I knew them, Horatio=2E=2E=2E Ahh, good ol' SPARC! >=20 > It looks like Red Hat is indeed reporting Power9 to be vulnerable: >=20 > https://access=2Eredhat=2Ecom/security/vulnerabilities/speculativeexecution >=20 > Unfortunate=2E I hope they get fixed silicon out in time for the Talos II > workstation=2E What *I* want to know; is whether they're going to drastically reduce the price on all the affected processors? As it stands, they should be practically giving them away=2E How is it that the burden lies on the OS vendors, and not the manufacturers?! --Chris From owner-freebsd-hackers@freebsd.org Fri Jan 5 19:11:53 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4BB74EBA7E3; Fri, 5 Jan 2018 19:11:53 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Received: from smtp-out-so.shaw.ca (smtp-out-so.shaw.ca [64.59.136.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "Client", Issuer "CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id EFB8E78A87; Fri, 5 Jan 2018 19:11:52 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Received: from spqr.komquats.com ([96.50.22.10]) by shaw.ca with ESMTPA id XXP8eEwCzS7BpXXP9eXXNd; Fri, 05 Jan 2018 12:11:51 -0700 X-Authority-Analysis: v=2.2 cv=NKylwwyg c=1 sm=1 tr=0 a=jvE2nwUzI0ECrNeyr98KWA==:117 a=jvE2nwUzI0ECrNeyr98KWA==:17 a=EA5itrwUPoEA:10 a=RgaUWeydRksA:10 a=YxBL1-UpAAAA:8 a=6I5d2MoRAAAA:8 a=Vd537ib8pVup7S2X7pQA:9 a=r4kWtJy-rJYS1cas:21 a=y9lL4hI3jn-rAOdh:21 a=pILNOxqGKmIA:10 a=COlSyhbwSJSHYInWLu4A:9 a=BC4vU4yZLJsX9d_B:21 a=NuoWA5lUaQeHgTHG:21 a=TUJ8KVh8p4i7ngx5:21 a=_W_S_7VecoQA:10 a=Ia-lj3WSrqcvXOmTRaiG:22 a=IjZwj45LgO3ly-622nXo:22 Received: from [10.168.3.146] (S0106d4ca6d8943b0.gv.shawcable.net [70.66.132.207]) by spqr.komquats.com (Postfix) with ESMTPSA id 404BC335; Fri, 5 Jan 2018 11:11:44 -0800 (PST) MIME-Version: 1.0 From: Cy Schubert Subject: RE: Intel hardware bug Date: Fri, 5 Jan 2018 11:11:49 -0800 To: Eric McCorkle , Jules Gilbert , "Ronald F. Guilmette" , Freebsd Security , Brett Glass , =?Windows-1252?Q?Dag-Erling_Sm=F8rgrav?= , Poul-Henning Kamp , "freebsd-arch@freebsd.org" , FreeBSD Hackers , Shawn Webb , Nathan Whitehorn Message-Id: <20180105191145.404BC335@spqr.komquats.com> X-CMAE-Envelope: MS4wfH322YrJxfBdQecR2LYFVCLIcE5yPI+FMR67fSdSbX5SZNmSHuWMZP7kLJHrNWkbgGHQJ/Or6pAKfo03lnxd/sr7jTjNhJOjwkd44edBoGATMMjkXkI3 QM1dGSB6Vp70ELNp2fudaiCoxqi/sSxgNbHOxHCZPKco7vbxWasxjK/xIeNfsQ90oMZy78xsGjH7/42XYf0s1tOdrO79NeTAaOgpEhVdcGycCsSKn1Vrsyf/ 0b7qFEzTR+OrUYKggiGe3R39N7Tsq7kRSH9ozokVOVGtkXPsIXHtFR4Ha+0FH6geOENSSPbrnLQDlqkVK1tB1JSwwxlfvtrLzqbnGebFHY7UbeABwcbKbREQ E957EaJEwBCxDZeXdMMRAz0wJG0rsDbL5/RqnCasMHNFNZHtXsB9gpwx6uFN5fe/x2JZp+z2wyYPcnfhOsDFKNnxIIsT93mQKpo0CGwZBe/aP/WNW108+A7J m4inBWLIpFHQhB+k3DJ7NuRQtWHCRRMsfRyDgciOvgFdJs5uInksvHNLeCs2dCVcxrd/0OEtesKcffNmJ202SvP9sfcio88q7p06NcgR5ZnbgIPasMXEyIkx A18St45WB/9j3BJcLTKyp/qHS5n+0oAkT6FYhBYcXnYjW7/clj7YZhl8vEKwc3DmV/wAV2OUS3GEUraS4JMGa5Dz X-Mailman-Approved-At: Fri, 05 Jan 2018 19:22:34 +0000 Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jan 2018 19:11:53 -0000 According to a Red Hat announcement, Power and Series z are also vulnerable= . --- Sent using a tiny phone keyboard. Apologies for any typos and autocorrect. Also, this old phone only supports top post. Apologies. Cy Schubert or The need of the many outweighs the greed of the few. --- -----Original Message----- From: Eric McCorkle Sent: 05/01/2018 04:48 To: Jules Gilbert; Ronald F. Guilmette; Freebsd Security; Brett Glass; Dag-= Erling Sm=F8rgrav; Poul-Henning Kamp; freebsd-arch@freebsd.org; FreeBSD Hac= kers; Shawn Webb; Nathan Whitehorn Subject: Re: Intel hardware bug On 01/05/2018 05:07, Jules Gilbert wrote: > Sorry guys, you just convinced me that no one, not the NSA, not the FSB, > no one!, has in the past, or will in the future be able to exploit this > to actually do something not nice. Attacks have already been demonstrated, pulling secrets out of kernel space with meltdown and http headers/passwords out of a browser with spectre. Javascript PoCs are already in existence, and we can expect them to find their way into adware-based malware within a week or two. Also, I'd be willing to bet you a year's rent that certain three-letter organizations have known about and used this for some time. > So what is this, really?, it's a market exploit opportunity for AMD. Don't bet on it. There's reports of AMD vulnerabilities, also for ARM. I doubt any major architecture is going to make it out unscathed. (But if one does, my money's on Power) _______________________________________________ freebsd-arch@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-arch To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" From owner-freebsd-hackers@freebsd.org Fri Jan 5 19:53:44 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 14885EBD0B2; Fri, 5 Jan 2018 19:53:44 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-lf0-x234.google.com (mail-lf0-x234.google.com [IPv6:2a00:1450:4010:c07::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8BBFD7B171; Fri, 5 Jan 2018 19:53:43 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: by mail-lf0-x234.google.com with SMTP id h137so6244147lfe.8; Fri, 05 Jan 2018 11:53:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=ky/ziy21wuNrx6LeM9kDYxphvOfDrWGuamPAcyLFoyQ=; b=d8mikptJfKSgq/Plw56qX0mkSPbAkIBgFcClyMWNHXZXn+wHIce4L6MLc1kzIh62O+ 0q4RdjMmwq59nCzRaOdHKKhiunrexnPIsHgfsBsdgqMGEDIbMeXcq9kH395EiZyBSWMi l5I22tyz+G+c9yLiLd4vHlnYCSKRr67+etdr7aUySD95fsDubjnJ3C27ftXZ/ZfcymuB wozHvReBFr//b6vtFTWDirbR9N7rDg3kwBhRJNh/i/j7a6uVm7epHtYwvbr5zX+Pi9d+ CJqfAvpFSIIOzJ7JEGDb2zNt7fBekkf4t9wPSwcM0zOTUsCdfbqXtDyDUxLhzZNvesPs 8Dqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=ky/ziy21wuNrx6LeM9kDYxphvOfDrWGuamPAcyLFoyQ=; b=XV2td7OqIzB3pAiVLMF3ZDrAR1t1wWA8F8RCFZdBn7KT1nQucK+Igu1HglVrkzP+Se YG+c8jeQlKIb+Vk2pZ+GuUe1kYF4IERh2suuWfG0Y29rhY39HleWIHWJ3wwn5TuBA/aB FN4a6CW13LKJsVKAeq504l/cfacJiDuR1OxlMyAp3fzjt0zi2NMCrdBehkruDfsTFlVf KPZbRSLuNkrA7qdubBxD/y7gSFpg9Gm52taYZErEINHD1bi9Jzfzmu5U6jjcdqRph/RI KGf7oV3AQkK1Zw2rg/gVfcTJF8SaNAN3Or48aktiwBqt6WOqbU3tDsU498fvLUUCzD9v lYvQ== X-Gm-Message-State: AKGB3mKU7PCS7WU6xVb5/uZ7EQUeeAE/wXUha2jxrM9jOjQ9nvcMZalV 2Q3JbN2Vj7JI09d3sr8FDG0rXAsgNfz1AlO0gUgHweck X-Google-Smtp-Source: ACJfBovbFN+wtgncUsoK8jC9T2axtY4jre5KXApipVpLUMad7mBkpUq+3SkJ9VzkZirFREU+19DtnlSJhitlqJl1FGA= X-Received: by 10.25.42.68 with SMTP id f65mr2037762lfl.25.1515182020362; Fri, 05 Jan 2018 11:53:40 -0800 (PST) MIME-Version: 1.0 Received: by 10.25.163.207 with HTTP; Fri, 5 Jan 2018 11:53:39 -0800 (PST) In-Reply-To: <20180105191145.404BC335@spqr.komquats.com> References: <20180105191145.404BC335@spqr.komquats.com> From: Freddie Cash Date: Fri, 5 Jan 2018 11:53:39 -0800 Message-ID: Subject: Re: Intel hardware bug To: Cy Schubert Cc: Freebsd Security , "freebsd-arch@freebsd.org" , FreeBSD Hackers Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jan 2018 19:53:44 -0000 On Fri, Jan 5, 2018 at 11:11 AM, Cy Schubert wrote: > According to a Red Hat announcement, Power and Series z are also > vulnerable. > =E2=80=8B > =E2=80=8BThere's a lot of confusion in the media, press releases, and annou= ncements due to conflating Spectre and Meltdown. Meltdown (aka CVE-2017-5754) is the issue that affects virtually all Intel CPUs and specific ARM Cortex-A CPUs. This allows read-access to kernel memory from unprivileged processes (ring 3 apps get read access to ring 0 memory).=E2=80=8B IBM POWER, Oracle Sparc, and AMD Zen are not affected by= this issue as they provide proper separation between kernel memory maps and userland memory maps; or they aren't OoO architectures that use speculative execution in this manner. Spectre (aka CVE-2017-5715 and CVE-2017-5753) is the issue that affects all CPUs (Intel, AMD, ARM, IBM, Oracle, etc) and allows userland processes to read memory assigned to other userland processes (but does NOT give access to kernel memory). =E2=80=8BIOW, POWER and Sparc are vulnerable to Spectre, but not vulnerable= to Meltdown. --=20 Freddie Cash fjwcash@gmail.com From owner-freebsd-hackers@freebsd.org Fri Jan 5 20:20:18 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BBA8EEBE46B for ; Fri, 5 Jan 2018 20:20:18 +0000 (UTC) (envelope-from jan@digitaldaemon.com) Received: from digitaldaemon.com (digitaldaemon.com [162.217.114.50]) by mx1.freebsd.org (Postfix) with SMTP id 57C3C7C423 for ; Fri, 5 Jan 2018 20:20:18 +0000 (UTC) (envelope-from jan@digitaldaemon.com) Received: (qmail 64581 invoked by uid 89); 5 Jan 2018 20:12:51 -0000 Received: from c-24-0-179-87.hsd1.nj.comcast.net (HELO iMac.local) (jan@digitaldaemon.com@24.0.179.87) by digitaldaemon.com with SMTP; 5 Jan 2018 20:12:51 -0000 Subject: Re: Intel hardware bug To: Freddie Cash Cc: Freebsd Security , FreeBSD Hackers , "freebsd-arch@freebsd.org" References: <20180105191145.404BC335@spqr.komquats.com> From: Jan Knepper Message-ID: <65e5dcae-b973-a54e-868e-bdc4abf007cb@digitaldaemon.com> Date: Fri, 5 Jan 2018 15:12:50 -0500 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:52.0) Gecko/20100101 Thunderbird/52.5.2 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jan 2018 20:20:18 -0000 Thank you! The news indeed does not properly understand the difference, nor which problem affects which hardware/CPU and in many ways acts like it is "the end of the world". On 01/05/2018 14:53, Freddie Cash wrote: > On Fri, Jan 5, 2018 at 11:11 AM, Cy Schubert > wrote: > >> According to a Red Hat announcement, Power and Series z are also >> vulnerable. >> ​ >> > ​There's a lot of confusion in the media, press releases, and announcements > due to conflating Spectre and Meltdown. > > Meltdown (aka CVE-2017-5754) is the issue that affects virtually all Intel > CPUs and specific ARM Cortex-A CPUs. This allows read-access to kernel > memory from unprivileged processes (ring 3 apps get read access to ring 0 > memory).​ IBM POWER, Oracle Sparc, and AMD Zen are not affected by this > issue as they provide proper separation between kernel memory maps and > userland memory maps; or they aren't OoO architectures that use speculative > execution in this manner. > > Spectre (aka CVE-2017-5715 and CVE-2017-5753) is the issue that affects all > CPUs (Intel, AMD, ARM, IBM, Oracle, etc) and allows userland processes to > read memory assigned to other userland processes (but does NOT give access > to kernel memory). > > ​IOW, POWER and Sparc are vulnerable to Spectre, but not vulnerable to > Meltdown. > From owner-freebsd-hackers@freebsd.org Fri Jan 5 19:37:18 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 91EA0EBC262; Fri, 5 Jan 2018 19:37:18 +0000 (UTC) (envelope-from kmacybsd@gmail.com) Received: from mail-oi0-x244.google.com (mail-oi0-x244.google.com [IPv6:2607:f8b0:4003:c06::244]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 50BC77A4F7; Fri, 5 Jan 2018 19:37:18 +0000 (UTC) (envelope-from kmacybsd@gmail.com) Received: by mail-oi0-x244.google.com with SMTP id j14so3774508oih.3; Fri, 05 Jan 2018 11:37:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc:content-transfer-encoding; bh=j5Lww5hOJTkHgnR5QlIw9v5SKLLqOgRo4YfIbtQFYNY=; b=YSvKHjrzNa3uzNxadLBIeSbptOZiF6XWNn+K/6xxbYxxmHAj5+zfQtUyUwjwj+BFNH 7gt+qqizXsgyDgTqr+hwK4XRuxIsRDdoo0C7tusyla3b/uM6TGK20KztmKG2M3Y3hz6W FEC0acah+MxEVSjn6NiVZVaCAVScADylhEG3NjIMhKCSszFGvCAMRCeM5dtGAMjroLa2 XOTDCphqH/gxAU+5xJ5OOvzk4aex/lNtUGgQpwxwFoWOXiSud7M43L6GGk2MNNKrQ52q MHHFom8BkpK9EI7E/9gBl0/FzbAgv7qGtJRMOwx/A3P8iTS6HupuPf5GGixr0ClyF62N +ASg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-transfer-encoding; bh=j5Lww5hOJTkHgnR5QlIw9v5SKLLqOgRo4YfIbtQFYNY=; b=lzET+UPxNoSBeWwbBymKzjfTyW1JKPRm4j4j0GC7IaFWpxzAiyO4cu8vOK4P6kHR6K McBGUOV/KxHCqWCADkIjD8ehtk6zjHtPO4G4bgUuaxu0GHBiLYxJYYT9I4kY1778Q4gY RFEUEH57gxq+SdZbHs04F7AfYGeagI025O21RMLqdt0ENc1hWyrKcLsI1xYDY+X/SVVr VypMiEuYBHGvKurAcNhvptHRbkZCOSxMMLAF3OJncho0eKOCPl5Cky3sSDiJ0a/UAirL /jI1irxEbalRVMBoiM33BDpgUcxHJ1ShsQnpgPwO9Sfd3izgvrn4WEx23QbMOIBelqMA 15ZQ== X-Gm-Message-State: AKGB3mIY3HV5Suq6vzxgaAim+/eqSfKjgmuT9CU40eWHCE/443KPWPtf ZyzWuyQO4H+S/B2kK8D+Se4l0EcIJPiE8JQa7aIxSg== X-Google-Smtp-Source: ACJfBovkWnm66+zvXb4XM73KT3niZsopBq7jwSyGn99gc6HbVmGi9ZgcIyGONAzc/eDT05C0HccurmJpmrajRkmJ3Vs= X-Received: by 10.202.57.87 with SMTP id g84mr2035556oia.201.1515181037442; Fri, 05 Jan 2018 11:37:17 -0800 (PST) MIME-Version: 1.0 Sender: kmacybsd@gmail.com Received: by 10.157.88.201 with HTTP; Fri, 5 Jan 2018 11:37:17 -0800 (PST) In-Reply-To: <20180105191145.404BC335@spqr.komquats.com> References: <20180105191145.404BC335@spqr.komquats.com> From: "K. Macy" Date: Fri, 5 Jan 2018 11:37:17 -0800 X-Google-Sender-Auth: YcqGLBjADoDtXpu6XXvSQHD1BXU Message-ID: Subject: Re: Intel hardware bug To: Cy Schubert Cc: Eric McCorkle , Jules Gilbert , "Ronald F. Guilmette" , Freebsd Security , Brett Glass , =?UTF-8?Q?Dag=2DErling_Sm=C3=B8rgrav?= , Poul-Henning Kamp , "freebsd-arch@freebsd.org" , FreeBSD Hackers , Shawn Webb , Nathan Whitehorn Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Mailman-Approved-At: Fri, 05 Jan 2018 21:31:29 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jan 2018 19:37:18 -0000 On Fri, Jan 5, 2018 at 11:11 AM, Cy Schubert wr= ote: > According to a Red Hat announcement, Power and Series z are also vulnerab= le. > Link? > --- > > -----Original Message----- > From: Eric McCorkle > Sent: 05/01/2018 04:48 > To: Jules Gilbert; Ronald F. Guilmette; Freebsd Security; Brett Glass; Da= g-Erling Sm=C3=B8rgrav; Poul-Henning Kamp; freebsd-arch@freebsd.org; FreeBS= D Hackers; Shawn Webb; Nathan Whitehorn > Subject: Re: Intel hardware bug > > On 01/05/2018 05:07, Jules Gilbert wrote: >> Sorry guys, you just convinced me that no one, not the NSA, not the FSB, >> no one!, has in the past, or will in the future be able to exploit this >> to actually do something not nice. > > Attacks have already been demonstrated, pulling secrets out of kernel > space with meltdown and http headers/passwords out of a browser with > spectre. Javascript PoCs are already in existence, and we can expect > them to find their way into adware-based malware within a week or two. > > Also, I'd be willing to bet you a year's rent that certain three-letter > organizations have known about and used this for some time. > >> So what is this, really?, it's a market exploit opportunity for AMD. > > Don't bet on it. There's reports of AMD vulnerabilities, also for ARM. > I doubt any major architecture is going to make it out unscathed. (But > if one does, my money's on Power) > _______________________________________________ > freebsd-arch@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > > _______________________________________________ > freebsd-arch@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" From owner-freebsd-hackers@freebsd.org Fri Jan 5 19:47:46 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C2408EBC993; Fri, 5 Jan 2018 19:47:46 +0000 (UTC) (envelope-from kmacybsd@gmail.com) Received: from mail-oi0-x22f.google.com (mail-oi0-x22f.google.com [IPv6:2607:f8b0:4003:c06::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7E84F7ABE9; Fri, 5 Jan 2018 19:47:46 +0000 (UTC) (envelope-from kmacybsd@gmail.com) Received: by mail-oi0-x22f.google.com with SMTP id w125so3778605oie.7; Fri, 05 Jan 2018 11:47:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc:content-transfer-encoding; bh=SBVbEq8aFAKbgpNgZlwITSCuachsMiVKCY3Y34M+Ok4=; b=icbzQZqRsdIYvp/iVR07/aVfxXJdvVjGAQTc2YCbFZv1//6BsJ+m8FXjLFzBDaKDhC 5twVcgXWEBf+gMWd6B7oBvZrZ7k42F3yj4K+SleKcBGU3/XFgjuJ0odnjmKpNSVfq4Et d5RqDi2khJZhkQ7NUwWVEGTg+zgGwJ6fw4y6EetTb9wb/llPeYNX1VKeYFvYxTtCNjnH 1XC17egYymlAixoGDcj4IwXTWAiSXsz5a5jC2F0y9+YEAX7WHc64i/DRAqM2V823keH8 nmjCQREqBKXzUV8V4qDxTnoftESHIibJQkLS8Syadi+2yODgdwyddHZyQUVHhkd5kqdE o/lA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-transfer-encoding; bh=SBVbEq8aFAKbgpNgZlwITSCuachsMiVKCY3Y34M+Ok4=; b=RITX5oNIw2bMGmQXllVjz9ZJCKiCmIScNR22vQs6jJL9wv+ebQjssS9CUNt9nyZOq3 Ry4BMiDsTwSIfH/n+ppTctlzVzuRcWy4pVgGBv4tDr+IQTRGRSFqQh7uDtodAq4cqd31 qd7dfb82GGVAw2CAQegeosROliTz+ez8yHO/l9VUChIlvU3cBb2iFw5IGrbM+Gb2UAVQ u11yI6yJvBSMNrF3HWNIhQ5wIKCrCu3Lz8139WkHuQq+YDse8XMm2QNrSYHtrTClza4l yMyoIQsXYutUQ6qEIF6e8qMNkEJ/RJauk5XwV/d7pQ+YtFxNd1b2uP7C1KH9gOSG15/e 8u7g== X-Gm-Message-State: AKGB3mJkaqJvEQE9ZS09s+Uap2zgtP9gTFP6gquNpPY22xHJHnpqAZd8 q6pklZL+FJQd9dfAa0c6F91pVQB70zjwKRye/zs= X-Google-Smtp-Source: ACJfBosiwiG1zx9hC6ZAi3zdYyVSAHiUbRLdW3Spf9Ljt82P0aumoVmMV+GWR08/NJk62dzyCnAj++/kie4Izd5IrcE= X-Received: by 10.202.57.87 with SMTP id g84mr2049019oia.201.1515181665680; Fri, 05 Jan 2018 11:47:45 -0800 (PST) MIME-Version: 1.0 Sender: kmacybsd@gmail.com Received: by 10.157.88.201 with HTTP; Fri, 5 Jan 2018 11:47:45 -0800 (PST) In-Reply-To: References: <20180105191145.404BC335@spqr.komquats.com> From: "K. Macy" Date: Fri, 5 Jan 2018 11:47:45 -0800 X-Google-Sender-Auth: bznOiYGbOMhIL_8UvY7BwimpOhY Message-ID: Subject: Re: Intel hardware bug To: Cy Schubert Cc: Eric McCorkle , Jules Gilbert , "Ronald F. Guilmette" , Freebsd Security , Brett Glass , =?UTF-8?Q?Dag=2DErling_Sm=C3=B8rgrav?= , Poul-Henning Kamp , "freebsd-arch@freebsd.org" , FreeBSD Hackers , Shawn Webb , Nathan Whitehorn Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Mailman-Approved-At: Fri, 05 Jan 2018 21:34:35 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jan 2018 19:47:46 -0000 On Fri, Jan 5, 2018 at 11:37 AM, K. Macy wrote: > On Fri, Jan 5, 2018 at 11:11 AM, Cy Schubert = wrote: >> According to a Red Hat announcement, Power and Series z are also vulnera= ble. >> > > Link? Spectre yes. Meltdown no. Spectre is a problem but much harder to exploit. It's Intel's handling of meltdown that is seriously grounds for table flipping. https://www.ibm.com/blogs/psirt/potential-impact-processors-power-family/ > > >> --- >> >> -----Original Message----- >> From: Eric McCorkle >> Sent: 05/01/2018 04:48 >> To: Jules Gilbert; Ronald F. Guilmette; Freebsd Security; Brett Glass; D= ag-Erling Sm=C3=B8rgrav; Poul-Henning Kamp; freebsd-arch@freebsd.org; FreeB= SD Hackers; Shawn Webb; Nathan Whitehorn >> Subject: Re: Intel hardware bug >> >> On 01/05/2018 05:07, Jules Gilbert wrote: >>> Sorry guys, you just convinced me that no one, not the NSA, not the FSB= , >>> no one!, has in the past, or will in the future be able to exploit this >>> to actually do something not nice. >> >> Attacks have already been demonstrated, pulling secrets out of kernel >> space with meltdown and http headers/passwords out of a browser with >> spectre. Javascript PoCs are already in existence, and we can expect >> them to find their way into adware-based malware within a week or two. >> >> Also, I'd be willing to bet you a year's rent that certain three-letter >> organizations have known about and used this for some time. >> >>> So what is this, really?, it's a market exploit opportunity for AMD. >> >> Don't bet on it. There's reports of AMD vulnerabilities, also for ARM. >> I doubt any major architecture is going to make it out unscathed. (But >> if one does, my money's on Power) >> _______________________________________________ >> freebsd-arch@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-arch >> To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" >> >> _______________________________________________ >> freebsd-arch@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-arch >> To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" From owner-freebsd-hackers@freebsd.org Fri Jan 5 19:47:48 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 278A4EBC99B; Fri, 5 Jan 2018 19:47:48 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Received: from smtp-out-so.shaw.ca (smtp-out-so.shaw.ca [64.59.136.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "Client", Issuer "CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id BFA097ABEA; Fri, 5 Jan 2018 19:47:47 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Received: from spqr.komquats.com ([96.50.22.10]) by shaw.ca with ESMTPA id XXxxeF6ugS7BpXXxyeXg6c; Fri, 05 Jan 2018 12:47:46 -0700 X-Authority-Analysis: v=2.2 cv=NKylwwyg c=1 sm=1 tr=0 a=jvE2nwUzI0ECrNeyr98KWA==:117 a=jvE2nwUzI0ECrNeyr98KWA==:17 a=RgaUWeydRksA:10 a=20KFwNOVAAAA:8 a=YxBL1-UpAAAA:8 a=6I5d2MoRAAAA:8 a=uzSovi_1LZDTydRZN2AA:9 a=8J4chuak_uIRPK4g:21 a=pVYAQAJm03auPRN6:21 a=QEXdDO2ut3YA:10 a=MMjiIUenpLf6ge6zeOQA:9 a=ZsGH8CjA44_So7fI:21 a=I18adWnKP6q6kCHM:21 a=Th99hHS62kTuvYDY:21 a=_W_S_7VecoQA:10 a=Ia-lj3WSrqcvXOmTRaiG:22 a=IjZwj45LgO3ly-622nXo:22 Received: from [10.168.3.146] (S0106d4ca6d8943b0.gv.shawcable.net [70.66.132.207]) by spqr.komquats.com (Postfix) with ESMTPSA id EAFBB3EA; Fri, 5 Jan 2018 11:47:43 -0800 (PST) MIME-Version: 1.0 From: Cy Schubert Subject: RE: Intel hardware bug Date: Fri, 5 Jan 2018 11:47:49 -0800 To: "K. Macy" , Cy Schubert CC: Eric McCorkle , Jules Gilbert , "Ronald F. Guilmette" , Freebsd Security , Brett Glass , =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= , Poul-Henning Kamp , "freebsd-arch@freebsd.org" , FreeBSD Hackers , Shawn Webb , Nathan Whitehorn Message-Id: <20180105194743.EAFBB3EA@spqr.komquats.com> X-CMAE-Envelope: MS4wfDfdNQ2Axb8uIQDJJKM+DFEBFjWsWorXKt0r8ks+tP0iKOErSoRXaeUrk2rDccno983U/CQgBtyh3RxCJ8FYzS+MJTKRz/sNso857+DlZa38tr0/3+VV c+vz41Fndb7Z5Hy3sIKDK7QnsHQdNUaCySGj5ONXpi1Yyl5x0iiGcN/9otJZkc26KDXwd2iJ9ZFok/nFmkacXxzE8TW4iDDV9qu5dI031RM9GDyQnZFAqfC/ m16owC1IwLIZs+FX0NcKdPZCDDS+X8GtOmMVeG5wnAj7YcqNBfUrUC8Vn8BHfpHoh1mkcQQ+UVEB5oU8tee/fYO7uuZCeGXdvyWbEAHWKnl+jj4++BfhudXA LnUklwoijzWov8l+P8h0dZl0K9Mv/7eQ/f4Ak5bJsQOY82hYGTHr8FZXoqjks345GWdMBUIGoLY7CgkI3qIpdgAmC+nz+mxNBsgp4qjR/D99e2jTznHpkukE dsk5H+6SULQcJ4XPRC0igRLJYvAWOpc+j7EgZYXs8shYOKtZNLJFlmPAMsFuMwm48eQU6CexknliTsA6/G8XYS8mDZ9ZwL7Q6U1SZiPEhUzgex4Yzo7wVIuC eyxaQfebLC9cLMj+FTKpQ9af70utNzbeL1nNqK5zaWQistOf/g99kRc7nJnjI2/XT+bhzBxL81UTDQSc542RpYoW5FywQPa4nqwAXdBH10aInabgLxudA5IY TyCIfunthjw= X-Mailman-Approved-At: Fri, 05 Jan 2018 21:39:57 +0000 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jan 2018 19:47:48 -0000 https://access.redhat.com/security/vulnerabilities/speculativeexecution?sc_= cid=3D701f2000000tsLNAAY& --- Sent using a tiny phone keyboard. Apologies for any typos and autocorrect. Also, this old phone only supports top post. Apologies. Cy Schubert or The need of the many outweighs the greed of the few. --- -----Original Message----- From: K. Macy Sent: 05/01/2018 11:37 To: Cy Schubert Cc: Eric McCorkle; Jules Gilbert; Ronald F. Guilmette; Freebsd Security; Br= ett Glass; Dag-Erling Sm=C3=B8rgrav; Poul-Henning Kamp; freebsd-arch@freebs= d.org; FreeBSD Hackers; Shawn Webb; Nathan Whitehorn Subject: Re: Intel hardware bug On Fri, Jan 5, 2018 at 11:11 AM, Cy Schubert wr= ote: > According to a Red Hat announcement, Power and Series z are also vulnerab= le. > Link? > --- > > -----Original Message----- > From: Eric McCorkle > Sent: 05/01/2018 04:48 > To: Jules Gilbert; Ronald F. Guilmette; Freebsd Security; Brett Glass; Da= g-Erling Sm=C3=B8rgrav; Poul-Henning Kamp; freebsd-arch@freebsd.org; FreeBS= D Hackers; Shawn Webb; Nathan Whitehorn > Subject: Re: Intel hardware bug > > On 01/05/2018 05:07, Jules Gilbert wrote: >> Sorry guys, you just convinced me that no one, not the NSA, not the FSB, >> no one!, has in the past, or will in the future be able to exploit this >> to actually do something not nice. > > Attacks have already been demonstrated, pulling secrets out of kernel > space with meltdown and http headers/passwords out of a browser with > spectre. Javascript PoCs are already in existence, and we can expect > them to find their way into adware-based malware within a week or two. > > Also, I'd be willing to bet you a year's rent that certain three-letter > organizations have known about and used this for some time. > >> So what is this, really?, it's a market exploit opportunity for AMD. > > Don't bet on it. There's reports of AMD vulnerabilities, also for ARM. > I doubt any major architecture is going to make it out unscathed. (But > if one does, my money's on Power) > _______________________________________________ > freebsd-arch@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > > _______________________________________________ > freebsd-arch@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" From owner-freebsd-hackers@freebsd.org Fri Jan 5 22:02:15 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 90F42EC347A; Fri, 5 Jan 2018 22:02:15 +0000 (UTC) (envelope-from eric@metricspace.net) Received: from mail.metricspace.net (mail.metricspace.net [IPv6:2001:470:1f11:617::107]) by mx1.freebsd.org (Postfix) with ESMTP id 622EA80BCA; Fri, 5 Jan 2018 22:02:15 +0000 (UTC) (envelope-from eric@metricspace.net) Received: from [192.168.43.57] (mobile-166-171-187-244.mycingular.net [166.171.187.244]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: eric) by mail.metricspace.net (Postfix) with ESMTPSA id 944EC8A2A; Fri, 5 Jan 2018 22:02:12 +0000 (UTC) Subject: Fwd: A more general possible meltdown/spectre countermeasure References: To: "freebsd-hackers@freebsd.org" , "freebsd-arch@freebsd.org" From: Eric McCorkle X-Forwarded-Message-Id: Message-ID: <33bcd281-4018-7075-1775-4dfcd58e5a48@metricspace.net> Date: Fri, 5 Jan 2018 17:02:11 -0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.5.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jan 2018 22:02:15 -0000 Re-posting to -hackers and -arch. I'm going to start working on something like this over the weekend. -------- Forwarded Message -------- Subject: A more general possible meltdown/spectre countermeasure Date: Thu, 4 Jan 2018 23:05:40 -0500 From: Eric McCorkle To: freebsd-security@freebsd.org I've thought more about how to deal with meltdown/spectre, and I have an idea I'd like to put forward. However, I'm still in something of a panic mode, so I'm not certain as to its effectiveness. Needless to say, I welcome any feedback on this, and I may be completely off-base. I'm calling this a "countermeasure" as opposed to a "mitigation", as it's something that requires modification of code as opposed to a drop-in patch. == Summary == Provide a kernel and userland API by which memory allocation can be done with extended attributes. In userland, this could be accomplished by extending MMAP flags, and I could imagine a malloc-with-attributes flag. In kernel space, this must already exist, as drivers need to allocate memory with various MTRR-type attributes set. The immediate aim here is to store sensitive information that must remain memory-resident in non-cacheable memory locations (or, if more effective attribute combinations exist, using those instead). See the rationale for the argument why this should work. Assuming the rationale holds, then the attack surface should be greatly reduced. Attackers would need to grab sensitive data out of stack frames or similar locations if/when it gets copied there for faster use. Moreover, if this is done right, it could dovetail nicely into a framework for storing and processing sensitive assets in more secure hardware[0] (like smart cards, the FPGAs I posted earlier, or other options). The obvious downside is that you take a performance hit storing things in non-cacheable locations, especially if you plan on doing heavy computation in that memory (say, encryption/decryption). However, this is almost certainly going to be less than the projected 30-50% performance hit from other mitigations. Also, this technique should work against spectre as well as meltdown (assuming the rationale holds). The second downside is that you have to modify code for this to work, and you have to be careful not to keep copies of sensitive information around too long (this gets tricky in userland, where you might get interrupted and switched out). [0]: Full disclosure, enabling open hardware implementations of this kind of thing is something of an agenda of mine. == Rationale == (Again, I'm tired, rushed, and somewhat panicked so my logic could be faulty at any point, so please point it out if it is) The rationale for why this should work relies on assumptions about out-of-order pipelines that cannot be guaranteed to hold, but are extremely likely to be true. As background, these attacks depend on out-of-order execution performing operations that end up affecting cache and branch-prediction state, ultimately storing information about sensitive data in these side-channels before the fault conditions are detected and acted upon. I'll borrow terminology from the paper, using "transient instructions" to refer to speculatively executed instructions that will eventually be cancelled by a fault. These attacks depend entirely on transient instructions being able to get sensitive information into the processor core and then perform some kind of instruction on them before the fault condition cancels them. Therefore, anything that prevents them from doing this *should* counter the attack. If the actual sensitive data never makes it to the core before the fault is detected, the dependent memory accesses/branches never get executed and the data never makes it to the side-channels. Another assumption here is that CPU architects are going to want to squash faulted instructions ASAP and stop issuing along those speculative branches, so as to reclaim execution units. So I'm assuming once a fault comes back from address translation, then transient execution stops dead. Now, break down the cases for whether the address containing sensitive data is in cache and TLB or not. (I'm assuming here that caches are virtually-indexed, which enables cache lookups to bypass address translation.) * In cache, in TLB: You end up basically racing between the cache and TLB, which will very likely end up detecting the fault before the data arrives, but at the very worst, you get one or two cycles of transient instruction execution before the fault. * In cache, not in TLB: Virtually-indexed tagged means you get a cache lookup racing a page-table walk. The cache lookup beats the page table walk by potentially hundreds (maybe thousands) of cycles, giving you a bunch of transient instructions before a fault gets triggered. This is the main attack case. * Not in cache, in TLB: Memory access requires address translation, which comes back almost immediately as a fault. * Not in cache, not in TLB: You have to do a page table walk before you can fetch the location, as you have to go out to physical memory (and therefore need a physical address). The page table walk will come back with a fault, stopping the attack. So, unless I'm missing something here, both non-cached cases defeat the meltdown attack, as you *cannot* get the data unless you do address translation first (and therefore detect faults). As for why this defeats the spectre attack, the logic is similar: you've jumped into someone else's executable code, hoping to scoop up enough information into your branch predictor before the fault kicks you out. However, to capture anything about sensitive information in your side-channels, the transient instructions need to actually get it into the core before a fault gets detected. The same case analysis as above applies, so you never actually get the sensitive info into the core before a fault comes back and you get squashed. [1]: A physically-indexed cache would be largely immune to this attack, as you'd have to do address translation before doing a cache lookup. I have some ideas that can build on this, but I'd like to get some feedback first. _______________________________________________ freebsd-security@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-security To unsubscribe, send any mail to "freebsd-security-unsubscribe@freebsd.org" From owner-freebsd-hackers@freebsd.org Fri Jan 5 22:10:29 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 385D2EC3AF2 for ; Fri, 5 Jan 2018 22:10:29 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-io0-x22a.google.com (mail-io0-x22a.google.com [IPv6:2607:f8b0:4001:c06::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0BDEB1147 for ; Fri, 5 Jan 2018 22:10:28 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-io0-x22a.google.com with SMTP id t63so7234869iod.0 for ; Fri, 05 Jan 2018 14:10:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=jci+Z9Xe8awaerfggkkSkVqYjBtP7tk6M6Xctaj+xa8=; b=tEUd4qLuvqDE18qbUAndFd/UJbs+9ElRtkDhAW03Waa72e9NNDtzoud7r2QujU8IZw XoIriXE4Nf3yzsuO+W0sw5CLkMmIRQlwR4DXPwkNLw17jpwL/x0xUmUb2LmH1X43w6Qc yILyXS+ZZouUwuF1ZRiSjkhLXZd/a1J7V8s8dSlqbekpfljh+QGqGIy+B7+5VFBu/z1W jeXw4dkF0ldUPPpflN1GLGwr8yaFazq6acW8OIaqhEGot8ifhrlnSik/oPtVJEndgCPZ vzNnOhMHWsvPK35YfhytYq6CR8jPvf8Yjm4KjTSE54VptFb84mgXYKVv4cdBqRdKWS3Z NThA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=jci+Z9Xe8awaerfggkkSkVqYjBtP7tk6M6Xctaj+xa8=; b=msyjb2/iTDT2ju2W3lvl5kfAaf+JrAxveAev6zTicMlmAFa1Ciukvt2WXacOTGXWwT nOfTk+Hr5Xvz3qbtlIQuY2fN+gjheTutuqnmmJDGb2s4407TYy4laX3JrvTEfEO/ca3p a/uw7Q8ee1Np6+9Git21kvgyQFJTVLSgguvsqBhCoVGHpOmgoc0XYOhVh2p0UonLV5lS Sfctp07Rwv0bmGIbdPveh6kz8WWB5TMCI++z4IhRGbNEdNep+5vuuTMtRVvz31uYnGGS nVLiTBKkt92DY93z74YWOiVGiFLZeoV7O4XNQs5YrzcLo3NUVYzauRNcAp/TeVL3q0uS +niA== X-Gm-Message-State: AKwxytfsSx2TIcNiRnpybE9WfO7DNMaB54p5Y9GVJwFmGYjO7BHBecNO qKDwpVSZ+2sQQnY/jb2llGbQasNNw+t9vgl1A7hNHA== X-Google-Smtp-Source: ACJfBotloPRlXMoKAWC+qNeHs2vWdnYqKBHbANltsOo6Ymgt9q4iQG5LVcYj3Zxk9O2j2tzmzS9KnHqJH5XAZdr/X1o= X-Received: by 10.107.78.12 with SMTP id c12mr4360284iob.63.1515190228036; Fri, 05 Jan 2018 14:10:28 -0800 (PST) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 10.79.160.217 with HTTP; Fri, 5 Jan 2018 14:10:27 -0800 (PST) X-Originating-IP: [2603:300b:6:5100:1052:acc7:f9de:2b6d] In-Reply-To: <33bcd281-4018-7075-1775-4dfcd58e5a48@metricspace.net> References: <33bcd281-4018-7075-1775-4dfcd58e5a48@metricspace.net> From: Warner Losh Date: Fri, 5 Jan 2018 15:10:27 -0700 X-Google-Sender-Auth: gx4T-5845RC6oqVpg5Hx_fwx1ls Message-ID: Subject: Re: A more general possible meltdown/spectre countermeasure To: Eric McCorkle Cc: "freebsd-hackers@freebsd.org" , "freebsd-arch@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jan 2018 22:10:29 -0000 I think this is fatally flawed. The side channel is the cache. Not the data at risk. Any mapped memory, cached or not, can be used to influence the cache. Storing stuff in uncached memory won't affect the side channel one bit. Basically, all attacks boil down to tricking the processor, at elevated privs, to doing something like a = foo[offset]; where foo + offset are designed to communicate information by populating a cache line. offset need not be cached itself and can be the result of simple computations that depend on anything accessible at all in the kernel. Warner On Fri, Jan 5, 2018 at 3:02 PM, Eric McCorkle wrote: > Re-posting to -hackers and -arch. I'm going to start working on > something like this over the weekend. > > -------- Forwarded Message -------- > Subject: A more general possible meltdown/spectre countermeasure > Date: Thu, 4 Jan 2018 23:05:40 -0500 > From: Eric McCorkle > To: freebsd-security@freebsd.org > > I've thought more about how to deal with meltdown/spectre, and I have an > idea I'd like to put forward. However, I'm still in something of a > panic mode, so I'm not certain as to its effectiveness. Needless to > say, I welcome any feedback on this, and I may be completely off-base. > > I'm calling this a "countermeasure" as opposed to a "mitigation", as > it's something that requires modification of code as opposed to a > drop-in patch. > > == Summary == > > Provide a kernel and userland API by which memory allocation can be done > with extended attributes. In userland, this could be accomplished by > extending MMAP flags, and I could imagine a malloc-with-attributes flag. > In kernel space, this must already exist, as drivers need to allocate > memory with various MTRR-type attributes set. > > The immediate aim here is to store sensitive information that must > remain memory-resident in non-cacheable memory locations (or, if more > effective attribute combinations exist, using those instead). See the > rationale for the argument why this should work. > > Assuming the rationale holds, then the attack surface should be greatly > reduced. Attackers would need to grab sensitive data out of stack > frames or similar locations if/when it gets copied there for faster use. > Moreover, if this is done right, it could dovetail nicely into a > framework for storing and processing sensitive assets in more secure > hardware[0] (like smart cards, the FPGAs I posted earlier, or other > options). > > The obvious downside is that you take a performance hit storing things > in non-cacheable locations, especially if you plan on doing heavy > computation in that memory (say, encryption/decryption). However, this > is almost certainly going to be less than the projected 30-50% > performance hit from other mitigations. Also, this technique should > work against spectre as well as meltdown (assuming the rationale holds). > > The second downside is that you have to modify code for this to work, > and you have to be careful not to keep copies of sensitive information > around too long (this gets tricky in userland, where you might get > interrupted and switched out). > > > [0]: Full disclosure, enabling open hardware implementations of this > kind of thing is something of an agenda of mine. > > == Rationale == > > (Again, I'm tired, rushed, and somewhat panicked so my logic could be > faulty at any point, so please point it out if it is) > > The rationale for why this should work relies on assumptions about > out-of-order pipelines that cannot be guaranteed to hold, but are > extremely likely to be true. > > As background, these attacks depend on out-of-order execution performing > operations that end up affecting cache and branch-prediction state, > ultimately storing information about sensitive data in these > side-channels before the fault conditions are detected and acted upon. > I'll borrow terminology from the paper, using "transient instructions" > to refer to speculatively executed instructions that will eventually be > cancelled by a fault. > > These attacks depend entirely on transient instructions being able to > get sensitive information into the processor core and then perform some > kind of instruction on them before the fault condition cancels them. > Therefore, anything that prevents them from doing this *should* counter > the attack. If the actual sensitive data never makes it to the core > before the fault is detected, the dependent memory accesses/branches > never get executed and the data never makes it to the side-channels. > > Another assumption here is that CPU architects are going to want to > squash faulted instructions ASAP and stop issuing along those > speculative branches, so as to reclaim execution units. So I'm assuming > once a fault comes back from address translation, then transient > execution stops dead. > > Now, break down the cases for whether the address containing sensitive > data is in cache and TLB or not. (I'm assuming here that caches are > virtually-indexed, which enables cache lookups to bypass address > translation.) > > * In cache, in TLB: You end up basically racing between the cache and > TLB, which will very likely end up detecting the fault before the data > arrives, but at the very worst, you get one or two cycles of transient > instruction execution before the fault. > > * In cache, not in TLB: Virtually-indexed tagged means you get a cache > lookup racing a page-table walk. The cache lookup beats the page table > walk by potentially hundreds (maybe thousands) of cycles, giving you a > bunch of transient instructions before a fault gets triggered. This is > the main attack case. > > * Not in cache, in TLB: Memory access requires address translation, > which comes back almost immediately as a fault. > > * Not in cache, not in TLB: You have to do a page table walk before you > can fetch the location, as you have to go out to physical memory (and > therefore need a physical address). The page table walk will come back > with a fault, stopping the attack. > > So, unless I'm missing something here, both non-cached cases defeat the > meltdown attack, as you *cannot* get the data unless you do address > translation first (and therefore detect faults). > > As for why this defeats the spectre attack, the logic is similar: you've > jumped into someone else's executable code, hoping to scoop up enough > information into your branch predictor before the fault kicks you out. > However, to capture anything about sensitive information in your > side-channels, the transient instructions need to actually get it into > the core before a fault gets detected. The same case analysis as above > applies, so you never actually get the sensitive info into the core > before a fault comes back and you get squashed. > > > [1]: A physically-indexed cache would be largely immune to this attack, > as you'd have to do address translation before doing a cache lookup. > > > I have some ideas that can build on this, but I'd like to get some > feedback first. > _______________________________________________ > freebsd-security@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-security > To unsubscribe, send any mail to "freebsd-security-unsubscribe@freebsd.org > " > _______________________________________________ > freebsd-arch@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > From owner-freebsd-hackers@freebsd.org Fri Jan 5 22:13:37 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C1860EC3E69 for ; Fri, 5 Jan 2018 22:13:37 +0000 (UTC) (envelope-from brooks@spindle.one-eyed-alien.net) Received: from spindle.one-eyed-alien.net (spindle.one-eyed-alien.net [199.48.129.229]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8F476169E; Fri, 5 Jan 2018 22:13:37 +0000 (UTC) (envelope-from brooks@spindle.one-eyed-alien.net) Received: by spindle.one-eyed-alien.net (Postfix, from userid 3001) id 4894E5A9F14; Fri, 5 Jan 2018 22:13:30 +0000 (UTC) Date: Fri, 5 Jan 2018 22:13:30 +0000 From: Brooks Davis To: Alan Somers Cc: Ian Lepore , Yuri , Freebsd hackers list Subject: Re: Is it considered to be ok to not check the return code of close(2) in base? Message-ID: <20180105221330.GD95035@spindle.one-eyed-alien.net> References: <24acbd94-c52f-e71a-8a96-d608a10963c6@rawbw.com> <1514572041.12000.7.camel@freebsd.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="+nBD6E3TurpgldQp" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.9.1 (2017-09-22) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jan 2018 22:13:37 -0000 --+nBD6E3TurpgldQp Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Dec 29, 2017 at 11:48:43AM -0700, Alan Somers wrote: > On Fri, Dec 29, 2017 at 11:27 AM, Ian Lepore wrote: >=20 > > On Fri, 2017-12-29 at 10:19 -0800, Yuri wrote: > > > Some base utilities sometimes close files that they open for their > > > purposes without checking the error code of close(2). > > > > > > Is this considered to be ok, because it's just a close call and we > > > are > > > done with that file descriptor, or is it considered to be more > > > appropriate to check close's error code? > > > > > > Maybe there is some policy that covers this? > > > > > > IMO, every system call's return value should be checked, just in > > > case. > > > > > > > > > Yuri > > > > > > > There's really no point in checking on a close from a file opened only > > for reading. You can argue it should be checked on a file open for > > writing, but often isn't because you're then confronted with the > > question "what should/can I do if there is an error?" If you report > > the error and exit, then what about other files that were open at the > > time? They're going to be closed by the kernel as part of process > > cleanup, with no error checking or reporting. > > > > Also, with the async nature of filesystems, IO errors can still happen > > after the close, unless fsync() was used. So if you're going to miss > > most of the errors because of that, why bother to check at all? >=20 > I would argue the opposite. There are very few reasons why close(s) would > ever fail, and the most likely is EBADF. EBADF indicates a programming > bug, like a double close or use of an uninitialized variable. Those could > easily turn into worse bugs in the future. So I think the best course of > action is to check the return code, assert() on EBADF, and ignore, or > possibly log, other errors. For this specific case, I think there would be value in an option to have the kernel kill any process that calls close(fd) where fd !=3D -1 where EBADF would be returned. -- Brooks --+nBD6E3TurpgldQp Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEcBAEBAgAGBQJaT/iJAAoJEKzQXbSebgfA7H0H/15jVVj8i+VQiKgNArRqgSXN aYWjS+ED53M9Vzca/1/clCmApw0qujYCDzhcKOQ7icIw0hnRYcB2pDTWoQDQ/NDd UC8y/PsLE2sSxHBefsCJUb2RX72BpVgiuW5ZrcCjZP6hm2ovxRAVbuMPbVWDMSa7 lID8x+kLT+8pE1Dq6zRwJPfWxZZnvwGZVROSt0rReXCjvreVid59YYbXYYfobHk2 /DqeFiCb7YgsC1YTyS3BhFnFUtYF+R5ZKLSKEAEAFWM1SAKepb8a/u09kpSry7R2 9K5dBHymjk4LbgwZ1MatrFpbEJkjwWmHaGVj8k1cyi0KqlSNxYx/UViH7mewV+w= =bbht -----END PGP SIGNATURE----- --+nBD6E3TurpgldQp-- From owner-freebsd-hackers@freebsd.org Fri Jan 5 22:15:30 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 91986EC4059; Fri, 5 Jan 2018 22:15:30 +0000 (UTC) (envelope-from eric@metricspace.net) Received: from mail.metricspace.net (mail.metricspace.net [IPv6:2001:470:1f11:617::107]) by mx1.freebsd.org (Postfix) with ESMTP id 48089190F; Fri, 5 Jan 2018 22:15:30 +0000 (UTC) (envelope-from eric@metricspace.net) Received: from [192.168.43.57] (mobile-166-171-187-244.mycingular.net [166.171.187.244]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: eric) by mail.metricspace.net (Postfix) with ESMTPSA id 01C848A55; Fri, 5 Jan 2018 22:15:27 +0000 (UTC) Subject: Re: A more general possible meltdown/spectre countermeasure To: Warner Losh Cc: "freebsd-hackers@freebsd.org" , "freebsd-arch@freebsd.org" References: <33bcd281-4018-7075-1775-4dfcd58e5a48@metricspace.net> From: Eric McCorkle Message-ID: <4ec1f3b1-f4b0-80ab-0e68-0dd679dd9e37@metricspace.net> Date: Fri, 5 Jan 2018 17:15:26 -0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.5.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jan 2018 22:15:30 -0000 Right, but you have to get the value "foo" into the pipeline in order for it to affect the side-channels. This technique attempts to stop that from happening. Unless I made a mistake, non-cached memory reads force address translation to happen first, which detects faults and blocks the meltdown attack. It also stops spectre with very high probability, as it's very unlikely that an uncached load will arrive before the speculative thread gets squashed. On 01/05/2018 17:10, Warner Losh wrote: > I think this is fatally flawed. > > The side channel is the cache. Not the data at risk. > > Any mapped memory, cached or not, can be used to influence the cache. > Storing stuff in uncached memory won't affect the side channel one bit. > > Basically, all attacks boil down to tricking the processor, at elevated > privs, to doing something like > > a = foo[offset]; > > where foo + offset are designed to communicate information by populating > a cache line. offset need not be cached itself and can be the result of > simple computations that depend on anything accessible at all in the kernel. > > Warner > > On Fri, Jan 5, 2018 at 3:02 PM, Eric McCorkle > wrote: > > Re-posting to -hackers and -arch.  I'm going to start working on > something like this over the weekend. > > -------- Forwarded Message -------- > Subject: A more general possible meltdown/spectre countermeasure > Date: Thu, 4 Jan 2018 23:05:40 -0500 > From: Eric McCorkle > > To: freebsd-security@freebsd.org > > > > I've thought more about how to deal with meltdown/spectre, and I have an > idea I'd like to put forward.  However, I'm still in something of a > panic mode, so I'm not certain as to its effectiveness.  Needless to > say, I welcome any feedback on this, and I may be completely off-base. > > I'm calling this a "countermeasure" as opposed to a "mitigation", as > it's something that requires modification of code as opposed to a > drop-in patch. > > == Summary == > > Provide a kernel and userland API by which memory allocation can be done > with extended attributes.  In userland, this could be accomplished by > extending MMAP flags, and I could imagine a malloc-with-attributes flag. >  In kernel space, this must already exist, as drivers need to allocate > memory with various MTRR-type attributes set. > > The immediate aim here is to store sensitive information that must > remain memory-resident in non-cacheable memory locations (or, if more > effective attribute combinations exist, using those instead).  See the > rationale for the argument why this should work. > > Assuming the rationale holds, then the attack surface should be greatly > reduced.  Attackers would need to grab sensitive data out of stack > frames or similar locations if/when it gets copied there for faster use. >  Moreover, if this is done right, it could dovetail nicely into a > framework for storing and processing sensitive assets in more secure > hardware[0] (like smart cards, the FPGAs I posted earlier, or other > options). > > The obvious downside is that you take a performance hit storing things > in non-cacheable locations, especially if you plan on doing heavy > computation in that memory (say, encryption/decryption).  However, this > is almost certainly going to be less than the projected 30-50% > performance hit from other mitigations.  Also, this technique should > work against spectre as well as meltdown (assuming the rationale holds). > > The second downside is that you have to modify code for this to work, > and you have to be careful not to keep copies of sensitive information > around too long (this gets tricky in userland, where you might get > interrupted and switched out). > > > [0]: Full disclosure, enabling open hardware implementations of this > kind of thing is something of an agenda of mine. > > == Rationale == > > (Again, I'm tired, rushed, and somewhat panicked so my logic could be > faulty at any point, so please point it out if it is) > > The rationale for why this should work relies on assumptions about > out-of-order pipelines that cannot be guaranteed to hold, but are > extremely likely to be true. > > As background, these attacks depend on out-of-order execution performing > operations that end up affecting cache and branch-prediction state, > ultimately storing information about sensitive data in these > side-channels before the fault conditions are detected and acted upon. > I'll borrow terminology from the paper, using "transient instructions" > to refer to speculatively executed instructions that will eventually be > cancelled by a fault. > > These attacks depend entirely on transient instructions being able to > get sensitive information into the processor core and then perform some > kind of instruction on them before the fault condition cancels them. > Therefore, anything that prevents them from doing this *should* counter > the attack.  If the actual sensitive data never makes it to the core > before the fault is detected, the dependent memory accesses/branches > never get executed and the data never makes it to the side-channels. > > Another assumption here is that CPU architects are going to want to > squash faulted instructions ASAP and stop issuing along those > speculative branches, so as to reclaim execution units.  So I'm assuming > once a fault comes back from address translation, then transient > execution stops dead. > > Now, break down the cases for whether the address containing sensitive > data is in cache and TLB or not.  (I'm assuming here that caches are > virtually-indexed, which enables cache lookups to bypass address > translation.) > > * In cache, in TLB: You end up basically racing between the cache and > TLB, which will very likely end up detecting the fault before the data > arrives, but at the very worst, you get one or two cycles of transient > instruction execution before the fault. > > * In cache, not in TLB: Virtually-indexed tagged means you get a cache > lookup racing a page-table walk.  The cache lookup beats the page table > walk by potentially hundreds (maybe thousands) of cycles, giving you a > bunch of transient instructions before a fault gets triggered.  This is > the main attack case. > > * Not in cache, in TLB: Memory access requires address translation, > which comes back almost immediately as a fault. > > * Not in cache, not in TLB: You have to do a page table walk before you > can fetch the location, as you have to go out to physical memory (and > therefore need a physical address).  The page table walk will come back > with a fault, stopping the attack. > > So, unless I'm missing something here, both non-cached cases defeat the > meltdown attack, as you *cannot* get the data unless you do address > translation first (and therefore detect faults). > > As for why this defeats the spectre attack, the logic is similar: you've > jumped into someone else's executable code, hoping to scoop up enough > information into your branch predictor before the fault kicks you out. > However, to capture anything about sensitive information in your > side-channels, the transient instructions need to actually get it into > the core before a fault gets detected.  The same case analysis as above > applies, so you never actually get the sensitive info into the core > before a fault comes back and you get squashed. > > > [1]: A physically-indexed cache would be largely immune to this attack, > as you'd have to do address translation before doing a cache lookup. > > > I have some ideas that can build on this, but I'd like to get some > feedback first. > _______________________________________________ > freebsd-security@freebsd.org > mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-security > > To unsubscribe, send any mail to > "freebsd-security-unsubscribe@freebsd.org > " > _______________________________________________ > freebsd-arch@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-arch > > To unsubscribe, send any mail to > "freebsd-arch-unsubscribe@freebsd.org > " > > From owner-freebsd-hackers@freebsd.org Fri Jan 5 22:18:15 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C3923EC42B1 for ; Fri, 5 Jan 2018 22:18:15 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [78.47.246.247]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 6B6021C83; Fri, 5 Jan 2018 22:18:14 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (root@eg.sd.rdtc.ru [62.231.161.221] (may be forged)) by hz.grosbein.net (8.15.2/8.15.2) with ESMTPS id w05MHxRn041958 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 5 Jan 2018 23:18:00 +0100 (CET) (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: brooks@freebsd.org Received: from [10.58.0.4] ([10.58.0.4]) by eg.sd.rdtc.ru (8.15.2/8.15.2) with ESMTPS id w05MHtNE019435 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Sat, 6 Jan 2018 05:17:55 +0700 (+07) (envelope-from eugen@grosbein.net) Subject: Re: Is it considered to be ok to not check the return code of close(2) in base? To: Brooks Davis , Alan Somers References: <24acbd94-c52f-e71a-8a96-d608a10963c6@rawbw.com> <1514572041.12000.7.camel@freebsd.org> <20180105221330.GD95035@spindle.one-eyed-alien.net> Cc: Yuri , Ian Lepore , Freebsd hackers list From: Eugene Grosbein Message-ID: <5A4FF989.1040709@grosbein.net> Date: Sat, 6 Jan 2018 05:17:45 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.7.2 MIME-Version: 1.0 In-Reply-To: <20180105221330.GD95035@spindle.one-eyed-alien.net> Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="WaX0PKN5PUEovsUibQhwjaF1BhrNSms3T" X-Spam-Status: No, score=2.2 required=5.0 tests=BAYES_00, LOCAL_FROM, RDNS_NONE autolearn=no autolearn_force=no version=3.4.1 X-Spam-Report: * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 2.6 LOCAL_FROM From my domains * 1.9 RDNS_NONE Delivered to internal network by a host with no rDNS X-Spam-Level: ** X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on hz.grosbein.net X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jan 2018 22:18:15 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --WaX0PKN5PUEovsUibQhwjaF1BhrNSms3T Content-Type: multipart/mixed; boundary="9s0LvRWHs38dRhcArdcFTcnu7qtmMJMoq"; protected-headers="v1" From: Eugene Grosbein To: Brooks Davis , Alan Somers Cc: Yuri , Ian Lepore , Freebsd hackers list Message-ID: <5A4FF989.1040709@grosbein.net> Subject: Re: Is it considered to be ok to not check the return code of close(2) in base? References: <24acbd94-c52f-e71a-8a96-d608a10963c6@rawbw.com> <1514572041.12000.7.camel@freebsd.org> <20180105221330.GD95035@spindle.one-eyed-alien.net> In-Reply-To: <20180105221330.GD95035@spindle.one-eyed-alien.net> --9s0LvRWHs38dRhcArdcFTcnu7qtmMJMoq Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable 06.01.2018 5:13, Brooks Davis wrote: >> I would argue the opposite. There are very few reasons why close(s) w= ould >> ever fail, and the most likely is EBADF. EBADF indicates a programmin= g >> bug, like a double close or use of an uninitialized variable. Those c= ould >> easily turn into worse bugs in the future. So I think the best course= of >> action is to check the return code, assert() on EBADF, and ignore, or >> possibly log, other errors. >=20 > For this specific case, I think there would be value in an option to > have the kernel kill any process that calls close(fd) where fd !=3D -1 > where EBADF would be returned. A medicine should not be worse worse than the disease, imho. --9s0LvRWHs38dRhcArdcFTcnu7qtmMJMoq-- --WaX0PKN5PUEovsUibQhwjaF1BhrNSms3T Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQEcBAEBAgAGBQJaT/mPAAoJELDNGvImmIsoJo0IAIqmmum8PyLuAvK7B9HFVu1c hQxA+avmB4NwqeyZhe1U6i+IqcGvIM1zrVkCje7eAqBbHEbSO7i5zmvI11warxPx 2JFQ1oayQQ26xILPgOb7kwdsIMb4RuTWa3F7EU2kszO8o3C8K1EpPTuLhpD3T2eU +LWOchaZJrMP7OSBo/b4OmyUKdmqv35674x8qR6ApgViFxABblW42vlN9wuvlPl0 30/jFG8hXFH3lq4sv9RGzKcU5u8Jyl6JtmucsRsYx5HFOhgj0SXWCkpkZ0FWEGLF TeBTDgxSPBWxqMEzzRKeoNhFH9prPB5m0kwlyMwCGckdBlSYSAStYAQyqzu3c+w= =ezxP -----END PGP SIGNATURE----- --WaX0PKN5PUEovsUibQhwjaF1BhrNSms3T-- From owner-freebsd-hackers@freebsd.org Fri Jan 5 21:01:59 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 04A41EC07A4; Fri, 5 Jan 2018 21:01:59 +0000 (UTC) (envelope-from amvandemore@gmail.com) Received: from mail-it0-x22c.google.com (mail-it0-x22c.google.com [IPv6:2607:f8b0:4001:c0b::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B9B2C7E572; Fri, 5 Jan 2018 21:01:58 +0000 (UTC) (envelope-from amvandemore@gmail.com) Received: by mail-it0-x22c.google.com with SMTP id m11so5617286iti.1; Fri, 05 Jan 2018 13:01:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=XgGGWIXJEyUixN2cD60ywZtybr4KTK3Y8243ZQ53278=; b=TnQt2LdirHwWS48ve8NiXhPwBgme9ZxKaV8Z2csqjCA1S0kQLQf3B11nwa9oNsMlFX M/xCknHXQzVVO+h09ZJazmiNsvS8EqenX9vPgHnjY/zMFeda54dVXTg4ljzjlJqeWGqk j9FnOkxaCgtZRAU35CUODWdzVm+t4pyVFekp2mnkPB2Y7lo53g1g9yyXexjrYdsy+B3N Z9hcu2Itmrm/NHlGyVnqvwbbAarb1qoAW1wM3AREWjAQeNe50ty3ARGDzDGL07WvOjV+ MhMvBw0ux4i71Iv2Xur9J/ujNn44+gyO82JTw+nA2cj/j67rvA5tBtS+ErNdioC9oTXa Rc+Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=XgGGWIXJEyUixN2cD60ywZtybr4KTK3Y8243ZQ53278=; b=J7vTH+LsX0R6v3m4sUaCn2rsGY1BWcbxpWYuHIh7RU9Sux/VEOhOcRfmZ/eFq8y0gD MClD3gxGMrxq5A5gvjAQ2EhK3LQKGrK7dOm+EOGH9D9zHfuG2rEMajovM2vHt8nRHrsm +dZveW7xioMYGKREA7GB+ougut2G0jemIBuMABvA/Lq0aJFB0JvEs0A7Fz+OQjn2vcxf 5DPZ6oQErZZlJnl1IHn5EcsVTQDdws/Cq7hcsQK0hwgANfKZb62YbDycPhgabMptCEUu Z0EzdH3xkROzSYrf3+Pr0E8DZq1wreokZbY8BUlX8EL4SxuycBLDNDHOUxl1lqhO+E4f kzXw== X-Gm-Message-State: AKGB3mJyNfRh4UqwzZY2c0lld2X/x/iPSVD9HQGqzprTHssEVFz8J5Ja aqeSY96BSTHCCObVmrAavS8tKcStDHGHeYlw0lRxww== X-Google-Smtp-Source: ACJfBov5EyZIUpx+1q3qOpBJGKLLD3n4O2v6Y8BjJMVCJ5kRxNgEukReHCxElPjxPClWUwCA9KTdiwWbeGxA0f8/Ek8= X-Received: by 10.36.138.134 with SMTP id v128mr4604359itd.153.1515186117757; Fri, 05 Jan 2018 13:01:57 -0800 (PST) MIME-Version: 1.0 Received: by 10.2.180.34 with HTTP; Fri, 5 Jan 2018 13:01:57 -0800 (PST) In-Reply-To: References: <20180105191145.404BC335@spqr.komquats.com> From: Adam Vande More Date: Fri, 5 Jan 2018 15:01:57 -0600 Message-ID: Subject: Re: Intel hardware bug To: "K. Macy" Cc: Cy Schubert , Eric McCorkle , FreeBSD Hackers , "freebsd-arch@freebsd.org" , Shawn Webb , Freebsd Security , Poul-Henning Kamp , Nathan Whitehorn , "Ronald F. Guilmette" , =?UTF-8?Q?Dag=2DErling_Sm=C3=B8rgrav?= , Brett Glass , Jules Gilbert X-Mailman-Approved-At: Fri, 05 Jan 2018 22:19:14 +0000 Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jan 2018 21:01:59 -0000 On Fri, Jan 5, 2018 at 1:37 PM, K. Macy wrote: > On Fri, Jan 5, 2018 at 11:11 AM, Cy Schubert > wrote: > > According to a Red Hat announcement, Power and Series z are also > vulnerable. > > > > Link? > https://access.redhat.com/security/vulnerabilities/speculativeexecution -- Adam From owner-freebsd-hackers@freebsd.org Fri Jan 5 22:22:23 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 15932EC483F for ; Fri, 5 Jan 2018 22:22:23 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-io0-x229.google.com (mail-io0-x229.google.com [IPv6:2607:f8b0:4001:c06::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C4B7E22CE for ; Fri, 5 Jan 2018 22:22:22 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-io0-x229.google.com with SMTP id f6so7236701ioh.8 for ; Fri, 05 Jan 2018 14:22:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=wvzaGdMzjJqu1Pqz+bsTMZ5lBVm4cRDLEetf2um4frc=; b=C4k+vKT4i1WxIG6rZL2Duj5jkuzy2agePNJKm+qDnsidA48B7MtvrDPNpL5EMwGUtN T5ClgGyQ6MrRf3/Jdzi8LiinrBs/bVVKbaYH3HJ9fOF1CdYFI5zO98qVgJ2hsSrbpAd2 YDlrdJkOkxpVHyAHxlyWeN2M7IgFRgn1O099DgOAnHoOLCHxzLCfMwk8cl8A0eJ3eA6v Tj+RTt6BT2yPXVOMqZ5lZpL6QgdmvZQbPW++hVxLgJmkEf5yfnjbmHNLdcjqgsqOVlG5 nyb5tSI6/kjCaPdL6FGhb5MhSjdR7MeF0fmg4Uwv0glie/Vb0ZHce5+0V2k8xdB4ipP+ Dszw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=wvzaGdMzjJqu1Pqz+bsTMZ5lBVm4cRDLEetf2um4frc=; b=fbmgJiuGwRwwrjqmeRP4vT8D/k6tTRJxXV/rKbiBOsMXVF9iU7hkwRofDtGqfDvYxw rJNmJnQnwg7JPN3SbEgC4C4suboSkbx/6b+qc8H421VBMit8sEXnD8f60F0N/70XC5pf u3UZNRbkWMcuCwHmiiyLd39OU8FpnHI/nGa/X3HXNZo5M1bBNy+515PWxOI4dG2Hd38g FuGoJWhUo5I7dpqNiQwRbrvK5QKG/XgVtS14CA4x560/RHVIdnx+Z9d6q0dbHEvN064j ShHx/LZKTRsG6XZ4xnqAto8JqKbTy2np0RPxnHsnqVrtUIf8hpJCLlmLXx3jtLBEM1lm fJ9w== X-Gm-Message-State: AKwxytfH32Ea5Usejx2s18H7XkyhTfRXrZAMovYEzdTDkzlksc2hSTFv 0VSHJ1YId3eVEQe9y6c4N+pbHUQom9bNyNbYkyoOiw== X-Google-Smtp-Source: ACJfBotWWLMZX6peHiF5E7bcio4qPyN8zHJt2SXwK+tH6TWCAjL9bEIDuJVn7Pyzvy4/dn0aD50I6TzXFT4sqp911qg= X-Received: by 10.107.78.12 with SMTP id c12mr4387650iob.63.1515190942020; Fri, 05 Jan 2018 14:22:22 -0800 (PST) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 10.79.160.217 with HTTP; Fri, 5 Jan 2018 14:22:21 -0800 (PST) X-Originating-IP: [2603:300b:6:5100:1052:acc7:f9de:2b6d] In-Reply-To: <4ec1f3b1-f4b0-80ab-0e68-0dd679dd9e37@metricspace.net> References: <33bcd281-4018-7075-1775-4dfcd58e5a48@metricspace.net> <4ec1f3b1-f4b0-80ab-0e68-0dd679dd9e37@metricspace.net> From: Warner Losh Date: Fri, 5 Jan 2018 15:22:21 -0700 X-Google-Sender-Auth: InOhVvYEc7Gg0JlflKXzLo25Ivg Message-ID: Subject: Re: A more general possible meltdown/spectre countermeasure To: Eric McCorkle Cc: "freebsd-hackers@freebsd.org" , "freebsd-arch@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jan 2018 22:22:23 -0000 While you might be right, I've seen no indication that a cache miss would defeat these attacks in the public and non-public data I've looked at, even though a large number of alternatives to the published workarounds have been discussed. I'm therefore somewhat skeptical this would be effective. I'm open, however, to data that changes that skepticism... Warner On Fri, Jan 5, 2018 at 3:15 PM, Eric McCorkle wrote: > Right, but you have to get the value "foo" into the pipeline in order > for it to affect the side-channels. This technique attempts to stop > that from happening. > > Unless I made a mistake, non-cached memory reads force address > translation to happen first, which detects faults and blocks the > meltdown attack. > > It also stops spectre with very high probability, as it's very unlikely > that an uncached load will arrive before the speculative thread gets > squashed. > > On 01/05/2018 17:10, Warner Losh wrote: > > I think this is fatally flawed. > > > > The side channel is the cache. Not the data at risk. > > > > Any mapped memory, cached or not, can be used to influence the cache. > > Storing stuff in uncached memory won't affect the side channel one bit. > > > > Basically, all attacks boil down to tricking the processor, at elevated > > privs, to doing something like > > > > a = foo[offset]; > > > > where foo + offset are designed to communicate information by populating > > a cache line. offset need not be cached itself and can be the result of > > simple computations that depend on anything accessible at all in the > kernel. > > > > Warner > > > > On Fri, Jan 5, 2018 at 3:02 PM, Eric McCorkle > > wrote: > > > > Re-posting to -hackers and -arch. I'm going to start working on > > something like this over the weekend. > > > > -------- Forwarded Message -------- > > Subject: A more general possible meltdown/spectre countermeasure > > Date: Thu, 4 Jan 2018 23:05:40 -0500 > > From: Eric McCorkle eric@metricspace.net>> > > To: freebsd-security@freebsd.org > > > > > > > > I've thought more about how to deal with meltdown/spectre, and I > have an > > idea I'd like to put forward. However, I'm still in something of a > > panic mode, so I'm not certain as to its effectiveness. Needless to > > say, I welcome any feedback on this, and I may be completely > off-base. > > > > I'm calling this a "countermeasure" as opposed to a "mitigation", as > > it's something that requires modification of code as opposed to a > > drop-in patch. > > > > == Summary == > > > > Provide a kernel and userland API by which memory allocation can be > done > > with extended attributes. In userland, this could be accomplished by > > extending MMAP flags, and I could imagine a malloc-with-attributes > flag. > > In kernel space, this must already exist, as drivers need to > allocate > > memory with various MTRR-type attributes set. > > > > The immediate aim here is to store sensitive information that must > > remain memory-resident in non-cacheable memory locations (or, if more > > effective attribute combinations exist, using those instead). See > the > > rationale for the argument why this should work. > > > > Assuming the rationale holds, then the attack surface should be > greatly > > reduced. Attackers would need to grab sensitive data out of stack > > frames or similar locations if/when it gets copied there for faster > use. > > Moreover, if this is done right, it could dovetail nicely into a > > framework for storing and processing sensitive assets in more secure > > hardware[0] (like smart cards, the FPGAs I posted earlier, or other > > options). > > > > The obvious downside is that you take a performance hit storing > things > > in non-cacheable locations, especially if you plan on doing heavy > > computation in that memory (say, encryption/decryption). However, > this > > is almost certainly going to be less than the projected 30-50% > > performance hit from other mitigations. Also, this technique should > > work against spectre as well as meltdown (assuming the rationale > holds). > > > > The second downside is that you have to modify code for this to work, > > and you have to be careful not to keep copies of sensitive > information > > around too long (this gets tricky in userland, where you might get > > interrupted and switched out). > > > > > > [0]: Full disclosure, enabling open hardware implementations of this > > kind of thing is something of an agenda of mine. > > > > == Rationale == > > > > (Again, I'm tired, rushed, and somewhat panicked so my logic could be > > faulty at any point, so please point it out if it is) > > > > The rationale for why this should work relies on assumptions about > > out-of-order pipelines that cannot be guaranteed to hold, but are > > extremely likely to be true. > > > > As background, these attacks depend on out-of-order execution > performing > > operations that end up affecting cache and branch-prediction state, > > ultimately storing information about sensitive data in these > > side-channels before the fault conditions are detected and acted > upon. > > I'll borrow terminology from the paper, using "transient > instructions" > > to refer to speculatively executed instructions that will eventually > be > > cancelled by a fault. > > > > These attacks depend entirely on transient instructions being able to > > get sensitive information into the processor core and then perform > some > > kind of instruction on them before the fault condition cancels them. > > Therefore, anything that prevents them from doing this *should* > counter > > the attack. If the actual sensitive data never makes it to the core > > before the fault is detected, the dependent memory accesses/branches > > never get executed and the data never makes it to the side-channels. > > > > Another assumption here is that CPU architects are going to want to > > squash faulted instructions ASAP and stop issuing along those > > speculative branches, so as to reclaim execution units. So I'm > assuming > > once a fault comes back from address translation, then transient > > execution stops dead. > > > > Now, break down the cases for whether the address containing > sensitive > > data is in cache and TLB or not. (I'm assuming here that caches are > > virtually-indexed, which enables cache lookups to bypass address > > translation.) > > > > * In cache, in TLB: You end up basically racing between the cache and > > TLB, which will very likely end up detecting the fault before the > data > > arrives, but at the very worst, you get one or two cycles of > transient > > instruction execution before the fault. > > > > * In cache, not in TLB: Virtually-indexed tagged means you get a > cache > > lookup racing a page-table walk. The cache lookup beats the page > table > > walk by potentially hundreds (maybe thousands) of cycles, giving you > a > > bunch of transient instructions before a fault gets triggered. This > is > > the main attack case. > > > > * Not in cache, in TLB: Memory access requires address translation, > > which comes back almost immediately as a fault. > > > > * Not in cache, not in TLB: You have to do a page table walk before > you > > can fetch the location, as you have to go out to physical memory (and > > therefore need a physical address). The page table walk will come > back > > with a fault, stopping the attack. > > > > So, unless I'm missing something here, both non-cached cases defeat > the > > meltdown attack, as you *cannot* get the data unless you do address > > translation first (and therefore detect faults). > > > > As for why this defeats the spectre attack, the logic is similar: > you've > > jumped into someone else's executable code, hoping to scoop up enough > > information into your branch predictor before the fault kicks you > out. > > However, to capture anything about sensitive information in your > > side-channels, the transient instructions need to actually get it > into > > the core before a fault gets detected. The same case analysis as > above > > applies, so you never actually get the sensitive info into the core > > before a fault comes back and you get squashed. > > > > > > [1]: A physically-indexed cache would be largely immune to this > attack, > > as you'd have to do address translation before doing a cache lookup. > > > > > > I have some ideas that can build on this, but I'd like to get some > > feedback first. > > _______________________________________________ > > freebsd-security@freebsd.org > > mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-security > > > > To unsubscribe, send any mail to > > "freebsd-security-unsubscribe@freebsd.org > > " > > _______________________________________________ > > freebsd-arch@freebsd.org mailing > list > > https://lists.freebsd.org/mailman/listinfo/freebsd-arch > > > > To unsubscribe, send any mail to > > "freebsd-arch-unsubscribe@freebsd.org > > " > > > > > From owner-freebsd-hackers@freebsd.org Fri Jan 5 22:26:43 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4C677EC4DC8 for ; Fri, 5 Jan 2018 22:26:43 +0000 (UTC) (envelope-from eric@vangyzen.net) Received: from smtp.vangyzen.net (hotblack.vangyzen.net [199.48.133.146]) by mx1.freebsd.org (Postfix) with ESMTP id AA19A2B14; Fri, 5 Jan 2018 22:26:40 +0000 (UTC) (envelope-from eric@vangyzen.net) Received: from sweettea.beer.town (unknown [76.164.8.130]) by smtp.vangyzen.net (Postfix) with ESMTPSA id 43F045646F; Fri, 5 Jan 2018 16:26:25 -0600 (CST) Subject: Re: Is it considered to be ok to not check the return code of close(2) in base? To: Eugene Grosbein , Brooks Davis , Alan Somers Cc: Yuri , Ian Lepore , Freebsd hackers list References: <24acbd94-c52f-e71a-8a96-d608a10963c6@rawbw.com> <1514572041.12000.7.camel@freebsd.org> <20180105221330.GD95035@spindle.one-eyed-alien.net> <5A4FF989.1040709@grosbein.net> From: Eric van Gyzen Message-ID: <7b977409-96ee-5acb-60d0-3b0e124495f0@vangyzen.net> Date: Fri, 5 Jan 2018 16:26:23 -0600 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.4.0 MIME-Version: 1.0 In-Reply-To: <5A4FF989.1040709@grosbein.net> Content-Type: text/plain; charset=windows-1252 Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jan 2018 22:26:43 -0000 On 01/05/2018 16:17, Eugene Grosbein wrote: > 06.01.2018 5:13, Brooks Davis wrote: > >>> I would argue the opposite. There are very few reasons why close(s) would >>> ever fail, and the most likely is EBADF. EBADF indicates a programming >>> bug, like a double close or use of an uninitialized variable. Those could >>> easily turn into worse bugs in the future. So I think the best course of >>> action is to check the return code, assert() on EBADF, and ignore, or >>> possibly log, other errors. >> >> For this specific case, I think there would be value in an option to >> have the kernel kill any process that calls close(fd) where fd != -1 >> where EBADF would be returned. > > A medicine should not be worse worse than the disease, imho. In a multi-threaded application, a double-close can close completely unrelated file descriptors, which can be a nightmare to diagnose. In that case, death by signal is far better than the disease. Eric From owner-freebsd-hackers@freebsd.org Fri Jan 5 22:31:57 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D1DC8EC5317; Fri, 5 Jan 2018 22:31:57 +0000 (UTC) (envelope-from eric@metricspace.net) Received: from mail.metricspace.net (mail.metricspace.net [IPv6:2001:470:1f11:617::107]) by mx1.freebsd.org (Postfix) with ESMTP id 8AF82304D; Fri, 5 Jan 2018 22:31:57 +0000 (UTC) (envelope-from eric@metricspace.net) Received: from [192.168.43.57] (mobile-166-171-187-244.mycingular.net [166.171.187.244]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: eric) by mail.metricspace.net (Postfix) with ESMTPSA id B616E8A89; Fri, 5 Jan 2018 22:31:55 +0000 (UTC) Subject: Re: A more general possible meltdown/spectre countermeasure To: Warner Losh Cc: "freebsd-hackers@freebsd.org" , "freebsd-arch@freebsd.org" References: <33bcd281-4018-7075-1775-4dfcd58e5a48@metricspace.net> <4ec1f3b1-f4b0-80ab-0e68-0dd679dd9e37@metricspace.net> From: Eric McCorkle Message-ID: <72f6097e-c71e-b53f-6885-cfe5a5a56586@metricspace.net> Date: Fri, 5 Jan 2018 17:31:54 -0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.5.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jan 2018 22:31:57 -0000 Well, the only way to find out would be to try it out. However, unless I'm missing something, if you're trying to pull a meltdown attack, you try and fetch from the kernel. If that location isn't cached (or if your cache is physically indexed), you need the physical address (otherwise you don't know where to look), and thus have to go through address translation, at which point you detect that the page isn't accessible and fault. In the mean time, you can't speculatively execute any of the operations that load up the side-channels, because you don't have the sensitive data. The reason you can pull off a meltdown attack at all is that a virtually-indexed cache lets you get the data in parallel with address translation (breaking the dependency between address translation and fetching data), which takes 1000s of cycles for a TLB miss, during which you have the data and can launch a whole bunch of transient ops. Again, these are uncharted waters we're in; so it's entirely possible I'm missing something here. On 01/05/2018 17:22, Warner Losh wrote: > While you might be right, I've seen no indication that a cache miss > would defeat these attacks in the public and non-public data I've looked > at, even though a large number of alternatives to the published > workarounds have been discussed. I'm therefore somewhat skeptical this > would be effective. I'm open, however, to data that changes that > skepticism... > > Warner > > On Fri, Jan 5, 2018 at 3:15 PM, Eric McCorkle > wrote: > > Right, but you have to get the value "foo" into the pipeline in order > for it to affect the side-channels.  This technique attempts to stop > that from happening. > > Unless I made a mistake, non-cached memory reads force address > translation to happen first, which detects faults and blocks the > meltdown attack. > > It also stops spectre with very high probability, as it's very unlikely > that an uncached load will arrive before the speculative thread gets > squashed. > > On 01/05/2018 17:10, Warner Losh wrote: > > I think this is fatally flawed. > > > > The side channel is the cache. Not the data at risk. > > > > Any mapped memory, cached or not, can be used to influence the cache. > > Storing stuff in uncached memory won't affect the side channel one bit. > > > > Basically, all attacks boil down to tricking the processor, at elevated > > privs, to doing something like > > > > a = foo[offset]; > > > > where foo + offset are designed to communicate information by populating > > a cache line. offset need not be cached itself and can be the result of > > simple computations that depend on anything accessible at all in the kernel. > > > > Warner > > > > On Fri, Jan 5, 2018 at 3:02 PM, Eric McCorkle > > >> wrote: > > > >     Re-posting to -hackers and -arch.  I'm going to start working on > >     something like this over the weekend. > > > >     -------- Forwarded Message -------- > >     Subject: A more general possible meltdown/spectre countermeasure > >     Date: Thu, 4 Jan 2018 23:05:40 -0500 > >     From: Eric McCorkle >> > >     To: freebsd-security@freebsd.org > > >      > > >      >> > > > >     I've thought more about how to deal with meltdown/spectre, and > I have an > >     idea I'd like to put forward.  However, I'm still in something > of a > >     panic mode, so I'm not certain as to its effectiveness.  > Needless to > >     say, I welcome any feedback on this, and I may be completely > off-base. > > > >     I'm calling this a "countermeasure" as opposed to a > "mitigation", as > >     it's something that requires modification of code as opposed to a > >     drop-in patch. > > > >     == Summary == > > > >     Provide a kernel and userland API by which memory allocation > can be done > >     with extended attributes.  In userland, this could be > accomplished by > >     extending MMAP flags, and I could imagine a > malloc-with-attributes flag. > >      In kernel space, this must already exist, as drivers need to > allocate > >     memory with various MTRR-type attributes set. > > > >     The immediate aim here is to store sensitive information that must > >     remain memory-resident in non-cacheable memory locations (or, > if more > >     effective attribute combinations exist, using those instead).  > See the > >     rationale for the argument why this should work. > > > >     Assuming the rationale holds, then the attack surface should > be greatly > >     reduced.  Attackers would need to grab sensitive data out of stack > >     frames or similar locations if/when it gets copied there for > faster use. > >      Moreover, if this is done right, it could dovetail nicely into a > >     framework for storing and processing sensitive assets in more > secure > >     hardware[0] (like smart cards, the FPGAs I posted earlier, or > other > >     options). > > > >     The obvious downside is that you take a performance hit > storing things > >     in non-cacheable locations, especially if you plan on doing heavy > >     computation in that memory (say, encryption/decryption).  > However, this > >     is almost certainly going to be less than the projected 30-50% > >     performance hit from other mitigations.  Also, this technique > should > >     work against spectre as well as meltdown (assuming the > rationale holds). > > > >     The second downside is that you have to modify code for this > to work, > >     and you have to be careful not to keep copies of sensitive > information > >     around too long (this gets tricky in userland, where you might get > >     interrupted and switched out). > > > > > >     [0]: Full disclosure, enabling open hardware implementations > of this > >     kind of thing is something of an agenda of mine. > > > >     == Rationale == > > > >     (Again, I'm tired, rushed, and somewhat panicked so my logic > could be > >     faulty at any point, so please point it out if it is) > > > >     The rationale for why this should work relies on assumptions about > >     out-of-order pipelines that cannot be guaranteed to hold, but are > >     extremely likely to be true. > > > >     As background, these attacks depend on out-of-order execution > performing > >     operations that end up affecting cache and branch-prediction > state, > >     ultimately storing information about sensitive data in these > >     side-channels before the fault conditions are detected and > acted upon. > >     I'll borrow terminology from the paper, using "transient > instructions" > >     to refer to speculatively executed instructions that will > eventually be > >     cancelled by a fault. > > > >     These attacks depend entirely on transient instructions being > able to > >     get sensitive information into the processor core and then > perform some > >     kind of instruction on them before the fault condition cancels > them. > >     Therefore, anything that prevents them from doing this > *should* counter > >     the attack.  If the actual sensitive data never makes it to > the core > >     before the fault is detected, the dependent memory > accesses/branches > >     never get executed and the data never makes it to the > side-channels. > > > >     Another assumption here is that CPU architects are going to > want to > >     squash faulted instructions ASAP and stop issuing along those > >     speculative branches, so as to reclaim execution units.  So > I'm assuming > >     once a fault comes back from address translation, then transient > >     execution stops dead. > > > >     Now, break down the cases for whether the address containing > sensitive > >     data is in cache and TLB or not.  (I'm assuming here that > caches are > >     virtually-indexed, which enables cache lookups to bypass address > >     translation.) > > > >     * In cache, in TLB: You end up basically racing between the > cache and > >     TLB, which will very likely end up detecting the fault before > the data > >     arrives, but at the very worst, you get one or two cycles of > transient > >     instruction execution before the fault. > > > >     * In cache, not in TLB: Virtually-indexed tagged means you get > a cache > >     lookup racing a page-table walk.  The cache lookup beats the > page table > >     walk by potentially hundreds (maybe thousands) of cycles, > giving you a > >     bunch of transient instructions before a fault gets > triggered.  This is > >     the main attack case. > > > >     * Not in cache, in TLB: Memory access requires address > translation, > >     which comes back almost immediately as a fault. > > > >     * Not in cache, not in TLB: You have to do a page table walk > before you > >     can fetch the location, as you have to go out to physical > memory (and > >     therefore need a physical address).  The page table walk will > come back > >     with a fault, stopping the attack. > > > >     So, unless I'm missing something here, both non-cached cases > defeat the > >     meltdown attack, as you *cannot* get the data unless you do > address > >     translation first (and therefore detect faults). > > > >     As for why this defeats the spectre attack, the logic is > similar: you've > >     jumped into someone else's executable code, hoping to scoop up > enough > >     information into your branch predictor before the fault kicks > you out. > >     However, to capture anything about sensitive information in your > >     side-channels, the transient instructions need to actually get > it into > >     the core before a fault gets detected.  The same case analysis > as above > >     applies, so you never actually get the sensitive info into the > core > >     before a fault comes back and you get squashed. > > > > > >     [1]: A physically-indexed cache would be largely immune to > this attack, > >     as you'd have to do address translation before doing a cache > lookup. > > > > > >     I have some ideas that can build on this, but I'd like to get some > >     feedback first. > >     _______________________________________________ > >     freebsd-security@freebsd.org > > > > >     mailing list > >     https://lists.freebsd.org/mailman/listinfo/freebsd-security > > >      > > >     To unsubscribe, send any mail to > >     "freebsd-security-unsubscribe@freebsd.org > > >      >" > >     _______________________________________________ > >     freebsd-arch@freebsd.org > > > mailing list > >     https://lists.freebsd.org/mailman/listinfo/freebsd-arch > > >      > > >     To unsubscribe, send any mail to > >     "freebsd-arch-unsubscribe@freebsd.org > > >      >" > > > > > > From owner-freebsd-hackers@freebsd.org Fri Jan 5 22:34:54 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 65628EC5582 for ; Fri, 5 Jan 2018 22:34:54 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id 2A0143373; Fri, 5 Jan 2018 22:34:53 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (unknown [192.168.55.3]) by phk.freebsd.dk (Postfix) with ESMTP id 8E73927374; Fri, 5 Jan 2018 22:34:51 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.15.2/8.15.2) with ESMTPS id w05MYach024175 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 5 Jan 2018 22:34:36 GMT (envelope-from phk@critter.freebsd.dk) Received: (from phk@localhost) by critter.freebsd.dk (8.15.2/8.15.2/Submit) id w05MYZow024174; Fri, 5 Jan 2018 22:34:35 GMT (envelope-from phk) To: Brooks Davis cc: Alan Somers , Yuri , Ian Lepore , Freebsd hackers list Subject: Re: Is it considered to be ok to not check the return code of close(2) in base? In-reply-to: <20180105221330.GD95035@spindle.one-eyed-alien.net> From: "Poul-Henning Kamp" References: <24acbd94-c52f-e71a-8a96-d608a10963c6@rawbw.com> <1514572041.12000.7.camel@freebsd.org> <20180105221330.GD95035@spindle.one-eyed-alien.net> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <24172.1515191675.1@critter.freebsd.dk> Content-Transfer-Encoding: quoted-printable Date: Fri, 05 Jan 2018 22:34:35 +0000 Message-ID: <24173.1515191675@critter.freebsd.dk> X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jan 2018 22:34:54 -0000 -------- In message <20180105221330.GD95035@spindle.one-eyed-alien.net>, Brooks Dav= is writes: >> I would argue the opposite. There are very few reasons why close(s) wo= uld >> ever fail, and the most likely is EBADF. EBADF indicates a programming >> bug, like a double close or use of an uninitialized variable. Those co= uld >> easily turn into worse bugs in the future. So I think the best course = of >> action is to check the return code, assert() on EBADF, and ignore, or >> possibly log, other errors. I fully agree on the seriousness of EBADF from close, but want to note that close (2) used on sockets return a bestiarium of errors depending on how the remote slammed the door. >For this specific case, I think there would be value in an option to >have the kernel kill any process that calls close(fd) where fd !=3D -1 >where EBADF would be returned. I think -current ought to be much more paranoid towards bugs. When I added phkmalloc to -current many years ago, I set the default flags to AJ and the first program which exploded on me was fsck(8), and the next dozen or so were all setuid or setgid. Brookes suggestion, while well intentioned, wouldn't get very far, because it is common for shells and shell-like programs to do: for (i =3D 3; i < ALOT; i++) (void)close(i); To get rid of unwanted filedescriptors from syslog(3), getpwent(3) etc. in the child process. Yes, I know about closefrom(2), but a lot of programs still don't use it. So yes please, make libc and the kernel complain about anything which looks like programming errors in -current, but hold off on the process-killing, and make it possible to silence these complaints at boot-time. Maybe we should simply add a "bootparanoia" flag like we have "bootverbose= " ? -- = Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe = Never attribute to malice what can adequately be explained by incompetence= . From owner-freebsd-hackers@freebsd.org Fri Jan 5 22:57:45 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 73590EA4898 for ; Fri, 5 Jan 2018 22:57:45 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [78.47.246.247]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 0011A6389A; Fri, 5 Jan 2018 22:57:44 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (root@eg.sd.rdtc.ru [62.231.161.221] (may be forged)) by hz.grosbein.net (8.15.2/8.15.2) with ESMTPS id w05MvaP6042267 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 5 Jan 2018 23:57:37 +0100 (CET) (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: eric@vangyzen.net Received: from [10.58.0.4] ([10.58.0.4]) by eg.sd.rdtc.ru (8.15.2/8.15.2) with ESMTPS id w05MvXxX030890 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Sat, 6 Jan 2018 05:57:33 +0700 (+07) (envelope-from eugen@grosbein.net) Subject: Re: Is it considered to be ok to not check the return code of close(2) in base? To: Eric van Gyzen , Brooks Davis , Alan Somers References: <24acbd94-c52f-e71a-8a96-d608a10963c6@rawbw.com> <1514572041.12000.7.camel@freebsd.org> <20180105221330.GD95035@spindle.one-eyed-alien.net> <5A4FF989.1040709@grosbein.net> <7b977409-96ee-5acb-60d0-3b0e124495f0@vangyzen.net> Cc: Yuri , Ian Lepore , Freebsd hackers list From: Eugene Grosbein Message-ID: <5A5002D9.9080205@grosbein.net> Date: Sat, 6 Jan 2018 05:57:29 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.7.2 MIME-Version: 1.0 In-Reply-To: <7b977409-96ee-5acb-60d0-3b0e124495f0@vangyzen.net> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=2.2 required=5.0 tests=BAYES_00, LOCAL_FROM, RDNS_NONE autolearn=no autolearn_force=no version=3.4.1 X-Spam-Report: * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 2.6 LOCAL_FROM From my domains * 1.9 RDNS_NONE Delivered to internal network by a host with no rDNS X-Spam-Level: ** X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on hz.grosbein.net X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jan 2018 22:57:45 -0000 06.01.2018 5:26, Eric van Gyzen wrote: >>>> I would argue the opposite. There are very few reasons why close(s) would >>>> ever fail, and the most likely is EBADF. EBADF indicates a programming >>>> bug, like a double close or use of an uninitialized variable. Those could >>>> easily turn into worse bugs in the future. So I think the best course of >>>> action is to check the return code, assert() on EBADF, and ignore, or >>>> possibly log, other errors. >>> >>> For this specific case, I think there would be value in an option to >>> have the kernel kill any process that calls close(fd) where fd != -1 >>> where EBADF would be returned. >> >> A medicine should not be worse worse than the disease, imho. > > In a multi-threaded application, a double-close can close completely > unrelated file descriptors, which can be a nightmare to diagnose. In > that case, death by signal is far better than the disease. A kernel cannot know if close() was called for "unrealated" but correct file descriptor. From owner-freebsd-hackers@freebsd.org Fri Jan 5 23:08:21 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B2D49EA562D for ; Fri, 5 Jan 2018 23:08:21 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-it0-x235.google.com (mail-it0-x235.google.com [IPv6:2607:f8b0:4001:c0b::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5E9E4643D0 for ; Fri, 5 Jan 2018 23:08:21 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-it0-x235.google.com with SMTP id c16so3259184itc.5 for ; Fri, 05 Jan 2018 15:08:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=7xFTD5DLONb1wsY6ElzO43YqgPVZ7fUSBkUxbNwkFNE=; b=QjKqaUaJ0zF+B7Wmafw+NafpFGz/hPPIqv91FEzr/s0ZWjWobuHTa/eYlaEZuD1YAZ Wx+0CzYEOxs1UslzEkv4nQMYW/0I0j15dqgET221coqG2hpfNGekBXOzFhG/HoqHqebN eNLIDlmrZa8Jq0ziGib5+dj6J7pnOIy3oE9QH0dkXDqEHVOebIlhleIsWPpNCMmb9mpu tJEpMjCGtQJvQGSVY9lkQPnwyAuywCCnKuXJaSavaWmpQOWbpSWQhUBWZmbYkVUaSyfH BLE6Ba1ymj5Fb45qaPtLBd1M4MAsVbnv6B0lsz1fPYsvwV1rl69sdEyw7I4yBSv+Ex+k W73A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=7xFTD5DLONb1wsY6ElzO43YqgPVZ7fUSBkUxbNwkFNE=; b=op5jL1yjxhTKQ2Z5MQV0Zi/eHH0nQOU9kwFqjQZDNtsvoXhj9RixBgKwi+G45AGfw/ 847wZKV6n2O0SEfXj7tfA4jd+6dUcdg3C7O3Tn2yhEgl5NC4uHaCPNF0wB2D/nUPr0DO dOvnoJzAiI1iAzxTgckeR38g4m9LCBPblGKB2eufPbHkyZZAdgEMxPVn/zo+ZPzKsIgN /ZNSKL4wCk8UwJvOOhErUwqTTX4QOsGebUSSlFZLfyubg6ZVo3NrXcJ27fz+RFoeaKlA wq5qjZ9bYKxY8Asvp6pwuBk2RMYY6kF4V76Iv/OlDIYPPTvhAcKfqYF7E2GrL3729u7L MLTA== X-Gm-Message-State: AKGB3mIyc47ucb+V1qx3jLWDMVJod5rER/h9S3EU9SPabYEPtQAql6s/ lMjJdB75A4Ybl5iEyVoBOhp+pRYp2mHlVNMbyTtDiw== X-Google-Smtp-Source: ACJfBosDmT/gqelm6+SREn2hv4byfPTI8OYeqUjMDdp8DaHGMPAm0pqFR3cqm1hkaD54Iio+Z5ApgNJK9m15k2JASu4= X-Received: by 10.36.131.8 with SMTP id d8mr4806255ite.115.1515193700441; Fri, 05 Jan 2018 15:08:20 -0800 (PST) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 10.79.160.217 with HTTP; Fri, 5 Jan 2018 15:08:19 -0800 (PST) X-Originating-IP: [2603:300b:6:5100:1052:acc7:f9de:2b6d] In-Reply-To: <72f6097e-c71e-b53f-6885-cfe5a5a56586@metricspace.net> References: <33bcd281-4018-7075-1775-4dfcd58e5a48@metricspace.net> <4ec1f3b1-f4b0-80ab-0e68-0dd679dd9e37@metricspace.net> <72f6097e-c71e-b53f-6885-cfe5a5a56586@metricspace.net> From: Warner Losh Date: Fri, 5 Jan 2018 16:08:19 -0700 X-Google-Sender-Auth: ipJBatNBLHw8pt4DKZfnbJTfRIA Message-ID: Subject: Re: A more general possible meltdown/spectre countermeasure To: Eric McCorkle Cc: "freebsd-hackers@freebsd.org" , "freebsd-arch@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jan 2018 23:08:21 -0000 Wouldn't you have to also unmap it from the direct map for this to be effective? Warner On Fri, Jan 5, 2018 at 3:31 PM, Eric McCorkle wrote: > Well, the only way to find out would be to try it out. > > However, unless I'm missing something, if you're trying to pull a > meltdown attack, you try and fetch from the kernel. If that location > isn't cached (or if your cache is physically indexed), you need the > physical address (otherwise you don't know where to look), and thus have > to go through address translation, at which point you detect that the > page isn't accessible and fault. In the mean time, you can't > speculatively execute any of the operations that load up the > side-channels, because you don't have the sensitive data. > > The reason you can pull off a meltdown attack at all is that a > virtually-indexed cache lets you get the data in parallel with address > translation (breaking the dependency between address translation and > fetching data), which takes 1000s of cycles for a TLB miss, during which > you have the data and can launch a whole bunch of transient ops. > > Again, these are uncharted waters we're in; so it's entirely possible > I'm missing something here. > > On 01/05/2018 17:22, Warner Losh wrote: > > While you might be right, I've seen no indication that a cache miss > > would defeat these attacks in the public and non-public data I've looked > > at, even though a large number of alternatives to the published > > workarounds have been discussed. I'm therefore somewhat skeptical this > > would be effective. I'm open, however, to data that changes that > > skepticism... > > > > Warner > > > > On Fri, Jan 5, 2018 at 3:15 PM, Eric McCorkle > > wrote: > > > > Right, but you have to get the value "foo" into the pipeline in order > > for it to affect the side-channels. This technique attempts to stop > > that from happening. > > > > Unless I made a mistake, non-cached memory reads force address > > translation to happen first, which detects faults and blocks the > > meltdown attack. > > > > It also stops spectre with very high probability, as it's very > unlikely > > that an uncached load will arrive before the speculative thread gets > > squashed. > > > > On 01/05/2018 17:10, Warner Losh wrote: > > > I think this is fatally flawed. > > > > > > The side channel is the cache. Not the data at risk. > > > > > > Any mapped memory, cached or not, can be used to influence the > cache. > > > Storing stuff in uncached memory won't affect the side channel one > bit. > > > > > > Basically, all attacks boil down to tricking the processor, at > elevated > > > privs, to doing something like > > > > > > a = foo[offset]; > > > > > > where foo + offset are designed to communicate information by > populating > > > a cache line. offset need not be cached itself and can be the > result of > > > simple computations that depend on anything accessible at all in > the kernel. > > > > > > Warner > > > > > > On Fri, Jan 5, 2018 at 3:02 PM, Eric McCorkle < > eric@metricspace.net > > > >> > wrote: > > > > > > Re-posting to -hackers and -arch. I'm going to start working > on > > > something like this over the weekend. > > > > > > -------- Forwarded Message -------- > > > Subject: A more general possible meltdown/spectre > countermeasure > > > Date: Thu, 4 Jan 2018 23:05:40 -0500 > > > From: Eric McCorkle > > >> > > > To: freebsd-security@freebsd.org > > > > > > > > > > > > >> > > > > > > I've thought more about how to deal with meltdown/spectre, and > > I have an > > > idea I'd like to put forward. However, I'm still in something > > of a > > > panic mode, so I'm not certain as to its effectiveness. > > Needless to > > > say, I welcome any feedback on this, and I may be completely > > off-base. > > > > > > I'm calling this a "countermeasure" as opposed to a > > "mitigation", as > > > it's something that requires modification of code as opposed > to a > > > drop-in patch. > > > > > > == Summary == > > > > > > Provide a kernel and userland API by which memory allocation > > can be done > > > with extended attributes. In userland, this could be > > accomplished by > > > extending MMAP flags, and I could imagine a > > malloc-with-attributes flag. > > > In kernel space, this must already exist, as drivers need to > > allocate > > > memory with various MTRR-type attributes set. > > > > > > The immediate aim here is to store sensitive information that > must > > > remain memory-resident in non-cacheable memory locations (or, > > if more > > > effective attribute combinations exist, using those instead). > > See the > > > rationale for the argument why this should work. > > > > > > Assuming the rationale holds, then the attack surface should > > be greatly > > > reduced. Attackers would need to grab sensitive data out of > stack > > > frames or similar locations if/when it gets copied there for > > faster use. > > > Moreover, if this is done right, it could dovetail nicely > into a > > > framework for storing and processing sensitive assets in more > > secure > > > hardware[0] (like smart cards, the FPGAs I posted earlier, or > > other > > > options). > > > > > > The obvious downside is that you take a performance hit > > storing things > > > in non-cacheable locations, especially if you plan on doing > heavy > > > computation in that memory (say, encryption/decryption). > > However, this > > > is almost certainly going to be less than the projected 30-50% > > > performance hit from other mitigations. Also, this technique > > should > > > work against spectre as well as meltdown (assuming the > > rationale holds). > > > > > > The second downside is that you have to modify code for this > > to work, > > > and you have to be careful not to keep copies of sensitive > > information > > > around too long (this gets tricky in userland, where you might > get > > > interrupted and switched out). > > > > > > > > > [0]: Full disclosure, enabling open hardware implementations > > of this > > > kind of thing is something of an agenda of mine. > > > > > > == Rationale == > > > > > > (Again, I'm tired, rushed, and somewhat panicked so my logic > > could be > > > faulty at any point, so please point it out if it is) > > > > > > The rationale for why this should work relies on assumptions > about > > > out-of-order pipelines that cannot be guaranteed to hold, but > are > > > extremely likely to be true. > > > > > > As background, these attacks depend on out-of-order execution > > performing > > > operations that end up affecting cache and branch-prediction > > state, > > > ultimately storing information about sensitive data in these > > > side-channels before the fault conditions are detected and > > acted upon. > > > I'll borrow terminology from the paper, using "transient > > instructions" > > > to refer to speculatively executed instructions that will > > eventually be > > > cancelled by a fault. > > > > > > These attacks depend entirely on transient instructions being > > able to > > > get sensitive information into the processor core and then > > perform some > > > kind of instruction on them before the fault condition cancels > > them. > > > Therefore, anything that prevents them from doing this > > *should* counter > > > the attack. If the actual sensitive data never makes it to > > the core > > > before the fault is detected, the dependent memory > > accesses/branches > > > never get executed and the data never makes it to the > > side-channels. > > > > > > Another assumption here is that CPU architects are going to > > want to > > > squash faulted instructions ASAP and stop issuing along those > > > speculative branches, so as to reclaim execution units. So > > I'm assuming > > > once a fault comes back from address translation, then > transient > > > execution stops dead. > > > > > > Now, break down the cases for whether the address containing > > sensitive > > > data is in cache and TLB or not. (I'm assuming here that > > caches are > > > virtually-indexed, which enables cache lookups to bypass > address > > > translation.) > > > > > > * In cache, in TLB: You end up basically racing between the > > cache and > > > TLB, which will very likely end up detecting the fault before > > the data > > > arrives, but at the very worst, you get one or two cycles of > > transient > > > instruction execution before the fault. > > > > > > * In cache, not in TLB: Virtually-indexed tagged means you get > > a cache > > > lookup racing a page-table walk. The cache lookup beats the > > page table > > > walk by potentially hundreds (maybe thousands) of cycles, > > giving you a > > > bunch of transient instructions before a fault gets > > triggered. This is > > > the main attack case. > > > > > > * Not in cache, in TLB: Memory access requires address > > translation, > > > which comes back almost immediately as a fault. > > > > > > * Not in cache, not in TLB: You have to do a page table walk > > before you > > > can fetch the location, as you have to go out to physical > > memory (and > > > therefore need a physical address). The page table walk will > > come back > > > with a fault, stopping the attack. > > > > > > So, unless I'm missing something here, both non-cached cases > > defeat the > > > meltdown attack, as you *cannot* get the data unless you do > > address > > > translation first (and therefore detect faults). > > > > > > As for why this defeats the spectre attack, the logic is > > similar: you've > > > jumped into someone else's executable code, hoping to scoop up > > enough > > > information into your branch predictor before the fault kicks > > you out. > > > However, to capture anything about sensitive information in > your > > > side-channels, the transient instructions need to actually get > > it into > > > the core before a fault gets detected. The same case analysis > > as above > > > applies, so you never actually get the sensitive info into the > > core > > > before a fault comes back and you get squashed. > > > > > > > > > [1]: A physically-indexed cache would be largely immune to > > this attack, > > > as you'd have to do address translation before doing a cache > > lookup. > > > > > > > > > I have some ideas that can build on this, but I'd like to get > some > > > feedback first. > > > _______________________________________________ > > > freebsd-security@freebsd.org > > > > > > > > > mailing list > > > https://lists.freebsd.org/mailman/listinfo/freebsd-security > > > > > > > > > > To unsubscribe, send any mail to > > > "freebsd-security-unsubscribe@freebsd.org > > > > > > >" > > > _______________________________________________ > > > freebsd-arch@freebsd.org > > > > > mailing list > > > https://lists.freebsd.org/mailman/listinfo/freebsd-arch > > > > > > > > > > To unsubscribe, send any mail to > > > "freebsd-arch-unsubscribe@freebsd.org > > > > > > >" > > > > > > > > > > > From owner-freebsd-hackers@freebsd.org Fri Jan 5 23:10:56 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 86E96EA5987; Fri, 5 Jan 2018 23:10:56 +0000 (UTC) (envelope-from eric@metricspace.net) Received: from mail.metricspace.net (mail.metricspace.net [IPv6:2001:470:1f11:617::107]) by mx1.freebsd.org (Postfix) with ESMTP id 2E8C064698; Fri, 5 Jan 2018 23:10:55 +0000 (UTC) (envelope-from eric@metricspace.net) Received: from [192.168.43.57] (mobile-166-171-187-244.mycingular.net [166.171.187.244]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: eric) by mail.metricspace.net (Postfix) with ESMTPSA id F00C08AB6; Fri, 5 Jan 2018 23:10:54 +0000 (UTC) Subject: Re: A more general possible meltdown/spectre countermeasure To: Warner Losh Cc: "freebsd-hackers@freebsd.org" , "freebsd-arch@freebsd.org" References: <33bcd281-4018-7075-1775-4dfcd58e5a48@metricspace.net> <4ec1f3b1-f4b0-80ab-0e68-0dd679dd9e37@metricspace.net> <72f6097e-c71e-b53f-6885-cfe5a5a56586@metricspace.net> From: Eric McCorkle Message-ID: Date: Fri, 5 Jan 2018 18:10:53 -0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.5.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jan 2018 23:10:56 -0000 I'm not sure what you mean by direct map. Do you mean TLB? On 01/05/2018 18:08, Warner Losh wrote: > Wouldn't you have to also unmap it from the direct map for this to be > effective? > > Warner > > > On Fri, Jan 5, 2018 at 3:31 PM, Eric McCorkle > wrote: > > Well, the only way to find out would be to try it out. > > However, unless I'm missing something, if you're trying to pull a > meltdown attack, you try and fetch from the kernel.  If that location > isn't cached (or if your cache is physically indexed), you need the > physical address (otherwise you don't know where to look), and thus have > to go through address translation, at which point you detect that the > page isn't accessible and fault.  In the mean time, you can't > speculatively execute any of the operations that load up the > side-channels, because you don't have the sensitive data. > > The reason you can pull off a meltdown attack at all is that a > virtually-indexed cache lets you get the data in parallel with address > translation (breaking the dependency between address translation and > fetching data), which takes 1000s of cycles for a TLB miss, during which > you have the data and can launch a whole bunch of transient ops. > > Again, these are uncharted waters we're in; so it's entirely possible > I'm missing something here. > > On 01/05/2018 17:22, Warner Losh wrote: > > While you might be right, I've seen no indication that a cache miss > > would defeat these attacks in the public and non-public data I've looked > > at, even though a large number of alternatives to the published > > workarounds have been discussed. I'm therefore somewhat skeptical this > > would be effective. I'm open, however, to data that changes that > > skepticism... > > > > Warner > > > > On Fri, Jan 5, 2018 at 3:15 PM, Eric McCorkle > > >> wrote: > > > >     Right, but you have to get the value "foo" into the pipeline in order > >     for it to affect the side-channels.  This technique attempts to stop > >     that from happening. > > > >     Unless I made a mistake, non-cached memory reads force address > >     translation to happen first, which detects faults and blocks the > >     meltdown attack. > > > >     It also stops spectre with very high probability, as it's very unlikely > >     that an uncached load will arrive before the speculative thread gets > >     squashed. > > > >     On 01/05/2018 17:10, Warner Losh wrote: > >     > I think this is fatally flawed. > >     > > >     > The side channel is the cache. Not the data at risk. > >     > > >     > Any mapped memory, cached or not, can be used to influence the cache. > >     > Storing stuff in uncached memory won't affect the side channel one bit. > >     > > >     > Basically, all attacks boil down to tricking the processor, at elevated > >     > privs, to doing something like > >     > > >     > a = foo[offset]; > >     > > >     > where foo + offset are designed to communicate information by populating > >     > a cache line. offset need not be cached itself and can be the result of > >     > simple computations that depend on anything accessible at all in the kernel. > >     > > >     > Warner > >     > > >     > On Fri, Jan 5, 2018 at 3:02 PM, Eric McCorkle > > > >     > > >>> wrote: > >     > > >     >     Re-posting to -hackers and -arch.  I'm going to start working on > >     >     something like this over the weekend. > >     > > >     >     -------- Forwarded Message -------- > >     >     Subject: A more general possible meltdown/spectre countermeasure > >     >     Date: Thu, 4 Jan 2018 23:05:40 -0500 > >     >     From: Eric McCorkle > >     > > > >     >>> > >     >     To: freebsd-security@freebsd.org > >     > > >     >      > >     >> > > >     > > >     >      > >      >>> > >     > > >     >     I've thought more about how to deal with > meltdown/spectre, and > >     I have an > >     >     idea I'd like to put forward.  However, I'm still in > something > >     of a > >     >     panic mode, so I'm not certain as to its effectiveness.  > >     Needless to > >     >     say, I welcome any feedback on this, and I may be completely > >     off-base. > >     > > >     >     I'm calling this a "countermeasure" as opposed to a > >     "mitigation", as > >     >     it's something that requires modification of code as > opposed to a > >     >     drop-in patch. > >     > > >     >     == Summary == > >     > > >     >     Provide a kernel and userland API by which memory allocation > >     can be done > >     >     with extended attributes.  In userland, this could be > >     accomplished by > >     >     extending MMAP flags, and I could imagine a > >     malloc-with-attributes flag. > >     >      In kernel space, this must already exist, as drivers > need to > >     allocate > >     >     memory with various MTRR-type attributes set. > >     > > >     >     The immediate aim here is to store sensitive information > that must > >     >     remain memory-resident in non-cacheable memory locations > (or, > >     if more > >     >     effective attribute combinations exist, using those > instead).  > >     See the > >     >     rationale for the argument why this should work. > >     > > >     >     Assuming the rationale holds, then the attack surface should > >     be greatly > >     >     reduced.  Attackers would need to grab sensitive data > out of stack > >     >     frames or similar locations if/when it gets copied there for > >     faster use. > >     >      Moreover, if this is done right, it could dovetail > nicely into a > >     >     framework for storing and processing sensitive assets in > more > >     secure > >     >     hardware[0] (like smart cards, the FPGAs I posted > earlier, or > >     other > >     >     options). > >     > > >     >     The obvious downside is that you take a performance hit > >     storing things > >     >     in non-cacheable locations, especially if you plan on > doing heavy > >     >     computation in that memory (say, encryption/decryption).  > >     However, this > >     >     is almost certainly going to be less than the projected > 30-50% > >     >     performance hit from other mitigations.  Also, this > technique > >     should > >     >     work against spectre as well as meltdown (assuming the > >     rationale holds). > >     > > >     >     The second downside is that you have to modify code for this > >     to work, > >     >     and you have to be careful not to keep copies of sensitive > >     information > >     >     around too long (this gets tricky in userland, where you > might get > >     >     interrupted and switched out). > >     > > >     > > >     >     [0]: Full disclosure, enabling open hardware implementations > >     of this > >     >     kind of thing is something of an agenda of mine. > >     > > >     >     == Rationale == > >     > > >     >     (Again, I'm tired, rushed, and somewhat panicked so my logic > >     could be > >     >     faulty at any point, so please point it out if it is) > >     > > >     >     The rationale for why this should work relies on > assumptions about > >     >     out-of-order pipelines that cannot be guaranteed to > hold, but are > >     >     extremely likely to be true. > >     > > >     >     As background, these attacks depend on out-of-order > execution > >     performing > >     >     operations that end up affecting cache and branch-prediction > >     state, > >     >     ultimately storing information about sensitive data in these > >     >     side-channels before the fault conditions are detected and > >     acted upon. > >     >     I'll borrow terminology from the paper, using "transient > >     instructions" > >     >     to refer to speculatively executed instructions that will > >     eventually be > >     >     cancelled by a fault. > >     > > >     >     These attacks depend entirely on transient instructions > being > >     able to > >     >     get sensitive information into the processor core and then > >     perform some > >     >     kind of instruction on them before the fault condition > cancels > >     them. > >     >     Therefore, anything that prevents them from doing this > >     *should* counter > >     >     the attack.  If the actual sensitive data never makes it to > >     the core > >     >     before the fault is detected, the dependent memory > >     accesses/branches > >     >     never get executed and the data never makes it to the > >     side-channels. > >     > > >     >     Another assumption here is that CPU architects are going to > >     want to > >     >     squash faulted instructions ASAP and stop issuing along > those > >     >     speculative branches, so as to reclaim execution units.  So > >     I'm assuming > >     >     once a fault comes back from address translation, then > transient > >     >     execution stops dead. > >     > > >     >     Now, break down the cases for whether the address containing > >     sensitive > >     >     data is in cache and TLB or not.  (I'm assuming here that > >     caches are > >     >     virtually-indexed, which enables cache lookups to bypass > address > >     >     translation.) > >     > > >     >     * In cache, in TLB: You end up basically racing between the > >     cache and > >     >     TLB, which will very likely end up detecting the fault > before > >     the data > >     >     arrives, but at the very worst, you get one or two cycles of > >     transient > >     >     instruction execution before the fault. > >     > > >     >     * In cache, not in TLB: Virtually-indexed tagged means > you get > >     a cache > >     >     lookup racing a page-table walk.  The cache lookup beats the > >     page table > >     >     walk by potentially hundreds (maybe thousands) of cycles, > >     giving you a > >     >     bunch of transient instructions before a fault gets > >     triggered.  This is > >     >     the main attack case. > >     > > >     >     * Not in cache, in TLB: Memory access requires address > >     translation, > >     >     which comes back almost immediately as a fault. > >     > > >     >     * Not in cache, not in TLB: You have to do a page table walk > >     before you > >     >     can fetch the location, as you have to go out to physical > >     memory (and > >     >     therefore need a physical address).  The page table walk > will > >     come back > >     >     with a fault, stopping the attack. > >     > > >     >     So, unless I'm missing something here, both non-cached cases > >     defeat the > >     >     meltdown attack, as you *cannot* get the data unless you do > >     address > >     >     translation first (and therefore detect faults). > >     > > >     >     As for why this defeats the spectre attack, the logic is > >     similar: you've > >     >     jumped into someone else's executable code, hoping to > scoop up > >     enough > >     >     information into your branch predictor before the fault > kicks > >     you out. > >     >     However, to capture anything about sensitive information > in your > >     >     side-channels, the transient instructions need to > actually get > >     it into > >     >     the core before a fault gets detected.  The same case > analysis > >     as above > >     >     applies, so you never actually get the sensitive info > into the > >     core > >     >     before a fault comes back and you get squashed. > >     > > >     > > >     >     [1]: A physically-indexed cache would be largely immune to > >     this attack, > >     >     as you'd have to do address translation before doing a cache > >     lookup. > >     > > >     > > >     >     I have some ideas that can build on this, but I'd like > to get some > >     >     feedback first. > >     >     _______________________________________________ > >     >     freebsd-security@freebsd.org > > >      > > >      > >     >> > >     >     mailing list > >     >     https://lists.freebsd.org/mailman/listinfo/freebsd-security > > >      > > >     >      > >      >> > >     >     To unsubscribe, send any mail to > >     >     "freebsd-security-unsubscribe@freebsd.org > > >      > > >     >      > >      >>" > >     >     _______________________________________________ > >     >     freebsd-arch@freebsd.org > > > >      >> > >     mailing list > >     >     https://lists.freebsd.org/mailman/listinfo/freebsd-arch > > >      > > >     >      > >      >> > >     >     To unsubscribe, send any mail to > >     >     "freebsd-arch-unsubscribe@freebsd.org > > >      > > >     >      > >      >>" > >     > > >     > > > > > > > From owner-freebsd-hackers@freebsd.org Fri Jan 5 23:17:39 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4A9B6EA60BC for ; Fri, 5 Jan 2018 23:17:39 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: from mail-it0-f42.google.com (mail-it0-f42.google.com [209.85.214.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 18BF164C12 for ; Fri, 5 Jan 2018 23:17:38 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: by mail-it0-f42.google.com with SMTP id b77so108422itd.0 for ; Fri, 05 Jan 2018 15:17:38 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:reply-to:in-reply-to:references :from:date:message-id:subject:to:cc; bh=8pFJ10xLWQixEl5Bh26lwm5KQhdLH86d7NX5KGdZLeg=; b=Xj/qXUgRhfRv6Tt4SRCTrrR932WJk9DhTtjviQ7TBxYfgkOiheeufI2+FGK6MOORjk +PGPYw6EMw4ZOqDE/mMCfu/AxL/+8gLjLdABw3T9zjmPXAE2bbRF5I+qhwFU6JtHRPwE zvJji77fdwqK/2jPIFeQ2Ygxt/etrNF73GbXNBaLhsEy0d5icsSufGlLQG9Oymk9n8nM gZB7rXpgIfeWs/VDegJR2GUDq3kvAo0zmF+5fUD43nSvNpr2leinZnezQQ7cct3542J0 iSNc7+k2wrAcJzSFydJeS9x+dI4LdaJXpMUNUKDG+TuXGK0HB8edBA972KGCwwHaKyUZ dfew== X-Gm-Message-State: AKGB3mJiB0HXEe9KCK023s6u7DO0nDIcqOgJXx4YpOP8J+utJyVISfWx 0jMTqji/6c4/bDla4/fH+PCjZu/t X-Google-Smtp-Source: ACJfBosAOvU4gilJqNlm08xpcH/6xZSsENGriYpWsTcy60GUwdb80m9TLVvvUBER93gfJJoN1AOmvw== X-Received: by 10.36.250.202 with SMTP id v193mr4977041ith.114.1515194252614; Fri, 05 Jan 2018 15:17:32 -0800 (PST) Received: from mail-it0-f52.google.com (mail-it0-f52.google.com. [209.85.214.52]) by smtp.gmail.com with ESMTPSA id l28sm4133138iod.32.2018.01.05.15.17.32 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 05 Jan 2018 15:17:32 -0800 (PST) Received: by mail-it0-f52.google.com with SMTP id f143so3398257itb.0 for ; Fri, 05 Jan 2018 15:17:32 -0800 (PST) X-Received: by 10.36.7.65 with SMTP id f62mr4900098itf.75.1515194252167; Fri, 05 Jan 2018 15:17:32 -0800 (PST) MIME-Version: 1.0 Reply-To: cem@freebsd.org Received: by 10.2.15.193 with HTTP; Fri, 5 Jan 2018 15:17:31 -0800 (PST) In-Reply-To: <24173.1515191675@critter.freebsd.dk> References: <24acbd94-c52f-e71a-8a96-d608a10963c6@rawbw.com> <1514572041.12000.7.camel@freebsd.org> <20180105221330.GD95035@spindle.one-eyed-alien.net> <24173.1515191675@critter.freebsd.dk> From: Conrad Meyer Date: Fri, 5 Jan 2018 15:17:31 -0800 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Is it considered to be ok to not check the return code of close(2) in base? To: Poul-Henning Kamp Cc: Freebsd hackers list Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jan 2018 23:17:39 -0000 On Fri, Jan 5, 2018 at 2:34 PM, Poul-Henning Kamp wrote: > Brookes suggestion, while well intentioned, wouldn't get very far, > because it is common for shells and shell-like programs to do: > > for (i = 3; i < ALOT; i++) > (void)close(i); > > To get rid of unwanted filedescriptors from syslog(3), getpwent(3) etc. > in the child process. > > Yes, I know about closefrom(2), but a lot of programs still don't use it. Hi, That seems like a good way to quickly identify programs in base that still do not use closefrom(). Best, Conrad From owner-freebsd-hackers@freebsd.org Fri Jan 5 23:24:16 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 28072EA6779 for ; Fri, 5 Jan 2018 23:24:16 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-it0-x232.google.com (mail-it0-x232.google.com [IPv6:2607:f8b0:4001:c0b::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DC2BF65321 for ; Fri, 5 Jan 2018 23:24:15 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-it0-x232.google.com with SMTP id o130so5889881itg.0 for ; Fri, 05 Jan 2018 15:24:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=GvG3oejfW9xjGvyN+mpfbk9gY590zsaEmxr2Bt4IGBs=; b=UXVoLH4lulIUsy+NYRKNw0AwThI0h3D+KEWlWL7JY9DfoobR58rNZjEB9J0z7LiyVv wyCICrFHU3Ohfsqe5QnAzADhQGAN5oOQSZFKAiktpWu3XUzy3RCo9KISM+GPBbaNgpwu U3D2Q63D/KP4PdZw0VrAmy8dAy6QwuLcnz6szYan3943jXP4bSALFzdWHupItwLCRe83 rdMRzF/gLM5Lu/n33CVNlIPfv3DHgePH3sxLDA0r0eQjwtUHzpc32tiHZ1Jt7Uc7R88+ XCd5NwMKEwDHOIABaW1y9JioJBafHYdCWhFPe9GF6Af5k5m3wVE7j4Y6sYBo0K9IJosc G4+Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=GvG3oejfW9xjGvyN+mpfbk9gY590zsaEmxr2Bt4IGBs=; b=H7W7KzKp4qrhqHThxoJNpAh6WOkCHLI6U5b7144paaGaS2I9dUBNhllPjbPquEMUe0 3ye7zUPhNHOo60l9IuJC9gan94ypaxS4RPrQmfNDnbXNeKKDhagQk7vbejzQvNrPL5Ad 3ba6245EY5/ZKOjAJcqsXlf4ss6ow6l0kU16ZRpo29m+4Dgsz0JsB8mqx2YEzAKU2RIK Rqp+2BE3JLCYDoWQfpQ99WC6ANCj47WlUgfA8wtgaEFIz31uT5DB7is+xP4TF0K9hqZn zLwNSIcS9mXHGjhQJwTVCI40P6VwXzFA7sTB4nBqv6qxnpN/yFzsSSh0qcrtvDaY2aji CkYw== X-Gm-Message-State: AKGB3mJuMWOGdKFG6HXRK/kFjgie7OdlRaleZ89H/KdybFg0mCDdVH9r +jAs+ETBWplgAzf8B/zF2tay3wG4z1yJZH9Wu8wJxHt1 X-Google-Smtp-Source: ACJfBotay+hdT0zImMRgcDqFBUqPj9lPbxTCPzQIaeyKtI7szzc/eXzmbasFn0RJh6o900rWOnWQIXKVhnr95api23E= X-Received: by 10.36.74.200 with SMTP id k191mr5053802itb.69.1515194654806; Fri, 05 Jan 2018 15:24:14 -0800 (PST) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 10.79.160.217 with HTTP; Fri, 5 Jan 2018 15:24:14 -0800 (PST) X-Originating-IP: [2603:300b:6:5100:1052:acc7:f9de:2b6d] In-Reply-To: References: <33bcd281-4018-7075-1775-4dfcd58e5a48@metricspace.net> <4ec1f3b1-f4b0-80ab-0e68-0dd679dd9e37@metricspace.net> <72f6097e-c71e-b53f-6885-cfe5a5a56586@metricspace.net> From: Warner Losh Date: Fri, 5 Jan 2018 16:24:14 -0700 X-Google-Sender-Auth: Q92uxMC-UbzBj8sh-MwHrMUtbsg Message-ID: Subject: Re: A more general possible meltdown/spectre countermeasure To: Eric McCorkle Cc: "freebsd-hackers@freebsd.org" , "freebsd-arch@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jan 2018 23:24:16 -0000 I mean the mappings we have in the kernel that map all of memory to a specific page using 512GB pages in sys/amd64/amd64/pmap.c:create_pagetables. This allows us to map any PA to a VA with simple math rather than a page table walk. Warner On Fri, Jan 5, 2018 at 4:10 PM, Eric McCorkle wrote: > I'm not sure what you mean by direct map. Do you mean TLB? > > On 01/05/2018 18:08, Warner Losh wrote: > > Wouldn't you have to also unmap it from the direct map for this to be > > effective? > > > > Warner > > > > > > On Fri, Jan 5, 2018 at 3:31 PM, Eric McCorkle > > wrote: > > > > Well, the only way to find out would be to try it out. > > > > However, unless I'm missing something, if you're trying to pull a > > meltdown attack, you try and fetch from the kernel. If that location > > isn't cached (or if your cache is physically indexed), you need the > > physical address (otherwise you don't know where to look), and thus > have > > to go through address translation, at which point you detect that the > > page isn't accessible and fault. In the mean time, you can't > > speculatively execute any of the operations that load up the > > side-channels, because you don't have the sensitive data. > > > > The reason you can pull off a meltdown attack at all is that a > > virtually-indexed cache lets you get the data in parallel with > address > > translation (breaking the dependency between address translation and > > fetching data), which takes 1000s of cycles for a TLB miss, during > which > > you have the data and can launch a whole bunch of transient ops. > > > > Again, these are uncharted waters we're in; so it's entirely possible > > I'm missing something here. > > > > On 01/05/2018 17:22, Warner Losh wrote: > > > While you might be right, I've seen no indication that a cache miss > > > would defeat these attacks in the public and non-public data I've > looked > > > at, even though a large number of alternatives to the published > > > workarounds have been discussed. I'm therefore somewhat skeptical > this > > > would be effective. I'm open, however, to data that changes that > > > skepticism... > > > > > > Warner > > > > > > On Fri, Jan 5, 2018 at 3:15 PM, Eric McCorkle < > eric@metricspace.net > > > >> > wrote: > > > > > > Right, but you have to get the value "foo" into the pipeline > in order > > > for it to affect the side-channels. This technique attempts > to stop > > > that from happening. > > > > > > Unless I made a mistake, non-cached memory reads force address > > > translation to happen first, which detects faults and blocks > the > > > meltdown attack. > > > > > > It also stops spectre with very high probability, as it's very > unlikely > > > that an uncached load will arrive before the speculative > thread gets > > > squashed. > > > > > > On 01/05/2018 17:10, Warner Losh wrote: > > > > I think this is fatally flawed. > > > > > > > > The side channel is the cache. Not the data at risk. > > > > > > > > Any mapped memory, cached or not, can be used to influence > the cache. > > > > Storing stuff in uncached memory won't affect the side > channel one bit. > > > > > > > > Basically, all attacks boil down to tricking the processor, > at elevated > > > > privs, to doing something like > > > > > > > > a = foo[offset]; > > > > > > > > where foo + offset are designed to communicate information > by populating > > > > a cache line. offset need not be cached itself and can be > the result of > > > > simple computations that depend on anything accessible at > all in the kernel. > > > > > > > > Warner > > > > > > > > On Fri, Jan 5, 2018 at 3:02 PM, Eric McCorkle < > eric@metricspace.net > > > > > > > > > >>> wrote: > > > > > > > > Re-posting to -hackers and -arch. I'm going to start > working on > > > > something like this over the weekend. > > > > > > > > -------- Forwarded Message -------- > > > > Subject: A more general possible meltdown/spectre > countermeasure > > > > Date: Thu, 4 Jan 2018 23:05:40 -0500 > > > > From: Eric McCorkle eric@metricspace.net> > > > > > > > > > >>> > > > > To: freebsd-security@freebsd.org freebsd-security@freebsd.org> > > > freebsd.org>> > > > > freebsd-security@freebsd.org> > > > freebsd.org>>> > > > > > freebsd.org>> > > > > > > > > > >>> > > > > > > > > I've thought more about how to deal with > > meltdown/spectre, and > > > I have an > > > > idea I'd like to put forward. However, I'm still in > > something > > > of a > > > > panic mode, so I'm not certain as to its effectiveness. > > > Needless to > > > > say, I welcome any feedback on this, and I may be > completely > > > off-base. > > > > > > > > I'm calling this a "countermeasure" as opposed to a > > > "mitigation", as > > > > it's something that requires modification of code as > > opposed to a > > > > drop-in patch. > > > > > > > > == Summary == > > > > > > > > Provide a kernel and userland API by which memory > allocation > > > can be done > > > > with extended attributes. In userland, this could be > > > accomplished by > > > > extending MMAP flags, and I could imagine a > > > malloc-with-attributes flag. > > > > In kernel space, this must already exist, as drivers > > need to > > > allocate > > > > memory with various MTRR-type attributes set. > > > > > > > > The immediate aim here is to store sensitive information > > that must > > > > remain memory-resident in non-cacheable memory locations > > (or, > > > if more > > > > effective attribute combinations exist, using those > > instead). > > > See the > > > > rationale for the argument why this should work. > > > > > > > > Assuming the rationale holds, then the attack surface > should > > > be greatly > > > > reduced. Attackers would need to grab sensitive data > > out of stack > > > > frames or similar locations if/when it gets copied there > for > > > faster use. > > > > Moreover, if this is done right, it could dovetail > > nicely into a > > > > framework for storing and processing sensitive assets in > > more > > > secure > > > > hardware[0] (like smart cards, the FPGAs I posted > > earlier, or > > > other > > > > options). > > > > > > > > The obvious downside is that you take a performance hit > > > storing things > > > > in non-cacheable locations, especially if you plan on > > doing heavy > > > > computation in that memory (say, encryption/decryption). > > > However, this > > > > is almost certainly going to be less than the projected > > 30-50% > > > > performance hit from other mitigations. Also, this > > technique > > > should > > > > work against spectre as well as meltdown (assuming the > > > rationale holds). > > > > > > > > The second downside is that you have to modify code for > this > > > to work, > > > > and you have to be careful not to keep copies of > sensitive > > > information > > > > around too long (this gets tricky in userland, where you > > might get > > > > interrupted and switched out). > > > > > > > > > > > > [0]: Full disclosure, enabling open hardware > implementations > > > of this > > > > kind of thing is something of an agenda of mine. > > > > > > > > == Rationale == > > > > > > > > (Again, I'm tired, rushed, and somewhat panicked so my > logic > > > could be > > > > faulty at any point, so please point it out if it is) > > > > > > > > The rationale for why this should work relies on > > assumptions about > > > > out-of-order pipelines that cannot be guaranteed to > > hold, but are > > > > extremely likely to be true. > > > > > > > > As background, these attacks depend on out-of-order > > execution > > > performing > > > > operations that end up affecting cache and > branch-prediction > > > state, > > > > ultimately storing information about sensitive data in > these > > > > side-channels before the fault conditions are detected > and > > > acted upon. > > > > I'll borrow terminology from the paper, using "transient > > > instructions" > > > > to refer to speculatively executed instructions that will > > > eventually be > > > > cancelled by a fault. > > > > > > > > These attacks depend entirely on transient instructions > > being > > > able to > > > > get sensitive information into the processor core and > then > > > perform some > > > > kind of instruction on them before the fault condition > > cancels > > > them. > > > > Therefore, anything that prevents them from doing this > > > *should* counter > > > > the attack. If the actual sensitive data never makes it > to > > > the core > > > > before the fault is detected, the dependent memory > > > accesses/branches > > > > never get executed and the data never makes it to the > > > side-channels. > > > > > > > > Another assumption here is that CPU architects are going > to > > > want to > > > > squash faulted instructions ASAP and stop issuing along > > those > > > > speculative branches, so as to reclaim execution units. > So > > > I'm assuming > > > > once a fault comes back from address translation, then > > transient > > > > execution stops dead. > > > > > > > > Now, break down the cases for whether the address > containing > > > sensitive > > > > data is in cache and TLB or not. (I'm assuming here that > > > caches are > > > > virtually-indexed, which enables cache lookups to bypass > > address > > > > translation.) > > > > > > > > * In cache, in TLB: You end up basically racing between > the > > > cache and > > > > TLB, which will very likely end up detecting the fault > > before > > > the data > > > > arrives, but at the very worst, you get one or two > cycles of > > > transient > > > > instruction execution before the fault. > > > > > > > > * In cache, not in TLB: Virtually-indexed tagged means > > you get > > > a cache > > > > lookup racing a page-table walk. The cache lookup beats > the > > > page table > > > > walk by potentially hundreds (maybe thousands) of cycles, > > > giving you a > > > > bunch of transient instructions before a fault gets > > > triggered. This is > > > > the main attack case. > > > > > > > > * Not in cache, in TLB: Memory access requires address > > > translation, > > > > which comes back almost immediately as a fault. > > > > > > > > * Not in cache, not in TLB: You have to do a page table > walk > > > before you > > > > can fetch the location, as you have to go out to physical > > > memory (and > > > > therefore need a physical address). The page table walk > > will > > > come back > > > > with a fault, stopping the attack. > > > > > > > > So, unless I'm missing something here, both non-cached > cases > > > defeat the > > > > meltdown attack, as you *cannot* get the data unless you > do > > > address > > > > translation first (and therefore detect faults). > > > > > > > > As for why this defeats the spectre attack, the logic is > > > similar: you've > > > > jumped into someone else's executable code, hoping to > > scoop up > > > enough > > > > information into your branch predictor before the fault > > kicks > > > you out. > > > > However, to capture anything about sensitive information > > in your > > > > side-channels, the transient instructions need to > > actually get > > > it into > > > > the core before a fault gets detected. The same case > > analysis > > > as above > > > > applies, so you never actually get the sensitive info > > into the > > > core > > > > before a fault comes back and you get squashed. > > > > > > > > > > > > [1]: A physically-indexed cache would be largely immune > to > > > this attack, > > > > as you'd have to do address translation before doing a > cache > > > lookup. > > > > > > > > > > > > I have some ideas that can build on this, but I'd like > > to get some > > > > feedback first. > > > > _______________________________________________ > > > > freebsd-security@freebsd.org > > > > > > > > > > > > > > freebsd.org>>> > > > > mailing list > > > > https://lists.freebsd.org/mailman/listinfo/freebsd- > security > > > > > > > > > > > security > > > > > > >> > > > > To unsubscribe, send any mail to > > > > "freebsd-security-unsubscribe@freebsd.org > > > > > > > > > > > > > > > > >>" > > > > _______________________________________________ > > > > freebsd-arch@freebsd.org org> > > > > > > > > >> > > > mailing list > > > > https://lists.freebsd.org/mailman/listinfo/freebsd-arch > > > > > > > > > > > > > > > > >> > > > > To unsubscribe, send any mail to > > > > "freebsd-arch-unsubscribe@freebsd.org > > > > > > > > > > > > > > > > >>" > > > > > > > > > > > > > > > > > > > From owner-freebsd-hackers@freebsd.org Fri Jan 5 23:30:21 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 946F4EA6C9A; Fri, 5 Jan 2018 23:30:21 +0000 (UTC) (envelope-from eric@metricspace.net) Received: from mail.metricspace.net (mail.metricspace.net [IPv6:2001:470:1f11:617::107]) by mx1.freebsd.org (Postfix) with ESMTP id CAFD56582F; Fri, 5 Jan 2018 23:30:20 +0000 (UTC) (envelope-from eric@metricspace.net) Received: from [10.148.202.109] (mobile-166-171-187-244.mycingular.net [166.171.187.244]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) (Authenticated sender: eric) by mail.metricspace.net (Postfix) with ESMTPSA id C40378AD8; Fri, 5 Jan 2018 23:30:16 +0000 (UTC) Date: Fri, 05 Jan 2018 18:30:14 -0500 User-Agent: K-9 Mail for Android In-Reply-To: References: <33bcd281-4018-7075-1775-4dfcd58e5a48@metricspace.net> <4ec1f3b1-f4b0-80ab-0e68-0dd679dd9e37@metricspace.net> <72f6097e-c71e-b53f-6885-cfe5a5a56586@metricspace.net> MIME-Version: 1.0 Subject: Re: A more general possible meltdown/spectre countermeasure To: Warner Losh CC: "freebsd-hackers@freebsd.org" , "freebsd-arch@freebsd.org" From: Eric McCorkle Message-ID: <9268C1F8-AD68-4B20-94D7-96B5FD6589B5@metricspace.net> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jan 2018 23:30:21 -0000 Ah, superpages=2E I wouldn't think so=2E The cpu still has to do a page tab= le walk (just stopping at the top level page table), and would discover tha= t it's not accessible=2E On January 5, 2018 6:24:14 PM EST, Warner Losh wrote: >I mean the mappings we have in the kernel that map all of memory to a >specific page using 512GB pages in >sys/amd64/amd64/pmap=2Ec:create_pagetables=2E This allows us to map any P= A >to a >VA with simple math rather than a page table walk=2E > >Warner > >On Fri, Jan 5, 2018 at 4:10 PM, Eric McCorkle >wrote: > >> I'm not sure what you mean by direct map=2E Do you mean TLB? >> >> On 01/05/2018 18:08, Warner Losh wrote: >> > Wouldn't you have to also unmap it from the direct map for this to >be >> > effective? >> > >> > Warner >> > >> > >> > On Fri, Jan 5, 2018 at 3:31 PM, Eric McCorkle > > > wrote: >> > >> > Well, the only way to find out would be to try it out=2E >> > >> > However, unless I'm missing something, if you're trying to pull >a >> > meltdown attack, you try and fetch from the kernel=2E If that >location >> > isn't cached (or if your cache is physically indexed), you need >the >> > physical address (otherwise you don't know where to look), and >thus >> have >> > to go through address translation, at which point you detect >that the >> > page isn't accessible and fault=2E In the mean time, you can't >> > speculatively execute any of the operations that load up the >> > side-channels, because you don't have the sensitive data=2E >> > >> > The reason you can pull off a meltdown attack at all is that a >> > virtually-indexed cache lets you get the data in parallel with >> address >> > translation (breaking the dependency between address >translation and >> > fetching data), which takes 1000s of cycles for a TLB miss, >during >> which >> > you have the data and can launch a whole bunch of transient >ops=2E >> > >> > Again, these are uncharted waters we're in; so it's entirely >possible >> > I'm missing something here=2E >> > >> > On 01/05/2018 17:22, Warner Losh wrote: >> > > While you might be right, I've seen no indication that a >cache miss >> > > would defeat these attacks in the public and non-public data >I've >> looked >> > > at, even though a large number of alternatives to the >published >> > > workarounds have been discussed=2E I'm therefore somewhat >skeptical >> this >> > > would be effective=2E I'm open, however, to data that changes >that >> > > skepticism=2E=2E=2E >> > > >> > > Warner >> > > >> > > On Fri, Jan 5, 2018 at 3:15 PM, Eric McCorkle < >> eric@metricspace=2Enet >> > > >= > >> wrote: >> > > >> > > Right, but you have to get the value "foo" into the >pipeline >> in order >> > > for it to affect the side-channels=2E This technique >attempts >> to stop >> > > that from happening=2E >> > > >> > > Unless I made a mistake, non-cached memory reads force >address >> > > translation to happen first, which detects faults and >blocks >> the >> > > meltdown attack=2E >> > > >> > > It also stops spectre with very high probability, as it's >very >> unlikely >> > > that an uncached load will arrive before the speculative >> thread gets >> > > squashed=2E >> > > >> > > On 01/05/2018 17:10, Warner Losh wrote: >> > > > I think this is fatally flawed=2E >> > > > >> > > > The side channel is the cache=2E Not the data at risk=2E >> > > > >> > > > Any mapped memory, cached or not, can be used to >influence >> the cache=2E >> > > > Storing stuff in uncached memory won't affect the side >> channel one bit=2E >> > > > >> > > > Basically, all attacks boil down to tricking the >processor, >> at elevated >> > > > privs, to doing something like >> > > > >> > > > a =3D foo[offset]; >> > > > >> > > > where foo + offset are designed to communicate >information >> by populating >> > > > a cache line=2E offset need not be cached itself and can >be >> the result of >> > > > simple computations that depend on anything accessible >at >> all in the kernel=2E >> > > > >> > > > Warner >> > > > >> > > > On Fri, Jan 5, 2018 at 3:02 PM, Eric McCorkle < >> eric@metricspace=2Enet >> > > >> > > > >> > >>> >wrote: >> > > > >> > > > Re-posting to -hackers and -arch=2E I'm going to >start >> working on >> > > > something like this over the weekend=2E >> > > > >> > > > -------- Forwarded Message -------- >> > > > Subject: A more general possible meltdown/spectre >> countermeasure >> > > > Date: Thu, 4 Jan 2018 23:05:40 -0500 >> > > > From: Eric McCorkle > eric@metricspace=2Enet> >> > > > >> > >> > > >>> >> > > > To: freebsd-security@freebsd=2Eorg > freebsd-security@freebsd=2Eorg> >> > > > freebsd=2Eorg>> >> > > > > freebsd-security@freebsd=2Eorg> >> > > > freebsd=2Eorg>>> >> > >> > > > freebsd=2Eorg>> >> > > > > > >> > > > > >>> >> > > > >> > > > I've thought more about how to deal with >> > meltdown/spectre, and >> > > I have an >> > > > idea I'd like to put forward=2E However, I'm still >in >> > something >> > > of a >> > > > panic mode, so I'm not certain as to its >effectiveness=2E >> > > Needless to >> > > > say, I welcome any feedback on this, and I may be >> completely >> > > off-base=2E >> > > > >> > > > I'm calling this a "countermeasure" as opposed to a >> > > "mitigation", as >> > > > it's something that requires modification of code >as >> > opposed to a >> > > > drop-in patch=2E >> > > > >> > > > =3D=3D Summary =3D=3D >> > > > >> > > > Provide a kernel and userland API by which memory >> allocation >> > > can be done >> > > > with extended attributes=2E In userland, this could >be >> > > accomplished by >> > > > extending MMAP flags, and I could imagine a >> > > malloc-with-attributes flag=2E >> > > > In kernel space, this must already exist, as >drivers >> > need to >> > > allocate >> > > > memory with various MTRR-type attributes set=2E >> > > > >> > > > The immediate aim here is to store sensitive >information >> > that must >> > > > remain memory-resident in non-cacheable memory >locations >> > (or, >> > > if more >> > > > effective attribute combinations exist, using those >> > instead)=2E >> > > See the >> > > > rationale for the argument why this should work=2E >> > > > >> > > > Assuming the rationale holds, then the attack >surface >> should >> > > be greatly >> > > > reduced=2E Attackers would need to grab sensitive >data >> > out of stack >> > > > frames or similar locations if/when it gets copied >there >> for >> > > faster use=2E >> > > > Moreover, if this is done right, it could dovetail >> > nicely into a >> > > > framework for storing and processing sensitive >assets in >> > more >> > > secure >> > > > hardware[0] (like smart cards, the FPGAs I posted >> > earlier, or >> > > other >> > > > options)=2E >> > > > >> > > > The obvious downside is that you take a performance >hit >> > > storing things >> > > > in non-cacheable locations, especially if you plan >on >> > doing heavy >> > > > computation in that memory (say, >encryption/decryption)=2E >> > > However, this >> > > > is almost certainly going to be less than the >projected >> > 30-50% >> > > > performance hit from other mitigations=2E Also, this >> > technique >> > > should >> > > > work against spectre as well as meltdown (assuming >the >> > > rationale holds)=2E >> > > > >> > > > The second downside is that you have to modify code >for >> this >> > > to work, >> > > > and you have to be careful not to keep copies of >> sensitive >> > > information >> > > > around too long (this gets tricky in userland, >where you >> > might get >> > > > interrupted and switched out)=2E >> > > > >> > > > >> > > > [0]: Full disclosure, enabling open hardware >> implementations >> > > of this >> > > > kind of thing is something of an agenda of mine=2E >> > > > >> > > > =3D=3D Rationale =3D=3D >> > > > >> > > > (Again, I'm tired, rushed, and somewhat panicked so >my >> logic >> > > could be >> > > > faulty at any point, so please point it out if it >is) >> > > > >> > > > The rationale for why this should work relies on >> > assumptions about >> > > > out-of-order pipelines that cannot be guaranteed to >> > hold, but are >> > > > extremely likely to be true=2E >> > > > >> > > > As background, these attacks depend on out-of-order >> > execution >> > > performing >> > > > operations that end up affecting cache and >> branch-prediction >> > > state, >> > > > ultimately storing information about sensitive data >in >> these >> > > > side-channels before the fault conditions are >detected >> and >> > > acted upon=2E >> > > > I'll borrow terminology from the paper, using >"transient >> > > instructions" >> > > > to refer to speculatively executed instructions >that will >> > > eventually be >> > > > cancelled by a fault=2E >> > > > >> > > > These attacks depend entirely on transient >instructions >> > being >> > > able to >> > > > get sensitive information into the processor core >and >> then >> > > perform some >> > > > kind of instruction on them before the fault >condition >> > cancels >> > > them=2E >> > > > Therefore, anything that prevents them from doing >this >> > > *should* counter >> > > > the attack=2E If the actual sensitive data never >makes it >> to >> > > the core >> > > > before the fault is detected, the dependent memory >> > > accesses/branches >> > > > never get executed and the data never makes it to >the >> > > side-channels=2E >> > > > >> > > > Another assumption here is that CPU architects are >going >> to >> > > want to >> > > > squash faulted instructions ASAP and stop issuing >along >> > those >> > > > speculative branches, so as to reclaim execution >units=2E >> So >> > > I'm assuming >> > > > once a fault comes back from address translation, >then >> > transient >> > > > execution stops dead=2E >> > > > >> > > > Now, break down the cases for whether the address >> containing >> > > sensitive >> > > > data is in cache and TLB or not=2E (I'm assuming >here that >> > > caches are >> > > > virtually-indexed, which enables cache lookups to >bypass >> > address >> > > > translation=2E) >> > > > >> > > > * In cache, in TLB: You end up basically racing >between >> the >> > > cache and >> > > > TLB, which will very likely end up detecting the >fault >> > before >> > > the data >> > > > arrives, but at the very worst, you get one or two >> cycles of >> > > transient >> > > > instruction execution before the fault=2E >> > > > >> > > > * In cache, not in TLB: Virtually-indexed tagged >means >> > you get >> > > a cache >> > > > lookup racing a page-table walk=2E The cache lookup >beats >> the >> > > page table >> > > > walk by potentially hundreds (maybe thousands) of >cycles, >> > > giving you a >> > > > bunch of transient instructions before a fault gets >> > > triggered=2E This is >> > > > the main attack case=2E >> > > > >> > > > * Not in cache, in TLB: Memory access requires >address >> > > translation, >> > > > which comes back almost immediately as a fault=2E >> > > > >> > > > * Not in cache, not in TLB: You have to do a page >table >> walk >> > > before you >> > > > can fetch the location, as you have to go out to >physical >> > > memory (and >> > > > therefore need a physical address)=2E The page table >walk >> > will >> > > come back >> > > > with a fault, stopping the attack=2E >> > > > >> > > > So, unless I'm missing something here, both >non-cached >> cases >> > > defeat the >> > > > meltdown attack, as you *cannot* get the data >unless you >> do >> > > address >> > > > translation first (and therefore detect faults)=2E >> > > > >> > > > As for why this defeats the spectre attack, the >logic is >> > > similar: you've >> > > > jumped into someone else's executable code, hoping >to >> > scoop up >> > > enough >> > > > information into your branch predictor before the >fault >> > kicks >> > > you out=2E >> > > > However, to capture anything about sensitive >information >> > in your >> > > > side-channels, the transient instructions need to >> > actually get >> > > it into >> > > > the core before a fault gets detected=2E The same >case >> > analysis >> > > as above >> > > > applies, so you never actually get the sensitive >info >> > into the >> > > core >> > > > before a fault comes back and you get squashed=2E >> > > > >> > > > >> > > > [1]: A physically-indexed cache would be largely >immune >> to >> > > this attack, >> > > > as you'd have to do address translation before >doing a >> cache >> > > lookup=2E >> > > > >> > > > >> > > > I have some ideas that can build on this, but I'd >like >> > to get some >> > > > feedback first=2E >> > > > _______________________________________________ >> > > > freebsd-security@freebsd=2Eorg >> > >> > > > > > >> > > > > >> > > > freebsd=2Eorg>>> >> > > > mailing list >> > > > https://lists=2Efreebsd=2Eorg/mailman/listinfo/freebs= d- >> security >> > >> > > =20 >> > = > >> > > > =20 >> security >> > >> > > =20 >> > = >> >> > > > To unsubscribe, send any mail to >> > > > "freebsd-security-unsubscribe@freebsd=2Eorg >> > >> > > > > > >> > > > > > >> > > > > >>" >> > > > _______________________________________________ >> > > > freebsd-arch@freebsd=2Eorg >> org> >> > > >> > > > > >> > >> >> > > mailing list >> > > > =20 >https://lists=2Efreebsd=2Eorg/mailman/listinfo/freebsd-arch >> > >> > > > > > >> > > > =20 >> > >> > > > > >> >> > > > To unsubscribe, send any mail to >> > > > "freebsd-arch-unsubscribe@freebsd=2Eorg >> > >> > > > > > >> > > > > > >> > > > > >>" >> > > > >> > > > >> > > >> > > >> > >> > >> --=20 Sent from my Android device with K-9 Mail=2E Please excuse my brevity=2E From owner-freebsd-hackers@freebsd.org Fri Jan 5 23:34:03 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DD57DEA7125 for ; Fri, 5 Jan 2018 23:34:03 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id 9F4EA65FB6; Fri, 5 Jan 2018 23:34:03 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (unknown [192.168.55.3]) by phk.freebsd.dk (Postfix) with ESMTP id 463782739C; Fri, 5 Jan 2018 23:33:58 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.15.2/8.15.2) with ESMTPS id w05NXgls024456 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 5 Jan 2018 23:33:42 GMT (envelope-from phk@critter.freebsd.dk) Received: (from phk@localhost) by critter.freebsd.dk (8.15.2/8.15.2/Submit) id w05NXgj0024455; Fri, 5 Jan 2018 23:33:42 GMT (envelope-from phk) To: cem@freebsd.org cc: Freebsd hackers list Subject: Re: Is it considered to be ok to not check the return code of close(2) in base? In-reply-to: From: "Poul-Henning Kamp" References: <24acbd94-c52f-e71a-8a96-d608a10963c6@rawbw.com> <1514572041.12000.7.camel@freebsd.org> <20180105221330.GD95035@spindle.one-eyed-alien.net> <24173.1515191675@critter.freebsd.dk> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <24453.1515195222.1@critter.freebsd.dk> Content-Transfer-Encoding: quoted-printable Date: Fri, 05 Jan 2018 23:33:42 +0000 Message-ID: <24454.1515195222@critter.freebsd.dk> X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jan 2018 23:34:04 -0000 -------- In message , Conra d Meyer writes: >On Fri, Jan 5, 2018 at 2:34 PM, Poul-Henning Kamp wr= ote: >> Brookes suggestion, while well intentioned, wouldn't get very far, >> because it is common for shells and shell-like programs to do: >> >> for (i =3D 3; i < ALOT; i++) >> (void)close(i); >> >> To get rid of unwanted filedescriptors from syslog(3), getpwent(3) etc. >> in the child process. >> >> Yes, I know about closefrom(2), but a lot of programs still don't use i= t. > >Hi, > >That seems like a good way to quickly identify programs in base that >still do not use closefrom(). Absolutely, by all means *identify* these programs, but before you start *killing* them, make sure you your system can actually function: $ cd /usr/src/bin/sh $ find . -name '*.c' -print | xargs grep closefrom $ cd /usr/src/contrib/tcsh $ find . -name '*.c' -print | xargs grep closefrom $ = As I said: I'm all for making the -current kernel more paranoid about userland, but log the results to syslog (with rate-limiting!), don't just kill the process. -- = Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe = Never attribute to malice what can adequately be explained by incompetence= . From owner-freebsd-hackers@freebsd.org Sat Jan 6 01:11:03 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D1647EABE63 for ; Sat, 6 Jan 2018 01:11:03 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from outbound1b.ore.mailhop.org (outbound1b.ore.mailhop.org [54.200.247.200]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B32926988B for ; Sat, 6 Jan 2018 01:11:03 +0000 (UTC) (envelope-from ian@freebsd.org) X-MHO-User: 6859a23e-f27e-11e7-8486-0934409070aa X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 73.78.92.27 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [73.78.92.27]) by outbound1.ore.mailhop.org (Halon) with ESMTPSA id 6859a23e-f27e-11e7-8486-0934409070aa; Sat, 06 Jan 2018 01:10:39 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id w061AtIH002449; Fri, 5 Jan 2018 18:10:55 -0700 (MST) (envelope-from ian@freebsd.org) Message-ID: <1515201055.1865.5.camel@freebsd.org> Subject: Re: Is it considered to be ok to not check the return code of close(2) in base? From: Ian Lepore To: cem@freebsd.org, Poul-Henning Kamp Cc: Freebsd hackers list Date: Fri, 05 Jan 2018 18:10:55 -0700 In-Reply-To: References: <24acbd94-c52f-e71a-8a96-d608a10963c6@rawbw.com> <1514572041.12000.7.camel@freebsd.org> <20180105221330.GD95035@spindle.one-eyed-alien.net> <24173.1515191675@critter.freebsd.dk> Content-Type: text/plain; charset="ISO-8859-1" X-Mailer: Evolution 3.18.5.1 FreeBSD GNOME Team Port Mime-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Jan 2018 01:11:03 -0000 On Fri, 2018-01-05 at 15:17 -0800, Conrad Meyer wrote: > On Fri, Jan 5, 2018 at 2:34 PM, Poul-Henning Kamp > wrote: > > > > Brookes suggestion, while well intentioned, wouldn't get very far, > > because it is common for shells and shell-like programs to do: > > > >         for (i = 3; i < ALOT; i++) > >                 (void)close(i); > > > > To get rid of unwanted filedescriptors from syslog(3), getpwent(3) > > etc. > > in the child process. > > > > Yes, I know about closefrom(2), but a lot of programs still don't > > use it. > Hi, > > That seems like a good way to quickly identify programs in base that > still do not use closefrom(). > > Best, > Conrad > Portable programs MUST iterate 3-ALOT, because closefrom() is not available in linux and other OSes. -- Ian From owner-freebsd-hackers@freebsd.org Sat Jan 6 05:28:10 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9056DEBC597; Sat, 6 Jan 2018 05:28:10 +0000 (UTC) (envelope-from eric@metricspace.net) Received: from mail.metricspace.net (mail.metricspace.net [IPv6:2001:470:1f11:617::107]) by mx1.freebsd.org (Postfix) with ESMTP id 56F3274F06; Sat, 6 Jan 2018 05:28:10 +0000 (UTC) (envelope-from eric@metricspace.net) Received: from [IPv6:2001:470:1f11:617:3210:b3ff:fe77:becd] (unknown [IPv6:2001:470:1f11:617:3210:b3ff:fe77:becd]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: eric) by mail.metricspace.net (Postfix) with ESMTPSA id 6235F8BCD; Sat, 6 Jan 2018 05:28:09 +0000 (UTC) Subject: Re: A more general possible meltdown/spectre countermeasure To: Nathan Dautenhahn Cc: Warner Losh , "freebsd-hackers@freebsd.org" , "freebsd-arch@freebsd.org" References: <33bcd281-4018-7075-1775-4dfcd58e5a48@metricspace.net> <4ec1f3b1-f4b0-80ab-0e68-0dd679dd9e37@metricspace.net> <72f6097e-c71e-b53f-6885-cfe5a5a56586@metricspace.net> <9268C1F8-AD68-4B20-94D7-96B5FD6589B5@metricspace.net> From: Eric McCorkle Message-ID: Date: Sat, 6 Jan 2018 00:28:08 -0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.5.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="V9KkvxoWVZPOhkSqZbg4YYxT3k6ucNyKb" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Jan 2018 05:28:10 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --V9KkvxoWVZPOhkSqZbg4YYxT3k6ucNyKb Content-Type: multipart/mixed; boundary="VvDKQxWj1LBLDsO8kW6mDydav7SWPi8CE"; protected-headers="v1" From: Eric McCorkle To: Nathan Dautenhahn Cc: Warner Losh , "freebsd-hackers@freebsd.org" , "freebsd-arch@freebsd.org" Message-ID: Subject: Re: A more general possible meltdown/spectre countermeasure References: <33bcd281-4018-7075-1775-4dfcd58e5a48@metricspace.net> <4ec1f3b1-f4b0-80ab-0e68-0dd679dd9e37@metricspace.net> <72f6097e-c71e-b53f-6885-cfe5a5a56586@metricspace.net> <9268C1F8-AD68-4B20-94D7-96B5FD6589B5@metricspace.net> In-Reply-To: --VvDKQxWj1LBLDsO8kW6mDydav7SWPi8CE Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable On 01/05/2018 23:32, Nathan Dautenhahn wrote: > Another solution, which would handle the more complex attack above, (I > know I'm piggybacking, not sure if that's bad) could be to partition a > subset of the kernel address space for secrets, and then map those in > only when needed, and flush out when done. I did some work a while > back on page table isolation and protection from potentially malicious > OSs called the nested kernel. I haven't reviewed these new > side-channel attacks in great detail yet, but I'm currently working on > pushing fine grained intra-address space isolation that might be a > nice solution for easily managing and subsets of kernel data. >=20 > The paper and associated code etc. is all linked on nestedkernel.org. > I think these attacks really motivate the nested kernel approach, > although we didn't consider secret protection from side-channels. This sounds more or less like what I had in mind: carve out some special region of kernel space for sensitive information. Ideally, this could be swapped out with an API for storing sensitive data in a secure device (TPM, smart card, etc). However, discussions of this approach over on the RISC-V lists suggest that Intel apparently does some rather crazy things that end up thwarting my proposed countermeasure. (Apparently, they don't acknowledge faults until the faulting instruction *commits*, which means any number of transients could have followed) I'll probably still put a PoC together, but I fear it may not work on Intel. --VvDKQxWj1LBLDsO8kW6mDydav7SWPi8CE-- --V9KkvxoWVZPOhkSqZbg4YYxT3k6ucNyKb Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iIsEARYIADMWIQTp6hWnRH4nHb9/QN/kI/o6qzq6mAUCWlBeaBUcZXJpY0BtZXRy aWNzcGFjZS5uZXQACgkQ5CP6Oqs6upjzSwEA70zkEBdc+WWD+OWW3GqXtw8rTZPs 3wEP+wuiThvhsoQBAKVcbCgztidemCIOCnlMO8QEyeZnG7v9YBjSnoeXeowN =czOH -----END PGP SIGNATURE----- --V9KkvxoWVZPOhkSqZbg4YYxT3k6ucNyKb-- From owner-freebsd-hackers@freebsd.org Sat Jan 6 09:19:15 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 79AA4EA4049 for ; Sat, 6 Jan 2018 09:19:15 +0000 (UTC) (envelope-from johalun0@gmail.com) Received: from mail-wm0-x229.google.com (mail-wm0-x229.google.com [IPv6:2a00:1450:400c:c09::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 10A4D7E1D8 for ; Sat, 6 Jan 2018 09:19:15 +0000 (UTC) (envelope-from johalun0@gmail.com) Received: by mail-wm0-x229.google.com with SMTP id t8so6213827wmc.3 for ; Sat, 06 Jan 2018 01:19:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=65JdumoBjEe7YxfUM8LYu5evRAdBZyf4UGsvOlQ9wS8=; b=rTfLEQGV7W9Ttbcj9V3mJQ5qAIO9AP9HFAh7rG3YGYa8tn9Nmbh04xR2dXzgLx2rnd Xht/MmAsvbJEibJihsqGX7fFyi3xHgiSNXp0nEyI9JADgp09zfcKjXRnGVh3LFfXjUE8 mUQitZ9mbQND7mHXwWgNnRvmTW0jT4J3jFRJ1DsSjcI2a9JDX8elOSiuMWxNXoUMU4j9 Y8axqQqXIBeMIhW1dB6r81loMdtjk53qKRzGjWAO1kbc5/97dE6PXgKJXMpeq0YrO7yy cGTQwDbbSrfEthPA2KXr+ZtXWVojXVV/dhn0r5Cx95C/F0RqzXH02pbV0k+EbdLAbABa QPzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=65JdumoBjEe7YxfUM8LYu5evRAdBZyf4UGsvOlQ9wS8=; b=Ip3YElsksTREMG7mA6jCd7t1TDIpKMld1/9dcORhvQo1dFe3YKYK9rj1oxIi7CqYap wXn3aAmxgBg/P6Ei4c9RSz6oX+20dp4oeobkHX4nkz3XNZR7oyqG1TKOmcw0gHSbo6yF 2m/g50XPfWp57gbjxmIdhwvdf8KtrM6J4led4ULosts9n0SbnFuUL4gnl2k+YZVg+ydo cqtypw+HV1zPNi2BFSCpMP5OE98EWa043twridtLRrA+6JxwYSqVgqvXxxvLUmF99DwJ K5yFyYnR/BDuvhs2shd8kvlcEUeOb/sG6xe84OUk0zGBWROP9v7gMGXpG14M/TC3fNoB T+0A== X-Gm-Message-State: AKGB3mKm1lkoP71VtNzxSCXvSdrLCWy6b1/rbdFf1cNXmiYphlapb0c8 Bv03sV2iKamIH13aWw2dlKb2tDHNNIF21GR7m+U= X-Google-Smtp-Source: ACJfBou0a+F3Bb/PygXSBTFerijP/KaFswHebH/AhL/svKgpj7bp4pn7umI7psOOzM8En9vTTJg9D+SGWvUin+m8TiU= X-Received: by 10.28.26.194 with SMTP id a185mr4609564wma.82.1515230352877; Sat, 06 Jan 2018 01:19:12 -0800 (PST) MIME-Version: 1.0 Received: by 10.223.197.70 with HTTP; Sat, 6 Jan 2018 01:18:32 -0800 (PST) From: Johannes Lundberg Date: Sat, 6 Jan 2018 09:18:32 +0000 Message-ID: Subject: callout_reset_on() with wait? To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Jan 2018 09:19:15 -0000 Hi I need to execute some code on another cpu (specified by cpu #) and wait for return. Is there any one function to call for this in freebsd 12 kernel? (or example where similar is implemented?) Cheers! From owner-freebsd-hackers@freebsd.org Sat Jan 6 10:48:41 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A4078EB24E0 for ; Sat, 6 Jan 2018 10:48:41 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3A37968B5B for ; Sat, 6 Jan 2018 10:48:41 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id w06AmX66044870 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sat, 6 Jan 2018 12:48:36 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua w06AmX66044870 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id w06AmX41044869; Sat, 6 Jan 2018 12:48:33 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 6 Jan 2018 12:48:33 +0200 From: Konstantin Belousov To: Johannes Lundberg Cc: freebsd-hackers@freebsd.org Subject: Re: callout_reset_on() with wait? Message-ID: <20180106104833.GB1684@kib.kiev.ua> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.9.2 (2017-12-15) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Jan 2018 10:48:41 -0000 On Sat, Jan 06, 2018 at 09:18:32AM +0000, Johannes Lundberg wrote: > Hi > > I need to execute some code on another cpu (specified by cpu #) and wait > for return. Is there any one function to call for this in freebsd 12 > kernel? (or example where similar is implemented?) There is smp_rendezvous_cpus() which does exactly that, but the context where the functions are called do not allow any synchronization, except carefully prepared spin locks. Your use of the word 'wait' leaves a lot to guess. From owner-freebsd-hackers@freebsd.org Sat Jan 6 04:33:28 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5DB57EB86CC for ; Sat, 6 Jan 2018 04:33:28 +0000 (UTC) (envelope-from ndd@cis.upenn.edu) Received: from hound.seas.upenn.edu (hound.seas.upenn.edu [IPv6:2607:f470:8:64:5ea5::23]) by mx1.freebsd.org (Postfix) with ESMTP id BB22C7214C for ; Sat, 6 Jan 2018 04:33:27 +0000 (UTC) (envelope-from ndd@cis.upenn.edu) Received: from mail-it0-f45.google.com (mail-it0-f45.google.com [209.85.214.45]) (authenticated bits=0) by hound.seas.upenn.edu (8.15.2/8.14.5) with ESMTPSA id w064XPGE031837 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 5 Jan 2018 23:33:26 -0500 Received: by mail-it0-f45.google.com with SMTP id c16so3736461itc.5 for ; Fri, 05 Jan 2018 20:33:26 -0800 (PST) X-Gm-Message-State: AKGB3mLPYLM7RlVoFkRteQuvCsyd/LE0ehXZutY3n93zKKBJwgdFLJE9 VRH4Dmi6RgMt8/N+nvRpLWLswL/86PwyFK8wCIkwqQ== X-Google-Smtp-Source: ACJfBos7DP42kbJk6US/uImtsQr1opuq3pic7I8T0mUJbgDwzX3/S/gQr4UePcKMqgsi5o02rRsCZHue0E3K9UxOKoA= X-Received: by 10.36.40.141 with SMTP id h135mr5120760ith.77.1515213200310; Fri, 05 Jan 2018 20:33:20 -0800 (PST) MIME-Version: 1.0 Received: by 10.2.132.76 with HTTP; Fri, 5 Jan 2018 20:32:59 -0800 (PST) In-Reply-To: <9268C1F8-AD68-4B20-94D7-96B5FD6589B5@metricspace.net> References: <33bcd281-4018-7075-1775-4dfcd58e5a48@metricspace.net> <4ec1f3b1-f4b0-80ab-0e68-0dd679dd9e37@metricspace.net> <72f6097e-c71e-b53f-6885-cfe5a5a56586@metricspace.net> <9268C1F8-AD68-4B20-94D7-96B5FD6589B5@metricspace.net> From: Nathan Dautenhahn Date: Fri, 5 Jan 2018 23:32:59 -0500 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: A more general possible meltdown/spectre countermeasure To: Eric McCorkle Cc: Warner Losh , "freebsd-hackers@freebsd.org" , "freebsd-arch@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-Proofpoint-Virus-Version: vendor=nai engine=5600 definitions=5800 signatures=585085 X-PP-Spam-Details: rule=add_spam_details policy=default score=0 spamscore=0 suspectscore=7 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1410280000 definitions=main-1703040151 X-Mailman-Approved-At: Sat, 06 Jan 2018 12:14:24 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Jan 2018 04:33:28 -0000 No this isn't quite superpages. There is a data structure that is a direct map (DMAP) that deterministically maps all physical pages in the system so that all physical pages are accessible through virtual memory. The DMAP has some simple offset calculations allowing it to find the right virtual address mapping (in the DMAP) for the physical address. The more important observation that Warner makes is that to mitigate the problem you have to make sure you get all aliases to each secret. How many exist for each one? How do you know you get them all? Can a attacker create bogus mappings (like ret2dir attack) and then use the new attacks? Another solution, which would handle the more complex attack above, (I know I'm piggybacking, not sure if that's bad) could be to partition a subset of the kernel address space for secrets, and then map those in only when needed, and flush out when done. I did some work a while back on page table isolation and protection from potentially malicious OSs called the nested kernel. I haven't reviewed these new side-channel attacks in great detail yet, but I'm currently working on pushing fine grained intra-address space isolation that might be a nice solution for easily managing and subsets of kernel data. The paper and associated code etc. is all linked on nestedkernel.org. I think these attacks really motivate the nested kernel approach, although we didn't consider secret protection from side-channels. Cheers, -- :: Nathan Dautenhahn :: -- On Fri, Jan 5, 2018 at 6:30 PM, Eric McCorkle wrote: > Ah, superpages. I wouldn't think so. The cpu still has to do a page table walk (just stopping at the top level page table), and would discover that it's not accessible. > > On January 5, 2018 6:24:14 PM EST, Warner Losh wrote: >>I mean the mappings we have in the kernel that map all of memory to a >>specific page using 512GB pages in >>sys/amd64/amd64/pmap.c:create_pagetables. This allows us to map any PA >>to a >>VA with simple math rather than a page table walk. >> >>Warner >> >>On Fri, Jan 5, 2018 at 4:10 PM, Eric McCorkle >>wrote: >> >>> I'm not sure what you mean by direct map. Do you mean TLB? >>> >>> On 01/05/2018 18:08, Warner Losh wrote: >>> > Wouldn't you have to also unmap it from the direct map for this to >>be >>> > effective? >>> > >>> > Warner >>> > >>> > >>> > On Fri, Jan 5, 2018 at 3:31 PM, Eric McCorkle >> > > wrote: >>> > >>> > Well, the only way to find out would be to try it out. >>> > >>> > However, unless I'm missing something, if you're trying to pull >>a >>> > meltdown attack, you try and fetch from the kernel. If that >>location >>> > isn't cached (or if your cache is physically indexed), you need >>the >>> > physical address (otherwise you don't know where to look), and >>thus >>> have >>> > to go through address translation, at which point you detect >>that the >>> > page isn't accessible and fault. In the mean time, you can't >>> > speculatively execute any of the operations that load up the >>> > side-channels, because you don't have the sensitive data. >>> > >>> > The reason you can pull off a meltdown attack at all is that a >>> > virtually-indexed cache lets you get the data in parallel with >>> address >>> > translation (breaking the dependency between address >>translation and >>> > fetching data), which takes 1000s of cycles for a TLB miss, >>during >>> which >>> > you have the data and can launch a whole bunch of transient >>ops. >>> > >>> > Again, these are uncharted waters we're in; so it's entirely >>possible >>> > I'm missing something here. >>> > >>> > On 01/05/2018 17:22, Warner Losh wrote: >>> > > While you might be right, I've seen no indication that a >>cache miss >>> > > would defeat these attacks in the public and non-public data >>I've >>> looked >>> > > at, even though a large number of alternatives to the >>published >>> > > workarounds have been discussed. I'm therefore somewhat >>skeptical >>> this >>> > > would be effective. I'm open, however, to data that changes >>that >>> > > skepticism... >>> > > >>> > > Warner >>> > > >>> > > On Fri, Jan 5, 2018 at 3:15 PM, Eric McCorkle < >>> eric@metricspace.net >>> > > >> >>> wrote: >>> > > >>> > > Right, but you have to get the value "foo" into the >>pipeline >>> in order >>> > > for it to affect the side-channels. This technique >>attempts >>> to stop >>> > > that from happening. >>> > > >>> > > Unless I made a mistake, non-cached memory reads force >>address >>> > > translation to happen first, which detects faults and >>blocks >>> the >>> > > meltdown attack. >>> > > >>> > > It also stops spectre with very high probability, as it's >>very >>> unlikely >>> > > that an uncached load will arrive before the speculative >>> thread gets >>> > > squashed. >>> > > >>> > > On 01/05/2018 17:10, Warner Losh wrote: >>> > > > I think this is fatally flawed. >>> > > > >>> > > > The side channel is the cache. Not the data at risk. >>> > > > >>> > > > Any mapped memory, cached or not, can be used to >>influence >>> the cache. >>> > > > Storing stuff in uncached memory won't affect the side >>> channel one bit. >>> > > > >>> > > > Basically, all attacks boil down to tricking the >>processor, >>> at elevated >>> > > > privs, to doing something like >>> > > > >>> > > > a = foo[offset]; >>> > > > >>> > > > where foo + offset are designed to communicate >>information >>> by populating >>> > > > a cache line. offset need not be cached itself and can >>be >>> the result of >>> > > > simple computations that depend on anything accessible >>at >>> all in the kernel. >>> > > > >>> > > > Warner >>> > > > >>> > > > On Fri, Jan 5, 2018 at 3:02 PM, Eric McCorkle < >>> eric@metricspace.net >>> > > >>> > > > > >>> > >>> >>wrote: >>> > > > >>> > > > Re-posting to -hackers and -arch. I'm going to >>start >>> working on >>> > > > something like this over the weekend. >>> > > > >>> > > > -------- Forwarded Message -------- >>> > > > Subject: A more general possible meltdown/spectre >>> countermeasure >>> > > > Date: Thu, 4 Jan 2018 23:05:40 -0500 >>> > > > From: Eric McCorkle >> eric@metricspace.net> >>> > > >> >>> > >>> > > >>>> >>> > > > To: freebsd-security@freebsd.org >> freebsd-security@freebsd.org> >>> > > >>> freebsd.org>> >>> > > > >> freebsd-security@freebsd.org> >>> > > >>> freebsd.org>>> >>> > > >>> > > >>> freebsd.org>> >>> > > > >> > >>> > > >> > >>> >>> > > > >>> > > > I've thought more about how to deal with >>> > meltdown/spectre, and >>> > > I have an >>> > > > idea I'd like to put forward. However, I'm still >>in >>> > something >>> > > of a >>> > > > panic mode, so I'm not certain as to its >>effectiveness. >>> > > Needless to >>> > > > say, I welcome any feedback on this, and I may be >>> completely >>> > > off-base. >>> > > > >>> > > > I'm calling this a "countermeasure" as opposed to a >>> > > "mitigation", as >>> > > > it's something that requires modification of code >>as >>> > opposed to a >>> > > > drop-in patch. >>> > > > >>> > > > == Summary == >>> > > > >>> > > > Provide a kernel and userland API by which memory >>> allocation >>> > > can be done >>> > > > with extended attributes. In userland, this could >>be >>> > > accomplished by >>> > > > extending MMAP flags, and I could imagine a >>> > > malloc-with-attributes flag. >>> > > > In kernel space, this must already exist, as >>drivers >>> > need to >>> > > allocate >>> > > > memory with various MTRR-type attributes set. >>> > > > >>> > > > The immediate aim here is to store sensitive >>information >>> > that must >>> > > > remain memory-resident in non-cacheable memory >>locations >>> > (or, >>> > > if more >>> > > > effective attribute combinations exist, using those >>> > instead). >>> > > See the >>> > > > rationale for the argument why this should work. >>> > > > >>> > > > Assuming the rationale holds, then the attack >>surface >>> should >>> > > be greatly >>> > > > reduced. Attackers would need to grab sensitive >>data >>> > out of stack >>> > > > frames or similar locations if/when it gets copied >>there >>> for >>> > > faster use. >>> > > > Moreover, if this is done right, it could dovetail >>> > nicely into a >>> > > > framework for storing and processing sensitive >>assets in >>> > more >>> > > secure >>> > > > hardware[0] (like smart cards, the FPGAs I posted >>> > earlier, or >>> > > other >>> > > > options). >>> > > > >>> > > > The obvious downside is that you take a performance >>hit >>> > > storing things >>> > > > in non-cacheable locations, especially if you plan >>on >>> > doing heavy >>> > > > computation in that memory (say, >>encryption/decryption). >>> > > However, this >>> > > > is almost certainly going to be less than the >>projected >>> > 30-50% >>> > > > performance hit from other mitigations. Also, this >>> > technique >>> > > should >>> > > > work against spectre as well as meltdown (assuming >>the >>> > > rationale holds). >>> > > > >>> > > > The second downside is that you have to modify code >>for >>> this >>> > > to work, >>> > > > and you have to be careful not to keep copies of >>> sensitive >>> > > information >>> > > > around too long (this gets tricky in userland, >>where you >>> > might get >>> > > > interrupted and switched out). >>> > > > >>> > > > >>> > > > [0]: Full disclosure, enabling open hardware >>> implementations >>> > > of this >>> > > > kind of thing is something of an agenda of mine. >>> > > > >>> > > > == Rationale == >>> > > > >>> > > > (Again, I'm tired, rushed, and somewhat panicked so >>my >>> logic >>> > > could be >>> > > > faulty at any point, so please point it out if it >>is) >>> > > > >>> > > > The rationale for why this should work relies on >>> > assumptions about >>> > > > out-of-order pipelines that cannot be guaranteed to >>> > hold, but are >>> > > > extremely likely to be true. >>> > > > >>> > > > As background, these attacks depend on out-of-order >>> > execution >>> > > performing >>> > > > operations that end up affecting cache and >>> branch-prediction >>> > > state, >>> > > > ultimately storing information about sensitive data >>in >>> these >>> > > > side-channels before the fault conditions are >>detected >>> and >>> > > acted upon. >>> > > > I'll borrow terminology from the paper, using >>"transient >>> > > instructions" >>> > > > to refer to speculatively executed instructions >>that will >>> > > eventually be >>> > > > cancelled by a fault. >>> > > > >>> > > > These attacks depend entirely on transient >>instructions >>> > being >>> > > able to >>> > > > get sensitive information into the processor core >>and >>> then >>> > > perform some >>> > > > kind of instruction on them before the fault >>condition >>> > cancels >>> > > them. >>> > > > Therefore, anything that prevents them from doing >>this >>> > > *should* counter >>> > > > the attack. If the actual sensitive data never >>makes it >>> to >>> > > the core >>> > > > before the fault is detected, the dependent memory >>> > > accesses/branches >>> > > > never get executed and the data never makes it to >>the >>> > > side-channels. >>> > > > >>> > > > Another assumption here is that CPU architects are >>going >>> to >>> > > want to >>> > > > squash faulted instructions ASAP and stop issuing >>along >>> > those >>> > > > speculative branches, so as to reclaim execution >>units. >>> So >>> > > I'm assuming >>> > > > once a fault comes back from address translation, >>then >>> > transient >>> > > > execution stops dead. >>> > > > >>> > > > Now, break down the cases for whether the address >>> containing >>> > > sensitive >>> > > > data is in cache and TLB or not. (I'm assuming >>here that >>> > > caches are >>> > > > virtually-indexed, which enables cache lookups to >>bypass >>> > address >>> > > > translation.) >>> > > > >>> > > > * In cache, in TLB: You end up basically racing >>between >>> the >>> > > cache and >>> > > > TLB, which will very likely end up detecting the >>fault >>> > before >>> > > the data >>> > > > arrives, but at the very worst, you get one or two >>> cycles of >>> > > transient >>> > > > instruction execution before the fault. >>> > > > >>> > > > * In cache, not in TLB: Virtually-indexed tagged >>means >>> > you get >>> > > a cache >>> > > > lookup racing a page-table walk. The cache lookup >>beats >>> the >>> > > page table >>> > > > walk by potentially hundreds (maybe thousands) of >>cycles, >>> > > giving you a >>> > > > bunch of transient instructions before a fault gets >>> > > triggered. This is >>> > > > the main attack case. >>> > > > >>> > > > * Not in cache, in TLB: Memory access requires >>address >>> > > translation, >>> > > > which comes back almost immediately as a fault. >>> > > > >>> > > > * Not in cache, not in TLB: You have to do a page >>table >>> walk >>> > > before you >>> > > > can fetch the location, as you have to go out to >>physical >>> > > memory (and >>> > > > therefore need a physical address). The page table >>walk >>> > will >>> > > come back >>> > > > with a fault, stopping the attack. >>> > > > >>> > > > So, unless I'm missing something here, both >>non-cached >>> cases >>> > > defeat the >>> > > > meltdown attack, as you *cannot* get the data >>unless you >>> do >>> > > address >>> > > > translation first (and therefore detect faults). >>> > > > >>> > > > As for why this defeats the spectre attack, the >>logic is >>> > > similar: you've >>> > > > jumped into someone else's executable code, hoping >>to >>> > scoop up >>> > > enough >>> > > > information into your branch predictor before the >>fault >>> > kicks >>> > > you out. >>> > > > However, to capture anything about sensitive >>information >>> > in your >>> > > > side-channels, the transient instructions need to >>> > actually get >>> > > it into >>> > > > the core before a fault gets detected. The same >>case >>> > analysis >>> > > as above >>> > > > applies, so you never actually get the sensitive >>info >>> > into the >>> > > core >>> > > > before a fault comes back and you get squashed. >>> > > > >>> > > > >>> > > > [1]: A physically-indexed cache would be largely >>immune >>> to >>> > > this attack, >>> > > > as you'd have to do address translation before >>doing a >>> cache >>> > > lookup. >>> > > > >>> > > > >>> > > > I have some ideas that can build on this, but I'd >>like >>> > to get some >>> > > > feedback first. >>> > > > _______________________________________________ >>> > > > freebsd-security@freebsd.org >>> > >>> > > >> > > >>> > > >> > >>> > > >>> freebsd.org>>> >>> > > > mailing list >>> > > > https://lists.freebsd.org/mailman/listinfo/freebsd- >>> security >>> > >>> > > >>>> > > >>> > > > >>>> security >>> > >>> > > >>>> > >> >>> > > > To unsubscribe, send any mail to >>> > > > "freebsd-security-unsubscribe@freebsd.org >>> > >>> > > >> > > >>> > > > >> > >>> > > >> > >>" >>> > > > _______________________________________________ >>> > > > freebsd-arch@freebsd.org >>>> org> >>> > >> >>> > > >> > >>>> > >> >>> > > mailing list >>> > > > >>https://lists.freebsd.org/mailman/listinfo/freebsd-arch >>> > >>> > > >> > > >>> > > > >>>> > >>> > > >> > >> >>> > > > To unsubscribe, send any mail to >>> > > > "freebsd-arch-unsubscribe@freebsd.org >>> > >>> > > >> > > >>> > > > >> > >>> > > >> > >>" >>> > > > >>> > > > >>> > > >>> > > >>> > >>> > >>> > > -- > Sent from my Android device with K-9 Mail. Please excuse my brevity. > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@freebsd.org Sat Jan 6 16:12:11 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E6188EBFEEF; Sat, 6 Jan 2018 16:12:11 +0000 (UTC) (envelope-from eric@metricspace.net) Received: from mail.metricspace.net (mail.metricspace.net [IPv6:2001:470:1f11:617::107]) by mx1.freebsd.org (Postfix) with ESMTP id C075D74AE8; Sat, 6 Jan 2018 16:12:11 +0000 (UTC) (envelope-from eric@metricspace.net) Received: from [IPv6:2001:470:1f11:617:3210:b3ff:fe77:ca3f] (unknown [IPv6:2001:470:1f11:617:3210:b3ff:fe77:ca3f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: eric) by mail.metricspace.net (Postfix) with ESMTPSA id 202868C54; Sat, 6 Jan 2018 16:12:11 +0000 (UTC) Subject: Re: Fwd: A more general possible meltdown/spectre countermeasure To: Wojciech Puchar Cc: "freebsd-hackers@freebsd.org" , "freebsd-arch@freebsd.org" References: <33bcd281-4018-7075-1775-4dfcd58e5a48@metricspace.net> From: Eric McCorkle Message-ID: <73d2f1a5-55f7-0ae7-7660-3e680ba3d32e@metricspace.net> Date: Sat, 6 Jan 2018 11:12:10 -0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.5.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Jan 2018 16:12:12 -0000 On 01/06/2018 11:07, Wojciech Puchar wrote: > sorry for stupid question but for my understanding these attacks works > as below: > > 1) perform access to byte not allowed virtual address and use next > instruction to store relative to private space so cache is filled > depending on value that one shouldn't be able to access. > > 2) as kernel get trap on access violation it will generate SIGSEGV or > SIGBUS which is directed by application using signal(2) so it can be > ignored. > > 3) other part of code perform some timing magic and detects this way > where cache is filled - so byte  value can be guessed properly. > > > My question is - why simply any access attempts to kernel space cannot > generate SIGKILL? Of course it would harm program development, but as > today developers doesn't usually use timesharing machine but have > private computers, simple sysctl variable would suffice. I'd thought of this myself. The problem is that the cache effects could still be observed by another process. While is doesn't defeat the attack, tt does still complicate attacks, so I think it's worth considering. From owner-freebsd-hackers@freebsd.org Sat Jan 6 16:31:44 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8751BEC0B77 for ; Sat, 6 Jan 2018 16:31:44 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-io0-x22c.google.com (mail-io0-x22c.google.com [IPv6:2607:f8b0:4001:c06::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 52D2475733 for ; Sat, 6 Jan 2018 16:31:44 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-io0-x22c.google.com with SMTP id f6so8812425ioh.8 for ; Sat, 06 Jan 2018 08:31:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=h6OI70j7BH1CsMhyDXPLWMy6nSQUdJa1qNrd5c24qZU=; b=p26tdeuP4XDdEjuSXCUSaY+ZFRObPgLfslUFrIWHAU8u2infsVe/NcJIIRVymGl0Wj TV8iuhHNAZsqfOPJu4xKEYo8crMHxgSerOcnyW58JCyPWucGlB6bj/TB9gZmQ5yxIRKn gA/yJSW27om3lSRsHbApzq1FxcCabugMK2I9XALcjOJe0kCZN+Zls2FEv0+cXDAtPa50 zWNb1HQWURQ24W+UOJxmBgIHdWf9duqe+5da7P2zz+8IcRwc/n5i5O5cyE8HOk2kXWlw 9Mkvft/ml5RFWzxxuYBck2emzQXiI+WhjvEtJl3yCkobOQOlByWGaQu1jwER7JwWWnkL 5MWQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=h6OI70j7BH1CsMhyDXPLWMy6nSQUdJa1qNrd5c24qZU=; b=rwCJwIt/8l0XUDPSoREX3i6VNqOmDIbRBDYKaWoJUKAuSoUyHu4jSD2NK/tsD/mTU4 o7hziqscotpcuTtbn9sWo3NK3YlQCU1G3B+CWtJlhEfUwokhGDqLJ+AtoAY7Rnr7YmoT MtbZvQkV2VuhampFZEzQbvhEA0hZjbNouRjshMkO5TKT9u9PaCw34HAWc0ISaxjVV7U9 HwQc/+0+XlXW5xZSQmf4yVciMMkjkYCpnnzM4zlBzcDEmmYZGU6mFMPaNiLdAoWg7Pir WF6a9xXBDjBc5fqpMwkoZGxuPZNrL1tgRFpw8bLi3V/EE7ufG6wwwCZWo2exFFlUJki4 70GQ== X-Gm-Message-State: AKwxytckHHw24wIlY6g06BXwXLEb+l//AUs+aS4/ybSNx7ImedY4os9Y UKWXOg14+sVrjoE+DA8KxtdjTERcWE88we09oNsX8w== X-Google-Smtp-Source: ACJfBosH70NxeKd9hGga//yXlTQA1lxCtwfiWMsBVfVjJSZZr2qxLI6XxpuTew9HIDFzHb6HYoE9OL7Ck6gMBlPHYs0= X-Received: by 10.107.78.12 with SMTP id c12mr6278684iob.63.1515256303292; Sat, 06 Jan 2018 08:31:43 -0800 (PST) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 10.79.160.217 with HTTP; Sat, 6 Jan 2018 08:31:42 -0800 (PST) X-Originating-IP: [2603:300b:6:5100:1052:acc7:f9de:2b6d] In-Reply-To: <73d2f1a5-55f7-0ae7-7660-3e680ba3d32e@metricspace.net> References: <33bcd281-4018-7075-1775-4dfcd58e5a48@metricspace.net> <73d2f1a5-55f7-0ae7-7660-3e680ba3d32e@metricspace.net> From: Warner Losh Date: Sat, 6 Jan 2018 09:31:42 -0700 X-Google-Sender-Auth: dqnExtGmM5mYdYak8_Ly-fGxTuQ Message-ID: Subject: Re: Fwd: A more general possible meltdown/spectre countermeasure To: Eric McCorkle Cc: Wojciech Puchar , "freebsd-hackers@freebsd.org" , "freebsd-arch@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Jan 2018 16:31:44 -0000 On Sat, Jan 6, 2018 at 9:12 AM, Eric McCorkle wrote: > On 01/06/2018 11:07, Wojciech Puchar wrote: > > sorry for stupid question but for my understanding these attacks works > > as below: > > > > 1) perform access to byte not allowed virtual address and use next > > instruction to store relative to private space so cache is filled > > depending on value that one shouldn't be able to access. > > > > 2) as kernel get trap on access violation it will generate SIGSEGV or > > SIGBUS which is directed by application using signal(2) so it can be > > ignored. > > > > 3) other part of code perform some timing magic and detects this way > > where cache is filled - so byte value can be guessed properly. > > > > > > My question is - why simply any access attempts to kernel space cannot > > generate SIGKILL? Of course it would harm program development, but as > > today developers doesn't usually use timesharing machine but have > > private computers, simple sysctl variable would suffice. > > I'd thought of this myself. The problem is that the cache effects could > still be observed by another process. > > While is doesn't defeat the attack, tt does still complicate attacks, so > I think it's worth considering. The problem is that the attempts to access kernel space are speculative. There's no way to get the 'speculative trap' that would have been generated had the code actually executed. There literally is no signal to the kernel this just happened. Warner From owner-freebsd-hackers@freebsd.org Sat Jan 6 16:34:10 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4702FEC0DE2; Sat, 6 Jan 2018 16:34:10 +0000 (UTC) (envelope-from wojtek@puchar.net) Received: from puchar.net (puchar.net [194.1.144.90]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "puchar.net", Issuer "puchar.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id DF3D175B22; Sat, 6 Jan 2018 16:34:09 +0000 (UTC) (envelope-from wojtek@puchar.net) Received: Received: from 127.0.0.1 (localhost [127.0.0.1]) by puchar.net (8.15.2/8.15.2) with ESMTPS id w06G7LqA041376 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sat, 6 Jan 2018 17:07:21 +0100 (CET) (envelope-from puchar-wojtek@puchar.net) Received: from localhost (puchar-wojtek@localhost) by puchar.net (8.15.2/8.15.2/Submit) with ESMTP id w06G7GwH041373; Sat, 6 Jan 2018 17:07:16 +0100 (CET) (envelope-from puchar-wojtek@puchar.net) Date: Sat, 6 Jan 2018 17:07:16 +0100 (CET) From: Wojciech Puchar To: Eric McCorkle cc: "freebsd-hackers@freebsd.org" , "freebsd-arch@freebsd.org" Subject: Re: Fwd: A more general possible meltdown/spectre countermeasure In-Reply-To: <33bcd281-4018-7075-1775-4dfcd58e5a48@metricspace.net> Message-ID: References: <33bcd281-4018-7075-1775-4dfcd58e5a48@metricspace.net> User-Agent: Alpine 2.20 (BSF 67 2015-01-07) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Jan 2018 16:34:10 -0000 sorry for stupid question but for my understanding these attacks works as below: 1) perform access to byte not allowed virtual address and use next instruction to store relative to private space so cache is filled depending on value that one shouldn't be able to access. 2) as kernel get trap on access violation it will generate SIGSEGV or SIGBUS which is directed by application using signal(2) so it can be ignored. 3) other part of code perform some timing magic and detects this way where cache is filled - so byte value can be guessed properly. My question is - why simply any access attempts to kernel space cannot generate SIGKILL? Of course it would harm program development, but as today developers doesn't usually use timesharing machine but have private computers, simple sysctl variable would suffice. On Fri, 5 Jan 2018, Eric McCorkle wrote: > Re-posting to -hackers and -arch. I'm going to start working on > something like this over the weekend. > > -------- Forwarded Message -------- > Subject: A more general possible meltdown/spectre countermeasure > Date: Thu, 4 Jan 2018 23:05:40 -0500 > From: Eric McCorkle > To: freebsd-security@freebsd.org > > I've thought more about how to deal with meltdown/spectre, and I have an > idea I'd like to put forward. However, I'm still in something of a > panic mode, so I'm not certain as to its effectiveness. Needless to > say, I welcome any feedback on this, and I may be completely off-base. > > I'm calling this a "countermeasure" as opposed to a "mitigation", as > it's something that requires modification of code as opposed to a > drop-in patch. > > == Summary == > > Provide a kernel and userland API by which memory allocation can be done > with extended attributes. In userland, this could be accomplished by > extending MMAP flags, and I could imagine a malloc-with-attributes flag. > In kernel space, this must already exist, as drivers need to allocate > memory with various MTRR-type attributes set. > > The immediate aim here is to store sensitive information that must > remain memory-resident in non-cacheable memory locations (or, if more > effective attribute combinations exist, using those instead). See the > rationale for the argument why this should work. > > Assuming the rationale holds, then the attack surface should be greatly > reduced. Attackers would need to grab sensitive data out of stack > frames or similar locations if/when it gets copied there for faster use. > Moreover, if this is done right, it could dovetail nicely into a > framework for storing and processing sensitive assets in more secure > hardware[0] (like smart cards, the FPGAs I posted earlier, or other > options). > > The obvious downside is that you take a performance hit storing things > in non-cacheable locations, especially if you plan on doing heavy > computation in that memory (say, encryption/decryption). However, this > is almost certainly going to be less than the projected 30-50% > performance hit from other mitigations. Also, this technique should > work against spectre as well as meltdown (assuming the rationale holds). > > The second downside is that you have to modify code for this to work, > and you have to be careful not to keep copies of sensitive information > around too long (this gets tricky in userland, where you might get > interrupted and switched out). > > > [0]: Full disclosure, enabling open hardware implementations of this > kind of thing is something of an agenda of mine. > > == Rationale == > > (Again, I'm tired, rushed, and somewhat panicked so my logic could be > faulty at any point, so please point it out if it is) > > The rationale for why this should work relies on assumptions about > out-of-order pipelines that cannot be guaranteed to hold, but are > extremely likely to be true. > > As background, these attacks depend on out-of-order execution performing > operations that end up affecting cache and branch-prediction state, > ultimately storing information about sensitive data in these > side-channels before the fault conditions are detected and acted upon. > I'll borrow terminology from the paper, using "transient instructions" > to refer to speculatively executed instructions that will eventually be > cancelled by a fault. > > These attacks depend entirely on transient instructions being able to > get sensitive information into the processor core and then perform some > kind of instruction on them before the fault condition cancels them. > Therefore, anything that prevents them from doing this *should* counter > the attack. If the actual sensitive data never makes it to the core > before the fault is detected, the dependent memory accesses/branches > never get executed and the data never makes it to the side-channels. > > Another assumption here is that CPU architects are going to want to > squash faulted instructions ASAP and stop issuing along those > speculative branches, so as to reclaim execution units. So I'm assuming > once a fault comes back from address translation, then transient > execution stops dead. > > Now, break down the cases for whether the address containing sensitive > data is in cache and TLB or not. (I'm assuming here that caches are > virtually-indexed, which enables cache lookups to bypass address > translation.) > > * In cache, in TLB: You end up basically racing between the cache and > TLB, which will very likely end up detecting the fault before the data > arrives, but at the very worst, you get one or two cycles of transient > instruction execution before the fault. > > * In cache, not in TLB: Virtually-indexed tagged means you get a cache > lookup racing a page-table walk. The cache lookup beats the page table > walk by potentially hundreds (maybe thousands) of cycles, giving you a > bunch of transient instructions before a fault gets triggered. This is > the main attack case. > > * Not in cache, in TLB: Memory access requires address translation, > which comes back almost immediately as a fault. > > * Not in cache, not in TLB: You have to do a page table walk before you > can fetch the location, as you have to go out to physical memory (and > therefore need a physical address). The page table walk will come back > with a fault, stopping the attack. > > So, unless I'm missing something here, both non-cached cases defeat the > meltdown attack, as you *cannot* get the data unless you do address > translation first (and therefore detect faults). > > As for why this defeats the spectre attack, the logic is similar: you've > jumped into someone else's executable code, hoping to scoop up enough > information into your branch predictor before the fault kicks you out. > However, to capture anything about sensitive information in your > side-channels, the transient instructions need to actually get it into > the core before a fault gets detected. The same case analysis as above > applies, so you never actually get the sensitive info into the core > before a fault comes back and you get squashed. > > > [1]: A physically-indexed cache would be largely immune to this attack, > as you'd have to do address translation before doing a cache lookup. > > > I have some ideas that can build on this, but I'd like to get some > feedback first. > _______________________________________________ > freebsd-security@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-security > To unsubscribe, send any mail to "freebsd-security-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > > From owner-freebsd-hackers@freebsd.org Sat Jan 6 16:53:47 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 75449EC1B43; Sat, 6 Jan 2018 16:53:47 +0000 (UTC) (envelope-from wojtek@puchar.net) Received: from puchar.net (puchar.net [194.1.144.90]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "puchar.net", Issuer "puchar.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 055BE7681A; Sat, 6 Jan 2018 16:53:46 +0000 (UTC) (envelope-from wojtek@puchar.net) Received: Received: from 127.0.0.1 (localhost [127.0.0.1]) by puchar.net (8.15.2/8.15.2) with ESMTPS id w06Grg1T046942 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sat, 6 Jan 2018 17:53:42 +0100 (CET) (envelope-from puchar-wojtek@puchar.net) Received: from localhost (puchar-wojtek@localhost) by puchar.net (8.15.2/8.15.2/Submit) with ESMTP id w06GrbFd046939; Sat, 6 Jan 2018 17:53:37 +0100 (CET) (envelope-from puchar-wojtek@puchar.net) Date: Sat, 6 Jan 2018 17:53:37 +0100 (CET) From: Wojciech Puchar To: Warner Losh cc: Eric McCorkle , Wojciech Puchar , "freebsd-hackers@freebsd.org" , "freebsd-arch@freebsd.org" Subject: Re: Fwd: A more general possible meltdown/spectre countermeasure In-Reply-To: Message-ID: References: <33bcd281-4018-7075-1775-4dfcd58e5a48@metricspace.net> <73d2f1a5-55f7-0ae7-7660-3e680ba3d32e@metricspace.net> User-Agent: Alpine 2.20 (BSF 67 2015-01-07) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8BIT X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Jan 2018 16:53:47 -0000 > While is doesn't defeat the attack, tt does still complicate attacks, so > I think it's worth considering. > > > The problem is that the attempts to access kernel space are speculative. There's no way to get the 'speculative trap' that would > have been generated had the code actually executed. There literally is no signal to the kernel this just happened. > > Warner  > > f..k. so there are no real workarounds. Anyway - if CPU companies would be honest they would replace at least all server CPUs that are on warranty From owner-freebsd-hackers@freebsd.org Sat Jan 6 17:04:56 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D764CEC2597 for ; Sat, 6 Jan 2018 17:04:56 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-io0-x22b.google.com (mail-io0-x22b.google.com [IPv6:2607:f8b0:4001:c06::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 984AA77061 for ; Sat, 6 Jan 2018 17:04:56 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-io0-x22b.google.com with SMTP id n14so8867035iob.4 for ; Sat, 06 Jan 2018 09:04:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=qxK6lHUIqiRkSC4De4cwFKYEOyQwjG9L2o6RG5K6ra4=; b=Bm3/brLnlFsGlX3cWd668tKRBgeXfyGd0Nxi8LZH0M2pJtmNueKSRgjX0WEpVmV12l DkDzwd7PmsxI2kCGotD8LYfsVylBbhNl3/DbCZy74U4VRCRRB3AVjKKadS7YK0FRhXDD S+In0OX1jtoFN1KbzafnqdIIbLm/5Jr6i/ysYALezZC2rGZmN4U+OTm09NrpfaQ0L9lS Lq8iJNPBVgUqmCkxagPxSW/Z125Z/1HlamI5vko3QUBSF9X4BWhjFYtcA6zcFXczA2AZ G04IOLy4tYIfvN6D59oaeBHQX29BVUIGaunAuIqhD/TIVAoEurOegvFAM6AMbN4McDfJ oayw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=qxK6lHUIqiRkSC4De4cwFKYEOyQwjG9L2o6RG5K6ra4=; b=FJGscfc/J7u227cQN2zmj/oPgowaS8l4BD7UOh4ivkekzvgt2LF7k8oGaY5dpRNUSe teTVplICAm5mQcvtI4wsSO42SjKDNCLFu9oWC+cr/I/vBWa/Nv7/OOHWJFFLKMHFgcIr d8VDicTPna00i2jGySsUuAuaw2uBGDgwiuP5nUDY2fyDz0qIn83nkUas6F79dCCoTyOX JJk6zyl1iBgSmNQ0wQ4kbwwSiTPD3URBNvTCp8Qy4k96oIrx9nFMcIxjOhD+oZjQF8B8 hMf8W4HofOb7R1o/HQVZkqaM7cm24DzdaQWuiwZ7e1lMbZcSflRdOmcGFeITDD4/Fsjd d/OQ== X-Gm-Message-State: AKwxytcFCxXvr6DZCJ9dNHM3dDriRURwQm0tXntpoa6YJsI0k/JeNgmj kRd1DO7Cr6U2P97MD5JdEXMJX200JEpE3rh3Avznmg== X-Google-Smtp-Source: ACJfBots7zwFP9w7ezasUUVZfZNLHrO8er3Sm5ib1lGQtE8vgsfs7WEl5LYD6cPFRhv3re/dmXN3DzXarhPu0UD8K0o= X-Received: by 10.107.78.12 with SMTP id c12mr6337340iob.63.1515258295739; Sat, 06 Jan 2018 09:04:55 -0800 (PST) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 10.79.160.217 with HTTP; Sat, 6 Jan 2018 09:04:54 -0800 (PST) X-Originating-IP: [2603:300b:6:5100:1052:acc7:f9de:2b6d] In-Reply-To: References: <33bcd281-4018-7075-1775-4dfcd58e5a48@metricspace.net> <73d2f1a5-55f7-0ae7-7660-3e680ba3d32e@metricspace.net> From: Warner Losh Date: Sat, 6 Jan 2018 10:04:54 -0700 X-Google-Sender-Auth: 3xtImcXG0eoMSSyWTPVcuOIfNSU Message-ID: Subject: Re: Fwd: A more general possible meltdown/spectre countermeasure To: Wojciech Puchar Cc: Eric McCorkle , "freebsd-hackers@freebsd.org" , "freebsd-arch@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Jan 2018 17:04:56 -0000 On Sat, Jan 6, 2018 at 9:53 AM, Wojciech Puchar wrote: > While is doesn't defeat the attack, tt does still complicate >> attacks, so >> I think it's worth considering. >> >> >> The problem is that the attempts to access kernel space are speculative. >> There's no way to get the 'speculative trap' that would >> have been generated had the code actually executed. There literally is no >> signal to the kernel this just happened. >> >> Warner >> >> >> f..k. so there are no real workarounds. Anyway - if CPU companies would > be honest they would replace at least all server CPUs that are on warranty The only workaround that's completely effective is to unmap all of kernel memory when running in userland. It's a bit tricky because there's small parts that have to stay mapped for various architectural reasons. This means KASLR on these CPUs likely can never be effective since meltdown will let you find what the trap address is and from that find the kernel (though there's some rumblings that the indirection Linux is doing will suffice). Warner From owner-freebsd-hackers@freebsd.org Sat Jan 6 19:00:24 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 68626EC69B2; Sat, 6 Jan 2018 19:00:24 +0000 (UTC) (envelope-from gljennjohn@gmail.com) Received: from mail-wr0-x22a.google.com (mail-wr0-x22a.google.com [IPv6:2a00:1450:400c:c0c::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EE1027AFAC; Sat, 6 Jan 2018 19:00:23 +0000 (UTC) (envelope-from gljennjohn@gmail.com) Received: by mail-wr0-x22a.google.com with SMTP id o15so7067306wrf.12; Sat, 06 Jan 2018 11:00:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:in-reply-to:references:reply-to :mime-version:content-transfer-encoding; bh=y/9jrmvhifUSJWS2amKTdaYeaaTWEWa/Ro0O/My4ppI=; b=ffajEtLxsUavF7T0nFW7kbSZnOd4+iWbhJSqkhMeCxB4l/Ie7s3IN60bO57WcT3KJ3 1ULme51H4GJPw5mwKc7V0LPbMDroJVoLlEsujv0gUQXifw31u3q/uC1OiBZ/b8IAhUrs C+9Bf4gyWkKZkYb3UeAluJpiN7n0i1+UdpFgF4oIzz48qwjcHKBp2APR7Hdedg0kxpdh 4VhVBl7ddYn7+tIKYPbLZmPCG/ZXjeGXroyCVGB/Gf34dw4ncaJwFJB/mrAf/Nf9Z3ls vU4Q93d2wxLgk+ujHyQd4NZmHQ7KRzm7iLYw3YP/9X+AaHFC94ju0tuXF9DmAO0RNEc6 cl1Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:reply-to:mime-version:content-transfer-encoding; bh=y/9jrmvhifUSJWS2amKTdaYeaaTWEWa/Ro0O/My4ppI=; b=I9PnPBi9bqFfRCQdUpzRHsAFrEmg3c3F8C6ukmL/2rl0Oh6prbMTemQpPWJg0IV1ML WDFIPMN5Tz1cKKgwy3dpc4XfgV41Rw/9e5yJcu55Ut2ZWkhOnD8qFSWTUSKiOpO3FBoq qdDRlN71HBVDzwuCEf96EDy34f5HUEGS6+51os0yZ8SYgh/2M2MG5XoSl0cSdNKpUiPj xbzJgz+HrBgFKfbKkAK7LZFvyz8IIigVFuTyHI4/5NCsiJESgpuGX5HTLJreb74VMRkU RD8n3fk8x6hIMkr0neW8KziF1zxkyucSHnu4ozKEAwBICnNxFUIE9UXT6USI1pXMI3Pc Jw2Q== X-Gm-Message-State: AKGB3mLxobfs+Tw5kfVdPPuRXIXwsDHx27m9hiKiCZXdouFg4sprDxdR X2mOPCfdK+wn1OY4/DpxlPE= X-Google-Smtp-Source: ACJfBosL9Zvtk0A2DjkMio8tfgJdKONQg9C7rAMY/VbWmqrkhJa/rpeNUKucpB9nJdSyfC1uxwnuNQ== X-Received: by 10.223.164.207 with SMTP id h15mr5897330wrb.59.1515265222539; Sat, 06 Jan 2018 11:00:22 -0800 (PST) Received: from ernst.home (p5B023419.dip0.t-ipconnect.de. [91.2.52.25]) by smtp.gmail.com with ESMTPSA id p107sm4457934wrc.61.2018.01.06.11.00.21 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sat, 06 Jan 2018 11:00:21 -0800 (PST) Date: Sat, 6 Jan 2018 20:00:20 +0100 From: Gary Jennejohn To: Warner Losh Cc: Wojciech Puchar , Eric McCorkle , "freebsd-hackers@freebsd.org" , "freebsd-arch@freebsd.org" Subject: Re: A more general possible meltdown/spectre countermeasure Message-ID: <20180106200020.051c4149@ernst.home> In-Reply-To: References: <33bcd281-4018-7075-1775-4dfcd58e5a48@metricspace.net> <73d2f1a5-55f7-0ae7-7660-3e680ba3d32e@metricspace.net> Reply-To: gljennjohn@gmail.com X-Mailer: Claws Mail 3.16.0 (GTK+ 2.24.31; amd64-portbld-freebsd12.0) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Jan 2018 19:00:24 -0000 On Sat, 6 Jan 2018 10:04:54 -0700 Warner Losh wrote: > On Sat, Jan 6, 2018 at 9:53 AM, Wojciech Puchar wrote: > > > While is doesn't defeat the attack, tt does still complicate > >> attacks, so > >> I think it's worth considering. > >> > >> > >> The problem is that the attempts to access kernel space are speculative. > >> There's no way to get the 'speculative trap' that would > >> have been generated had the code actually executed. There literally is no > >> signal to the kernel this just happened. > >> > >> Warner > >> > >> > >> f..k. so there are no real workarounds. Anyway - if CPU companies would > > be honest they would replace at least all server CPUs that are on warranty > > > The only workaround that's completely effective is to unmap all of kernel > memory when running in userland. It's a bit tricky because there's small > parts that have to stay mapped for various architectural reasons. This > means KASLR on these CPUs likely can never be effective since meltdown will > let you find what the trap address is and from that find the kernel (though > there's some rumblings that the indirection Linux is doing will suffice). > This point is addressed in one of the papers. KAISER only maps small parts of the address space, which are apparently required for special use, in both the kernel and user space. Otherwise, the kernel and user space do not share any part of the memory map. The conclusion in the paper is that, yes, a small part of memory is still common to both the kernel and user space, but if KASLR is used, then it will be very difficult to identify these ranges. -- Gary Jennejohn From owner-freebsd-hackers@freebsd.org Sat Jan 6 19:55:17 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 873CDDB8C72; Sat, 6 Jan 2018 19:55:17 +0000 (UTC) (envelope-from jmg@gold.funkthat.com) Received: from gold.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "gate2.funkthat.com", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6A0587CD63; Sat, 6 Jan 2018 19:55:17 +0000 (UTC) (envelope-from jmg@gold.funkthat.com) Received: from gold.funkthat.com (localhost [127.0.0.1]) by gold.funkthat.com (8.15.2/8.15.2) with ESMTPS id w06JtBre047913 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sat, 6 Jan 2018 11:55:11 -0800 (PST) (envelope-from jmg@gold.funkthat.com) Received: (from jmg@localhost) by gold.funkthat.com (8.15.2/8.15.2/Submit) id w06JtBG9047912; Sat, 6 Jan 2018 11:55:11 -0800 (PST) (envelope-from jmg) Date: Sat, 6 Jan 2018 11:55:11 -0800 From: John-Mark Gurney To: Freddie Cash Cc: Freebsd Security , FreeBSD Hackers , "freebsd-arch@freebsd.org" Subject: Re: Intel hardware bug Message-ID: <20180106195510.GH75576@funkthat.com> Mail-Followup-To: Freddie Cash , Freebsd Security , FreeBSD Hackers , "freebsd-arch@freebsd.org" References: <20180105191145.404BC335@spqr.komquats.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Operating-System: FreeBSD 11.0-RELEASE-p7 amd64 X-PGP-Fingerprint: D87A 235F FB71 1F3F 55B7 ED9B D5FF 5A51 C0AC 3D65 X-Files: The truth is out there X-URL: https://www.funkthat.com/ X-Resume: https://www.funkthat.com/~jmg/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? User-Agent: Mutt/1.6.1 (2016-04-27) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (gold.funkthat.com [127.0.0.1]); Sat, 06 Jan 2018 11:55:11 -0800 (PST) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Jan 2018 19:55:17 -0000 Freddie Cash wrote this message on Fri, Jan 05, 2018 at 11:53 -0800: > Spectre (aka CVE-2017-5715 and CVE-2017-5753) is the issue that affects all > CPUs (Intel, AMD, ARM, IBM, Oracle, etc) and allows userland processes to > read memory assigned to other userland processes (but does NOT give access > to kernel memory). No, Spectre does not allow one userland process to read another userland process's memory.. It allows an attacker to read any memory within the same process.. -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-hackers@freebsd.org Sat Jan 6 20:42:04 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C2A27DEE59D; Sat, 6 Jan 2018 20:42:04 +0000 (UTC) (envelope-from wojtek@puchar.net) Received: from puchar.net (puchar.net [194.1.144.90]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "puchar.net", Issuer "puchar.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 51D0D7E98C; Sat, 6 Jan 2018 20:42:03 +0000 (UTC) (envelope-from wojtek@puchar.net) Received: Received: from 127.0.0.1 (localhost [127.0.0.1]) by puchar.net (8.15.2/8.15.2) with ESMTPS id w06KfwvF072158 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sat, 6 Jan 2018 21:41:58 +0100 (CET) (envelope-from puchar-wojtek@puchar.net) Received: from localhost (puchar-wojtek@localhost) by puchar.net (8.15.2/8.15.2/Submit) with ESMTP id w06KfqJX072155; Sat, 6 Jan 2018 21:41:52 +0100 (CET) (envelope-from puchar-wojtek@puchar.net) Date: Sat, 6 Jan 2018 21:41:52 +0100 (CET) From: Wojciech Puchar To: Warner Losh cc: Wojciech Puchar , Eric McCorkle , "freebsd-hackers@freebsd.org" , "freebsd-arch@freebsd.org" Subject: Re: Fwd: A more general possible meltdown/spectre countermeasure In-Reply-To: Message-ID: References: <33bcd281-4018-7075-1775-4dfcd58e5a48@metricspace.net> <73d2f1a5-55f7-0ae7-7660-3e680ba3d32e@metricspace.net> User-Agent: Alpine 2.20 (BSF 67 2015-01-07) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Jan 2018 20:42:04 -0000 > The only workaround that's completely effective is to unmap all of kernel memory when running in userland. It's a bit tricky because this means on every syscall on interrupt: - memcopy part of top level PTE on enter, bzero on exit - TLB flush both on enter and exit. IMHO it would make much more than 30% overhead in many cases. am i wrong? > there's small parts that have to stay mapped for various architectural reasons. This means KASLR on these CPUs likely can never be > effective since meltdown will let you find what the trap address is and from that find the kernel (though there's some rumblings > that the indirection Linux is doing will suffice). > > Warner > > From owner-freebsd-hackers@freebsd.org Sat Jan 6 21:37:21 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C5CF4DF3837; Sat, 6 Jan 2018 21:37:21 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-lf0-x236.google.com (mail-lf0-x236.google.com [IPv6:2a00:1450:4010:c07::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48F7A81857; Sat, 6 Jan 2018 21:37:21 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: by mail-lf0-x236.google.com with SMTP id y71so8475234lfd.12; Sat, 06 Jan 2018 13:37:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=VeaCCrhahGTR5WjwlkG09R8dkN7k/hKE7TBnG6w2zuY=; b=SL72Fx5QJc0H5QWPX8SqdhZIv4VYWvvsv2eoOg5HIXeee4+rf7v6CR7X4qwezQf+kF 3iBiMitaya0BvyatHH4ntyal/bGMwxb8xzy/kMSAjXLB5Gbaow2tu9wxOlGA/tXuXKgX sdEGVFwG+MxBDZfZ6Rbm29AJXFTpJ7pbk3uyu5YWIyGJwLqo2eN8e7tjNB4hnvE8cXLz iQEtS1VrHlk9uG2IdmMhGbX++gPBP8uFN4BSN4dLsyqfhAsARWjdUA1BK5Hny+YX7bhs Ug9CMyFBJFqmCjPacByKwVnaSSPoFtonGZqqG+0O91Ql72TJ93RHcoil7E48hDIrjHg7 MPDg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=VeaCCrhahGTR5WjwlkG09R8dkN7k/hKE7TBnG6w2zuY=; b=uRtu6XrljA1Q3lqzzfROasjQYY3oWrmdx1N/ETCMXp4BUGeWDqrN/I9JEWIBQ/SIX1 VQkjF1rMr48prJHyPmVzO9JYU8ut4Y5XziA31iy4/zSqvi+10pDCwRq1lK8BolhbOjtF JAt7fL4+hImy2KaanIP2b167tuPewdtMcKwMr+0dVYVAS0eUddcFsR5BImTRwBmIqTwX OQaozql11kkSEcx/CpY3zEQMqIFg9k7uAcVfIdc9rc5qhtt7oXhJDzuzM0fWfo20h6gl mwF51x9SVqKk0Xed7GpDBy2SmHDjsjCFo+TbQtp+ROd2D1sK1NtOJ2+NTgwvGCXiTl7W PRgw== X-Gm-Message-State: AKwxytdXWHFzrODCtfRxE8LEaCUTiswtcN1uaqsj0pADxWB9BiJSDtJO WlPpFOy/TuPZJhts56iPUEMvYa5OF7uDFOV6//Ny5A== X-Google-Smtp-Source: ACJfBotMFotIM1tbfdLMCAnsnlHdMM/WhEcPYKtsMIOHy83w/zxoB7a4xL3lsLZRDtHhtyohi/n0nRxgDRDTqui4OLQ= X-Received: by 10.25.86.17 with SMTP id k17mr3860615lfb.67.1515274638724; Sat, 06 Jan 2018 13:37:18 -0800 (PST) MIME-Version: 1.0 Received: by 10.25.163.207 with HTTP; Sat, 6 Jan 2018 13:37:17 -0800 (PST) Received: by 10.25.163.207 with HTTP; Sat, 6 Jan 2018 13:37:17 -0800 (PST) In-Reply-To: <20180106195510.GH75576@funkthat.com> References: <20180105191145.404BC335@spqr.komquats.com> <20180106195510.GH75576@funkthat.com> From: Freddie Cash Date: Sat, 6 Jan 2018 13:37:17 -0800 Message-ID: Subject: Re: Intel hardware bug To: Freebsd Security , FreeBSD Hackers , "freebsd-arch@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Jan 2018 21:37:21 -0000 On Jan 6, 2018 11:55 AM, "John-Mark Gurney" wrote: Freddie Cash wrote this message on Fri, Jan 05, 2018 at 11:53 -0800: > Spectre (aka CVE-2017-5715 and CVE-2017-5753) is the issue that affects all > CPUs (Intel, AMD, ARM, IBM, Oracle, etc) and allows userland processes to > read memory assigned to other userland processes (but does NOT give access > to kernel memory). No, Spectre does not allow one userland process to read another userland process's memory.. It allows an attacker to read any memory within the same process. That's variant 1 of Spectre. Variant 2 crosses process boundaries. It's the one that has VM hosting systems worried as a process running in VM1 can read memory assigned to VM2. Cheers, Freddie From owner-freebsd-hackers@freebsd.org Sat Jan 6 21:59:50 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0925FDF4FEA for ; Sat, 6 Jan 2018 21:59:50 +0000 (UTC) (envelope-from brooks@spindle.one-eyed-alien.net) Received: from spindle.one-eyed-alien.net (spindle.one-eyed-alien.net [199.48.129.229]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D4A8FD8C; Sat, 6 Jan 2018 21:59:49 +0000 (UTC) (envelope-from brooks@spindle.one-eyed-alien.net) Received: by spindle.one-eyed-alien.net (Postfix, from userid 3001) id 4A3B95A9F15; Sat, 6 Jan 2018 21:59:47 +0000 (UTC) Date: Sat, 6 Jan 2018 21:59:47 +0000 From: Brooks Davis To: Ian Lepore Cc: cem@freebsd.org, Poul-Henning Kamp , Freebsd hackers list Subject: Re: Is it considered to be ok to not check the return code of close(2) in base? Message-ID: <20180106215947.GG95035@spindle.one-eyed-alien.net> References: <24acbd94-c52f-e71a-8a96-d608a10963c6@rawbw.com> <1514572041.12000.7.camel@freebsd.org> <20180105221330.GD95035@spindle.one-eyed-alien.net> <24173.1515191675@critter.freebsd.dk> <1515201055.1865.5.camel@freebsd.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="QNDPHrPUIc00TOLW" Content-Disposition: inline In-Reply-To: <1515201055.1865.5.camel@freebsd.org> User-Agent: Mutt/1.9.1 (2017-09-22) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Jan 2018 21:59:50 -0000 --QNDPHrPUIc00TOLW Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Jan 05, 2018 at 06:10:55PM -0700, Ian Lepore wrote: > On Fri, 2018-01-05 at 15:17 -0800, Conrad Meyer wrote: > > On Fri, Jan 5, 2018 at 2:34 PM, Poul-Henning Kamp > > wrote: > > >=20 > > > Brookes suggestion, while well intentioned, wouldn't get very far, > > > because it is common for shells and shell-like programs to do: > > >=20 > > > ????????for (i =3D 3; i < ALOT; i++) > > > ????????????????(void)close(i); > > >=20 > > > To get rid of unwanted filedescriptors from syslog(3), getpwent(3) > > > etc. > > > in the child process. > > >=20 > > > Yes, I know about closefrom(2), but a lot of programs still don't > > > use it. > >=20 > > That seems like a good way to quickly identify programs in base that > > still do not use closefrom(). >=20 > Portable programs MUST iterate 3-ALOT, because closefrom() is not > available in linux and other OSes. The portable API is terrible and closefrom() with arguments other than 3 (or maybe 4) is in many ways worse with require dup2(2) shenanigans. A non-crap (but harder to adopt) API would be an extended execve(2) that takes an array of FDs to be installed in the given positions. (I'd probably make the array be name-value pairs, as I believe cloudabi does, since magic offsets into a kernel array have no business in a 21st century IPC API.) -- Brooks --QNDPHrPUIc00TOLW Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEcBAEBAgAGBQJaUUbSAAoJEKzQXbSebgfAC1kH+wUkAsZ0B17uxoL0Bvisj8nO cLTwJ0H0iY8zwsTXKpUFK/4yLcML+CO0MJta3xHDXnQcDbzd4UCLGt+dybTeP6bW 967Zb9PWrM/ZFQ7R+7LimaHeQnJTEz0g7M+fxuaAFfCVfvBSMWpiGm3Ic3PeCXs2 +CgI7zjVFWrawTOaPZOM4xxlxE+NU1tc5YMvhyZxClUNWOC7I1DVeCNcU14XIJgd 16FlzU6WaAPVL7T17U0biHjB7CPHzhZsiiE2IR8HQEFS2Z2LMdFY5jwdZhJYT4ry d10X7CKwSSYmRgopokKFgE0NzIamc3N1sdgQRtvuiPIulCsaXarv7ZOcog1CzPU= =QuZ2 -----END PGP SIGNATURE----- --QNDPHrPUIc00TOLW--