Date: Fri, 5 Jan 2018 16:24:14 -0700 From: Warner Losh <imp@bsdimp.com> To: Eric McCorkle <eric@metricspace.net> Cc: "freebsd-hackers@freebsd.org" <freebsd-hackers@freebsd.org>, "freebsd-arch@freebsd.org" <freebsd-arch@freebsd.org> Subject: Re: A more general possible meltdown/spectre countermeasure Message-ID: <CANCZdfq8UZWXdd9fc3yDTETPrCOYYojKyAzyjywrkG=d8OA6Jg@mail.gmail.com> In-Reply-To: <b4f8b84c-88f4-0b03-40c8-7fa07a496e26@metricspace.net> References: <c98b7ac3-26f0-81ee-2769-432697f876e5@metricspace.net> <33bcd281-4018-7075-1775-4dfcd58e5a48@metricspace.net> <CANCZdfq2iHErm-pPU1cQ35zrvfdVFPBi1sKYshOyyWUgm3VHBA@mail.gmail.com> <4ec1f3b1-f4b0-80ab-0e68-0dd679dd9e37@metricspace.net> <CANCZdfo2MPbYxrOyuwSO82FAx9ur_kxgwUijoZCaRL2MDxFxFQ@mail.gmail.com> <72f6097e-c71e-b53f-6885-cfe5a5a56586@metricspace.net> <CANCZdfoYY5ACwbT0gUW-u=%2BHyG-Y2McoN%2B%2BNZZvqED=wU5MEUA@mail.gmail.com> <b4f8b84c-88f4-0b03-40c8-7fa07a496e26@metricspace.net>
next in thread | previous in thread | raw e-mail | index | archive | help
I mean the mappings we have in the kernel that map all of memory to a specific page using 512GB pages in sys/amd64/amd64/pmap.c:create_pagetables. This allows us to map any PA to a VA with simple math rather than a page table walk. Warner On Fri, Jan 5, 2018 at 4:10 PM, Eric McCorkle <eric@metricspace.net> wrote: > I'm not sure what you mean by direct map. Do you mean TLB? > > On 01/05/2018 18:08, Warner Losh wrote: > > Wouldn't you have to also unmap it from the direct map for this to be > > effective? > > > > Warner > > > > > > On Fri, Jan 5, 2018 at 3:31 PM, Eric McCorkle <eric@metricspace.net > > <mailto:eric@metricspace.net>> wrote: > > > > Well, the only way to find out would be to try it out. > > > > However, unless I'm missing something, if you're trying to pull a > > meltdown attack, you try and fetch from the kernel. If that location > > isn't cached (or if your cache is physically indexed), you need the > > physical address (otherwise you don't know where to look), and thus > have > > to go through address translation, at which point you detect that the > > page isn't accessible and fault. In the mean time, you can't > > speculatively execute any of the operations that load up the > > side-channels, because you don't have the sensitive data. > > > > The reason you can pull off a meltdown attack at all is that a > > virtually-indexed cache lets you get the data in parallel with > address > > translation (breaking the dependency between address translation and > > fetching data), which takes 1000s of cycles for a TLB miss, during > which > > you have the data and can launch a whole bunch of transient ops. > > > > Again, these are uncharted waters we're in; so it's entirely possible > > I'm missing something here. > > > > On 01/05/2018 17:22, Warner Losh wrote: > > > While you might be right, I've seen no indication that a cache miss > > > would defeat these attacks in the public and non-public data I've > looked > > > at, even though a large number of alternatives to the published > > > workarounds have been discussed. I'm therefore somewhat skeptical > this > > > would be effective. I'm open, however, to data that changes that > > > skepticism... > > > > > > Warner > > > > > > On Fri, Jan 5, 2018 at 3:15 PM, Eric McCorkle < > eric@metricspace.net <mailto:eric@metricspace.net> > > > <mailto:eric@metricspace.net <mailto:eric@metricspace.net>>> > wrote: > > > > > > Right, but you have to get the value "foo" into the pipeline > in order > > > for it to affect the side-channels. This technique attempts > to stop > > > that from happening. > > > > > > Unless I made a mistake, non-cached memory reads force address > > > translation to happen first, which detects faults and blocks > the > > > meltdown attack. > > > > > > It also stops spectre with very high probability, as it's very > unlikely > > > that an uncached load will arrive before the speculative > thread gets > > > squashed. > > > > > > On 01/05/2018 17:10, Warner Losh wrote: > > > > I think this is fatally flawed. > > > > > > > > The side channel is the cache. Not the data at risk. > > > > > > > > Any mapped memory, cached or not, can be used to influence > the cache. > > > > Storing stuff in uncached memory won't affect the side > channel one bit. > > > > > > > > Basically, all attacks boil down to tricking the processor, > at elevated > > > > privs, to doing something like > > > > > > > > a = foo[offset]; > > > > > > > > where foo + offset are designed to communicate information > by populating > > > > a cache line. offset need not be cached itself and can be > the result of > > > > simple computations that depend on anything accessible at > all in the kernel. > > > > > > > > Warner > > > > > > > > On Fri, Jan 5, 2018 at 3:02 PM, Eric McCorkle < > eric@metricspace.net <mailto:eric@metricspace.net> > > <mailto:eric@metricspace.net <mailto:eric@metricspace.net>> > > > > <mailto:eric@metricspace.net <mailto:eric@metricspace.net> > > <mailto:eric@metricspace.net <mailto:eric@metricspace.net>>>> wrote: > > > > > > > > Re-posting to -hackers and -arch. I'm going to start > working on > > > > something like this over the weekend. > > > > > > > > -------- Forwarded Message -------- > > > > Subject: A more general possible meltdown/spectre > countermeasure > > > > Date: Thu, 4 Jan 2018 23:05:40 -0500 > > > > From: Eric McCorkle <eric@metricspace.net <mailto: > eric@metricspace.net> > > > <mailto:eric@metricspace.net <mailto:eric@metricspace.net>> > > <mailto:eric@metricspace.net <mailto:eric@metricspace.net> > > > <mailto:eric@metricspace.net <mailto:eric@metricspace.net>>>> > > > > To: freebsd-security@freebsd.org <mailto: > freebsd-security@freebsd.org> > > > <mailto:freebsd-security@freebsd.org <mailto:freebsd-security@ > freebsd.org>> > > > > <mailto:freebsd-security@freebsd.org <mailto: > freebsd-security@freebsd.org> > > > <mailto:freebsd-security@freebsd.org <mailto:freebsd-security@ > freebsd.org>>> > > <freebsd-security@freebsd.org <mailto:freebsd-security@freebsd.org> > > > <mailto:freebsd-security@freebsd.org <mailto:freebsd-security@ > freebsd.org>> > > > > <mailto:freebsd-security@freebsd.org > > <mailto:freebsd-security@freebsd.org> > > > <mailto:freebsd-security@freebsd.org > > <mailto:freebsd-security@freebsd.org>>>> > > > > > > > > I've thought more about how to deal with > > meltdown/spectre, and > > > I have an > > > > idea I'd like to put forward. However, I'm still in > > something > > > of a > > > > panic mode, so I'm not certain as to its effectiveness. > > > Needless to > > > > say, I welcome any feedback on this, and I may be > completely > > > off-base. > > > > > > > > I'm calling this a "countermeasure" as opposed to a > > > "mitigation", as > > > > it's something that requires modification of code as > > opposed to a > > > > drop-in patch. > > > > > > > > == Summary == > > > > > > > > Provide a kernel and userland API by which memory > allocation > > > can be done > > > > with extended attributes. In userland, this could be > > > accomplished by > > > > extending MMAP flags, and I could imagine a > > > malloc-with-attributes flag. > > > > In kernel space, this must already exist, as drivers > > need to > > > allocate > > > > memory with various MTRR-type attributes set. > > > > > > > > The immediate aim here is to store sensitive information > > that must > > > > remain memory-resident in non-cacheable memory locations > > (or, > > > if more > > > > effective attribute combinations exist, using those > > instead). > > > See the > > > > rationale for the argument why this should work. > > > > > > > > Assuming the rationale holds, then the attack surface > should > > > be greatly > > > > reduced. Attackers would need to grab sensitive data > > out of stack > > > > frames or similar locations if/when it gets copied there > for > > > faster use. > > > > Moreover, if this is done right, it could dovetail > > nicely into a > > > > framework for storing and processing sensitive assets in > > more > > > secure > > > > hardware[0] (like smart cards, the FPGAs I posted > > earlier, or > > > other > > > > options). > > > > > > > > The obvious downside is that you take a performance hit > > > storing things > > > > in non-cacheable locations, especially if you plan on > > doing heavy > > > > computation in that memory (say, encryption/decryption). > > > However, this > > > > is almost certainly going to be less than the projected > > 30-50% > > > > performance hit from other mitigations. Also, this > > technique > > > should > > > > work against spectre as well as meltdown (assuming the > > > rationale holds). > > > > > > > > The second downside is that you have to modify code for > this > > > to work, > > > > and you have to be careful not to keep copies of > sensitive > > > information > > > > around too long (this gets tricky in userland, where you > > might get > > > > interrupted and switched out). > > > > > > > > > > > > [0]: Full disclosure, enabling open hardware > implementations > > > of this > > > > kind of thing is something of an agenda of mine. > > > > > > > > == Rationale == > > > > > > > > (Again, I'm tired, rushed, and somewhat panicked so my > logic > > > could be > > > > faulty at any point, so please point it out if it is) > > > > > > > > The rationale for why this should work relies on > > assumptions about > > > > out-of-order pipelines that cannot be guaranteed to > > hold, but are > > > > extremely likely to be true. > > > > > > > > As background, these attacks depend on out-of-order > > execution > > > performing > > > > operations that end up affecting cache and > branch-prediction > > > state, > > > > ultimately storing information about sensitive data in > these > > > > side-channels before the fault conditions are detected > and > > > acted upon. > > > > I'll borrow terminology from the paper, using "transient > > > instructions" > > > > to refer to speculatively executed instructions that will > > > eventually be > > > > cancelled by a fault. > > > > > > > > These attacks depend entirely on transient instructions > > being > > > able to > > > > get sensitive information into the processor core and > then > > > perform some > > > > kind of instruction on them before the fault condition > > cancels > > > them. > > > > Therefore, anything that prevents them from doing this > > > *should* counter > > > > the attack. If the actual sensitive data never makes it > to > > > the core > > > > before the fault is detected, the dependent memory > > > accesses/branches > > > > never get executed and the data never makes it to the > > > side-channels. > > > > > > > > Another assumption here is that CPU architects are going > to > > > want to > > > > squash faulted instructions ASAP and stop issuing along > > those > > > > speculative branches, so as to reclaim execution units. > So > > > I'm assuming > > > > once a fault comes back from address translation, then > > transient > > > > execution stops dead. > > > > > > > > Now, break down the cases for whether the address > containing > > > sensitive > > > > data is in cache and TLB or not. (I'm assuming here that > > > caches are > > > > virtually-indexed, which enables cache lookups to bypass > > address > > > > translation.) > > > > > > > > * In cache, in TLB: You end up basically racing between > the > > > cache and > > > > TLB, which will very likely end up detecting the fault > > before > > > the data > > > > arrives, but at the very worst, you get one or two > cycles of > > > transient > > > > instruction execution before the fault. > > > > > > > > * In cache, not in TLB: Virtually-indexed tagged means > > you get > > > a cache > > > > lookup racing a page-table walk. The cache lookup beats > the > > > page table > > > > walk by potentially hundreds (maybe thousands) of cycles, > > > giving you a > > > > bunch of transient instructions before a fault gets > > > triggered. This is > > > > the main attack case. > > > > > > > > * Not in cache, in TLB: Memory access requires address > > > translation, > > > > which comes back almost immediately as a fault. > > > > > > > > * Not in cache, not in TLB: You have to do a page table > walk > > > before you > > > > can fetch the location, as you have to go out to physical > > > memory (and > > > > therefore need a physical address). The page table walk > > will > > > come back > > > > with a fault, stopping the attack. > > > > > > > > So, unless I'm missing something here, both non-cached > cases > > > defeat the > > > > meltdown attack, as you *cannot* get the data unless you > do > > > address > > > > translation first (and therefore detect faults). > > > > > > > > As for why this defeats the spectre attack, the logic is > > > similar: you've > > > > jumped into someone else's executable code, hoping to > > scoop up > > > enough > > > > information into your branch predictor before the fault > > kicks > > > you out. > > > > However, to capture anything about sensitive information > > in your > > > > side-channels, the transient instructions need to > > actually get > > > it into > > > > the core before a fault gets detected. The same case > > analysis > > > as above > > > > applies, so you never actually get the sensitive info > > into the > > > core > > > > before a fault comes back and you get squashed. > > > > > > > > > > > > [1]: A physically-indexed cache would be largely immune > to > > > this attack, > > > > as you'd have to do address translation before doing a > cache > > > lookup. > > > > > > > > > > > > I have some ideas that can build on this, but I'd like > > to get some > > > > feedback first. > > > > _______________________________________________ > > > > freebsd-security@freebsd.org > > <mailto:freebsd-security@freebsd.org> > > > <mailto:freebsd-security@freebsd.org > > <mailto:freebsd-security@freebsd.org>> > > > <mailto:freebsd-security@freebsd.org > > <mailto:freebsd-security@freebsd.org> > > > <mailto:freebsd-security@freebsd.org <mailto:freebsd-security@ > freebsd.org>>> > > > > mailing list > > > > https://lists.freebsd.org/mailman/listinfo/freebsd- > security > > <https://lists.freebsd.org/mailman/listinfo/freebsd-security>; > > > <https://lists.freebsd.org/mailman/listinfo/freebsd-security > > <https://lists.freebsd.org/mailman/listinfo/freebsd-security>>; > > > > <https://lists.freebsd.org/mailman/listinfo/freebsd- > security > > <https://lists.freebsd.org/mailman/listinfo/freebsd-security>; > > > <https://lists.freebsd.org/mailman/listinfo/freebsd-security > > <https://lists.freebsd.org/mailman/listinfo/freebsd-security>>>; > > > > To unsubscribe, send any mail to > > > > "freebsd-security-unsubscribe@freebsd.org > > <mailto:freebsd-security-unsubscribe@freebsd.org> > > > <mailto:freebsd-security-unsubscribe@freebsd.org > > <mailto:freebsd-security-unsubscribe@freebsd.org>> > > > > <mailto:freebsd-security-unsubscribe@freebsd.org > > <mailto:freebsd-security-unsubscribe@freebsd.org> > > > <mailto:freebsd-security-unsubscribe@freebsd.org > > <mailto:freebsd-security-unsubscribe@freebsd.org>>>" > > > > _______________________________________________ > > > > freebsd-arch@freebsd.org <mailto:freebsd-arch@freebsd. > org> > > <mailto:freebsd-arch@freebsd.org <mailto:freebsd-arch@freebsd.org>> > > > <mailto:freebsd-arch@freebsd.org > > <mailto:freebsd-arch@freebsd.org> <mailto:freebsd-arch@freebsd.org > > <mailto:freebsd-arch@freebsd.org>>> > > > mailing list > > > > https://lists.freebsd.org/mailman/listinfo/freebsd-arch > > <https://lists.freebsd.org/mailman/listinfo/freebsd-arch>; > > > <https://lists.freebsd.org/mailman/listinfo/freebsd-arch > > <https://lists.freebsd.org/mailman/listinfo/freebsd-arch>>; > > > > <https://lists.freebsd.org/mailman/listinfo/freebsd-arch > > <https://lists.freebsd.org/mailman/listinfo/freebsd-arch>; > > > <https://lists.freebsd.org/mailman/listinfo/freebsd-arch > > <https://lists.freebsd.org/mailman/listinfo/freebsd-arch>>>; > > > > To unsubscribe, send any mail to > > > > "freebsd-arch-unsubscribe@freebsd.org > > <mailto:freebsd-arch-unsubscribe@freebsd.org> > > > <mailto:freebsd-arch-unsubscribe@freebsd.org > > <mailto:freebsd-arch-unsubscribe@freebsd.org>> > > > > <mailto:freebsd-arch-unsubscribe@freebsd.org > > <mailto:freebsd-arch-unsubscribe@freebsd.org> > > > <mailto:freebsd-arch-unsubscribe@freebsd.org > > <mailto:freebsd-arch-unsubscribe@freebsd.org>>>" > > > > > > > > > > > > > > > > > > >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CANCZdfq8UZWXdd9fc3yDTETPrCOYYojKyAzyjywrkG=d8OA6Jg>