Date: Mon, 30 Mar 2015 19:33:58 +0200 From: Mateusz Guzik <mjguzik@gmail.com> To: Oliver Pinter <oliver.pinter@hardenedbsd.org> Cc: PaX Team <pageexec@freemail.hu>, Benjamin Kaduk <bjk@freebsd.org>, HardenedBSD Core <core@hardenedbsd.org>, Yue Chen <ycyc321@gmail.com>, "freebsd-hackers@freebsd.org" <freebsd-hackers@freebsd.org> Subject: Re: How to traverse kernel threads? Message-ID: <20150330173358.GB9095@dft-labs.eu> In-Reply-To: <CAPQ4fft%2BmNJObMSw8SFvsweZsB-UKNUV2yghg4jB%2BdvSg9ho2w@mail.gmail.com> References: <CAKtBrB4h13ZFJ=V0fvkDeTG-L6=e5Uz9%2BHfYc8vY523Y3X6N0A@mail.gmail.com> <20150321220246.GE14650@dft-labs.eu> <CAKtBrB5KNqt6UJ1R_BQpPfTvQZdUzGvZZtT7Uz5qd4VrrfgEdw@mail.gmail.com> <20150321232622.GF14650@dft-labs.eu> <alpine.GSO.1.10.1503221644440.22210@multics.mit.edu> <CAPQ4ffuszSi%2B_SopJdCkoFr4OoY9=BZVbO6oo_s0sKrn8Rgjrw@mail.gmail.com> <CAKtBrB6ZF2FVExmDd%2Bt8yFpN0H7xHwaieWgvryR535Vc2cNBjw@mail.gmail.com> <20150327194920.GB18158@dft-labs.eu> <CAKtBrB6qeNTyemG-ws7X_OKcB2t-6muxJrOZ1Eyr0CRjjso5mQ@mail.gmail.com> <CAPQ4fft%2BmNJObMSw8SFvsweZsB-UKNUV2yghg4jB%2BdvSg9ho2w@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Fri, Mar 27, 2015 at 11:20:39PM +0100, Oliver Pinter wrote: > On Fri, Mar 27, 2015 at 9:55 PM, Yue Chen <ycyc321@gmail.com> wrote: > >> Except it seems there routines are supposed to be only used when > >> execution is 'frozen' (e.g. when escaped to the debugger). > > > > It means probably we can run the code in ``smp_rendezvous()'' function, > > right? > > > >> Still nobody knows what you are trying to do. > > > > We are trying to enhance FreeBSD's security by randomizing kernel code basic > > blocks periodically at runtime, to mitigate the attacks like return-oriented > > programming (ROP). It is basically a stronger form of ASLR. > > After each randomization procedure, the function return addresses saved in > > the stack are the ``old'' addresses before randomization, so we need to > > update them to the new addresses. > > That's why we need to get all the stack ranges to find those addresses. > > > > Also, in kernel, we believe that not only the return addresses in stacks > > need to be updated, there may exist other ``old'' saved instruction (PC) > > addresses in memory. Like in exception handling (maybe, do not know), > > debugging-purpose code and restartable atomic sequences (RAS) > > implementation. That's why I asked how to traverse all the kernel pages and > > get their virtual addresses here: > > https://lists.freebsd.org/pipermail/freebsd-hackers/2015-March/047336.html > > > > Now we found that it seems needed to traverse the ``pv_entry'' structure for > > x86_64 MMU. > > > > Another problem is that we do not know if FreeBSD has any form of special > > encodings or offset form for 64-bit instruction addresses (e.g., saved %RIP) > > on X86_64, instead of hard-coded addresses. For example, using a 32-bit > > offset instead of the 64-bit full address; and doing what glibc does for the > > setjmp/longjmp jmp_buf (special encodings (PTR_MANGLE) for the saved > > register values). > > > > Any suggestion or correction are highly appreciated. > > > > Best, > > Yue > > (Added HardenedBSD core and PaXTeam to CC.) > > Until you can not fixed all of the infoleaks from kernel (try sysctl > -a | grep 0x or similar command) the KASLR and other kernel address > space randomization techniques are easily bypass-able... > I do not believe this can be implemented reliably without some serious tinkering around the kernel. Surely I'm not a live patching expert (not any other kind of expert), but hear my out. It seems proposed approach is to move the kernel around and then updated all relevant pointers in all pages. I do not believe this can be done reliably without corrupting data, unless a major surgery is performed on the kernel. Consider: struct meh { func_t *m_func; size_t m_len; data *m_buf; }; Here we have 'm_func' which needs to be updated, but we don't know if that's the only pointer so we have to scan the entire struct. But m_buf can have data which looks like kernel pointers (i.e. matches some kernel funcs) and how will you know not to modify it? What if this is some sort of a hack and in fact you *should* modify it? CTF or some other solutions like that don't help if you just traverse all allocated buffers since you have no idea what the object you found really is. So, assuming this has to be done in runtime, the only somewhat workable solution I see would require leaving function entry points at contant addresses and only insert jumps to new locations. This would be a performance hit and would leave a lot of data at constant addresses, defeating the point to some extent. Also runtime relocation of everything definitely has a lot of unknown unknowns, so all in all I would say this is a non-starter. One could consider a different approach where kernel data is randomly shuffled around with some granularity and relevant symbol relocated prior to booting. This should provide unique enough layout, which paired with big likelyhood of a kernel panic on first bad exploit attempt may be an acceptable solution. But then the kernel may still have enough info leaks for this to not matter, so I would not be so eager to implement it. That said, what prompted this entire effort? Is there an operating system which got this to work or what? > > > > > > > > On Fri, Mar 27, 2015 at 3:49 PM, Mateusz Guzik <mjguzik@gmail.com> wrote: > >> > >> On Fri, Mar 27, 2015 at 02:35:55PM -0400, Yue Chen wrote: > >> > When using the following code on kernel module loading: > >> > > >> > ------------------------------------------------------------------------------------------ > >> > struct thread *td = kdb_thr_first(); > >> > td = kdb_thr_next(td); > >> > > >> > ------------------------------------------------------------------------------------------ > >> > The kernel panics. > >> > > >> > >> Panics how? > >> > >> Also you can easily see these functions don't lock anything, so it would > >> be assumed you took appropriate locks. > >> > >> Except it seems there routines are supposed to be only used when > >> execution is 'frozen' (e.g. when escaped to the debugger). > >> > >> > > >> > And when printing all threads in proc0 (all kernel threads?): > >> > > >> > ------------------------------------------------------------------------------------------ > >> > struct proc *p = pfind(0); > >> > FOREACH_THREAD_IN_PROC(p, td) { > >> > uprintf("td: %x\n", td); > >> > } > >> > > >> > >> proc0 is an exported symbol, no need to pfind. > >> > >> > td = curthread; > >> > uprintf("cur td: %x\n", td); > >> > > >> > ------------------------------------------------------------------------------------------ > >> > The ``curthread'' (from this kernel module running the above code) is > >> > not > >> > in the 0 process group. > >> > > >> > >> There is no 'curthread from kernel module'. > >> > >> My guess is you do this work from module initializator, and in that case > >> curthread is the thread which loads the module, and such a thread is > >> definitely not linked into proc0. > >> > >> Still nobody knows what you are trying to do. > >> > >> -- > >> Mateusz Guzik <mjguzik gmail.com> > > > > -- Mateusz Guzik <mjguzik gmail.com>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20150330173358.GB9095>