Date: Mon, 27 Jan 2014 15:34:32 +0530 From: "Jayachandran C." <c.jayachandran@gmail.com> To: Juli Mallett <jmallett@freebsd.org> Cc: "freebsd-mips@freebsd.org" <freebsd-mips@freebsd.org> Subject: Re: More trapframe panics Message-ID: <CA%2B7sy7BEgvMNJKiR4bSSGZ_BE1T5W3Hujq8_2C6JjPC1bFwO6A@mail.gmail.com> In-Reply-To: <CACVs6=8ie2x16c4bafG4zaJS4UV%2BXT1ooJkm%2BdwT5%2BdSS%2B-p0Q@mail.gmail.com> References: <52E42A1B.3040907@rewt.org.uk> <CACVs6=--Qy_8poWdHdCXYKqkO22=dvHhW8=Uma8kLR%2BhCoZDxw@mail.gmail.com> <CA%2B7sy7ATMTjdWD5%2BNYCJPb=AWmeJoUwbgRdf=%2Bb4uYsJO0UUfw@mail.gmail.com> <CACVs6=8ie2x16c4bafG4zaJS4UV%2BXT1ooJkm%2BdwT5%2BdSS%2B-p0Q@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, Jan 27, 2014 at 12:49 PM, Juli Mallett <jmallett@freebsd.org> wrote: > On Sun, Jan 26, 2014 at 11:06 PM, Jayachandran C. <c.jayachandran@gmail.com> > wrote: >> >> Hi Juli, >> >> On Sun, Jan 26, 2014 at 3:36 AM, Juli Mallett <jmallett@freebsd.org> >> wrote: >> > >> > This happens easily on 64-bit MIPS because due to slightly-crummy >> > design >> > on our part there's proportionally less room on the stack than on a >> > 32-bit >> > system. Several people have nebulous plans to address the problem of >> > the >> > stack being too small, but I don't know of anyone intending concrete >> > action >> > going forward. >> >> I had not seen the issue so far. > > > Honestly, I haven't seen it much either, certainly not as much as many > people seem to report on the lists, and I built lots of packages, etc. But > clearly my workload was different or something has changed in the kernel, > but it's definitely a real problem. It would be interesting to see the stacktrace when the overflow happens. If this is some specific driver or codepath, fixing that would be a better solution than to waste 8K per task. >> >> When I had done the pmap changes ealier, I had thought of adding the >> option to use a higher order page for kernel stack and using the KSEG0 >> address as the stack pointer, instead of using the wired entry as we >> do now. Is there any reason this will not work. > > I don't have a problem with physically-contiguous kernel stacks and use of > KSEG0/XKPHYS. It's better in some ways, and I'd worry about a system on > which finding 4 physically-contiguous pages was impossible. We can't swap > out kstacks then, but that's probably okay. My major concern is that then a > kernel stack overflow is probably a lot harder to detect, and an attacker > probably has other interesting kernel data structures nearby. I'm sure > there's other drawbacks to consider, but it'd simplify a lot of bad code, > and it seems worthwhile as an intermediate step anyway, even if it does mean > MIPS can't support guard pages for the kernel stack. Getting contiguous memory - maybe a custom UMA zone for kstack pages would be useful (although I have not really looked at this). And stack overflow detection: this will have to move to software if really needed. Or we should have a wired entry with a larger pagemask for the kstack page. > You could write very careful code to load the stack pointer from PCPU and > bounds check sp and proceed very carefully, but I'm not sure we really want > or need to go there. That said, it only matters for faults taken within the > kernel; anything coming from userland including interrupts and syscalls will > load sp and start at the top of the stack anyway, so you could at least > avoid adding the overhead for the most performance-critical exception > handlers. > > I know that on some CPUs we care about, having to use the extra wired TLB > entry is a real pain. There are things we could do to lessen this pain, > such as getting rid of the wired PCPU TLB entry. There's two possibilities > I'd suggest for that, and only one of them relies on kstacks not being > shared between CPUs: > > (1) Move to keeping the PCPU pointer in "gp". > (2) Always store a PCPU pointer at the top of the kernel stack, adding a > little indirection to load PCPU, but not a lot. > > It's nice to have the PCPU wired because you do want to be able to access it > quickly from things like exception handlers, which gets messier with the > "gp" approach, since then you have to load "gp" in some annoying manner when > coming from userland. I would like to move the PCPU out of wired entry as well. Using gp is one option. I wonder why we did not use the simpler option of having a array of pointers indexed by cpuid. Some MIPS processors also have COP0 kscratch registers. which can be used in a lot of theses cases. For example, we can keep the frequently used pointers like the current task's page table pointer, the current thread's kernel stack and current CPU's pcpu ptr in scratch. > Just some thoughts. I'd be delighted to see any forward-moving approach > that's actually taken :) The only unknown here would be to see how difficult it is to get a higher order page from the VM, and whether it will slow down task creation when the contiguous pages are not available. JC.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CA%2B7sy7BEgvMNJKiR4bSSGZ_BE1T5W3Hujq8_2C6JjPC1bFwO6A>