Date: Thu, 06 Dec 2012 11:48:16 -0600 From: Alan Cox <alc@rice.edu> To: Davide Italiano <davide@freebsd.org> Cc: alc@freebsd.org, FreeBSD Hackers <freebsd-hackers@freebsd.org>, Andriy Gapon <avg@freebsd.org> Subject: Re: huge ktr buffer Message-ID: <50C0DA60.9090706@rice.edu> In-Reply-To: <CACYV=-HhQcPx4nM=Ytc3wKS-oZqvtZCquJZSL6BooDHvRpuvGA@mail.gmail.com> References: <50C0B763.8000004@FreeBSD.org> <CACYV=-HhQcPx4nM=Ytc3wKS-oZqvtZCquJZSL6BooDHvRpuvGA@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 12/06/2012 09:43, Davide Italiano wrote: > On Thu, Dec 6, 2012 at 4:18 PM, Andriy Gapon <avg@freebsd.org> wrote: >> So I configured a kernel with the following option: >> options KTR_ENTRIES="(1024UL*1024)" >> then booted the kernel and did >> $ sysctl debug.ktr.clear=1 >> and got an insta-reboot. >> >> No panic, nothing, just a reset. >> >> I suspect that the huge static buffer resulting from the above option could be a >> cause. But I would like to understand the details, if possible. >> >> Also, perhaps ktr could be a little bit more sophisticated with its buffer than >> just using a static array. >> >> -- >> Andriy Gapon >> _______________________________________________ >> freebsd-hackers@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > It was a while ago, but running r238886 built using the following > kernel configuration file: > http://people.freebsd.org/~davide/DEBUG I found a similar issue. > The machine paniced: fatal trap 12 with interrupt disable in early boot > (even before the appareance of the Berkley logo). > Basically, my configuration file is just GENERIC with slight > modifications, in particular debugging options (WITNESS, INVARIANTS, > etc..) turned on and the following KTR options enabled: > > options KTR > options KTR_COMPILE=(KTR_CALLOUT|KTR_PROC) > options KTR_MASK=(KTR_CALLOUT|KTR_PROC) > options KTR_ENTRIES=524288 > > It seems the issue is related to KTR itself, and in particular to the > value of KTR_ENTRIES. As long as this value is little (e.g. 2048) > everything works fine and the boot sequence ends. If I choose 524288 > (the value you can also see from the kernel conf file) the fatal trap > occurs. > > Even though it was really difficult to me to get some informations > because the fail happens too early, I put some printf() within the > code and I isolated the point in which the kernel dies: > (sys/amd64/amd64/machdep.c, in getmemsize()) > > 1540 /* > 1541 * map page into kernel: valid, read/write,non-cacheable > 1542 */ > 1543 *pte = pa | PG_V | PG_RW | PG_N; > > > As also Alan suggested, a way to workaround the problem is to increase > NKPT value (e.g. from 32 to 64). Obviously, this is not a proper fix. > For a proper fix the kernel needs to be able to dynamically set the > size of NKPT. In this particular case, this wouldn't be too hard, but > there is a different case, where people preload a large memory disk > image at boot time that isn't so easy to fix. Andriy makes a good suggestion. One that I think should be easy to implement. The KTR code already supports the use of a dynamically allocated KTR buffer. (See sysctl_debug_ktr_entries().) Let's take advantage of this. Place a cap on the size of the (compile-time) statically allocated buffer. However, use this buffer early in the kernel initialization process, specifically, up until SI_ORDER_KMEM has completed. At that point, switch to a dynamically allocated buffer and copy over the entries from the statically allocated buffer. Relatively speaking, SI_ORDER_KMEM is early enough in the boot process, that I doubt many people wanting an enormous KTR buffer will be impacted by the cap. In fact, I think you could implement overflow detection without pessimizing the KTR code. Alan P.S. There are other reasons that having an enormous statically allocated array in the kernel is undesirable. The first that comes to mind is that it eats up memory at low physical addresses, which is sometimes needed for special purposes. So, I think there are good reasons besides the NKPT issue to shift the KTR code to dynamic allocation.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?50C0DA60.9090706>