From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 3 07:35:23 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6DA0A14C; Fri, 3 Oct 2014 07:35:23 +0000 (UTC) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id EC4B5263; Fri, 3 Oct 2014 07:35:22 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.9/8.14.9) with ESMTP id s937ZH5w065146 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 3 Oct 2014 10:35:17 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.9.2 kib.kiev.ua s937ZH5w065146 Received: (from kostik@localhost) by tom.home (8.14.9/8.14.9/Submit) id s937ZHWC065143; Fri, 3 Oct 2014 10:35:17 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 3 Oct 2014 10:35:17 +0300 From: Konstantin Belousov To: Larry Baird Subject: Re: Kernel/Compiler bug Message-ID: <20141003073517.GC26076@kib.kiev.ua> References: <20141001031553.GA14360@gta.com> <20141001134044.GA57022@gta.com> <542C8C75.30007@FreeBSD.org> <20141002075537.GU26076@kib.kiev.ua> <20141002140232.GA52387@gta.com> <20141002143345.GY26076@kib.kiev.ua> <20141003015456.GA27080@gta.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20141003015456.GA27080@gta.com> User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on tom.home Cc: "freebsd-hackers@freebsd.org" , Ryan Stone , Dimitry Andric , Bryan Drewery X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Oct 2014 07:35:23 -0000 On Thu, Oct 02, 2014 at 09:54:56PM -0400, Larry Baird wrote: > > The easiest thing to do is to record the stack depth for kernel mode > > on entry into interrupt. Interrupt handlers are usually well written > > and do not consume a lot of stack. > > > > Look at the intr_event_handle(), which is the entry point. The mode can > > be deduced from trapframe passed. The kernel stack for the thread is > > described by td->td_kstack (base, i.e. bottom) and td->td_kstack_pages > > (size), so the top of the stack is at td_kstack + td_kstack_size [*]. > > The current stack consumption could be taken from reading %rsp register, > > or you may take the address of any local variable as well. > > > > * - there are pcb and usermode fpu save area at the top of the stack, and > > actual kernel stack top is right below fpu save area. This should not > > be important for your measurements, since you are looking at how close > > the %rsp gets to the bottom. > > This idea worked very well. Booting a GENERIC 10.1-BETA3 kernel I get a > maximum stack used of 5247 bytes. This was with a minimal virtual box > configuration. It would be interesting to hear about users with more exotic > hardware and or configurations. Not sure if I have the KASSERT correct. > I have several notes. Mostly, it comes from my desire to make the patch committable. A global one is that the profiling of the stack use should be hidden under some kernel config option. > > > Index: kern_intr.c > =================================================================== > --- kern_intr.c (revision 44897) > +++ kern_intr.c (working copy) > @@ -1386,6 +1386,12 @@ > } > } > > +static int max_kern_thread_stack; Add 'usage' somewhere in the name of the var and sysctl ? > + > +SYSCTL_INT(_kern, OID_AUTO, max_kern_thread_stack, CTLFLAG_RD, > + &max_kern_thread_stack, 0, > + "Maxiumum stack used by a kernel thread"); > + > /* > * Main interrupt handling body. > * > @@ -1407,6 +1413,22 @@ > > td = curthread; > > + /* > + * Check for maximum stack used bya kernel thread. > + */ > + if (!TRAPF_USERMODE(frame)) { Just a note, the test for interruption of the usermode is not strictly needed, it only optimizes the execution, since interrupt from usermode would have only the trap frame on the stack. Might be, this should be commented. > + char *top = (char *)(td->td_kstack + td->td_kstack_pages * > + PAGE_SIZE - 1); > + char *current = (char *)&ih; Use the address of top ? It should be deeper in the stack, and account for the normal current function stack use, assuming compiler did not flatten out the frame. Anyway, it assumes that the stack grows down. Also, there are some situations, where the hardware might switch to dedicated stack for interrupt handling. It is impossible right now on amd64 and hw interrupts, but might become used in future, or on other arches. It makes sense to check that current value falls into the td stack region, before using it. > + int used = top - current; > + > + if (used > max_kern_thread_stack) { > + max_kern_thread_stack = used; This should be a loop with atomic cas, to not loose update from other thread. > + KASSERT(max_kern_thread_stack < KSTACK_PAGES * PAGE_SIZE, > + "Maximum kernel thread stack exxceeded"); Assert is not needed, we put a guard page below the thread stack to catch the overflow. You have seen it already with double fault on x86. > + } > + } > + > /* An interrupt with no event or handlers is a stray interrupt. */ > if (ie == NULL || TAILQ_EMPTY(&ie->ie_handlers)) > return (EINVAL);