From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 2 07:55:45 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0B62E827; Thu, 2 Oct 2014 07:55:45 +0000 (UTC) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A12E987B; Thu, 2 Oct 2014 07:55:44 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.9/8.14.9) with ESMTP id s927tcGT030413 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 2 Oct 2014 10:55:38 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.9.2 kib.kiev.ua s927tcGT030413 Received: (from kostik@localhost) by tom.home (8.14.9/8.14.9/Submit) id s927tbCm030410; Thu, 2 Oct 2014 10:55:37 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 2 Oct 2014 10:55:37 +0300 From: Konstantin Belousov To: Bryan Drewery Subject: Re: Kernel/Compiler bug Message-ID: <20141002075537.GU26076@kib.kiev.ua> References: <20141001031553.GA14360@gta.com> <20141001134044.GA57022@gta.com> <542C8C75.30007@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <542C8C75.30007@FreeBSD.org> User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on tom.home Cc: "freebsd-hackers@freebsd.org" , Ryan Stone , Dimitry Andric , Larry Baird X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Oct 2014 07:55:45 -0000 On Wed, Oct 01, 2014 at 06:21:25PM -0500, Bryan Drewery wrote: > On 10/1/2014 2:37 PM, Dimitry Andric wrote: > > On 01 Oct 2014, at 15:40, Larry Baird wrote: > >> Ryan, > >> > >> On Wed, Oct 01, 2014 at 12:46:35AM -0400, Ryan Stone wrote: > >>> This may not be a compiler bug. A quick look at the esp values > >>> provided in that backtrace shows that at least 7KB has been used on > >>> the stack. The stack for kernel threads is only 8KB, and a stack > >>> overflow can cause a double fault like that. > >>> > >>> My suspicion would be that without optimizations on clang uses a lot > >>> more stack space and you push over the limit. There's a kernel build > >>> option for the stack size that you could change to confirm. I believe > >>> that it's called KSTACK_PAGES. Try increasing it to 4. > >> Good catch. Increasing KSTACK_PAGES does fix the issue. I wonder with > >> optimization, how close to stack overflow does the kernel get during boot? > > > > It obviously depends on which optimization flags you use, which drivers > > you include, and so on. There was a thread some time ago about somebody > > banging into the limit when mounting certain ZFS filesystems, here: > > > > https://lists.freebsd.org/pipermail/freebsd-current/2012-December/038208.html > > > > This is why Kostik added printing of the frame addresses to the panic > > backtrace output, so you can easily see if you hit the stack limit. > > > > That said, 8k is not much these days, especially not with fairly > > complicated code like ZFS, combined with high optimization, which can > > inline a lot of functions, causing even more stack usage. I would just > > bump KSTACK_PAGES. > > > > -Dimitry > > > > Is this something that can be bumped in the tree for GENERIC? The cost of the increased size for kernel stack is significant, even on architectures with ample KVA. This must not be done just because some non-default kernel settings cause stack overflow. If somebody feels himself qualified enough to tune compiler options, it must understand the consequences and do other required adjustments, including kernel stack size tuning. FWIW, there was old reason why -O0 did not worked for the kernel. The cpufunc.h inlines are not provided in non-inline version, and at least gcc at -O0 level sometimes generated the call to nonexisting function, leading to linking failure. It is curious that clang always inlines at -O0, but it is possible, although unlikely, that kernel source was changed to be immune.