Date: Tue, 19 Dec 2017 16:34:25 -0800 From: Nathan Whitehorn <nwhitehorn@freebsd.org> To: John Baldwin <jhb@freebsd.org>, Eugene Grosbein <eugen@grosbein.net>, Konstantin Belousov <kostikbel@gmail.com>, Conrad Meyer <cem@freebsd.org> Cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r326758 - in head/sys/i386: conf include Message-ID: <9b4b021b-dbf5-b788-2e23-6a49efa95da8@freebsd.org> In-Reply-To: <4a9c76c9-8063-9420-b198-14487b089840@FreeBSD.org> References: <201712110432.vBB4WbnE021090@repo.freebsd.org> <20171211091943.GF2272@kib.kiev.ua> <5A2E5D44.9030904@grosbein.net> <4a9c76c9-8063-9420-b198-14487b089840@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On 12/12/17 11:32, John Baldwin wrote: > On 12/11/17 5:26 AM, Eugene Grosbein wrote: >> On 11.12.2017 16:19, Konstantin Belousov wrote: >>> On Mon, Dec 11, 2017 at 04:32:37AM +0000, Conrad Meyer wrote: >>>> Author: cem >>>> Date: Mon Dec 11 04:32:37 2017 >>>> New Revision: 326758 >>>> URL: https://svnweb.freebsd.org/changeset/base/326758 >>>> >>>> Log: >>>> i386: Bump KSTACK_PAGES default to match amd64 >>> i386 is not amd64, the change is wrong. >>> >>> i386 has the word size two times smaller than amd64, which makes typical >>> frame smaller by 30-40% over same code on amd64. Also i386 has much >>> smaller available KVA size (tens of MB) and KVA fragmentation is both >>> more severe and more fatal due to this. I expect that your change will >>> make any non-trivial load which creates enough threads to either fail >>> randomly or deadlock. >>> >>> If somebody tries to fit large load onto i386 machine, he must know what to >>> do and how to configure the kernel to adapt to the load (which does not >>> require the recompilation). >> Its very easy to get kernel stack overflow with 11-STABLE/i386 >> without any significant load due to abuse of kernel stack in many kernel subsystems >> as shown in the https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=219476 >> >> Contrary, "enough threads" seems to be very non-trivial number of threads >> and pretty unusual load pattern for i386 as I run several such systems >> with kern.kstack_pages=4 for quite long time and have no problems. >> No random fails, no deadlocks. And with 2 pages only 11-STABLE/i386 is just unusable >> if one utilizes SCTP, IPv6, some WiFi connectivity, IPSEC or even very small ZFS pool. >> >> I still wonder if there is really such load pattern that creates "enough threads" >> for i386 to make 4-pages stack troublesome. > I suspect two things are at play in running out of stack in 10.x and later. > 1) Usage of XSAVE for AVX, etc. state uses more of the kstack (as kib@ alludes > to), and 2) clang. Certainly for MIPS I have found that compiling with clang > instead of gcc for mips64 gives a kernel that panics for stack overflow for any > use of NFS. It might be that this is due to something MIPS-specific, but it > might be worthwhile retesting with kstack_pages=2 and building the kernel > with CROSS_TOOLCHAIN=i386-gcc after installing the appropriate package. For what it's worth, I see the same NFS-related stack overflows on mips64 with in-tree GCC 4.2.1. -Nathan
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?9b4b021b-dbf5-b788-2e23-6a49efa95da8>