From owner-freebsd-stable@FreeBSD.ORG Tue May 3 09:21:15 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 15359106566C for ; Tue, 3 May 2011 09:21:15 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta11.emeryville.ca.mail.comcast.net (qmta11.emeryville.ca.mail.comcast.net [76.96.27.211]) by mx1.freebsd.org (Postfix) with ESMTP id ED1D88FC1C for ; Tue, 3 May 2011 09:21:14 +0000 (UTC) Received: from omta07.emeryville.ca.mail.comcast.net ([76.96.30.59]) by qmta11.emeryville.ca.mail.comcast.net with comcast id exMB1g0021GXsucABxMElS; Tue, 03 May 2011 09:21:14 +0000 Received: from koitsu.dyndns.org ([67.180.84.87]) by omta07.emeryville.ca.mail.comcast.net with comcast id exMD1g0061t3BNj8UxMDqa; Tue, 03 May 2011 09:21:14 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 0FE709B418; Tue, 3 May 2011 02:21:13 -0700 (PDT) Date: Tue, 3 May 2011 02:21:13 -0700 From: Jeremy Chadwick To: Olaf Seibert Message-ID: <20110503092113.GA39704@icarus.home.lan> References: <20110502143230.GW6733@twoquid.cs.ru.nl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20110502143230.GW6733@twoquid.cs.ru.nl> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-stable@freebsd.org Subject: Re: Automatic reboot doesn't reboot X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 May 2011 09:21:15 -0000 On Mon, May 02, 2011 at 04:32:30PM +0200, Olaf Seibert wrote: > I have a FreeBSD/amd64 8.2 server that has a few ZFS file systems served > over NFS. It has 8 GB of memory. There are 6 disks of 1,5 TB each > forming a pool with raidz2. > > >From time to time it crashes with some stack backtrace (included below). > This already happened before the upgrade to 8.2. > > Now a crash of a file server is annoying, but if it reboots > automatically, there is just a few minutes of downtime (most of it is > even spent by the BIOS before it gets to boot the OS). > > However, it doesn't automatically reboot in 15 seconds, as promised. > It just sits there the whole weekend, until I log onto the IPMI console > and press the virtual reset button. There are two things you might try fiddling with. These are sysctls so you can try them on the fly: hw.acpi.disable_on_reboot hw.acpi.handle_reboot On our systems we set hw.acpi.handle_reboot=1 to speed up the reboot process. I remember hearing long ago how some people had issues getting their machines to reboot (sometimes 100% of the time, other times occasionally); using ACPI to reboot the machine fixed their issues. > This was visible before I did that (4-finger copy): > > panic: kmem_alloc(131072): kmem_map too small: 3428782080 total allocated > cpuid = 0 Check out the thread Peter Jeremy provided. This is a near-sure indicator of ZFS ARC exhaustion, and you seem to know of that. What's very interesting to me is this part of your mail: > There is some tuning in /boot/loader.conf from previous attempts tune to > avoid crashes. > > vm.kmem_size="16G" > vfs.zfs.arc_max="4G" > > Is that still useful, or does it harm by now? Real memory is 8 GB. > I note that if I look with sysctl, I see > > vm.kmem_size: 3739230208 > vfs.zfs.arc_max: 2665488384 > > which doesn't seem to match these attempted settings. Is this box running i386 or amd64? If amd64, I can't explain why your /boot/loader.conf settings aren't taking -- they should be for sure. Maybe provide us a full dmesg and XXX out things you consider sensitive. If i386, I'm not too surprised that some automatic defaults get chosen instead of what you ask. -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP 4BD6C0CB |