From owner-freebsd-fs@FreeBSD.ORG Fri Jan 22 13:22:13 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 90CA3106568B for ; Fri, 22 Jan 2010 13:22:13 +0000 (UTC) (envelope-from doug@polands.org) Received: from hrndva-omtalb.mail.rr.com (hrndva-omtalb.mail.rr.com [71.74.56.122]) by mx1.freebsd.org (Postfix) with ESMTP id 488A08FC15 for ; Fri, 22 Jan 2010 13:22:12 +0000 (UTC) X-Authority-Analysis: v=1.0 c=1 a=GSN_Y9T6cv4A:10 a=5oGaD+IacabtnYYqBVNCkQ==:17 a=bqq2Vc5EAAAA:8 a=5zptC--39CfEUlsaC4YA:9 a=-c6Oll0x8xvuYqk6BTcA:7 a=hCRTB8BdqILmrSuMg6odhxlswz4A:4 a=5ERLOmoKdHQA:10 X-Cloudmark-Score: 0 X-Originating-IP: 75.87.219.217 Received: from [75.87.219.217] ([75.87.219.217:57808] helo=haran.polands.org) by hrndva-oedge04.mail.rr.com (envelope-from ) (ecelerity 2.2.2.39 r()) with ESMTP id 71/C8-10659-486A95B4; Fri, 22 Jan 2010 13:22:12 +0000 Received: from email.polands.org (ammon.polands.org [172.16.1.7]) by haran.polands.org (8.14.3/8.14.3) with ESMTP id o0MDMBUC030444; Fri, 22 Jan 2010 07:22:11 -0600 (CST) (envelope-from doug@polands.org) Received: from 209.103.214.34 (SquirrelMail authenticated user djp) by email.polands.org with HTTP; Fri, 22 Jan 2010 07:22:11 -0600 Message-ID: In-Reply-To: References: <4B58976E.1020402@polands.org> <4B58A069.8000802@egr.msu.edu> <4B58BD2D.30803@rcn.com> <4B58D4D3.80009@egr.msu.edu> <20100122042843.GA8858@polands.org> Date: Fri, 22 Jan 2010 07:22:11 -0600 From: "Doug Poland" To: "Artem Belevich" User-Agent: SquirrelMail/1.4.20-RC2 MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal Cc: freebsd-fs@freebsd.org Subject: Re: Repeatable ZFS "kmem map too small" panic on 8.0-STABLE X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 Jan 2010 13:22:13 -0000 On Fri, January 22, 2010 00:09, Artem Belevich wrote: > On Thu, Jan 21, 2010 at 8:28 PM, Doug Poland > wrote: >> Interesting discussion :)  I added vm.kmem_size="20G" to >> /boot/loader.conf per your instructions.  This time, it didn't >> panic at the same point in the test, however, it appears the >> filesystem is "hanging". >> >> On the fdisk test, I hit T and get: >> cmd: fsdisk 37066 [zio->io_cv)] 245.62r 0.12u 25.10s 0 >> >> My various metrics are still running, but anything that needs the >> filesystem appears "stuck". >> >> The memory usage of the item "solaris" (vmstat -m | grep solaris) >> spiked at 3334781952 (3180.30 MiB). >> >> # zpool iostat 2, a T shows: >> load: 0.00  cmd: zpool 934 [tx->tx_quiesce_done_cv)] 2052.45r 0.06 u 0.39s 0% 0k >> >> # vmstat -v | grep solaris to disk every second and it's hung at: >> load: 0.00  cmd: sh 38551 [zfs] 909.85r 0.00u 0.00s 0% 16k >> >> Any suggestions! >> > Next step would be to set vfs.zfs.arc_max to a value that's somewhat > below your physical memory size. Let's say - 1G or so. On my box with > 8GB of RAM I set these values: > > vfs.zfs.arc_max="6500M" > vfs.zfs.arc_min="4G" > OK, I'll give that a shot... > You didn't mention how recent is your kernel. There were quite a few > bugfixes committed to -8. Make sure your kernel is r201987 or newer. > Right now I'm running a stock 8.0-RELEASE. I had tried it on -STABLE a couple of days ago with similar results. I'll get the box back up to -STABLE again. Do you advise to upgrade the pool to v14 for this testing? -- Regards, Doug