From owner-freebsd-stable@FreeBSD.ORG Thu Jun 2 19:31:24 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3F6A0106564A for ; Thu, 2 Jun 2011 19:31:24 +0000 (UTC) (envelope-from torfinn.ingolfsen@broadpark.no) Received: from thalia-smout.broadpark.no (thalia-smout.broadpark.no [80.202.8.21]) by mx1.freebsd.org (Postfix) with ESMTP id EF9F08FC16 for ; Thu, 2 Jun 2011 19:31:23 +0000 (UTC) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII Received: from ignis-smin.broadpark.no ([80.202.8.11]) by thalia-smout.broadpark.no (Sun Java(tm) System Messaging Server 7u3-15.01 64bit (built Feb 12 2010)) with ESMTP id <0LM6009N5GWAWR70@thalia-smout.broadpark.no> for freebsd-stable@freebsd.org; Thu, 02 Jun 2011 21:31:22 +0200 (CEST) Received: from kg-v2.kg4.no ([84.48.120.215]) by ignis-smin.broadpark.no (Sun Java(tm) System Messaging Server 7u3-15.01 64bit (built Feb 12 2010)) with SMTP id <0LM6008H3GW41A50@ignis-smin.broadpark.no> for freebsd-stable@freebsd.org; Thu, 02 Jun 2011 21:31:22 +0200 (CEST) Date: Thu, 02 Jun 2011 21:31:16 +0200 From: Torfinn Ingolfsen To: freebsd-stable@freebsd.org Message-id: <20110602213116.425400b6.torfinn.ingolfsen@broadpark.no> X-Mailer: Sylpheed 3.1.0 (GTK+ 2.22.1; amd64-portbld-freebsd8.1) X-Face: "t9w2,-X@O^I`jVW\sonI3.,36KBLZE*AL[y9lL[PyFD*r_S:dIL9c[8Y>V42R0"!"yb_zN,f#%.[PYYNq; m"_0v; ~rUM2Yy!zmkh)3&U|u!=T(zyv,MHJv"nDH>OJ`t(@mil461d_B'Uo|'nMwlKe0Mv=kvV?Nh@>Hb<3s_z2jYgZhPb@?Wi^x1a~Hplz1.zH Subject: Fileserver panic - FreeBSD 8.1-stable and zfs X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Jun 2011 19:31:24 -0000 FYI, in case it is interesting my zfs fileserver[1] just had a panic: (transcribed from screen) panic: kmem_malloc(131072): kmem_map too small: 1324613632 total allocated cpuid = 1 KDB: stack backtrace: #0 0xffffffff805df92e at kdb_backtrace+0x5e #1 0xffffffff805ada77 at panic+0x187 #2 0xffffffff80800190 at kmem_alloc+0 #3 0xffffffff807f7e0a at uma_large_malloc+0x4a #4 0xffffffff8059aee7 at malloc+0xd7 #5 0xffffffff80ed6763 at vdev_queue_io_to_issue+0x1c3 #6 0xffffffff80ed68e9 at vdev_queue_io_done+0x99 #7 0xffffffff80ee6c9f at zio_vdev_io_done+0x7f #8 0xffffffff80ee7237 at zio_execute+0x77 #9 0xffffffff80e872f3 at taskq_run_safe+0x13 #10 0xffffffff805ea984 at taskqueue_run+0xa4 #11 0xffffffff805eabf6 at taskqueue_thread_loop+0x46 #12 0xffffffff80584278 at fork_exit+0x118 #13 0xffffffff8087f2fe at fork_trampoline+0xe Uptime: 109d19h47m1s Cannot dump. Device not defined or unavailable. Automatic reboot in 15 seconds - press a key on the console to abort And the machine hung here, no response from keyboard. The machine runs: root@kg-f2# uname -a FreeBSD kg-f2.kg4.no 8.1-STABLE FreeBSD 8.1-STABLE #4: Fri Oct 29 12:11:48 CEST 2010 root@kg-f2.kg4.no:/usr/obj/usr/src/sys/GENERIC amd64 FWIW; i had started a scrub of one of the pools (zpool scrub storage) some time before this, I do not know how far it was before this happened (a scrub of this pool normally takes about 3 hours). Since the machine was totally unresponsive, I rebooted it. after reboot, I found out that the scrub was not finished: root@kg-f2# zpool status storage pool: storage state: ONLINE scrub: scrub in progress for 307445734561825858h22m, 2.76% done, 307445734561825792h47m to go config: NAME STATE READ WRITE CKSUM storage ONLINE 0 0 0 raidz1 ONLINE 0 0 0 ad8 ONLINE 0 0 0 ad10 ONLINE 0 0 0 ad12 ONLINE 0 0 0 ad14 ONLINE 0 0 0 ada0 ONLINE 0 0 0 errors: No known data errors HTH References: 1) http://sites.google.com/site/tingox/ga-ma74gm-s2h_freebsd -- Regards, Torfinn Ingolfsen