From owner-freebsd-current@FreeBSD.ORG Tue Nov 6 20:34:14 2007 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C7BE916A4FA for ; Tue, 6 Nov 2007 20:34:14 +0000 (UTC) (envelope-from stevenschlansker@berkeley.edu) Received: from smtp-out1.berkeley.edu (smtp-out1.Berkeley.EDU [128.32.61.106]) by mx1.freebsd.org (Postfix) with ESMTP id 664AD13C4A3 for ; Tue, 6 Nov 2007 20:34:14 +0000 (UTC) (envelope-from stevenschlansker@berkeley.edu) Received: from eva-wlan-253-209.airbears.berkeley.edu ([169.229.253.209]) by fe2.calmail with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.68) (auth plain:stevenschlansker@berkeley.edu) (envelope-from ) id 1IpUt7-000773-84 for freebsd-current@freebsd.org; Tue, 06 Nov 2007 12:19:05 -0800 Message-ID: <4730CC39.8020804@berkeley.edu> Date: Tue, 06 Nov 2007 12:19:05 -0800 From: Steven Schlansker User-Agent: Thunderbird 2.0.0.6 (X11/20071022) MIME-Version: 1.0 To: freebsd-current@freebsd.org References: <20071005000046.GC92272@garage.freebsd.pl> <20071008121523.GM2327@garage.freebsd.pl> <20071105215035.GC26730@heff.fud.org.nz> <2e77fc10711051531k41e7224dq6aaedb35cad8d9f2@mail.gmail.com> <6214AB9C-9F9B-4B9D-8B05-0B3DF5F6C16D@SARENET.ES> <20071106100015.GB5268@garage.freebsd.pl> In-Reply-To: <20071106100015.GB5268@garage.freebsd.pl> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: ZFS kmem_map too small. X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Nov 2007 20:34:14 -0000 Pawel Jakub Dawidek wrote: > On Tue, Nov 06, 2007 at 10:04:43AM +0100, Borja Marcos wrote: >> On Nov 6, 2007, at 12:31 AM, Niki Denev wrote: >> the kernel memory limits to 1.5 GB but it still crashes with the >> kmem_too_small panic. I also >> tried the patch supplied by Pawel. >> >> The machine has 6 hard disks configured as a raidz2 pool. >> >> Please let me know of any test anyone might need. > > If you use vm_kern.c.2.patch, can you show loader.conf and exact command > that can provke the panic? > I'd like to mention that I have the same problem - running FreeBSD ganymede.dailycal.org 7.0-BETA1 FreeBSD 7.0-BETA1 #0: Tue Oct 30 12:43:09 PDT 2007 root@ganymede.dailycal.org:/usr/obj/usr/src/sys/GENERIC i386 on a CPU: Intel(R) Xeon(TM) CPU 2.80GHz (2793.20-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0xf41 Stepping = 1 with 1GB ram. I also have the vm patch applied. It helped, but it still panics. I run a huge rsync from a remote machine to a zpool set up as follows: [sschlansker@ganymede /usr/obj/usr/src/sys/GENERIC]$ sudo zpool status pool: dailycal state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM dailycal ONLINE 0 0 0 raidz2 ONLINE 0 0 0 da1s1 ONLINE 0 0 0 da1s2 ONLINE 0 0 0 ad4 ONLINE 0 0 0 ad6 ONLINE 0 0 0 ad8 ONLINE 0 0 0 ad10 ONLINE 0 0 0 /boot/loader.conf: vfs.zfs.arc_max="104857600" vm.kmem_size_max="838860800" vm.kmem_size="402653184" The panic is as follows: Unread portion of the kernel message buffer: panic: kmem_malloc(131072): kmem_map too small: 358408192 total allocated cpuid = 0 Uptime: 20h0m56s Physical memory: 1011 MB Dumping 430 MB: 415 399 383 367 351 335 319 303 287 271 255 239 223 207 191 175 159 143 127 111 95 79 63 47 31 15 #0 doadump () at pcpu.h:195 195 __asm __volatile("movl %%fs:0,%0" : "=r" (td)); (kgdb) bt #0 doadump () at pcpu.h:195 #1 0xc07506a7 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:409 #2 0xc0750969 in panic (fmt=Variable "fmt" is not available. ) at /usr/src/sys/kern/kern_shutdown.c:563 #3 0xc096b9fa in kmem_malloc (map=0xc147108c, size=131072, flags=2) at /usr/src/sys/vm/vm_kern.c:312 #4 0xc0962217 in page_alloc (zone=0x0, bytes=131072, pflag=0xe892da33 "\002", wait=2) at /usr/src/sys/vm/uma_core.c:955 #5 0xc0964ce0 in uma_large_malloc (size=131072, wait=2) at /usr/src/sys/vm/uma_core.c:2709 #6 0xc0741d98 in malloc (size=131072, mtp=0xc4405400, flags=2) at /usr/src/sys/kern/kern_malloc.c:364 #7 0xc4384a60 in ?? () #8 0x00020000 in ?? () #9 0xc4405400 in ?? () ... It survives a scrub, but the rsync kills it every time. I'm also willing to help test out patches and whatnot - hope something here helps figure this problem out. Steven