Date: Sun, 24 Dec 2006 23:28:55 +0900 From: Norikatsu Shigemura <nork@FreeBSD.org> To: Pawel Jakub Dawidek <pjd@FreeBSD.org> Cc: freebsd-fs@FreeBSD.org Subject: Re: [REPORT] Some panics on ZFS Message-ID: <20061224232855.b0786250.nork@FreeBSD.org> In-Reply-To: <20061223142332.GA99259@garage.freebsd.pl> References: <20061223111738.6292605a.nork@FreeBSD.org> <20061223142332.GA99259@garage.freebsd.pl>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sat, 23 Dec 2006 15:23:32 +0100 Pawel Jakub Dawidek <pjd@freebsd.org> wrote: > > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - > > ZFS filesystem version 3 > > ZFS storage pool version 3 > > Warning: memory type solaris leaked memory on destroy (116 allocations, 7328 bytes leaked). > > ZFS filesystem version 3 > > ZFS storage pool version 3 > > Warning: memory type solaris leaked memory on destroy (116 allocations, 7328 bytes leaked). > > ZFS filesystem version 3 > > ZFS storage pool version 3 > > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - > This one is already fixed. Thank you. > > But 'kldload zfs; mount -t zfs tank /mnt' didn't panic. f(?_?)? > It can be easly fixed, but bascially in zfs world you should mount file I saw perforce/112119. Thank you. (http://perforce.freebsd.org/chv.cgi?CH=112119) > systems after loading zfs.ko via: > # zfs mount -a > and activate zvols: > # zfs volinit Yes. So I'm using following configuration:-). /etc/rc.conf.d/mountlate - - - - - - - - - - - - - - - - - - - - - - - zfsinit () { kldstat -q -m zfs || kldload zfs } zfsmount () { zfs mount -a zfs volinit } zfsunmount () { zfs unmount -a } start_precmd=zfsinit start_postcmd=zfsmount stop_precmd=zfsunmount /etc/rc.conf.d/mountlate - - - - - - - - - - - - - - - - - - - - - - - I think that /etc/rc.d/mountzfs will be installed. > > 3. When I'm testing with iozone, ZFS panic-ed. > > I saw some memory modified after free. > Are you able to setup memguard(9)? Or maybe you can give me procedure to > reproduce it? I did setup memguard(9). But I cannot get any data. Because I got following panics:-). - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - panic: MEMGUARD: You must increase MAX_PAGES_PER_ITEM in memguard.c (requested: 512 pages) cpuid = 0 KDB: enter: panic [thread pid 817 tid 100070 ] Stopped at kdb_enter+0x30: leave db> bt Tracing pid 817 tid 100070 td 0x8e80a360 kdb_enter(806b6baf,0,806d113f,f9ef6a8c,8e80a360,...) at kdb_enter+0x30 panic(806d113f,200,0,f9ef6aa4,200000,...) at panic+0x14e memguard_alloc(200000,1,8ebd359f,200000,20000,...) at memguard_alloc+0x31 malloc(200000,8ec4cb20,1,f9ef6af8,8ebd349e,...) at malloc+0xaf kmem_alloc(200000,1,80000,0,f9ef6b38,...) at kmem_alloc+0x21 kmem_zalloc(200000,1,0,0,100,...) at kmem_zalloc+0x1e dbuf_init(f9ef6b5c,8ec09cc7,8ec56638,8ec08950,1c,...) at dbuf_init+0x81 dmu_init(8ec56638,8ec08950,1c,8,8ebacc80,...) at dmu_init+0x8 spa_init(3,0,0,0,180,...) at spa_init+0x97 zfs_modevent(8ebacc80,0,0,75,f9ef6ba4,...) at zfs_modevent+0x58 module_register_init(8ec4da30,0,806b44ad,dc,0,...) at module_register_init+0xa5 linker_file_sysinit(8eb8a500,8ebad180,f9ef6bec,175,8ebad180,...) at linker_file_sysinit+0xe0 linker_load_file(8ebad180,f9ef6c14,0,72c,d0,...) at linker_load_file+0x125 linker_load_module(0,8e04c400,0,0,f9ef6c40,...) at linker_load_module+0xf9 kern_kldload(8e80a360,8e04c400,f9ef6c68,0,2f8,...) at kern_kldload+0xc4 kldload(8e80a360,f9ef6d00,4,c,f9ef6d38,...) at kldload+0x74 syscall(f9ef6d38) at syscall+0x2e3 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (1, FreeBSD ELF32, sys_exit), eip = 0x2, esp = 0x213, ebp = 0x7fbfeee4 --- db> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - I increased MAX_PAGES_PER_ITEM to 65536. But... - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - panic: MEMGUARD: Failed with M_WAITOK: memguard_map too small cpuid = 0 KDB: enter: panic [thread pid 1027 tid 100126 ] Stopped at kdb_enter+0x30: leave db> bt Tracing pid 1027 tid 100126 td 0x8ecd9d80 kdb_enter(806b6b8f,0,806d1172,f9fe3830,8ecd9d80,...) at kdb_enter+0x30 panic(806d1172,0,806d104e,f3,240,...) at panic+0x14e memguard_alloc(240,2,0,240,8ecd8d80,...) at memguard_alloc+0x156 malloc(240,8ec50b20,2,f9fe389c,8ebd749e,...) at malloc+0xaf kmem_alloc(240,2,84607000,9ee80400,f9fe38e4,...) at kmem_alloc+0x21 kmem_zalloc(240,2,8db77000,8db78000,7,...) at kmem_zalloc+0x1e zio_create(25b9,0,9ee80400,aaf2b000,4000,...) at zio_create+0x2d zio_write(8d416000,84607000,6,2,1,...) at zio_write+0x77 arc_write(8d416000,84607000,6,2,1,...) at arc_write+0x105 dbuf_sync(aafb2a50,8d416000,87109000,0,8ec48bf8,...) at dbuf_sync+0x3ca dnode_sync(906d68f0,0,8d416000,87109000,8d416000,...) at dnode_sync+0x51b dmu_objset_sync_dnodes(87109000,400,8ec48b11,806b5ba7,ae,...) at dmu_objset_sync_dnodes+0x97 dmu_objset_sync(85abc000,87109000,25b9,0,f9fe3bac,...) at dmu_objset_sync+0x60 dsl_dataset_sync(85aba000,87109000,0,84f8e1cc,84f8e184,...) at dsl_dataset_sync+0x24 dsl_pool_sync(84f8e000,25b9,0,1,0,...) at dsl_pool_sync+0x8f spa_sync(84607000,25b9,0,84f8e15c,84f8e154,...) at spa_sync+0x453 txg_sync_thread(84f8e000,f9fe3d38,806b3a8a,328,8ecd8d80,...) at txg_sync_thread+0x1d7 fork_exit(8ec0fd20,84f8e000,f9fe3d38) at fork_exit+0xd1 fork_trampoline() at fork_trampoline+0x8 --- trap 0x1, eip = 0, esp = 0xf9fe3d6c, ebp = 0 --- db> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - > I'll try to reproduce iozone panic, I'd also like to reproduce "Memory > modified after free" and "mi_switch: switch in a critical section" > panics, can you tell me how to do it? All cases are using iozone. 1. zpool create tank da0 2. cd /tank; iozone -a |& tee iozone.1.log 3. (panic) get backtrace and reset 4. cd /tank; iozone -a |& tee iozone.2.log 5. (panic) get backtrace and reset 6. cd /tank; ...... : And I got many type of panic messages. So I don't know how to reproduce these behaviors.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20061224232855.b0786250.nork>