From owner-freebsd-current@FreeBSD.ORG Sat Apr 23 09:50:33 2005 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id AB3D216A4CE for ; Sat, 23 Apr 2005 09:50:33 +0000 (GMT) Received: from cs1.cs.huji.ac.il (cs1.cs.huji.ac.il [132.65.16.10]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2822543D5F for ; Sat, 23 Apr 2005 09:50:33 +0000 (GMT) (envelope-from danny@cs.huji.ac.il) Received: from pampa.cs.huji.ac.il ([132.65.80.32]) by cs1.cs.huji.ac.il with esmtp id 1DPHHS-000GdD-J4; Sat, 23 Apr 2005 12:50:30 +0300 X-Mailer: exmh version 2.7.0 06/18/2004 with nmh-1.0.4 To: Doug White In-Reply-To: Message from Doug White <20050422200953.N10333@carver.gumbysoft.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Sat, 23 Apr 2005 12:50:30 +0300 From: Danny Braniss Message-ID: cc: current@freebsd.org Subject: Re: diskless/unionfs panics X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 23 Apr 2005 09:50:33 -0000 > On Fri, 22 Apr 2005, Danny Braniss wrote: > > > hi, > > after much debugging, it seems that the main problem with unionfs is > > that if it's called early in the boot process it will panic the kernel: > > > > trap 12: page fault while in kernel mode > > cpuid = 0; apic id = 00 > > fault virtual address = 0x0 > > fault code = supervisor read, page not present > > instruction pointer = 0x8:0xffffffff8038e3f5 > > stack pointer = 0x10:0xffffffffb1eac7b0 > > frame pointer = 0x10:0xffffffffb1eac7e0 > > code segment = base 0x0, limit 0xfffff, type 0x1b > > = DPL 0, pres 1, long 1, def32 0, gran 1 > > processor eflags = interrupt enabled, resume, IOPL = 0 > > current process = 213 (sh) > > [thread pid 213 tid 100066 ] > > Stopped at _mtx_lock_flags+0x35: cmpq $0x80779d40,0(%rdi) > > unintialized mutex, probably, although it looks like it'd be the vm page > queue mutex which should be init'd by then. > > Is this -CURRENT? yes, cvs'ed a few days ago (but the problem is not new). > > > db> tr > > Tracing pid 213 tid 100066 td 0xffffff007b9b1000 > > _mtx_lock_flags() at _mtx_lock_flags+0x35 > > exec_map_first_page() at exec_map_first_page+0x60 > > If you have a debug kernel for this around, load it into gdb and 'disass > exec_map_first_page' and look around offset 96 to see if its referencing a > mutex (mtx) near there. arghh, gdb, is there a quick guide for this? im almost there, but can't sync speed (the console is at 38400). > > > kern_execve() at kern_execve+0x2a0 > > execve() at execve+0x5d > > syscall() at syscall+0x4ab > > Xfast_syscall() at Xfast_syscall+0xa8 > > --- syscall (59, FreeBSD ELF64, execve), rip = 0x80090630c, rsp = > > 0x7fffffffcbf8, rbp = 0 --- > > > > doing the unionfs stuff sometime later - after the single user prompt - seems > > do be ok. > > > > another thing, there are some LOR caused by the unionfs & md: > > extarct of rc.d/initdiskless: > > > > if [ -e /conf/union ]; then > > if ! sysctl vfs.uniondebug >/dev/null 2>&1; then > > echo Loading unionfs > > kldload unionfs > > fi > > echo Doing UNIONFS > > mount_md 4096 /conf/etc > > chmod 755 /conf/etc > > mount_unionfs /conf/etc /etc > > ls -R /etc > /dev/null > > touch /etc/.sentinel > > md_created_etc=created > > fi > > ... > > > > Loading unionfs > > lock order reversal > > 1st 0xffffff007c3b00f0 system map (system map) @ /r+d/6.0/src/sys/vm/vm_map.c: > > 1100 > > 2nd 0xffffff00623d8620 vm object (standard object) @ > > /r+d/6.0/src/sys/vm/vm_map.c:897 > > KDB: stack backtrace: > > witness_checkorder() at witness_checkorder+0x5f1 > > _mtx_lock_flags() at _mtx_lock_flags+0x4a > > vm_map_insert() at vm_map_insert+0x115 > > vm_map_find() at vm_map_find+0x9d > > link_elf_load_file() at link_elf_load_file+0x811 > > linker_load_module() at linker_load_module+0x50e > > kldload() at kldload+0xc3 > > syscall() at syscall+0x4ab > > Xfast_syscall() at Xfast_syscall+0xa8 > > --- syscall (304, FreeBSD ELF64, kldload), rip = 0x80067950c, rsp = > > 0x7fffffffedf8, rbp = 0x7fffffffee68 --- > > > > Doing UNIONFS > > lock order reversal > > 1st 0xffffffff80892040 vm page queue mutex (vm page queue mutex) @ > > /r+d/6.0/src/sys/kern/vfs_bio.c:1485 > > 2nd 0xffffff0061dc98b0 vnode interlock (vnode interlock) @ > > /r+d/6.0/src/sys/kern/vfs_subr.c:1992 > > KDB: stack backtrace: > > witness_checkorder() at witness_checkorder+0x5f1 > > _mtx_lock_flags() at _mtx_lock_flags+0x4a > > vdrop() at vdrop+0x31 > > vm_page_remove() at vm_page_remove+0x126 > > vm_page_free_toq() at vm_page_free_toq+0x61 > > vm_page_free() at vm_page_free+0x1e > > vfs_vmio_release() at vfs_vmio_release+0x18f > > brelse() at brelse+0x792 > > ffs_mount() at ffs_mount+0x121b > > vfs_donmount() at vfs_donmount+0xaa3 > > kernel_mount() at kernel_mount+0xaf > > ffs_cmount() at ffs_cmount+0x92 > > mount() at mount+0x132 > > syscall() at syscall+0x1fb > > Xfast_syscall() at Xfast_syscall+0xa8 > > --- syscall (21, FreeBSD ELF64, mount), rip = 0x80067e68c, rsp = > > 0x7fffffffdf98, rbp = 0x7fffffffea58 --- > > > > > > _______________________________________________ > > freebsd-current@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-current > > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" > > > > -- > Doug White | FreeBSD: The Power to Serve > dwhite@gumbysoft.com | www.FreeBSD.org