From owner-freebsd-current@FreeBSD.ORG Tue Jul 21 14:33:56 2009 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6E39B106566B; Tue, 21 Jul 2009 14:33:56 +0000 (UTC) (envelope-from spambox@haruhiism.net) Received: from fujibayashi.jp (karas.fujibayashi.jp [77.221.159.4]) by mx1.freebsd.org (Postfix) with ESMTP id DD8908FC0A; Tue, 21 Jul 2009 14:33:55 +0000 (UTC) (envelope-from spambox@haruhiism.net) Received: from [192.168.0.10] (datacenter.telecombusinessconsulting.net [77.221.137.211]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by fujibayashi.jp (Postfix) with ESMTPSA id A182778F50; Tue, 21 Jul 2009 18:33:54 +0400 (MSD) Message-ID: <4A65D1CD.40006@haruhiism.net> Date: Tue, 21 Jul 2009 18:33:49 +0400 From: Kamigishi Rei User-Agent: Thunderbird 2.0.0.22 (Windows/20090605) MIME-Version: 1.0 To: John Baldwin References: <4A659F98.2060007@haruhiism.net> <200907210857.01690.jhb@freebsd.org> <4A65C9D1.6080902@haruhiism.net> <200907211027.06589.jhb@freebsd.org> In-Reply-To: <200907211027.06589.jhb@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Lawrence Stewart , freebsd-current@freebsd.org Subject: Re: [follow-up] Fatal trap 12 in r195146+ in netisr_queue_internal X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Jul 2009 14:33:56 -0000 John Baldwin wrote: > Can you print out 'owner' as well? You won't get a panic until you actually > dereference 'owner' to get 'owner->td_state' even though gdb will show this > as the faulting line (gdb can sometimes get confused by compiler > optimization). You are seeing these values because mtx_lock was changed (due > to either a mtx_unlock() or a mtx_init()) while you were spinning. That > value of v is not what I have typically seen in these panics. Do you also > have the original fatal kernel trap messages? > Why does v change to a non-kernel address then? From what I see, it should never get assigned a value that's not MTX_UNOWNED or 0xfff......(address,flags). However, I can reproduce this trap in all revisions starting with 195146 up to 195484 (and probably more, didn't check yet; at 1956xx these traps stop occurring). vmcore.51 (all cores starting with .9 are related to mtx_lock_sleep() trap): Unread portion of the kernel message buffer: kernel trap 12 with interrupts disabled Fatal trap 12: page fault while in kernel mode cpuid = 0; apic id = 00 fault virtual address = 0x14ee288 fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff80586255 stack pointer = 0x28:0xffffff80787115f0 frame pointer = 0x28:0xffffff8078711620 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = resume, IOPL = 0 current process = 2438 (iperf) trap number = 12 panic: page fault cpuid = 0 Uptime: 43s Physical memory: 4014 MB (kgdb) fr 6 #6 0xffffffff80586255 in _mtx_lock_sleep (m=0xffffffff80e60823, tid=18446742977255365296, opts=Variable "opts" is not available. ) at /usr/src/sys/kern/kern_mutex.c:407 407 owner = (struct thread *)(v & ~MTX_FLAGMASK); (kgdb) print owner $1 = (volatile struct thread *) 0x14ee000 (kgdb) print v $2 = 21946368 (kgdb) print m->mtx_lock $3 = 4 (kgdb) print owner->td_state Cannot access memory at address 0x14ee288 vmcore.50: Unread portion of the kernel message buffer: kernel trap 12 with interrupts disabled Fatal trap 12: page fault while in kernel mode cpuid = 0; apic id = 00 fault virtual address = 0x14ee288 fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff80586255 stack pointer = 0x28:0xffffff80785005f0 frame pointer = 0x28:0xffffff8078500620 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = resume, IOPL = 0 current process = 2448 (iperf) trap number = 12 panic: page fault cpuid = 0 Uptime: 53s Physical memory: 4014 MB (kgdb) fr 6 #6 0xffffffff80586255 in _mtx_lock_sleep (m=0xffffffff80e60823, tid=18446742974555039520, opts=Variable "opts" is not available. ) at /usr/src/sys/kern/kern_mutex.c:407 407 owner = (struct thread *)(v & ~MTX_FLAGMASK); (kgdb) print owner $1 = (volatile struct thread *) 0x14ee000 (kgdb) print m->mtx_lock $2 = 4 (kgdb) print v $3 = 21946368 (kgdb) print owner->td_state Cannot access memory at address 0x14ee288 -- Kamigishi Rei KREI-RIPE