From owner-freebsd-amd64@FreeBSD.ORG Tue Feb 13 18:53:13 2007 Return-Path: X-Original-To: amd64@FreeBSD.org Delivered-To: freebsd-amd64@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 4D8DB16A401; Tue, 13 Feb 2007 18:53:13 +0000 (UTC) (envelope-from kris@obsecurity.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 44DF413C4AA; Tue, 13 Feb 2007 18:53:13 +0000 (UTC) (envelope-from kris@obsecurity.org) Received: from obsecurity.dyndns.org (elvis.mu.org [192.203.228.196]) by elvis.mu.org (Postfix) with ESMTP id F171D1A4D8B; Tue, 13 Feb 2007 10:53:12 -0800 (PST) Received: by obsecurity.dyndns.org (Postfix, from userid 1000) id 6DBA353588; Tue, 13 Feb 2007 13:53:12 -0500 (EST) Date: Tue, 13 Feb 2007 13:53:12 -0500 From: Kris Kennaway To: current@FreeBSD.org Message-ID: <20070213185312.GF67616@xor.obsecurity.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="NtwzykIc2mflq5ck" Content-Disposition: inline User-Agent: Mutt/1.4.2.2i Cc: amd64@FreeBSD.org Subject: Page fault in amd64 pmap_qremove from vm_thread_new() X-BeenThere: freebsd-amd64@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Porting FreeBSD to the AMD64 platform List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Feb 2007 18:53:13 -0000 --NtwzykIc2mflq5ck Content-Type: text/plain; charset=us-ascii Content-Disposition: inline I get this frequently when running stress2 on an 8-core amd64 system: Fatal trap 12: page fault while in kernel mode Fatal trap 12: page fault while in kernel mode cpuid = 2; apic id = 02 Fatal trap 12: page fault while in kernel mode cpuid = 5; fault virtual address = 0xffff807ffffff040 Fatal trap 12: page fault while in kernel mode Fatal trap 12: page fault while in kernel mode cpuid = 4; apic id = 05 apic id = 04 fault virtual address = 0xffff807ffffff0e0 fault virtual address = 0xffff807ffffff0b8 cpuid = 0; fault code = supervisor write data, page not present instruction pointer = 0x8:0xffffffff803deedd cpuid = 3; stack pointer = 0x10:0xffffffffc7647720 fault code = supervisor write data, page not present instruction pointer = 0x8:0xffffffff803deedd apic id = 00 stack pointer = 0x10:0xffffffffcfd7e720 fault code = supervisor write data, page not present frame pointer = 0x10:0xffffffffc7647730 frame pointer = 0x10:0xffffffffcfd7e730 Fatal trap 12: page fault while in kernel mode cpuid = 6; instruction pointer = 0x8:0xffffffff803deedd stack pointer = 0x10:0xffffffffb2b93720 frame pointer = 0x10:0xffffffffb2b93730 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, Fatal trap 12: page fault while in kernel mode apic id = 06 cpuid = 7; fault virtual address = 0xffff807ffffff108 apic id = 07 fault code = supervisor write data, page not present code segment = base 0x0, limit 0xfffff, type 0x1b apic id = 03 = DPL 0, pres 1, long 1, def32 0, gran 1 fault virtual address = 0xffff807ffffff068 IOPL = 0 fault code = supervisor write data, page not present fault virtual address = 0xffff807ffffff018 instruction pointer = 0x8:0xffffffff803deedd instruction pointer = 0x8:0xffffffff803deedd Fatal trap 12: page fault while in kernel mode stack pointer = 0x10:0xffffffffbf901720 cpuid = 4; stack pointer = 0x10:0xffffffffb1c11720 processor eflags = frame pointer = 0x10:0xffffffffb1c11730 interrupt enabled, resume, fault code = supervisor write data, page not present IOPL = 0 instruction pointer = 0x8:0xffffffff803deedd current process = stack pointer = 0x10:0xffffffffd5b25720 frame pointer = 0x10:0xffffffffbf901730 frame pointer = 0x10:0xffffffffd5b25730 code segment = base 0x0, limit 0xfffff, type 0x1b current process = = DPL 0, pres 1, long 1, def32 0, gran 1 code segment = base 0x0, limit 0xfffff, type 0x1b code segment = base 0x0, limit 0xfffff, type 0x1b 18747 (thr2) [thread pid 18747 tid 142909 ] Stopped at pmap_qremove+0x2d: movq $0,(%rcx,%rax,8) db> wh Tracing pid 18747 tid 142909 td 0xffffff0095710cd0 pmap_qremove() at pmap_qremove+0x2d vm_thread_new() at vm_thread_new+0x8d thread_init() at thread_init+0x16 slab_zalloc() at slab_zalloc+0x282 uma_zone_slab() at uma_zone_slab+0x1ae uma_zalloc_bucket() at uma_zalloc_bucket+0x19d uma_zalloc_arg() at uma_zalloc_arg+0x3a3 thread_alloc() at thread_alloc+0x1f create_thread() at create_thread+0xc5 kern_thr_new() at kern_thr_new+0x75 thr_new() at thr_new+0x62 syscall() at syscall+0x310 Xfast_syscall() at Xfast_syscall+0xab --- syscall (455, FreeBSD ELF64, thr_new), rip = 0x8007a1cac, rsp = 0x7fffffffdef8, rbp = 0 --- db> show allpcpu Current CPU: 2 cpuid = 0 curthread = 0xffffff00717e8290: pid 18944 "thr2" curpcb = 0xffffffffe2e33d50 fpcurthread = none idlethread = 0xffffff00b9aa6520: pid 17 "idle: cpu0" spin locks held: cpuid = 1 curthread = 0xffffff0015e9d7b0: pid 18736 "thr2" curpcb = 0xffffffffbceefd50 fpcurthread = none idlethread = 0xffffff00b9aa6290: pid 16 "idle: cpu1" spin locks held: exclusive spin mutex sio r = 0 (0xffffffff806bf3c0) locked @ dev/sio/sio.c:1390 cpuid = 2 curthread = 0xffffff0095710cd0: pid 18747 "thr2" curpcb = 0xffffffffcfd7ed50 fpcurthread = none idlethread = 0xffffff00b9aa6000: pid 15 "idle: cpu2" spin locks held: cpuid = 3 curthread = 0xffffff00ad485290: pid 18743 "thr2" curpcb = 0xffffffffd5b25d50 fpcurthread = none idlethread = 0xffffff00b9a63cd0: pid 14 "idle: cpu3" spin locks held: cpuid = 4 curthread = 0xffffff0098fc7000: pid 18942 "thr2" curpcb = 0xffffffffc77fad50 fpcurthread = none idlethread = 0xffffff00b9a63000: pid 13 "idle: cpu4" spin locks held: exclusive spin mutex turnstile chain r = 0 (0xffffffff80613ed8) locked @ kern/subr_turnstile.c:489 cpuid = 5 curthread = 0xffffff00215b8cd0: pid 18708 "thr2" curpcb = 0xffffffffb2b93d50 fpcurthread = none idlethread = 0xffffff00b9a8fcd0: pid 12 "idle: cpu5" spin locks held: cpuid = 6 curthread = 0xffffff005b72d520: pid 18718 "thr2" curpcb = 0xffffffffb1c11d50 fpcurthread = none idlethread = 0xffffff00b9a8fa40: pid 11 "idle: cpu6" spin locks held: cpuid = 7 curthread = 0xffffff0078aae7b0: pid 18782 "thr2" curpcb = 0xffffffffbf901d50 fpcurthread = none idlethread = 0xffffff00b9a8f7b0: pid 10 "idle: cpu7" spin locks held: For some reason ddb doesn't give sensible backtraces for the running threads: db> wh 18944 Tracing pid 18944 tid 130433 td 0xffffff009daa7290 fork_trampoline() at fork_trampoline db> wh 18736 Tracing pid 18736 tid 165977 td 0xffffff00632b2cd0 fork_trampoline() at fork_trampoline db> wh 18747 Tracing pid 18747 tid 165890 td 0xffffff0037403000 fork_trampoline() at fork_trampoline db> wh 18743 Tracing pid 18743 tid 165929 td 0xffffff004f59e000 fork_trampoline() at fork_trampoline db> wh 18942 Tracing pid 18942 tid 130531 td 0xffffff000a166520 fork_trampoline() at fork_trampoline db> wh 18708 Tracing pid 18708 tid 166269 td 0xffffff005c28a290 fork_trampoline() at fork_trampoline db> wh 18718 Tracing pid 18718 tid 111088 td 0xffffff0081f51a40 fork_trampoline() at fork_trampoline db> wh 18782 Tracing pid 18782 tid 166078 td 0xffffff0052b4c000 fork_trampoline() at fork_trampoline Kris --NtwzykIc2mflq5ck Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (FreeBSD) iD8DBQFF0gkXWry0BWjoQKURApFCAJ9Kvsu4QF5QvGjxgq7dWk/N1M0VNwCg3QS7 mpBhzPM1bASlmup733EvX74= =CwwH -----END PGP SIGNATURE----- --NtwzykIc2mflq5ck--