From owner-freebsd-stable@freebsd.org Sun Mar 3 11:33:38 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6E04F150722D for ; Sun, 3 Mar 2019 11:33:38 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 48E55812F7 for ; Sun, 3 Mar 2019 11:33:37 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1h0PN2-0001pk-Px for freebsd-stable@freebsd.org; Sun, 03 Mar 2019 14:33:28 +0300 Date: Sun, 3 Mar 2019 14:33:28 +0300 From: Slawa Olhovchenkov To: freebsd-stable@freebsd.org Subject: Re: FreeBSD-11: Fatal trap 9: general protection fault while in kernel mode (in key_addref()) Message-ID: <20190303113328.GD2178@zxy.spb.ru> References: <20190227205420.GC2178@zxy.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20190227205420.GC2178@zxy.spb.ru> User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-Rspamd-Queue-Id: 48E55812F7 X-Spamd-Bar: +++ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [3.71 / 15.00]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_SPAM_SHORT(0.88)[0.878,0]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; DMARC_NA(0.00)[zxy.spb.ru]; AUTH_NA(1.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_SPAM_MEDIUM(0.98)[0.976,0]; RCVD_TLS_LAST(0.00)[]; MX_GOOD(-0.01)[zxy.spb.ru]; NEURAL_SPAM_LONG(0.97)[0.966,0]; R_SPF_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:5495, ipnet:195.70.192.0/19, country:RU]; MID_RHS_MATCH_FROM(0.00)[]; IP_SCORE(0.00)[country: RU(0.00)]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Mar 2019 11:33:38 -0000 On Wed, Feb 27, 2019 at 11:54:20PM +0300, Slawa Olhovchenkov wrote: > Is this known issuse? > > Fatal trap 9: general protection fault while in kernel mode > cpuid = 13; apic id = 2a > instruction pointer = 0x20:0xffffffff806b6a94 > stack pointer = 0x28:0xfffffe2026e274f0 > frame pointer = 0x28:0xfffffe2026e274f0 > code segment = base 0x0, limit 0xfffff, type 0x1b > = DPL 0, pres 1, long 1, def32 0, gran 1 > processor eflags = interrupt enabled, resume, IOPL = 0 > current process = 12 (irq295: t5nex0:0a5) > trap number = 9 > panic: general protection fault > cpuid = 13 > KDB: stack backtrace: > db_trace_self_wrapper() at 0xffffffff8032667b = db_trace_self_wrapper+0x2b/frame 0xfffffe2026e27130 > vpanic() at 0xffffffff804c2006 = vpanic+0x186/frame 0xfffffe2026e271b0 > panic() at 0xffffffff804c1e73 = panic+0x43/frame 0xfffffe2026e27210 > trap_fatal() at 0xffffffff807503f2 = trap_fatal+0x322/frame 0xfffffe2026e27260 > trap() at 0xffffffff8074fa5e = trap+0x5e/frame 0xfffffe2026e27420 > calltrap() at 0xffffffff80735771 = calltrap+0x8/frame 0xfffffe2026e27420 > --- trap 0x9, rip = 0xffffffff806b6a94, rsp = 0xfffffe2026e274f0, rbp = 0xfffffe2026e274f0 --- > key_addref() at 0xffffffff806b6a94 = key_addref+0x4/frame 0xfffffe2026e274f0 > ipsec_getpcbpolicy() at 0xffffffff806b20b9 = ipsec_getpcbpolicy+0x49/frame 0xfffffe2026e27530 > ipsec4_getpolicy() at 0xffffffff806b10a5 = ipsec4_getpolicy+0x25/frame 0xfffffe2026e275d0 > ipsec4_in_reject() at 0xffffffff806b138b = ipsec4_in_reject+0x1b/frame 0xfffffe2026e27600 > tcp_input() at 0xffffffff8066127c = tcp_input+0x97c/frame 0xfffffe2026e27740 > ip_input() at 0xffffffff805e447f = ip_input+0x10f/frame 0xfffffe2026e277a0 > netisr_dispatch_src() at 0xffffffff805c4750 = netisr_dispatch_src+0xa0/frame 0xfffffe2026e277f0 > ether_demux() at 0xffffffff805b43ff = ether_demux+0x13f/frame 0xfffffe2026e27820 > ether_nh_input() at 0xffffffff805b506b = ether_nh_input+0x31b/frame 0xfffffe2026e27880 > netisr_dispatch_src() at 0xffffffff805c4750 = netisr_dispatch_src+0xa0/frame 0xfffffe2026e278d0 > ether_input() at 0xffffffff805b4676 = ether_input+0x26/frame 0xfffffe2026e278f0 > t4_eth_rx() at 0xffffffff816403b3 = t4_eth_rx+0x103/frame 0xfffffe2026e27910 > service_iq() at 0xffffffff81644886 = service_iq+0x4a6/frame 0xfffffe2026e279c0 > t4_intr() at 0xffffffff81644b3e = t4_intr+0x2e/frame 0xfffffe2026e279e0 > intr_event_execute_handlers() at 0xffffffff804871ac = intr_event_execute_handlers+0xec/frame 0xfffffe2026e27a20 > ithread_loop() at 0xffffffff80487846 = ithread_loop+0xd6/frame 0xfffffe2026e27a70 > fork_exit() at 0xffffffff80484805 = fork_exit+0x85/frame 0xfffffe2026e27ab0 > fork_trampoline() at 0xffffffff80735cae = fork_trampoline+0xe/frame 0xfffffe2026e27ab0 > --- trap 0, rip = 0, rsp = 0, rbp = 0 --- > Uptime: 657d14h33m52s kgdb decode: Unread portion of the kernel message buffer: Fatal trap 9: general protection fault while in kernel mode cpuid = 13; apic id = 2a instruction pointer = 0x20:0xffffffff806b6a94 stack pointer = 0x28:0xfffffe2026e274f0 frame pointer = 0x28:0xfffffe2026e274f0 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 12 (irq295: t5nex0:0a5) trap number = 9 panic: general protection fault cpuid = 13 KDB: stack backtrace: db_trace_self_wrapper() at 0xffffffff8032667b = db_trace_self_wrapper+0x2b/frame 0xfffffe2026e27130 vpanic() at 0xffffffff804c2006 = vpanic+0x186/frame 0xfffffe2026e271b0 panic() at 0xffffffff804c1e73 = panic+0x43/frame 0xfffffe2026e27210 trap_fatal() at 0xffffffff807503f2 = trap_fatal+0x322/frame 0xfffffe2026e27260 trap() at 0xffffffff8074fa5e = trap+0x5e/frame 0xfffffe2026e27420 calltrap() at 0xffffffff80735771 = calltrap+0x8/frame 0xfffffe2026e27420 --- trap 0x9, rip = 0xffffffff806b6a94, rsp = 0xfffffe2026e274f0, rbp = 0xfffffe2026e274f0 --- key_addref() at 0xffffffff806b6a94 = key_addref+0x4/frame 0xfffffe2026e274f0 ipsec_getpcbpolicy() at 0xffffffff806b20b9 = ipsec_getpcbpolicy+0x49/frame 0xfffffe2026e27530 ipsec4_getpolicy() at 0xffffffff806b10a5 = ipsec4_getpolicy+0x25/frame 0xfffffe2026e275d0 ipsec4_in_reject() at 0xffffffff806b138b = ipsec4_in_reject+0x1b/frame 0xfffffe2026e27600 tcp_input() at 0xffffffff8066127c = tcp_input+0x97c/frame 0xfffffe2026e27740 ip_input() at 0xffffffff805e447f = ip_input+0x10f/frame 0xfffffe2026e277a0 netisr_dispatch_src() at 0xffffffff805c4750 = netisr_dispatch_src+0xa0/frame 0xfffffe2026e277f0 ether_demux() at 0xffffffff805b43ff = ether_demux+0x13f/frame 0xfffffe2026e27820 ether_nh_input() at 0xffffffff805b506b = ether_nh_input+0x31b/frame 0xfffffe2026e27880 netisr_dispatch_src() at 0xffffffff805c4750 = netisr_dispatch_src+0xa0/frame 0xfffffe2026e278d0 ether_input() at 0xffffffff805b4676 = ether_input+0x26/frame 0xfffffe2026e278f0 t4_eth_rx() at 0xffffffff816403b3 = t4_eth_rx+0x103/frame 0xfffffe2026e27910 service_iq() at 0xffffffff81644886 = service_iq+0x4a6/frame 0xfffffe2026e279c0 t4_intr() at 0xffffffff81644b3e = t4_intr+0x2e/frame 0xfffffe2026e279e0 intr_event_execute_handlers() at 0xffffffff804871ac = intr_event_execute_handlers+0xec/frame 0xfffffe2026e27a20 ithread_loop() at 0xffffffff80487846 = ithread_loop+0xd6/frame 0xfffffe2026e27a70 fork_exit() at 0xffffffff80484805 = fork_exit+0x85/frame 0xfffffe2026e27ab0 fork_trampoline() at 0xffffffff80735cae = fork_trampoline+0xe/frame 0xfffffe2026e27ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Uptime: 657d14h33m52s (da1:mps0:0:23:0): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (da1:mps0:0:23:0): CAM status: Command timeout (da1:mps0:0:23:0): Error 5, Retries exhausted (da1:mps0:0:23:0): Synchronize cache failed (da2:mps0:0:18:0): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (da2:mps0:0:18:0): CAM status: Command timeout (da2:mps0:0:18:0): Error 5, Retries exhausted (da2:mps0:0:18:0): Synchronize cache failed (da3:mps0:0:20:0): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (da3:mps0:0:20:0): CAM status: Command timeout (da3:mps0:0:20:0): Error 5, Retries exhausted (da3:mps0:0:20:0): Synchronize cache failed (da4:mps0:0:19:0): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (da4:mps0:0:19:0): CAM status: Command timeout (da4:mps0:0:19:0): Error 5, Retries exhausted (da4:mps0:0:19:0): Synchronize cache failed (da5:mps0:0:16:0): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (da5:mps0:0:16:0): CAM status: Command timeout (da5:mps0:0:16:0): Error 5, Retries exhausted (da5:mps0:0:16:0): Synchronize cache failed (da6:mps0:0:21:0): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (da6:mps0:0:21:0): CAM status: Command timeout (da6:mps0:0:21:0): Error 5, Retries exhausted (da6:mps0:0:21:0): Synchronize cache failed Dumping 18704 out of 131018 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% __curthread () at ./machine/pcpu.h:222 222 __asm("movq %%gs:%1,%0" : "=r" (td) (kgdb) bt #0 __curthread () at ./machine/pcpu.h:222 #1 doadump (textdump=1) at /usr/src/sys/kern/kern_shutdown.c:298 #2 0xffffffff804c1b66 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:366 #3 0xffffffff804c2040 in vpanic (fmt=, ap=0xfffffe2026e271f0) at /usr/src/sys/kern/kern_shutdown.c:759 #4 0xffffffff804c1e73 in panic (fmt=) at /usr/src/sys/kern/kern_shutdown.c:690 #5 0xffffffff807503f2 in trap_fatal (frame=0xfffffe2026e27430, eva=0) at /usr/src/sys/amd64/amd64/trap.c:801 #6 0xffffffff8074fa5e in trap (frame=0xfffffe2026e27430) at /usr/src/sys/amd64/amd64/trap.c:197 #7 #8 atomic_add_barr_int (v=1, p=) at ./machine/atomic.h:460 #9 refcount_acquire (count=) at /usr/src/sys/sys/refcount.h:53 #10 key_addref (sp=0x601d473c0b4c70ce) at /usr/src/sys/netipsec/key.c:737 #11 0xffffffff806b20b9 in ipsec_getpcbpolicy (inp=0xfffff8157e311000, dir=) at /usr/src/sys/netipsec/ipsec.c:474 #12 0xffffffff806b10a5 in ipsec4_getpolicy (m=0xfffff80589ff9500, inp=0xfffff8157e311000, dir=1) at /usr/src/sys/netipsec/ipsec.c:571 #13 0xffffffff806b138b in ipsec4_in_reject (m=0xfffff80589ff9500, inp=0xfffff8157e311000) at /usr/src/sys/netipsec/ipsec.c:626 #14 0xffffffff8066127c in tcp_input (mp=, offp=, proto=) at /usr/src/sys/netinet/tcp_input.c:926 #15 0xffffffff805e447f in ip_input (m=0x0) at /usr/src/sys/netinet/ip_input.c:823 #16 0xffffffff805c4750 in netisr_dispatch_src (proto=1, source=, m=0xfffff816d1af42e0) at /usr/src/sys/net/netisr.c:1120 #17 0xffffffff805b43ff in ether_demux (ifp=0xfffff8012c13f000, m=0x1) at /usr/src/sys/net/if_ethersubr.c:850 #18 0xffffffff805b506b in ether_input_internal (ifp=0xfffff8012c13f000, m=0x1) at /usr/src/sys/net/if_ethersubr.c:639 #19 ether_nh_input (m=) at /usr/src/sys/net/if_ethersubr.c:669 #20 0xffffffff805c4750 in netisr_dispatch_src (proto=5, source=, m=0xfffff816d1af42e0) at /usr/src/sys/net/netisr.c:1120 #21 0xffffffff805b4676 in ether_input (ifp=, m=0x0) at /usr/src/sys/net/if_ethersubr.c:759 #22 0xffffffff816403b3 in t4_eth_rx (iq=, rss=, m0=0xfffff80589ff9500) at /usr/src/sys/modules/cxgbe/if_cxgbe/../../../dev/cxgbe/t4_sge.c:1833 #23 0xffffffff81644886 in service_iq (iq=, budget=0) at /usr/src/sys/modules/cxgbe/if_cxgbe/../../../dev/cxgbe/t4_sge.c:1431 #24 0xffffffff81644b3e in t4_intr (arg=0xfffffe001d474a00) at /usr/src/sys/modules/cxgbe/if_cxgbe/../../../dev/cxgbe/t4_sge.c:1332 #25 0xffffffff804871ac in intr_event_execute_handlers (p=, ie=0xfffff8012c382c00) at /usr/src/sys/kern/kern_intr.c:1262 #26 0xffffffff80487846 in ithread_execute_handlers (ie=, p=) at /usr/src/sys/kern/kern_intr.c:1275 #27 ithread_loop (arg=0xfffff8012c4573c0) at /usr/src/sys/kern/kern_intr.c:1356 #28 0xffffffff80484805 in fork_exit (callout=0xffffffff80487770 , arg=0xfffff8012c4573c0, frame=0xfffffe2026e27ac0) at /usr/src/sys/kern/kern_fork.c:1040 #29