Date: Wed, 18 Jul 2018 22:33:13 +0200 From: Alexander Leidinger <Alexander@leidinger.net> To: Marius Strobl <marius@freebsd.org> Cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r336313 - in head/sys: dev/bnxt dev/e1000 dev/ixgbe dev/ixl net sys Message-ID: <20180718223313.Horde.lYE8PRYqLdkrN3QMTTHx3aV@webmail.leidinger.net> In-Reply-To: <201807151904.w6FJ4NNg039896@repo.freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
[-- Attachment #1 --] Quoting Marius Strobl <marius@freebsd.org> (from Sun, 15 Jul 2018 19:04:23 +0000 (UTC)): > Author: marius > Date: Sun Jul 15 19:04:23 2018 > New Revision: 336313 > URL: https://svnweb.freebsd.org/changeset/base/336313 > > Log: > Assorted TSO fixes for em(4)/iflib(9) and dead code removal: [...] > Okayed by: sbruno@ at 201806 DevSummit Transport Working Group [1] > Reviewed by: sbruno (earlier version), erj > PR: 219428 (part of; comment #10) [1], 220997 (part of; comment #3) Hi Marius, thanks a lot for this change, it improves the situation (PR 220997) a lot. The system is running at r336329, as such I don't have your change r336356 yet on the system. Maybe the 2 panics (more below) I've seen are fixed by this. Before I try your second change (surely not before the WE), here at least the report in case it is related to your changes and not related to r336313: I got 2 panics, both within 6 minutes (based upon the timestamp of the coredumps in the filesystem): 1) panic: Assertion ifsd_m[next] == NULL failed at /usr/src/sys/net/iflib.c:3151 cpuid = 2 time = 1531944124 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe008af85850 vpanic() at vpanic+0x1a3/frame 0xfffffe008af858b0 doadump() at doadump/frame 0xfffffe008af85930 iflib_txq_drain() at iflib_txq_drain+0xe58/frame 0xfffffe008af85aa0 ifmp_ring_check_drainage() at ifmp_ring_check_drainage+0x16c/frame 0xfffffe008af85b00 _task_fn_tx() at _task_fn_tx+0x76/frame 0xfffffe008af85b30 gtaskqueue_run_locked() at gtaskqueue_run_locked+0x139/frame 0xfffffe008af85b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x88/frame 0xfffffe008af85bb0 fork_exit() at fork_exit+0x84/frame 0xfffffe008af85bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe008af85bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Uptime: 1d22h51m17s Dumping 2990 out of 8037 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% __curthread () at ./machine/pcpu.h:230 230 __asm("movq %%gs:%1,%0" : "=r" (td) (kgdb) #0 __curthread () at ./machine/pcpu.h:230 #1 doadump (textdump=1) at /usr/src/sys/kern/kern_shutdown.c:366 #2 0xffffffff80485ea1 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:446 #3 0xffffffff80486483 in vpanic (fmt=<optimized out>, ap=0xfffffe008af858f0) at /usr/src/sys/kern/kern_shutdown.c:863 #4 0xffffffff804861f0 in kassert_panic ( fmt=0xffffffff807e085f "Assertion %s failed at %s:%d") at /usr/src/sys/kern/kern_shutdown.c:749 #5 0xffffffff8059cd78 in iflib_busdma_load_mbuf_sg (flags=0, txq=<optimized out>, tag=<optimized out>, map=<optimized out>, m0=<optimized out>, segs=<optimized out>, nsegs=<optimized out>, max_segs=<optimized out>) at /usr/src/sys/net/iflib.c:3151 #6 iflib_encap (txq=0xfffff800028dc000, m_headp=0xfffffe00959bdd30) at /usr/src/sys/net/iflib.c:3321 #7 iflib_txq_drain (r=0xfffffe00959ba000, cidx=<optimized out>, pidx=41319936) at /usr/src/sys/net/iflib.c:3636 #8 0xffffffff805a0f4c in drain_ring_lockless (r=<optimized out>, os=..., prev=<optimized out>, budget=<optimized out>) at /usr/src/sys/net/mp_ring.c:199 #9 ifmp_ring_check_drainage (r=<optimized out>, budget=32) at /usr/src/sys/net/mp_ring.c:502 #10 0xffffffff80599c46 in _task_fn_tx (context=<optimized out>) at /usr/src/sys/net/iflib.c:3747 #11 0xffffffff804cd2c9 in gtaskqueue_run_locked (queue=0xfffff800025e0d00) at /usr/src/sys/kern/subr_gtaskqueue.c:332 #12 0xffffffff804cd048 in gtaskqueue_thread_loop (arg=<optimized out>) at /usr/src/sys/kern/subr_gtaskqueue.c:507 #13 0xffffffff8044cc34 in fork_exit ( callout=0xffffffff804ccfc0 <gtaskqueue_thread_loop>, arg=0xfffffe0007ffd038, frame=0xfffffe008af85c00) at /usr/src/sys/kern/kern_fork.c:1057 (kgdb) up 5 #5 0xffffffff8059cd78 in iflib_busdma_load_mbuf_sg (flags=0, txq=<optimized out>, tag=<optimized out>, map=<optimized out>, m0=<optimized out>, segs=<optimized out>, nsegs=<optimized out>, max_segs=<optimized out>) at /usr/src/sys/net/iflib.c:3151 3151 MPASS(ifsd_m[next] == NULL); (kgdb) list 3146 /* 3147 * see if we can't be smarter about physically 3148 * contiguous mappings 3149 */ 3150 next = (pidx + count) & (ntxd-1); 3151 MPASS(ifsd_m[next] == NULL); 3152 #if MEMORY_LOGGING 3153 txq->ift_enqueued++; 3154 #endif 3155 ifsd_m[next] = m; (kgdb) print ifsd_m $1 = (struct mbuf **) 0xfffffe00959b8000 (kgdb) print next $2 = <optimized out> (kgdb) print pidx $3 = 277 (kgdb) print count $4 = 0 (kgdb) print ntxd $5 = <optimized out> 2) Unread portion of the kernel message buffer: panic: Assertion ifsd_m[next] == NULL failed at /usr/src/sys/net/iflib.c:3151 cpuid = 2 time = 1531944550 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe008af85850 vpanic() at vpanic+0x1a3/frame 0xfffffe008af858b0 doadump() at doadump/frame 0xfffffe008af85930 iflib_txq_drain() at iflib_txq_drain+0xe58/frame 0xfffffe008af85aa0 ifmp_ring_check_drainage() at ifmp_ring_check_drainage+0x16c/frame 0xfffffe008af85b00 _task_fn_tx() at _task_fn_tx+0x76/frame 0xfffffe008af85b30 gtaskqueue_run_locked() at gtaskqueue_run_locked+0x139/frame 0xfffffe008af85b80 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x88/frame 0xfffffe008af85bb0 fork_exit() at fork_exit+0x84/frame 0xfffffe008af85bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe008af85bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Uptime: 5m27s Dumping 1555 out of 8037 MB:..2%..11%..21%..31%..41%..51%..61%..71%..81%..91% __curthread () at ./machine/pcpu.h:230 230 __asm("movq %%gs:%1,%0" : "=r" (td) (kgdb) bt #0 __curthread () at ./machine/pcpu.h:230 #1 doadump (textdump=1) at /usr/src/sys/kern/kern_shutdown.c:366 #2 0xffffffff80485ea1 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:446 #3 0xffffffff80486483 in vpanic (fmt=<optimized out>, ap=0xfffffe008af858f0) at /usr/src/sys/kern/kern_shutdown.c:863 #4 0xffffffff804861f0 in kassert_panic (fmt=0xffffffff807e085f "Assertion %s failed at %s:%d") at /usr/src/sys/kern/kern_shutdown.c:749 #5 0xffffffff8059cd78 in iflib_busdma_load_mbuf_sg (flags=0, txq=<optimized out>, tag=<optimized out>, map=<optimized out>, m0=<optimized out>, segs=<optimized out>, nsegs=<optimized out>, max_segs=<optimized out>) at /usr/src/sys/net/iflib.c:3151 #6 iflib_encap (txq=0xfffff800028fe000, m_headp=0xfffffe00959bdde8) at /usr/src/sys/net/iflib.c:3321 #7 iflib_txq_drain (r=0xfffffe00959ba000, cidx=<optimized out>, pidx=42948608) at /usr/src/sys/net/iflib.c:3636 #8 0xffffffff805a0f4c in drain_ring_lockless (r=<optimized out>, os=..., prev=<optimized out>, budget=<optimized out>) at /usr/src/sys/net/mp_ring.c:199 #9 ifmp_ring_check_drainage (r=<optimized out>, budget=32) at /usr/src/sys/net/mp_ring.c:502 #10 0xffffffff80599c46 in _task_fn_tx (context=<optimized out>) at /usr/src/sys/net/iflib.c:3747 #11 0xffffffff804cd2c9 in gtaskqueue_run_locked (queue=0xfffff800025a2200) at /usr/src/sys/kern/subr_gtaskqueue.c:332 #12 0xffffffff804cd048 in gtaskqueue_thread_loop (arg=<optimized out>) at /usr/src/sys/kern/subr_gtaskqueue.c:507 #13 0xffffffff8044cc34 in fork_exit (callout=0xffffffff804ccfc0 <gtaskqueue_thread_loop>, arg=0xfffffe0007ffd038, frame=0xfffffe008af85c00) at /usr/src/sys/kern/kern_fork.c:1057 #14 <signal handler called> (kgdb) up 5 #5 0xffffffff8059cd78 in iflib_busdma_load_mbuf_sg (flags=0, txq=<optimized out>, tag=<optimized out>, map=<optimized out>, m0=<optimized out>, segs=<optimized out>, nsegs=<optimized out>, max_segs=<optimized out>) at /usr/src/sys/net/iflib.c:3151 3151 MPASS(ifsd_m[next] == NULL); (kgdb) print ifsd_m $1 = (struct mbuf **) 0xfffffe00959b8000 (kgdb) print pidx $2 = 707 (kgdb) print count $3 = 0 Bye, Alexander. -- http://www.Leidinger.net Alexander@Leidinger.net: PGP 0x8F31830F9F2772BF http://www.FreeBSD.org netchild@FreeBSD.org : PGP 0x8F31830F9F2772BF [-- Attachment #2 --] -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAABAgAGBQJbT6QJAAoJEKrxQhqFIICEL0sP/21ishTejRzesyfAKXAfCykd yCl3edMsWb+62aLIicM21qfLqYvSQ13N4PyuuGknTAkRYUuXgHp/qeXP/2I9amh1 MxEgVK+OZPvcEs9awHyQzMVXIrPpAA9TIpLVt+UkPVvrwz9wKTMHTd7UOd16ADhF SY2nVdoL8rhp9czTod5atQXL2RtHsnn3VgOpg63a7GUH9jkZfmlWB8DaqAwYaL8n skUrt5yTENhXSqSbXV6XWnnBMJOvlQ1Xe2/QTCq3mb4tU9qEPSM4vM2qR7P/wsld 7rsRciwlJ3PrDeLwtjzTVed2IuqLmDRDW7HJRqQlDrgs7xGU0cK0ocUqlhC8dpWS Vq5SG2xzDCRb2S1nL5JOVSncuPsbs8TGubqQ+QTG3aUP/mutTa9zjBTHnrGb+PcV WdHrbkOGiTkev7lpRf8tsf6Rzmwi/G63cXuNFGyMG9TlV/L5qMkmR8PYGo8bscxZ BqzCKxZ2lBrdX/YrLHoaHGEGV2LzwhNrdXXS1hUDUcMBfzWbebENTw+0zdJ2uMyK a21wIt8K2WDg/IGc2KEu9GDFo1dMkfcGqGFaRlRUdgPOeJil/Hywy9PM5Jztgepw oWQhUr5VUjeozcstuEdaCyy7crrdEdTv8LcvhY0vUsvU6EHdmkp6BzcwQJb+/7U4 DqYfsRHetkoZqy9Yhlww =SQsB -----END PGP SIGNATURE-----
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20180718223313.Horde.lYE8PRYqLdkrN3QMTTHx3aV>