Date: Wed, 18 Jul 2018 22:33:13 +0200 From: Alexander Leidinger <Alexander@leidinger.net> To: Marius Strobl <marius@freebsd.org> Cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r336313 - in head/sys: dev/bnxt dev/e1000 dev/ixgbe dev/ixl net sys Message-ID: <20180718223313.Horde.lYE8PRYqLdkrN3QMTTHx3aV@webmail.leidinger.net> In-Reply-To: <201807151904.w6FJ4NNg039896@repo.freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
This message is in MIME format and has been PGP signed. --=_squ2rs7hgvM55G-LHO-CB7r Content-Type: text/plain; charset=utf-8; format=flowed; DelSp=Yes Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Quoting Marius Strobl <marius@freebsd.org> (from Sun, 15 Jul 2018=20=20 19:04:23=20+0000 (UTC)): > Author: marius > Date: Sun Jul 15 19:04:23 2018 > New Revision: 336313 > URL: https://svnweb.freebsd.org/changeset/base/336313 > > Log: > Assorted TSO fixes for em(4)/iflib(9) and dead code removal: [...] > Okayed by: sbruno@ at 201806 DevSummit Transport Working Group [1] > Reviewed by: sbruno (earlier version), erj > PR: 219428 (part of; comment #10) [1], 220997 (part of; comment #3) Hi Marius, thanks a lot for this change, it improves the situation (PR 220997) a=20=20 lot.=20The system is running at r336329, as such I don't have your=20=20 change=20r336356 yet on the system. Maybe the 2 panics (more below) I've=20= =20 seen=20are fixed by this. Before I try your second change (surely not=20=20 before=20the WE), here at least the report in case it is related to your=20= =20 changes=20and not related to r336313: I got 2 panics, both within 6 minutes (based upon the timestamp of the=20= =20 coredumps=20in the filesystem): 1) panic: Assertion ifsd_m[next] =3D=3D NULL failed at /usr/src/sys/net/iflib.= c:3151 cpuid =3D 2 time =3D 1531944124 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe008af85= 850 vpanic() at vpanic+0x1a3/frame 0xfffffe008af858b0 doadump() at doadump/frame 0xfffffe008af85930 iflib_txq_drain() at iflib_txq_drain+0xe58/frame 0xfffffe008af85aa0 ifmp_ring_check_drainage() at ifmp_ring_check_drainage+0x16c/frame=20=20 0xfffffe008af85b00 _task_fn_tx()=20at _task_fn_tx+0x76/frame 0xfffffe008af85b30 gtaskqueue_run_locked() at gtaskqueue_run_locked+0x139/frame=20=20 0xfffffe008af85b80 gtaskqueue_thread_loop()=20at gtaskqueue_thread_loop+0x88/frame=20=20 0xfffffe008af85bb0 fork_exit()=20at fork_exit+0x84/frame 0xfffffe008af85bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe008af85bf0 --- trap 0, rip =3D 0, rsp =3D 0, rbp =3D 0 --- Uptime: 1d22h51m17s Dumping 2990 out of 8037 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..9= 1% __curthread () at ./machine/pcpu.h:230 230 __asm("movq %%gs:%1,%0" : "=3Dr" (td) (kgdb) #0 __curthread () at ./machine/pcpu.h:230 #1 doadump (textdump=3D1) at /usr/src/sys/kern/kern_shutdown.c:366 #2 0xffffffff80485ea1 in kern_reboot (howto=3D260) at /usr/src/sys/kern/kern_shutdown.c:446 #3 0xffffffff80486483 in vpanic (fmt=3D<optimized out>, ap=3D0xfffffe008af= 858f0) at /usr/src/sys/kern/kern_shutdown.c:863 #4 0xffffffff804861f0 in kassert_panic ( fmt=3D0xffffffff807e085f "Assertion %s failed at %s:%d") at /usr/src/sys/kern/kern_shutdown.c:749 #5 0xffffffff8059cd78 in iflib_busdma_load_mbuf_sg (flags=3D0, txq=3D<optimized out>, tag=3D<optimized out>, map=3D<optimized out>, m0=3D<optimized out>, segs=3D<optimized out>, nsegs=3D<optimized out>, max_segs=3D<optimized out>) at /usr/src/sys/net/iflib.c:3151 #6 iflib_encap (txq=3D0xfffff800028dc000, m_headp=3D0xfffffe00959bdd30) at /usr/src/sys/net/iflib.c:3321 #7 iflib_txq_drain (r=3D0xfffffe00959ba000, cidx=3D<optimized out>, pidx=3D41319936) at /usr/src/sys/net/iflib.c:3636 #8 0xffffffff805a0f4c in drain_ring_lockless (r=3D<optimized out>, os=3D..= ., prev=3D<optimized out>, budget=3D<optimized out>) at /usr/src/sys/net/mp_ring.c:199 #9 ifmp_ring_check_drainage (r=3D<optimized out>, budget=3D32) at /usr/src/sys/net/mp_ring.c:502 #10 0xffffffff80599c46 in _task_fn_tx (context=3D<optimized out>) at /usr/src/sys/net/iflib.c:3747 #11 0xffffffff804cd2c9 in gtaskqueue_run_locked (queue=3D0xfffff800025e0d00= ) at /usr/src/sys/kern/subr_gtaskqueue.c:332 #12 0xffffffff804cd048 in gtaskqueue_thread_loop (arg=3D<optimized out>) at /usr/src/sys/kern/subr_gtaskqueue.c:507 #13 0xffffffff8044cc34 in fork_exit ( callout=3D0xffffffff804ccfc0 <gtaskqueue_thread_loop>, arg=3D0xfffffe0007ffd038, frame=3D0xfffffe008af85c00) at /usr/src/sys/kern/kern_fork.c:1057 (kgdb) up 5 #5 0xffffffff8059cd78 in iflib_busdma_load_mbuf_sg (flags=3D0,=20=20 txq=3D<optimized out>, tag=3D<optimized out>, map=3D<optimized out>, m0=3D<optimized out>, segs=3D<optimized out>,= =20=20 nsegs=3D<optimized out>, max_segs=3D<optimized out>) at /usr/src/sys/net/iflib.c:3151 3151 MPASS(ifsd_m[next] =3D=3D NULL); (kgdb) list 3146 /* 3147 * see if we can't be smarter about physica= lly 3148 * contiguous mappings 3149 */ 3150 next =3D (pidx + count) & (ntxd-1); 3151 MPASS(ifsd_m[next] =3D=3D NULL); 3152 #if MEMORY_LOGGING 3153 txq->ift_enqueued++; 3154 #endif 3155 ifsd_m[next] =3D m; (kgdb) print ifsd_m $1 =3D (struct mbuf **) 0xfffffe00959b8000 (kgdb) print next $2 =3D <optimized out> (kgdb) print pidx $3 =3D 277 (kgdb) print count $4 =3D 0 (kgdb) print ntxd $5 =3D <optimized out> 2) Unread portion of the kernel message buffer: panic: Assertion ifsd_m[next] =3D=3D NULL failed at /usr/src/sys/net/iflib.= c:3151 cpuid =3D 2 time =3D 1531944550 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe008af85= 850 vpanic() at vpanic+0x1a3/frame 0xfffffe008af858b0 doadump() at doadump/frame 0xfffffe008af85930 iflib_txq_drain() at iflib_txq_drain+0xe58/frame 0xfffffe008af85aa0 ifmp_ring_check_drainage() at ifmp_ring_check_drainage+0x16c/frame=20=20 0xfffffe008af85b00 _task_fn_tx()=20at _task_fn_tx+0x76/frame 0xfffffe008af85b30 gtaskqueue_run_locked() at gtaskqueue_run_locked+0x139/frame=20=20 0xfffffe008af85b80 gtaskqueue_thread_loop()=20at gtaskqueue_thread_loop+0x88/frame=20=20 0xfffffe008af85bb0 fork_exit()=20at fork_exit+0x84/frame 0xfffffe008af85bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe008af85bf0 --- trap 0, rip =3D 0, rsp =3D 0, rbp =3D 0 --- Uptime: 5m27s Dumping 1555 out of 8037 MB:..2%..11%..21%..31%..41%..51%..61%..71%..81%..9= 1% __curthread () at ./machine/pcpu.h:230 230 __asm("movq %%gs:%1,%0" : "=3Dr" (td) (kgdb) bt #0 __curthread () at ./machine/pcpu.h:230 #1 doadump (textdump=3D1) at /usr/src/sys/kern/kern_shutdown.c:366 #2 0xffffffff80485ea1 in kern_reboot (howto=3D260) at=20=20 /usr/src/sys/kern/kern_shutdown.c:446 #3=20 0xffffffff80486483 in vpanic (fmt=3D<optimized out>, ap=3D0xfffffe008= af858f0) at /usr/src/sys/kern/kern_shutdown.c:863 #4 0xffffffff804861f0 in kassert_panic (fmt=3D0xffffffff807e085f=20=20 "Assertion=20%s failed at %s:%d") at /usr/src/sys/kern/kern_shutdown.c:749 #5 0xffffffff8059cd78 in iflib_busdma_load_mbuf_sg (flags=3D0,=20=20 txq=3D<optimized out>, tag=3D<optimized out>, map=3D<optimized out>, m0=3D<optimized out>, segs=3D<optimized out>,= =20=20 nsegs=3D<optimized out>, max_segs=3D<optimized out>) at /usr/src/sys/net/iflib.c:3151 #6 iflib_encap (txq=3D0xfffff800028fe000, m_headp=3D0xfffffe00959bdde8)=20= =20 at=20/usr/src/sys/net/iflib.c:3321 #7 iflib_txq_drain (r=3D0xfffffe00959ba000, cidx=3D<optimized out>,=20=20 pidx=3D42948608) at /usr/src/sys/net/iflib.c:3636 #8 0xffffffff805a0f4c in drain_ring_lockless (r=3D<optimized out>,=20=20 os=3D..., prev=3D<optimized out>, budget=3D<optimized out>) at /usr/src/sys/net/mp_ring.c:199 #9 ifmp_ring_check_drainage (r=3D<optimized out>, budget=3D32) at=20=20 /usr/src/sys/net/mp_ring.c:502 #10=200xffffffff80599c46 in _task_fn_tx (context=3D<optimized out>) at=20= =20 /usr/src/sys/net/iflib.c:3747 #11=200xffffffff804cd2c9 in gtaskqueue_run_locked (queue=3D0xfffff800025a22= 00) at /usr/src/sys/kern/subr_gtaskqueue.c:332 #12 0xffffffff804cd048 in gtaskqueue_thread_loop (arg=3D<optimized out>)=20= =20 at=20/usr/src/sys/kern/subr_gtaskqueue.c:507 #13 0xffffffff8044cc34 in fork_exit (callout=3D0xffffffff804ccfc0=20=20 <gtaskqueue_thread_loop>,=20arg=3D0xfffffe0007ffd038, frame=3D0xfffffe008af85c00) at /usr/src/sys/kern/kern_fork.c:1057 #14 <signal handler called> (kgdb) up 5 #5 0xffffffff8059cd78 in iflib_busdma_load_mbuf_sg (flags=3D0,=20=20 txq=3D<optimized out>, tag=3D<optimized out>, map=3D<optimized out>, m0=3D<optimized out>, segs=3D<optimized out>,= =20=20 nsegs=3D<optimized out>, max_segs=3D<optimized out>) at /usr/src/sys/net/iflib.c:3151 3151 MPASS(ifsd_m[next] =3D=3D NULL); (kgdb) print ifsd_m $1 =3D (struct mbuf **) 0xfffffe00959b8000 (kgdb) print pidx $2 =3D 707 (kgdb) print count $3 =3D 0 Bye, Alexander. --=20 http://www.Leidinger.net=20Alexander@Leidinger.net: PGP 0x8F31830F9F2772BF http://www.FreeBSD.org netchild@FreeBSD.org : PGP 0x8F31830F9F2772BF --=_squ2rs7hgvM55G-LHO-CB7r Content-Type: application/pgp-signature Content-Description: Digitale PGP-Signatur Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAABAgAGBQJbT6QJAAoJEKrxQhqFIICEL0sP/21ishTejRzesyfAKXAfCykd yCl3edMsWb+62aLIicM21qfLqYvSQ13N4PyuuGknTAkRYUuXgHp/qeXP/2I9amh1 MxEgVK+OZPvcEs9awHyQzMVXIrPpAA9TIpLVt+UkPVvrwz9wKTMHTd7UOd16ADhF SY2nVdoL8rhp9czTod5atQXL2RtHsnn3VgOpg63a7GUH9jkZfmlWB8DaqAwYaL8n skUrt5yTENhXSqSbXV6XWnnBMJOvlQ1Xe2/QTCq3mb4tU9qEPSM4vM2qR7P/wsld 7rsRciwlJ3PrDeLwtjzTVed2IuqLmDRDW7HJRqQlDrgs7xGU0cK0ocUqlhC8dpWS Vq5SG2xzDCRb2S1nL5JOVSncuPsbs8TGubqQ+QTG3aUP/mutTa9zjBTHnrGb+PcV WdHrbkOGiTkev7lpRf8tsf6Rzmwi/G63cXuNFGyMG9TlV/L5qMkmR8PYGo8bscxZ BqzCKxZ2lBrdX/YrLHoaHGEGV2LzwhNrdXXS1hUDUcMBfzWbebENTw+0zdJ2uMyK a21wIt8K2WDg/IGc2KEu9GDFo1dMkfcGqGFaRlRUdgPOeJil/Hywy9PM5Jztgepw oWQhUr5VUjeozcstuEdaCyy7crrdEdTv8LcvhY0vUsvU6EHdmkp6BzcwQJb+/7U4 DqYfsRHetkoZqy9Yhlww =SQsB -----END PGP SIGNATURE----- --=_squ2rs7hgvM55G-LHO-CB7r--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20180718223313.Horde.lYE8PRYqLdkrN3QMTTHx3aV>