From owner-freebsd-current@freebsd.org Thu Nov 8 16:05:17 2018 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id ABA31112ED77 for ; Thu, 8 Nov 2018 16:05:17 +0000 (UTC) (envelope-from sg@efficientip.com) Received: from mail-ed1-x52f.google.com (mail-ed1-x52f.google.com [IPv6:2a00:1450:4864:20::52f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8CA1684D2A for ; Thu, 8 Nov 2018 16:05:16 +0000 (UTC) (envelope-from sg@efficientip.com) Received: by mail-ed1-x52f.google.com with SMTP id w39-v6so12399008edw.12 for ; Thu, 08 Nov 2018 08:05:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=efficientip.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=vzBxgErGgqZSmMjAcav8ZsSkBYbgm3/FeGP/4OBNnD0=; b=OQI6BWDn2ZoV9rfClhx69oLO/E5Ow/J7BvXMAVQiWY0eC6xgWPw6dkSmBvuFBDHldK i5MKq8raT1ak9mLfbmpyG7W63cmHpjlDv/RrlcMGkD52rtgx3I/PshbC+xEYwbPuyBHJ XwG1x6ttG9DSSSZlq48to8fmdJ/uhs5TEmOuQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=vzBxgErGgqZSmMjAcav8ZsSkBYbgm3/FeGP/4OBNnD0=; b=uLJEgKL7O0jh7+C2cCqMelsBdqXwmvzbu4EOTfBYaIN/lU+nPPftrtDdqsxqIK/39D rs0HvP7E9Ibfajm8ZVTtBdDDjeYpJDwXUhjyScCupErg6E3QQuxQmpOGuRDLz+C4z+Ws K88VNnmK/SoqHyfFqXDj1ldMt3plBXFOvkBIGDL8CM7adKiBoiChVtNCK516xQRDSxG8 72YkopCsHosN5doF17/X4cA1JyjR6rH5IM1QPHEbNCZNJYvWqDyqeK6gXIn+H/dgBbXL JB0ByfU6xbVnVRFo4qagoDdDGoZEQEybwDrQ8N9ykuLYL4m2nPeU8fWCIMZwvV16rjXv lSnw== X-Gm-Message-State: AGRZ1gKVtyGfXZVvlJGZ+fUCoraaFy16pwZaTnggfCYFeRdJxmchaxAx C2prEI2365OwnjIuRUYRlE8zzoAzlsMy0tlpnQr9cMTVZnUEWg== X-Google-Smtp-Source: AJdET5eTktSlvUV/ortvyiC41vHkm9WGv+AK7RIh8N8PkKOdwT0zr/Mz1kZX86LJG0YkdYb9by1RPUXjtPta3cdzEXg= X-Received: by 2002:a50:90fb:: with SMTP id d56-v6mr4539970eda.228.1541693115118; Thu, 08 Nov 2018 08:05:15 -0800 (PST) MIME-Version: 1.0 References: <20181107043503.GB30861@raichu> In-Reply-To: <20181107043503.GB30861@raichu> From: Sylvain GALLIANO Date: Thu, 8 Nov 2018 17:05:03 +0100 Message-ID: Subject: Re: Panic on kern_event.c To: freebsd-current@freebsd.org X-Rspamd-Queue-Id: 8CA1684D2A X-Spamd-Result: default: False [-5.72 / 200.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; R_DKIM_ALLOW(-0.20)[efficientip.com]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-0.998,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; DMARC_NA(0.00)[efficientip.com]; DKIM_TRACE(0.00)[efficientip.com:+]; MX_GOOD(-0.01)[alt1.aspmx.l.google.com,aspmx.l.google.com,aspmx5.googlemail.com,aspmx4.googlemail.com,aspmx3.googlemail.com,alt2.aspmx.l.google.com,aspmx2.googlemail.com]; RCVD_IN_DNSWL_NONE(0.00)[f.2.5.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0]; NEURAL_HAM_SHORT(-0.89)[-0.890,0]; R_SPF_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; IP_SCORE(-2.52)[ip: (-8.98), ipnet: 2a00:1450::/32(-2.00), asn: 15169(-1.52), country: US(-0.09)]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Server: mx1.freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Nov 2018 16:05:17 -0000 Hi, I replaced << printf("XXX knote %p already in tailq status:%x kq_count:%d [%p %p] %u\n",kn,kn->kn_status,kq->kq_count,kn->kn_tqe.tqe_next,kn->kn_tqe.tqe_prev= ,__LINE__); by >> panic("XXX knote %p already in tailq status:%x kq_count:%d [%p %p] %u\n",kn,kn->kn_status,kq->kq_count,kn->kn_tqe.tqe_next,kn->kn_tqe.tqe_prev= ,__LINE__); Here is the stack during panic: panic: XXX knote 0xfffff801e1c6ddc0 already in tailq status:1 kq_count:2 [0 0xfffff8000957a978] 2671 cpuid =3D 0 time =3D 1541688832 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2c/frame 0xfffffe0412258fd0 kdb_backtrace() at kdb_backtrace+0x53/frame 0xfffffe04122590a0 vpanic() at vpanic+0x277/frame 0xfffffe0412259170 doadump() at doadump/frame 0xfffffe04122591d0 knote_enqueue() at knote_enqueue+0xf2/frame 0xfffffe0412259210 kqueue_register() at kqueue_register+0xaed/frame 0xfffffe0412259340 kqueue_kevent() at kqueue_kevent+0x13c/frame 0xfffffe04122595b0 kern_kevent_fp() at kern_kevent_fp+0x66/frame 0xfffffe0412259610 kern_kevent() at kern_kevent+0x17f/frame 0xfffffe0412259700 kern_kevent_generic() at kern_kevent_generic+0xfe/frame 0xfffffe0412259780 sys_kevent() at sys_kevent+0xaa/frame 0xfffffe0412259810 syscallenter() at syscallenter+0x4e3/frame 0xfffffe04122598f0 amd64_syscall() at amd64_syscall+0x1b/frame 0xfffffe04122599b0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe04122599b0 --- syscall (560, FreeBSD ELF64, sys_kevent), rip =3D 0x406e3bfa, rsp =3D 0x7fffdf7e9db8, rbp =3D 0x7fffdf7e9e00 --- KDB: enter: panic you can get kernel.debug + vmcore at: https://drive.google.com/drive/folders/1MbqJQm12-KOYDbb4-9uNRTnAdsNqLaIP?us= p=3Dsharing Le mer. 7 nov. 2018 =C3=A0 05:35, Mark Johnston a =C3= =A9crit : > On Tue, Nov 06, 2018 at 10:50:06AM +0100, Sylvain GALLIANO wrote: > > Hi, > > > > I got random panic on Current & 11.2-STABLE on kern_event.c > > > > Panic occur in syslog-ng (logging at high rate) with the folloging line= s: > > > > Panic String: Bad tailq NEXT(0xfffff80039ae7a38->tqh_last) !=3D NULL > > Panic String: Bad tailq head 0xfffff80039f1a238 first->prev !=3D head > > > > It's look like knote_enqueue try to add and existings knote on TAILQ > > (confirmed by following patch). > > > > logs after apply patch: > > XXX knote 0xfffff8012e3d33c0 already in tailq status:1 kq_count:1 [0 > > 0xfffff800327d3538] 2671 > > XXX knote 0xfffff80032861780 already in tailq status:1 kq_count:1 [0 > > 0xfffff80032457938] 2671 > > Can you grab the stack when this happens as well, with kdb_backtrace()? > Or better, convert the print into a panic so that we can examine the > kernel dump. >