Date: Tue, 22 Apr 2008 18:36:27 +0300 From: Kostik Belousov <kostikbel@gmail.com> To: John-Mark Gurney <jmg@funkthat.com> Cc: current@freebsd.org Subject: Re: knlist_cleardel() panic Message-ID: <20080422153627.GS18958@deviant.kiev.zoral.com.ua> In-Reply-To: <20080303003705.GJ96595@funkthat.com> References: <20080302152523.GO57756@deviant.kiev.zoral.com.ua> <20080302195334.GI96595@funkthat.com> <20080302200647.GQ57756@deviant.kiev.zoral.com.ua> <20080303003705.GJ96595@funkthat.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--KRG9W6ukdHdpcXku
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
On Sun, Mar 02, 2008 at 04:37:06PM -0800, John-Mark Gurney wrote:
> Kostik Belousov wrote this message on Sun, Mar 02, 2008 at 22:06 +0200:
> > On Sun, Mar 02, 2008 at 11:53:34AM -0800, John-Mark Gurney wrote:
> > > Kostik Belousov wrote this message on Sun, Mar 02, 2008 at 17:25 +020=
0:
> > > > The panic below was already reported, but now I got it on my desktop
> > > > and was able to investigate further.
> > > >=20
> > > > #5 0xc06daf36 in trap (frame=3D0xe8093b1c)
> > > > at /usr/bsd/src/sys/i386/i386/trap.c:490
> > > > #6 0xc06c0b4b in calltrap () at /usr/bsd/src/sys/i386/i386/excepti=
on.s:139
> > > > #7 0xc0493968 in knlist_cleardel (knl=3D0xcabec128, td=3D0x0, islo=
cked=3D1,=20
> > > > killkn=3D0) at atomic.h:149
> > > > #8 0xc04f520c in pipeclose (cpipe=3D0xcabec0b8)
> > > > at /usr/bsd/src/sys/kern/sys_pipe.c:1508
> > > > #9 0xc04f5320 in pipe_close (fp=3D0xc5ce8630, td=3D0xcac01aa0)
> > > > at /usr/bsd/src/sys/kern/sys_pipe.c:1425
> > > > #10 0xc0489442 in fdrop (fp=3D0xc5ce8630, td=3D0xcac01aa0) at file.=
h:297
> > > > #11 0xc048accf in closef (fp=3D0xc5ce8630, td=3D0xcac01aa0)
> > > > at /usr/bsd/src/sys/kern/kern_descrip.c:1958
> > > > #12 0xc048b1ff in kern_close (td=3D0xcac01aa0, fd=3D10)
> > > > at /usr/bsd/src/sys/kern/kern_descrip.c:1054
> > > > #13 0xc048b2da in close (td=3D0xcac01aa0, uap=3D0xe8093cfc)
> > > > at /usr/bsd/src/sys/kern/kern_descrip.c:1006
> > > > ---Type <return> to continue, or q <return> to quit---
> > > > #14 0xc06da865 in syscall (frame=3D0xe8093d38)
> > > > at /usr/bsd/src/sys/i386/i386/trap.c:1035
> > > > #15 0xc06c0bb0 in Xint0x80_syscall ()
> > > > at /usr/bsd/src/sys/i386/i386/exception.s:196
> > > >=20
> > > > At the frame 8, we have
> > > > (kgdb) p/x *(knl->kl_list->slh_first)
> > > > $9 =3D {kn_link =3D {sle_next =3D 0x0}, kn_selnext =3D {sle_next =
=3D 0x0},
> > > > kn_knlist =3D 0x0, kn_tqe =3D {tqe_next =3D 0xc58de484, tqe_prev =
=3D 0xc5e9ab20},
> > > > kn_kq =3D 0x0, kn_kevent =3D {ident =3D 0x0, filter =3D 0x0, flag=
s =3D 0x0,
> > > > fflags =3D 0x0, data =3D 0x0, udata =3D 0x0}, kn_status =3D 0x2=
0,
> > > > kn_sfflags =3D 0x0, kn_sdata =3D 0x0, kn_ptr =3D {p_fp =3D 0x0, p=
_proc =3D 0x0,
> > > > p_aio =3D 0x0, p_lio =3D 0x0}, kn_fop =3D 0x0, kn_hook =3D 0x0}
> > > >=20
> > > > The knote is KN_MARKER, and the kn_kq is NULL. The result is that K=
Q_LOCK
> > > > in the knlist_cleardel()::SLIST_FOREACH_SAFE() loop dereferences NU=
LL and
> > > > panics.
> > > >=20
> > > > Does the following change makes any sense?
> > >=20
> > > I thought this was a bug, but upon further examination, there is
> > > something wrong... a KN_MARKER knote should never be on the knlist..
> > > It is only ever added to kq's even list, never to an object's list...
> > >=20
> > > If you could walk the knl->kl_list through kn_selnext, and ensure
> > > that the kn w/ kN_MARKER exists would be good... It should be
> > > last one as kn_selnext is NULL...
> >=20
> > As I shown above, the KN_MARKER was found on the cpipe->pipe_sel.si_not=
e,
> >=20
> > (kgdb) p cpipe->pipe_sel.si_note
> > $1 =3D {kl_list =3D {slh_first =3D 0xc58df330},=20
> > kl_lock =3D 0xc0493a20 <knlist_mtx_lock>,=20
> > kl_unlock =3D 0xc0493370 <knlist_mtx_unlock>,=20
> > kl_locked =3D 0xc0493350 <knlist_mtx_locked>, kl_lockarg =3D 0xcabec1=
70
> > }
> >=20
> > (kgdb) p/x (knl->kl_list->slh_first)
> > $2 =3D 0xc58df330
> >=20
> > (kgdb) p/x *(knl->kl_list->slh_first)
> > $3 =3D {kn_link =3D {sle_next =3D 0x0}, kn_selnext =3D {sle_next =3D 0x=
0},=20
> > kn_knlist =3D 0x0, kn_tqe =3D {tqe_next =3D 0xc58de484, tqe_prev =3D =
0xc5e9ab20},=20
> > kn_kq =3D 0x0, kn_kevent =3D {ident =3D 0x0, filter =3D 0x0, flags =
=3D 0x0,=20
> > fflags =3D 0x0, data =3D 0x0, udata =3D 0x0}, kn_status =3D 0x20,=
=20
> > kn_sfflags =3D 0x0, kn_sdata =3D 0x0, kn_ptr =3D {p_fp =3D 0x0, p_pro=
c =3D 0x0,=20
> > p_aio =3D 0x0, p_lio =3D 0x0}, kn_fop =3D 0x0, kn_hook =3D 0x0}
> > >=20
> > > I can't think of a way that this can happen, as the only way to get
> > > on the knlist is by calling knlist_add, and it should only ever be
> > > called from f_attach, which is called in one place, and has to have
> > > kn_kq set properly... Are there any custom patches on the system?
> >=20
> > This is the stock RELENG_7 as of today.
>=20
> Do you have a reproducable test case that demonstrates this?
>=20
> You can try the attached patch, but I'm pretty sure that it won't be
> triggered...
>=20
> --=20
> John-Mark Gurney Voice: +1 415 225 5579
>=20
> "All that I will do, has been done, All that I have, has not."
> --- kern_event.c.orig 2007-07-14 14:23:30.000000000 -0700
> +++ kern_event.c 2008-03-02 16:26:59.000000000 -0800
> @@ -1590,6 +1590,8 @@
> KQ_NOTOWNED(kn->kn_kq);
> KASSERT((kn->kn_status & (KN_INFLUX|KN_DETACHED)) =3D=3D
> (KN_INFLUX|KN_DETACHED), ("knote not KN_INFLUX and KN_DETACHED"));
> + KASSERT((kn->kn_status & (KN_MARKER)) =3D=3D 0,
> + ("knote has KN_MARKER set"));
> if (!islocked)
> knl->kl_lock(knl->kl_lockarg);
> SLIST_INSERT_HEAD(&knl->kl_list, kn, kn_selnext);
When the pipe_peer is being closed, and the pipeclose() has set the
pipe_present =3D 0 and enters knlist_cleardel(), the PIPE_MTX may be
dropped when knl->kl_list cannot be cleared due to influx knotes.
If the following often encountered code fragment
if (!(kn->kn_status & KN_DETACHED))
kn->kn_fop->f_detach(kn);
knote_drop(kn, td); [1]
is executed while the knlist lock is dropped, then the knote memory is freed
by the knote_drop() [1] without knote being removed from the knlist, since
the filt_pipedetach() contains the following:
if (kn->kn_filter =3D=3D EVFILT_WRITE) {
if (!cpipe->pipe_peer->pipe_present) {
PIPE_UNLOCK(cpipe);
return;
Now, the memory may be reused in the zone, for instance, for the marker
knote. And, we ending up the marker on the knlist.
Does this look plausible ?
Thanks.
--KRG9W6ukdHdpcXku
Content-Type: application/pgp-signature
Content-Disposition: inline
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (FreeBSD)
iEYEARECAAYFAkgOBXUACgkQC3+MBN1Mb4gUnACguIt9+YTd4sDynH4vFHyNGZiF
v7gAn0ctqZ1OYLH1fE4H31Qjpflh8wqT
=sZTX
-----END PGP SIGNATURE-----
--KRG9W6ukdHdpcXku--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20080422153627.GS18958>