Date: Tue, 4 Mar 2003 16:51:12 +0200 From: Peter Pentchev <roam@ringlet.net> To: Jonathan Lemon <jlemon@flugsvamp.com> Cc: stable@freebsd.org Subject: Re: 4.8-PRE (and earlier) crash - syncookies? Message-ID: <20030304145112.GE554@straylight.oblivion.bg> In-Reply-To: <200303041440.h24Ee9ha074389@mail.flugsvamp.com> References: <local.mail.freebsd-stable/20030304115918.GA554@straylight.oblivion.bg> <200303041440.h24Ee9ha074389@mail.flugsvamp.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--UKNXkkdQCYZ6W5l3
Content-Type: multipart/mixed; boundary="UBnjLfzoMQYIXCvq"
Content-Disposition: inline
--UBnjLfzoMQYIXCvq
Content-Type: text/plain; charset=windows-1251
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
On Tue, Mar 04, 2003 at 08:40:09AM -0600, Jonathan Lemon wrote:
> In article <local.mail.freebsd-stable/20030304115918.GA554@straylight.obl=
ivion.bg> you write:
> >-=3D-=3D-=3D-=3D-=3D-
> >-=3D-=3D-=3D-=3D-=3D-
> >
> >Hi,
> >
> >I think this has been going on for a couple of months; unfortunately, I
> >cannot remember exactly when it started :( Several machines, which have
> >always been updated synchronously at intervals of a couple of weeks or
> >so, have been experiencing panics every few days with recent 4.6-STABLE,
> >4.7-STABLE, and 4.8-PRERELEASE kernel+world.
> >
> >Attached is the kernel config and a gdb session of the most recent
> >crashdump. Maybe the most interesting part is at the end of the gdb
> >session: is there a normal reason why a TAILQ structure would have a
> >null start pointer, and an end pointer pointing back to itself?
>=20
> [ ... ]
>=20
> >#16 0xc01b2768 in syncache_insert (sc=3D0xcb870b80, sch=3D0xc0eeeb7c)
> > at /usr/src/sys/netinet/tcp_syncache.c:289
> >289 sc2 =3D TAILQ_FIRST(&sch->sch_bucket);
> >(kgdb) print sch
> >$1 =3D (struct syncache_head *) 0xc0eeeb7c
> >(kgdb) print *sch
> >$2 =3D {sch_bucket =3D {tqh_first =3D 0x0, tqh_last =3D 0xc0eeeb7c},=20
> > sch_length =3D 4294967295}
>=20
>=20
> It looks to me like the tailq is empty (first =3D=3D null, last =3D=3D &h=
ead),
> but the code path to remove an entry was taken because the length
> counter indicated it was over the bucket limit. sch_length =3D=3D -1.
>=20
> So my guess is the code dropped a syncache entry twice, and this is=20
> the resulting mess. Can you print out the contents of tcp_syncache
> and tcpstat?
See attached; I could provide the whole corefile on request, if it will
be helpful.
G'luck,
Peter
--=20
Peter Pentchev roam@ringlet.net roam@sbnd.net roam@FreeBSD.org
PGP key: http://people.FreeBSD.org/~roam/roam.key.asc
Key fingerprint FDBA FD79 C26F 3C51 C95E DF9E ED18 B68D 1619 4553
This sentence would be seven words long if it were six words shorter.
--UBnjLfzoMQYIXCvq
Content-Type: text/plain; charset=windows-1251
Content-Disposition: attachment; filename="pan-pl.tcpstat.txt"
Content-Transfer-Encoding: quoted-printable
(kgdb) fr 16
#16 0xc01b2768 in syncache_insert (sc=3D0xcb870b80, sch=3D0xc0eeeb7c)
at /usr/src/sys/netinet/tcp_syncache.c:289
289 sc2 =3D TAILQ_FIRST(&sch->sch_bucket);
(kgdb) print tcp_syncache
$1 =3D {hashbase =3D 0xc0eee000, zone =3D 0xc0d94c80, hashsize =3D 512,=20
hashmask =3D 511, bucket_limit =3D 30, cache_count =3D 4294967295,=20
cache_limit =3D 15359, rexmt_limit =3D 3, hash_secret =3D 1416729105,=20
next_reseed =3D 0, timerq =3D {{tqh_first =3D 0x0, tqh_last =3D 0xc02a874=
8}, {
tqh_first =3D 0x0, tqh_last =3D 0xc02a8750}, {tqh_first =3D 0x0,=20
tqh_last =3D 0xc02a8758}, {tqh_first =3D 0x0, tqh_last =3D 0xc02a8760=
}},=20
tt_timerq =3D {{c_links =3D {sle =3D {sle_next =3D 0xc603a2e8}, tqe =3D {
tqe_next =3D 0xc603a2e8, tqe_prev =3D 0xc603c620}}, c_time =3D 55=
852000,=20
c_arg =3D 0x0, c_func =3D 0, c_flags =3D 0}, {c_links =3D {sle =3D {
sle_next =3D 0x0}, tqe =3D {tqe_next =3D 0x0, tqe_prev =3D 0xc603=
e200}},=20
c_time =3D 55658332, c_arg =3D 0x1, c_func =3D 0, c_flags =3D 0}, {c_=
links =3D {
sle =3D {sle_next =3D 0xc603a2e8}, tqe =3D {tqe_next =3D 0xc603a2e8=
,=20
tqe_prev =3D 0xc603c130}}, c_time =3D 52482882, c_arg =3D 0x2, c_=
func =3D 0,=20
c_flags =3D 0}, {c_links =3D {sle =3D {sle_next =3D 0xc603a2e8}, tqe =
=3D {
tqe_next =3D 0xc603a2e8, tqe_prev =3D 0xc603cc30}}, c_time =3D 52=
485282,=20
c_arg =3D 0x3, c_func =3D 0, c_flags =3D 0}}}
(kgdb) print tcpstat
$2 =3D {tcps_connattempt =3D 598918, tcps_accepts =3D 32652, tcps_connects =
=3D 612607,=20
tcps_drops =3D 11716, tcps_conndrops =3D 13231, tcps_closed =3D 636232,=
=20
tcps_segstimed =3D 1956837, tcps_rttupdated =3D 2499750, tcps_delack =3D =
615940,=20
tcps_timeoutdrop =3D 23, tcps_rexmttimeo =3D 98711, tcps_persisttimeo =3D=
55,=20
tcps_keeptimeo =3D 6166, tcps_keepprobe =3D 204, tcps_keepdrops =3D 5962,=
=20
tcps_sndtotal =3D 5368356, tcps_sndpack =3D 1503648, tcps_sndbyte =3D 388=
668829,=20
tcps_sndrexmitpack =3D 13994, tcps_sndrexmitbyte =3D 10990401,=20
tcps_sndacks =3D 2587703, tcps_sndprobe =3D 48, tcps_sndurg =3D 0,=20
tcps_sndwinup =3D 53833, tcps_sndctrl =3D 1209130, tcps_rcvtotal =3D 4122=
112,=20
tcps_rcvpack =3D 2027883, tcps_rcvbyte =3D 301857290, tcps_rcvbadsum =3D =
45,=20
tcps_rcvbadoff =3D 0, tcps_rcvmemdrop =3D 0, tcps_rcvshort =3D 0,=20
tcps_rcvduppack =3D 571696, tcps_rcvdupbyte =3D 1922742,=20
tcps_rcvpartduppack =3D 27, tcps_rcvpartdupbyte =3D 14961,=20
tcps_rcvoopack =3D 23227, tcps_rcvoobyte =3D 24974272,=20
tcps_rcvpackafterwin =3D 637, tcps_rcvbyteafterwin =3D 677422,=20
tcps_rcvafterclose =3D 479, tcps_rcvwinprobe =3D 4, tcps_rcvdupack =3D 61=
2690,=20
tcps_rcvacktoomuch =3D 0, tcps_rcvackpack =3D 2551692,=20
tcps_rcvackbyte =3D 384639652, tcps_rcvwinupd =3D 12215, tcps_pawsdrop =
=3D 83,=20
tcps_predack =3D 58265, tcps_preddat =3D 255753, tcps_pcbcachemiss =3D 0,=
=20
tcps_cachedrtt =3D 852, tcps_cachedrttvar =3D 852, tcps_cachedssthresh =
=3D 282,=20
tcps_usedrtt =3D 11154, tcps_usedrttvar =3D 11154, tcps_usedssthresh =3D =
2053,=20
tcps_persistdrop =3D 0, tcps_badsyn =3D 24, tcps_mturesent =3D 0,=20
tcps_listendrop =3D 0, tcps_sc_added =3D 32716, tcps_sc_retransmitted =3D=
197,=20
tcps_sc_dupsyn =3D 165, tcps_sc_dropped =3D 0, tcps_sc_completed =3D 3265=
2,=20
tcps_sc_bucketoverflow =3D 0, tcps_sc_cacheoverflow =3D 0, tcps_sc_reset =
=3D 52,=20
tcps_sc_stale =3D 13, tcps_sc_aborted =3D 0, tcps_sc_badack =3D 0,=20
tcps_sc_unreach =3D 0, tcps_sc_zonefail =3D 0, tcps_sc_sendcookie =3D 0,=
=20
tcps_sc_recvcookie =3D 0}
(kgdb) q
--UBnjLfzoMQYIXCvq--
--UKNXkkdQCYZ6W5l3
Content-Type: application/pgp-signature
Content-Disposition: inline
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.1 (FreeBSD)
iD8DBQE+ZL1g7Ri2jRYZRVMRAjIvAJ4ntEc3PLrf/5GryfePorrZvldeNACfSasa
xFtp5E2D+otJOeDRAD4Goag=
=hWsO
-----END PGP SIGNATURE-----
--UKNXkkdQCYZ6W5l3--
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-stable" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20030304145112.GE554>