From owner-freebsd-stable@FreeBSD.ORG Tue Feb 28 13:09:37 2012 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 839FB106564A; Tue, 28 Feb 2012 13:09:37 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id BB2878FC18; Tue, 28 Feb 2012 13:09:36 +0000 (UTC) Received: from skuns.kiev.zoral.com.ua (localhost [127.0.0.1]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id q1SD8dVR042167; Tue, 28 Feb 2012 15:08:39 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5) with ESMTP id q1SD8den086437; Tue, 28 Feb 2012 15:08:39 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5/Submit) id q1SD8cKH086436; Tue, 28 Feb 2012 15:08:38 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Tue, 28 Feb 2012 15:08:38 +0200 From: Konstantin Belousov To: Hiroki Sato Message-ID: <20120228130838.GN55074@deviant.kiev.zoral.com.ua> References: <20120223.234558.1101656075598772176.hrs@allbsd.org> <20120224143336.GS55074@deviant.kiev.zoral.com.ua> <20120224150259.GV55074@deviant.kiev.zoral.com.ua> <20120225.025828.128418237042325597.hrs@allbsd.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="sEASj6BbPXAOAu+u" Content-Disposition: inline In-Reply-To: <20120225.025828.128418237042325597.hrs@allbsd.org> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-4.0 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: stable@freebsd.org Subject: Re: another panic in 8.3-PRERELEASE X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Feb 2012 13:09:37 -0000 --sEASj6BbPXAOAu+u Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Feb 25, 2012 at 02:58:28AM +0900, Hiroki Sato wrote: > Konstantin Belousov wrote > in <20120224150259.GV55074@deviant.kiev.zoral.com.ua>: >=20 > ko> > > #19 0x0000000800abecfc in ?? () > ko> > > Previous frame inner to this frame (corrupt stack?) > ko> > > (kgdb) > ko> > Can you, please, print out the content of *td, e.g. from the frame = 16 ? > ko>=20 > ko> And *req from the frame 11, please. >=20 > Here: >=20 > (kgdb) f 16 > #16 0xffffffff80675e3a in __sysctl (td=3D0xffffff0396ec5460,=20 > uap=3D0xffffff86c6389bc0) at /usr/src/sys/kern/kern_sysctl.c:1491 > 1491 error =3D userland_sysctl(td, name, uap->namelen, > (kgdb) print *td > $2 =3D {td_lock =3D 0xffffffff80d7f540, td_proc =3D 0xffffff03969bf470, t= d_plist =3D { > tqe_next =3D 0x0, tqe_prev =3D 0xffffff03969bf480}, td_runq =3D {tqe_= next =3D 0x0,=20 > tqe_prev =3D 0xffffffff80d7f788}, td_slpq =3D {tqe_next =3D 0x0,=20 > tqe_prev =3D 0xffffff0396ebe800}, td_lockq =3D {tqe_next =3D 0x0,=20 > tqe_prev =3D 0xffffff86c57b48a0}, td_cpuset =3D 0xffffff0005789dc8,= =20 > td_sel =3D 0xffffff01b5dd0500, td_sleepqueue =3D 0xffffff0396ebe800,=20 > td_turnstile =3D 0xffffff01334cf600, td_umtxq =3D 0xffffff0396ec3a80,= =20 > td_tid =3D 100763, td_sigqueue =3D {sq_signals =3D {__bits =3D {0, 0, 0= , 0}},=20 > sq_kill =3D {__bits =3D {0, 0, 0, 0}}, sq_list =3D {tqh_first =3D 0x0= ,=20 > tqh_last =3D 0xffffff0396ec5500}, sq_proc =3D 0xffffff03969bf470,= =20 > sq_flags =3D 1}, td_flags =3D 65540, td_inhibitors =3D 0, td_pflags = =3D 0,=20 > td_dupfd =3D 0, td_sqqueue =3D 0, td_wchan =3D 0x0, td_wmesg =3D 0x0,= =20 > td_lastcpu =3D 4 '\004', td_oncpu =3D 4 '\004', td_owepreempt =3D 0 '\0= ',=20 > td_tsqueue =3D 255 '?', td_locks =3D 4, td_rw_rlocks =3D 0, td_lk_slock= s =3D 0,=20 > td_blocked =3D 0x0, td_lockname =3D 0x0, td_contested =3D {lh_first =3D= 0x0},=20 > td_sleeplocks =3D 0xffffffff80ecebf0, td_intr_nesting_level =3D 0,=20 > td_pinned =3D 0, td_ucred =3D 0xffffff007d537b00, td_estcpu =3D 0, td_s= lptick =3D 0,=20 > td_blktick =3D 0, td_ru =3D {ru_utime =3D {tv_sec =3D 0, tv_usec =3D 0}= , ru_stime =3D { > tv_sec =3D 0, tv_usec =3D 0}, ru_maxrss =3D 1864, ru_ixrss =3D 6628= 8,=20 > ru_idrss =3D 1347856, ru_isrss =3D 176768, ru_minflt =3D 263901, ru_m= ajflt =3D 10,=20 > ru_nswap =3D 0, ru_inblock =3D 0, ru_oublock =3D 0, ru_msgsnd =3D 0,= =20 > ru_msgrcv =3D 0, ru_nsignals =3D 0, ru_nvcsw =3D 14937, ru_nivcsw =3D= 3286},=20 > td_incruntime =3D 0, td_runtime =3D 15204044088, td_pticks =3D 15, td_s= ticks =3D 15,=20 > td_iticks =3D 0, td_uticks =3D 0, td_intrval =3D 0, td_oldsigmask =3D {= __bits =3D {0,=20 > 0, 0, 0}}, td_sigmask =3D {__bits =3D {0, 0, 0, 0}}, td_generation = =3D 18223,=20 > td_sigstk =3D {ss_sp =3D 0x0, ss_size =3D 0, ss_flags =3D 4}, td_xsig = =3D 0,=20 > td_profil_addr =3D 0, td_profil_ticks =3D 0,=20 > td_name =3D "top", '\0' , td_fpop =3D 0x0, td_dbgflag= s =3D 0,=20 > td_dbgksi =3D {ksi_link =3D {tqe_next =3D 0x0, tqe_prev =3D 0x0}, ksi_i= nfo =3D { > si_signo =3D 0, si_errno =3D 0, si_code =3D 0, si_pid =3D 0, si_uid= =3D 0,=20 > si_status =3D 0, si_addr =3D 0x0, si_value =3D {sival_int =3D 0,=20 > sival_ptr =3D 0x0, sigval_int =3D 0, sigval_ptr =3D 0x0}, _reason= =3D { > _fault =3D {_trapno =3D 0}, _timer =3D {_timerid =3D 0, _overrun = =3D 0},=20 > _mesgq =3D {_mqd =3D 0}, _poll =3D {_band =3D 0}, __spare__ =3D {= __spare1__ =3D 0,=20 > __spare2__ =3D {0, 0, 0, 0, 0, 0, 0}}}}, ksi_flags =3D 0,=20 > ksi_sigq =3D 0x0}, td_ng_outbound =3D 0, td_osd =3D {osd_nslots =3D 0= ,=20 > osd_slots =3D 0x0, osd_next =3D {le_next =3D 0x0, le_prev =3D 0x0}},= =20 > td_rqindex =3D 32 ' ', td_base_pri =3D 128 '\200', td_priority =3D 128 = '\200',=20 > td_pri_class =3D 3 '\003', td_user_pri =3D 129 '\201',=20 > td_base_user_pri =3D 129 '\201', td_pcb =3D 0xffffff86c6389d10,=20 > td_state =3D TDS_RUNNING, td_retval =3D {0, 34375032832}, td_slpcallout= =3D { > c_links =3D {sle =3D {sle_next =3D 0x0}, tqe =3D {tqe_next =3D 0x0,= =20 > tqe_prev =3D 0xffffff800042ccd0}}, c_time =3D 51568077,=20 > c_arg =3D 0xffffff0396ec5460, c_func =3D 0xffffffff806a84c0 ,=20 > c_lock =3D 0x0, c_flags =3D 18, c_cpu =3D 4}, td_frame =3D 0xffffff86= c6389c50,=20 > td_kstack_obj =3D 0xffffff03410b20d8, td_kstack =3D 1844674355304912486= 4,=20 > td_kstack_pages =3D 4, td_unused1 =3D 0x0, td_unused2 =3D 0, td_unused3= =3D 0,=20 > td_critnest =3D 0, td_md =3D {md_spinlock_count =3D 0, md_saved_flags = =3D 70},=20 > td_sched =3D 0xffffff0396ec5890, td_ar =3D 0x0, td_syscalls =3D 469926,= =20 > td_lprof =3D {{lh_first =3D 0x0}, {lh_first =3D 0x0}}, td_dtrace =3D 0x= 0,=20 > td_errno =3D 0, td_vnet =3D 0x0, td_vnet_lpush =3D 0x0, td_rux =3D { > rux_runtime =3D 15204044088, rux_uticks =3D 226, rux_sticks =3D 1140,= =20 > rux_iticks =3D 0, rux_uu =3D 0, rux_su =3D 0, rux_tu =3D 0},=20 > td_map_def_user =3D 0x0, td_dbg_forked =3D 0} > (kgdb) f 11 > #11 0xffffffff8065f6a6 in sysctl_out_proc_copyout (ki=3D0xffffff86c638947= 0,=20 > req=3D0xffffff86c63899c0) at /usr/src/sys/kern/kern_proc.c:1085 > 1085 error =3D SYSCTL_OUT(req, ki, sizeof(struct kinfo_proc)); > (kgdb) print *req > $3 =3D {td =3D 0xffffff0396ec5460, lock =3D 2, oldptr =3D 0x800e96000, ol= dlen =3D 68217,=20 > oldidx =3D 1088, oldfunc =3D 0xffffffff80675e80 , newp= tr =3D 0x0,=20 > newlen =3D 0, newidx =3D 0, newfunc =3D 0xffffffff80675d10 ,=20 > validlen =3D 68217, flags =3D 0} > (kgdb) quit >=20 > -- Hiroki I can see the race in how the wiring of the sysctl buffers is done, but the race can only realize for the multithreaded process. Can you, please, further show me two things: - the p/x *(td->td_pcb) - (this is somewhat laborous) Please find the vm map entry in the process vm_map which covers the range [0x800e96000, 0x800ea6a79) and print it out. You need to walk the td->td_proc->p_vmspace.vm_map.header list using the next link, looking for the entry start/end values. --sEASj6BbPXAOAu+u Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (FreeBSD) iEYEARECAAYFAk9M0dYACgkQC3+MBN1Mb4i6LACcDG0tVBwEKUVuW19H7LVlPDXx uxsAoLa6r2njpLUhYaUbhhrHc3eiQ9UE =VBMZ -----END PGP SIGNATURE----- --sEASj6BbPXAOAu+u--