From owner-freebsd-current@freebsd.org Sun Mar 26 06:18:20 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 61957D1DBAC for ; Sun, 26 Mar 2017 06:18:20 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from cell.glebi.us (glebi.us [96.95.210.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "cell.glebi.us", Issuer "cell.glebi.us" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 335CD173E for ; Sun, 26 Mar 2017 06:18:19 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from cell.glebi.us (localhost [127.0.0.1]) by cell.glebi.us (8.15.2/8.15.2) with ESMTPS id v2Q6IDZH003640 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sat, 25 Mar 2017 23:18:13 -0700 (PDT) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebi.us (8.15.2/8.15.2/Submit) id v2Q6IDJ3003639; Sat, 25 Mar 2017 23:18:13 -0700 (PDT) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebi.us: glebius set sender to glebius@FreeBSD.org using -f Date: Sat, 25 Mar 2017 23:18:13 -0700 From: Gleb Smirnoff To: Konstantin Belousov Cc: Darren , "freebsd-current@freebsd.org" Subject: Re: r315684 panic: sleepq_add: td 0xfffff80003c01a40 to sleep on wchan 0xfffff80006f0873c with sleeping prohibited Message-ID: <20170326061813.GB23308@FreeBSD.org> References: <1824572972.3096988.1490377215756.ref@mail.yahoo.com> <1824572972.3096988.1490377215756@mail.yahoo.com> <20170325010314.GG43712@kib.kiev.ua> <20170325033142.GA23308@FreeBSD.org> <20170325094529.GH43712@kib.kiev.ua> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="x1F0m3RQhDZyj8sd" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20170325094529.GH43712@kib.kiev.ua> User-Agent: Mutt/1.7.2 (2016-11-26) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Mar 2017 06:18:20 -0000 --x1F0m3RQhDZyj8sd Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit On Sat, Mar 25, 2017 at 11:45:29AM +0200, Konstantin Belousov wrote: K> On Fri, Mar 24, 2017 at 08:31:42PM -0700, Gleb Smirnoff wrote: K> > Darren, K> > K> > On Sat, Mar 25, 2017 at 03:03:14AM +0200, Konstantin Belousov wrote: K> > K> On Fri, Mar 24, 2017 at 05:40:15PM +0000, Darren wrote: K> > K> > I am getting this panic every hour to every couple of hours. K> > K> > K> > K> > FreeBSD asrock 12.0-CURRENT FreeBSD 12.0-CURRENT #0 r315684: Thu Mar 23 14:56:45 EDT 2017     darren@asrock:/usr/obj/usr/src/sys/GENERIC  amd64 K> > K> > I manually typed out the following, apologize for any typos. K> > K> > K> > K> > K> > K> > panic: sleepq_add: td 0xfffff80003c01a40 to sleep on wchan 0xfffff80006f0873c with sleeping prohibited K> > K> > cpuid = 0 K> > K> > time = 1490372797 K> > K> > KDB: stack backtrace: K> > K> > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0072e33690 K> > K> > vpanic() at vpanic+0x19c/frame 0xfffffe0072e33710 K> > K> > kassert_panic() at kassert_panic+0x126/frame 0xfffffe0072e33780 K> > K> > sleepq_add() at sleepq_add+0x34f/frame 0xfffffe0072337d0 K> > K> > _sleep() at _sleep+0x28d/frame 0xfffffe0072e33870 K> > K> > soclose() at soclose+0xda/frame 0xfffffe0072e338b0 K> > K> > _fdrop() at _fdrop+0x1a/frame 0xfffffe0072e338d0 K> > K> > sendfile_iodone() at sendfile_iodone+0x19d/frame 0xfffffe0072e33910 K> > K> > vnode_pager_generic_getpages_done_async() at vnode_pager_generic_getpages_done_async+037/frame 0xfffffe0072e33930 K> > K> > bufdone() at bufdone+0x64/frame 0xfffffe0072e33960 K> > K> > g_io_deliver() at g_io_deliver+0x276/frame 0xfffffe0072e339b0 K> > K> > g_io_deliver() at g_io_deliver+0x276/frame 0xfffffe0072e33a00 K> > K> > g_disk_done() at g_disk_done+0x104/frame 0xfffffe0072e33a40 K> > K> > xpt_done_process() at xpt_done_process+0x35f/frame 0xfffffe0072e33a80 K> > K> > xpt_done_direct() at ahci_ch_intr_direct+0xd5/frame 0xfffffe0072e33af0 K> > K> > ahci_itr() at ahci_intr+0x102/frame 0xfffffe0072e33b20 K> > K> > intr_event_execute_handlers() at intr_event_execute_handlers+0x99/frame 0xfffffe0072e33b60 K> > K> > ithread_loop() at ithread_loop+0xb6/frame 0xfffffe0072e33bb0 K> > K> > fork_exit() at fork_exit+0x84/frame 0xfffffe0072e33bf0 K> > K> > fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0072e33bf0 K> > K> > --- trap 0, rip = 0, rsp = 0, rbp = 0 --- K> > K> > KDB: enter: panic K> > K> > [ thread pid 12 tid 100038 ] K> > K> > Stopped at      kdb_enter+0x3b: movq    $0,kdb_why K> > K> > db> K> > K> K> > K> Indeed, the context where sendfile_iodone() is executed, cannot call fdrop(). K> > K> > Can you please test the attached patch? K> > K> > -- K> > Totus tuus, Glebius. K> K> > Index: sys/kern/kern_sendfile.c K> > =================================================================== K> > --- sys/kern/kern_sendfile.c (revision 315926) K> > +++ sys/kern/kern_sendfile.c (working copy) K> > @@ -296,8 +296,9 @@ sendfile_iodone(void *arg, vm_page_t *pg, int coun K> > CURVNET_RESTORE(); K> > } K> > K> > - /* XXXGL: curthread */ K> > - fdrop(sfio->sock_fp, curthread); K> > + ACCEPT_LOCK(); K> > + SOCK_LOCK(so); K> > + sorele(so); K> > free(sfio, M_TEMP); K> > } K> > K> > @@ -860,7 +861,9 @@ prepend_header: K> > } else { K> > sfio->sock_fp = sock_fp; K> > sfio->npages = npages; K> > - fhold(sock_fp); K> > + SOCK_LOCK(so); K> > + soref(so); K> > + SOCK_UNLOCK(so); K> > error = (*so->so_proto->pr_usrreqs->pru_send) K> > (so, PRUS_NOTREADY, m, NULL, NULL, td); K> > sendfile_iodone(sfio, NULL, 0, 0); K> K> With this patch, what prevents a close of the sfio->sock_fp file, which is K> needed to get the pointer to socket ? You are right, patch is unfinished. Here is better one. -- Totus tuus, Glebius. --x1F0m3RQhDZyj8sd Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="sendfile_sleep.diff" Index: sys/kern/kern_sendfile.c =================================================================== --- sys/kern/kern_sendfile.c (revision 315926) +++ sys/kern/kern_sendfile.c (working copy) @@ -80,7 +80,7 @@ struct sf_io { volatile u_int nios; u_int error; int npages; - struct file *sock_fp; + struct socket *so; struct mbuf *m; vm_page_t pa[]; }; @@ -255,7 +255,7 @@ static void sendfile_iodone(void *arg, vm_page_t *pg, int count, int error) { struct sf_io *sfio = arg; - struct socket *so; + struct socket *so = sfio->so; for (int i = 0; i < count; i++) if (pg[i] != bogus_page) @@ -267,8 +267,6 @@ sendfile_iodone(void *arg, vm_page_t *pg, int coun if (!refcount_release(&sfio->nios)) return; - so = sfio->sock_fp->f_data; - if (sfio->error) { struct mbuf *m; @@ -296,8 +294,9 @@ sendfile_iodone(void *arg, vm_page_t *pg, int coun CURVNET_RESTORE(); } - /* XXXGL: curthread */ - fdrop(sfio->sock_fp, curthread); + ACCEPT_LOCK(); + SOCK_LOCK(so); + sorele(so); free(sfio, M_TEMP); } @@ -858,9 +857,11 @@ prepend_header: error = (*so->so_proto->pr_usrreqs->pru_send) (so, 0, m, NULL, NULL, td); } else { - sfio->sock_fp = sock_fp; + sfio->so = so; sfio->npages = npages; - fhold(sock_fp); + SOCK_LOCK(so); + soref(so); + SOCK_UNLOCK(so); error = (*so->so_proto->pr_usrreqs->pru_send) (so, PRUS_NOTREADY, m, NULL, NULL, td); sendfile_iodone(sfio, NULL, 0, 0); --x1F0m3RQhDZyj8sd--