From owner-svn-src-all@freebsd.org Mon Nov 2 21:43:41 2015 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 83C29A253F6; Mon, 2 Nov 2015 21:43:41 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-ig0-x230.google.com (mail-ig0-x230.google.com [IPv6:2607:f8b0:4001:c05::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FFC11B24; Mon, 2 Nov 2015 21:43:41 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: by igdg1 with SMTP id g1so67156540igd.1; Mon, 02 Nov 2015 13:43:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=kWN42B0i8Y29Zzj246iyryxhO6Cizj59FKHxSAcNfL0=; b=KGi7+h7gncTARfqx0+ePf3037kMHOSAA+kJz3vgk+doKF1Xpuw0ViAp4WJ9pvQ5DAj V5A/rz5ZJHYNFA/VUOgaqEZSIdoxYEhT2hzJgiESjem9KfLC70CES6+Irn2ph9OOOHde pPOql2PFYvaBWRbuMClMYN4abZBiARtWB8Zzpk3E9Eg8sQjmId/qg3pKz0GnhTzpeO3K kS51erAeMoENQPWj7xRnV++2CrHgSPxqDVyG9sPs2Z7U6WRduILuIrv2HpDqp9dNifTn au3x8d1lRYcR7ZGrbtXFf6SxqdnqfKXQLMGO7mPO5j17lTXYVDn94XxBVsSCk7yqY/l4 INtA== MIME-Version: 1.0 X-Received: by 10.50.155.41 with SMTP id vt9mr13040987igb.22.1446500620767; Mon, 02 Nov 2015 13:43:40 -0800 (PST) Received: by 10.36.46.66 with HTTP; Mon, 2 Nov 2015 13:43:40 -0800 (PST) In-Reply-To: <20151029212554.799f76eb@kalimero.tijl.coosemans.org> References: <201510140210.t9E2A79H056595@repo.freebsd.org> <20151029212554.799f76eb@kalimero.tijl.coosemans.org> Date: Mon, 2 Nov 2015 13:43:40 -0800 Message-ID: Subject: Re: svn commit: r289279 - in head/sys: kern vm From: Adrian Chadd To: Tijl Coosemans Cc: Jeff Roberson , "src-committers@freebsd.org" , "svn-src-all@freebsd.org" , "svn-src-head@freebsd.org" , Konstantin Belousov Content-Type: text/plain; charset=UTF-8 X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Nov 2015 21:43:41 -0000 Hi, try this? adrian@victoria:~/work/freebsd/head-embedded/src/sys/kern % svn diff vfs_bio.c Index: vfs_bio.c =================================================================== --- vfs_bio.c (revision 290048) +++ vfs_bio.c (working copy) @@ -3623,6 +3623,13 @@ if (bp == NULL) { if (slpflag || slptimeo) return NULL; + /* + * Yield so the bufdaemon has a chance to run and + * clean some buffers; otherwise we'll get stuck in + * a tight loop with no sleeping; we'll wake up the + * buffer daemon but never give it a slice to run. + */ + kern_yield(PRI_USER); goto loop; } -a On 29 October 2015 at 13:25, Tijl Coosemans wrote: > On Wed, 14 Oct 2015 02:10:07 +0000 (UTC) Jeff Roberson wrote: >> Author: jeff >> Date: Wed Oct 14 02:10:07 2015 >> New Revision: 289279 >> URL: https://svnweb.freebsd.org/changeset/base/289279 >> >> Log: >> Parallelize the buffer cache and rewrite getnewbuf(). This results in a >> 8x performance improvement in a micro benchmark on a 4 socket machine. >> >> - Get buffer headers from a per-cpu uma cache that sits in from of the >> free queue. >> - Use a per-cpu quantum cache in vmem to eliminate contention for kva. >> - Use multiple clean queues according to buffer cache size to eliminate >> clean queue lock contention. >> - Introduce a bufspace daemon that attempts to prevent getnewbuf() callers >> from blocking or doing direct recycling. >> - Close some bufspace allocation races that could lead to endless >> recycling. >> - Further the transition to a more modern style of small functions grouped >> by prefix in order to improve growing complexity. > > I have an i386 system that locks up easily after this commit. Booting > into single user and running make installkernel triggers it consistently. > I haven't been able to reproduce it on amd64. Examining threads with > DDB shows that they are all at sched_switch (a few at fork_trampoline). > The only lock being held is Giant (by the interrupt handler for > ctrl+alt+esc I think). So it doesn't look like a dead lock. It's more > a sleeping beauty situation. All threads in the castle are sleeping and > there's no prince to wake them up. > > (kgdb) info thread > Id Target Id Frame > > These are from make installkernel: > > 72 Thread 100071 (PID=107: install) sched_switch (td=0xc667d000, > newtd=0xc6407000, flags=) > at /usr/src/sys/kern/sched_ule.c:1969 > 71 Thread 100070 (PID=81: make) sched_switch (td=0xc667d340, > newtd=0xc667d000, flags=) > at /usr/src/sys/kern/sched_ule.c:1969 > 70 Thread 100067 (PID=30: make) sched_switch (td=0xc667e000, > newtd=0xc667d340, flags=) > at /usr/src/sys/kern/sched_ule.c:1969 > 69 Thread 100066 (PID=25: make) sched_switch (td=0xc667e340, > newtd=0xc667e000, flags=) > at /usr/src/sys/kern/sched_ule.c:1969 > > Single user shell: > > 68 Thread 100065 (PID=17: sh) sched_switch (td=0xc6406000, > newtd=0xc667e340, flags=) > at /usr/src/sys/kern/sched_ule.c:1969 > > Kernel threads: > > 67 Thread 100063 (PID=16: vnlru) sched_switch (td=0xc6406680, > newtd=0xc6407340, flags=) > at /usr/src/sys/kern/sched_ule.c:1969 > 66 Thread 100062 (PID=9: syncer) sched_switch (td=0xc64069c0, > newtd=0xc667d000, flags=) > at /usr/src/sys/kern/sched_ule.c:1969 > 65 Thread 100061 (PID=8: bufspacedaemon) sched_switch (td=0xc6407000, > newtd=0xc62dc000, flags=) > at /usr/src/sys/kern/sched_ule.c:1969 > 64 Thread 100060 (PID=7: bufdaemon) sched_switch (td=0xc6407340, > newtd=0xc6408000, flags=) > at /usr/src/sys/kern/sched_ule.c:1969 > 63 Thread 100068 (PID=7: bufdaemon//var worker) sched_switch ( > td=0xc667d9c0, newtd=0xc6407000, flags=) > at /usr/src/sys/kern/sched_ule.c:1969 > 62 Thread 100069 (PID=7: bufdaemon//usr worker) sched_switch ( > td=0xc667d680, newtd=0xc667d000, flags=) > at /usr/src/sys/kern/sched_ule.c:1969 > 61 Thread 100059 (PID=6: pagezero) sched_switch (td=0xc6407680, > newtd=0xc55ba680, flags=) > at /usr/src/sys/kern/sched_ule.c:1969 > 60 Thread 100058 (PID=5: vmdaemon) sched_switch (td=0xc64079c0, > newtd=0xc6407340, flags=) > at /usr/src/sys/kern/sched_ule.c:1969 > 59 Thread 100057 (PID=4: pagedaemon) sched_switch (td=0xc6408000, > newtd=0xc6407000, flags=) > at /usr/src/sys/kern/sched_ule.c:1969 > 58 Thread 100064 (PID=4: pagedaemon/uma) sched_switch (td=0xc6406340, > newtd=0xc55b9340, flags=) > at /usr/src/sys/kern/sched_ule.c:1969 > 57 Thread 100050 (PID=15: acpi_cooling0) sched_switch (td=0xc62dc340, > newtd=0xc6407000, flags=) > at /usr/src/sys/kern/sched_ule.c:1969 > .... > > Anything else you need to debug this? >