From owner-freebsd-arch@freebsd.org Thu Nov 30 18:37:37 2017 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7E163E685D8 for ; Thu, 30 Nov 2017 18:37:37 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-it0-x234.google.com (mail-it0-x234.google.com [IPv6:2607:f8b0:4001:c0b::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4B5BE6FA4A for ; Thu, 30 Nov 2017 18:37:37 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-it0-x234.google.com with SMTP id b5so9478263itc.3 for ; Thu, 30 Nov 2017 10:37:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=1nvLGoT174xftLsApbXJjVlzTBA7FKzLmGcOtUJ7q8A=; b=vfSJf4cu7SSdVrMrwgleFWH54/ABDRxXJVZhnWNRodcdgk8I2xz89vsdWYzAhSSnNd r+jtEi5D0lPZQgnSuTY/49V/BUo3lENDDzAzvdUK5qUonGfnbIpNoMS1AoOREkMcmM7M U73wqrGxWVDfMyxGPLhKTXzQBUxKFvc1xhsMyF6yWGWG6v3VIvQZVxB36DPNZqNQPXj/ R8wiwtcVvxZLnRR3QbyVH3eauVSqYN66yCuxmrNrn+QvSmR8AeRuYcrAWpsw4zE//Uyd h0eWV35m6LcdwqfHtzRFvt6KkI2WJNy5Z2fvDbsmWhzi0q/ZjIuGCGZYAg6elFXfKEJA Ygeg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=1nvLGoT174xftLsApbXJjVlzTBA7FKzLmGcOtUJ7q8A=; b=d5z5NrTJOv/jjwi586k/B0nHB5ceuZ1PjrtBmO1oWylZQuYzouds1U0430OlSKC1Lm c8+uwzExbykRXT23QKe+D4bEZwR01RGQPaVlijzWR7btIwdstfQlJmJh915zQMHV+GDm UBsto1Vs0nNZS0wFUcrIIJrCxmognUgvDKp4Kt+eQUiABpa/+MZcUXUx/GFRtRCzKeHV VUJmyUYztAbSZsNrFBxyR6JBI3TqiTPjuBLY9JcaLEFIYvk+IhXIpZawNM4NTsbIe7ph FkAFB0ct0F7n69piG7v79BCqiWXKzqoCHiBYOaS3EZqiC/Tz3VjVfqfua9t9LvcUNHjD YVPg== X-Gm-Message-State: AJaThX6B4krCSHWeNJOOaS1HazDhAOaBYqNcVbvPvycWCLizkM5eKuvZ cUG/6+wNDdd78wudYFQ6uhv4LosonVgKkLmeNfYq0Q== X-Google-Smtp-Source: AGs4zMY8+YyyeyxLG+xun+94JTelh8lrlsCflIQbd6I2ON2VPLd36GmidCxXpEc/u7sUSjaKHrqp43G77hZfYnKSyBg= X-Received: by 10.36.164.13 with SMTP id z13mr3805573ite.115.1512067056174; Thu, 30 Nov 2017 10:37:36 -0800 (PST) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 10.79.108.204 with HTTP; Thu, 30 Nov 2017 10:37:35 -0800 (PST) X-Originating-IP: [2603:300b:6:5100:1052:acc7:f9de:2b6d] In-Reply-To: <20171130173424.GA811@mcvoy.com> References: <20171130173424.GA811@mcvoy.com> From: Warner Losh Date: Thu, 30 Nov 2017 11:37:35 -0700 X-Google-Sender-Auth: wNc2l5AtKQooTpvvWkMH-S_vUcw Message-ID: Subject: Re: small patch for pageout. Comments? To: Larry McVoy Cc: "freebsd-arch@freebsd.org" , Scott Long , Kevin Bowling , Drew Gallatin Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Nov 2017 18:37:37 -0000 On Thu, Nov 30, 2017 at 10:34 AM, Larry McVoy wrote: > In a recent numa meeting that Scott called, Jeff suggested a small > patch to the pageout daemon (included below). > > It's rather dramatic the difference it makes for me. If I arrange to > thrash the crap out of memory, without this patch the kernel is so > borked with all the processes in disk wait that I can't kill them, > I can't reboot, my only option is to power off. > > With the patch there is still some borkage, the kernel is randomly > killing processes because of out of mem, it should kill one of my > processes that is causing the problem but it doesn't, it killed > random stuff like dhclient, getty (logged me out), etc. > > But the system is responsive. > > What the patch does is say "if we have more than one core, don't sleep > in pageout, just keep running until we freed enough mem". > > Comments? > Just to confirm why this patch works. For UP systems, we have to pause here to allow work to complete, otherwise we can't switch to their threads to complete the I/Os. For MP, however, we can continue to schedule more work because that work can be completed on other CPUs. This parallelism greatly increases the pageout rate, allowing the system to keep up better when some ass-hat process (or processes) is thrashing memory. I'm pretty sure the UP case was also designed to not flood the lower layers with work, starving other consumers. Does this result in undo flooding, and would we get better results if we could schedule up to the right amount of work rather flooding in the MP case? Warner --lm > > diff --git a/sys/vm/vm_pageout.c b/sys/vm/vm_pageout.c > index 4ecae5ad5fd..f59a09e96e2 100644 > --- a/sys/vm/vm_pageout.c > +++ b/sys/vm/vm_pageout.c > @@ -1815,10 +1815,18 @@ vm_pageout_worker(void *arg) > * (page reclamation) scan, then increase the level > * and scan again now. Otherwise, sleep a bit and > * try again later. > + * LM: per discussions with the numa team, don't > + * sleep if we have at least 2 cpus, just keep > + * scanning. This makes a HUGE difference when > + * the system is thrashing on memory, it's the > + * difference between usable and borked. > */ > mtx_unlock(&vm_page_queue_free_mtx); > - if (pass >= 1) > - pause("psleep", hz / VM_INACT_SCAN_RATE); > + if (pass >= 1) { > + if (mp_ncpus < 2) { > + pause("psleep", hz > /VM_INACT_SCAN_RATE); > + } > + } > pass++; > } else { > /* > _______________________________________________ > freebsd-arch@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" >