From owner-freebsd-arm@freebsd.org Fri Mar 2 17:04:58 2018 Return-Path: Delivered-To: freebsd-arm@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 96CE7F34E6B for ; Fri, 2 Mar 2018 17:04:58 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-io0-x236.google.com (mail-io0-x236.google.com [IPv6:2607:f8b0:4001:c06::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 198FF6C02E for ; Fri, 2 Mar 2018 17:04:58 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-io0-x236.google.com with SMTP id q24so11277260ioh.8 for ; Fri, 02 Mar 2018 09:04:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=AJzVSU6SwnUG1b4Tp7f+n2krdNybQXCFERjBseFAa5Q=; b=IfXdmq9YRigWva1/pkyVta8biSnABNAxDlZa29yIeWhZGAoJNTM9PdsVh7C+gdLn8K j/UYQS/zbDWi3n3rQ3hstQnoXo3jE7mF5AE8NzY8Rnfe5b5ICYwKH/Jsp19cj6mOgApw sbxSXEaJFimxHM1cadjyV6gXpaTeYOwTQmcb5bsrancc+eUgfpA7qDyH428tRgPpx6Rj BJiw7ML3WA8yBSDpTkND6iMTYC9/Gf3FPuWGkjYUC/+IgQeHT5/xtmfQDTk8ZwySqZp9 TWFdBEoDBUezrsYBizchXqGu+ha4Gxo+oQdqCPbDgg5TvQ0tRI8OB6yOJ+5V7vTaEA5+ C9TA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=AJzVSU6SwnUG1b4Tp7f+n2krdNybQXCFERjBseFAa5Q=; b=mUVC6n7kA/ytYHofZD5J+gEhCSvVEEQiJeWQPA5ysIIYWLjjmXOsMm1JDEdI3dPeDt 0QGvixwUFT1dmAR9FMwWt3fbnHEUcMJBBAXivHEdZKMXu6QB+blfluQfd9Cxs+7ww6KD aMsBeFsLUPluh3zKykeV+Uzs8ZM7Yx3pQ93ztlWhvQZKs0TU4+rXKeIdDOq4jK3kCg2N o+8gzhIFM1bjBGKDINfyCJ1JPUXYP46qOUCjAQ8nH4QvuZb5IXU5Vku1fyBZ2YIvQgrF I9VCwPe0e9fZQs0j7pTghMRcOyKJqMfpidLMYCF0eG18CYTvb9iElSi2UoPOADkMUY5l HsTQ== X-Gm-Message-State: AElRT7FQCej2I49RXG59lwzHf6feHD8Izq4E6b4UXzbhG/TQjaeNgYj7 mOuH7gczlN9QUJZMF3cJAtXzDZgLqzHe38QIbanHbA== X-Google-Smtp-Source: AG47ELvn63ZsOPBlLstN4BYBEP8foYUdFN3lA5h468zRr6ipYoR/SUnvDBMQPzXIelaCMQwQd83C6bvGIuWgl5fVziQ= X-Received: by 10.107.142.79 with SMTP id q76mr7297817iod.299.1520010297310; Fri, 02 Mar 2018 09:04:57 -0800 (PST) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 10.79.203.196 with HTTP; Fri, 2 Mar 2018 09:04:56 -0800 (PST) X-Originating-IP: [50.227.106.226] In-Reply-To: References: <20180228170311.GA26187@www.zefox.net> <20180228185517.GB26187@www.zefox.net> <8f422161-885e-aa91-eacd-018540222d65@mgm51.com> <20180228214301.GA29481@www.zefox.net> From: Warner Losh Date: Fri, 2 Mar 2018 10:04:56 -0700 X-Google-Sender-Auth: x2SorJgQEtw4G54r-SmrXIr6-ps Message-ID: Subject: Re: Is maximum swap usage tunable? To: Mike Cc: "freebsd-arm@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Mar 2018 17:04:58 -0000 On Fri, Mar 2, 2018 at 9:19 AM, Mike wrote: > On 2/28/2018 4:43 PM, bob prohaska wrote: > > On Wed, Feb 28, 2018 at 02:37:36PM -0500, Mike wrote: > >> On 2/28/2018 1:55 PM, bob prohaska wrote: > >>> On Wed, Feb 28, 2018 at 12:20:56PM -0500, Mike wrote: > >>>> On 2/28/2018 12:03 PM, bob prohaska wrote: > >>>>> In watching system compilations on an RPi3 it looks as if the > >>>>> system starts killing processes with "out of swap" warnings > >>>>> well below 50% of full utilization (in this case, 2 GB). One > >>>>> recent instance of make -j4 kernel-toolchain killed llvm-tblgen > >>>>> with only 34% of the swap in use. > >>>>> > >>>>> Is the maximum swap usage limit adjustable in any way? I didn't > >>>>> recognize anything useful in the page at > >>>>> https://www.freebsd.org/cgi/man.cgi?query=sysctl(8)& > sektion=&manpath=freebsd-release-ports > >>>>> > >>>>> It's possible the problem is really swap speed, rather than size, so > I'd > >>>>> like to try changing size limits if possible. The swap media claims > 2-3 > >>>>> MB/sec random write speed and observations with gstat seem to > support the > >>>>> claim, but transient stalls are hard to observe. An RPi2 with similar > >>>>> hardware seems to have no problems. > >>>>> > >>>> > >>>> > >>>>> It's possible the problem is really swap speed > >>>> > >>>> I was running into swap speed / timeout issues. There were messages > on > >>>> the console to that effect. > >>>> > >>>> Once I put the swap space on rotating rust, that part of the compile > >>>> problem disappeared. I use 1GB swap space. > >>>> > >>> > >>> The latest kernel versions seem to have largely done away with the > >>> "indefinite wait buffobj" warnings. They're few and far between, > >>> the compile proceeds unless they're abundant. The fact that armv7 > >>> has no problem, and the system is reporting "out of swap" with 34% > >>> in use strikes me as suspicious. Kernel and userland are in sync, > >>> so the figure of 34% swap usage is probably accurate. > >>> > >>> Perhaps my question is better phrased thus: How does FreeBSD-arm > >>> determine when it's out of swap? > >>> > >> > >> > >> Thanks for the follow-up. > >> > >> I was planning to download and try the > >> > >> RPI3-20180226-r330034 > >> > >> image file this evening. Per your comment, I'll not use the rusty swap > >> space, and see what happens. > >> > >> I'll report back, probably on the morrow... > >> > > FWIW, I've been using Sandisk Extreme USB 3 flash drives for > > /usr /var/ and swap on the Pi3. It seems that flash write > > speed is a tighter bottleneck than USB 2.0 ports on the Pi. > > Attempts to use older, slower USB 2 flash drives on a Pi2 > > didn't work, though the symptoms were never "out of swap". > > > > buildworld is still running. I started it around 3PM (EST) on March 1. > > I'm seeing a lot of swap messages on the console. The text is: > > swap_pager: indefinite wait buffer: bufobj: 0, > blkno: [varies], size: [varies] This message comes from the swap pager when the I/O doesn't complete quickly enough. It has nothing to do with sizes of the swap space. I've seen three causes for this: (1) The driver going crazy and not completing I/Os it gets notified as being done; (2) The driver not having a backstop timer on I/Os to fail them with timeout; (3) timeouts not firing for the driver to fail the I/O up the stack. (4) when a drive suddenly slow way way down (~100x or more slower), the I/O queues balloons and you can get these. Warner