From owner-freebsd-current@freebsd.org Fri Apr 6 15:25:10 2018 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 44FB1F8E369 for ; Fri, 6 Apr 2018 15:25:10 +0000 (UTC) (envelope-from chmeeedalf@gmail.com) Received: from mail-lf0-x232.google.com (mail-lf0-x232.google.com [IPv6:2a00:1450:4010:c07::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A57CA81736; Fri, 6 Apr 2018 15:25:09 +0000 (UTC) (envelope-from chmeeedalf@gmail.com) Received: by mail-lf0-x232.google.com with SMTP id e5-v6so1090084lfb.7; Fri, 06 Apr 2018 08:25:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=72DRAhTNJtiE73UkqYK2PZcQraHGIEh1l+RX7zLyWYA=; b=L5rA2Qzs44EKo04t2SmhYDNZYjXbvFVZgpeRCSPh+dIfHBRIGziKpIPfgtiOOJsxKv D1vKv7Pkk8RN4kruvsc/6FVMyzyFZ+zW7xgpmrUBUNTyn2SA3MuiPR835iTWVKabuAsL CNQWxg4RWSnGwbA2JepSx09dafjTC35NcasigVRZvDeFktHTc6hLkka/gfv5sATfT0Om CMVzQv6oZV5mJaVdXuwmnD9bNeyF9XkZuVx9vLPCdvLmAeCdVRtKzNoS6wE8mBzD8m/e d+6i2dTAzu+zSh6aRbLu2AXbEmvMtOttG8Nl33lCymi4jFk9e3hGc2zS1C9ILUi35y5w PytQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=72DRAhTNJtiE73UkqYK2PZcQraHGIEh1l+RX7zLyWYA=; b=Uu2vO7ldGNmCBycy/Lpt8xMkJPuO2CwScymzJKf3k8YQxhHxWftRj4eSUtB6FekzzU XgpOFCo+RFmnZNcVVy4lDrszdI0U0Cz0Sk2zA7vOZ5fI+gvd0vx/nhUlgtYscHe9PG96 xeV+BecgmkPnNXIHpTDQ3SZi59q82CDPhMM8DI9qkKE2rM4wLkLSnjAmeI6/DGnGJzH5 yj1XgYFqrs+cFgFcaA4yqMQZ4H7VddDYWe+7wQY4zAm/Xihtj2BJADNv9pZALUydctBB WqtddpHcAgy7YZP+Od+lNo1GXVBVUHurhHC+oCWCHoaZQ/DiORDnw+9zht7+QTpR7JTT pzgQ== X-Gm-Message-State: AElRT7FkntggXWIR6mP/1sNm0N9HrSTnZnpcwTVQFkUFYb43pw/Mgihu jgNAck8y+eYqbqxRk3EFBq0oCgVHXwsHjRWPqQU= X-Google-Smtp-Source: AIpwx48f6oZ0+5vMBFFjJcbHLY+Dq8rm6VUi3dCVQnTX/T6SMp2oP2vuHfz7EwRUE+ESlwJ4Rj0IL/zHqC6IyoBipEk= X-Received: by 10.46.112.24 with SMTP id l24mr17310931ljc.104.1523028307358; Fri, 06 Apr 2018 08:25:07 -0700 (PDT) MIME-Version: 1.0 Received: by 10.46.85.13 with HTTP; Fri, 6 Apr 2018 08:25:06 -0700 (PDT) In-Reply-To: <20180406150814.GB10017@raichu> References: <20180320070745.GA12880@server.rulingia.com> <2b3db2af-03c7-65ff-25e7-425cfd8815b6@FreeBSD.org> <1fd2b47b-b559-69f8-7e39-665f0f599c8f@FreeBSD.org> <20180404174949.GA12271@raichu> <20180406150814.GB10017@raichu> From: Justin Hibbits Date: Fri, 6 Apr 2018 10:25:06 -0500 Message-ID: Subject: Re: Strange ARC/Swap/CPU on yesterday's -CURRENT To: Mark Johnston Cc: Jeff Roberson , FreeBSD current Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Apr 2018 15:25:10 -0000 On Fri, Apr 6, 2018 at 10:08 AM, Mark Johnston wrote: > On Fri, Apr 06, 2018 at 12:47:14AM +0000, Justin Hibbits wrote: >> My powerpc64 embedded machine is virtually unusable since these vm changes. >> I tried setting vfs.zfs.arc_free_target as suggested, and that didn't help >> at all. Eventually the machine hangs and just gets stuck in vmdaemon, with >> many processes in wait channel btalloc. > > You don't really have the same symptoms that Don is reporting. Okay. I latched onto the thread because it seemed similar. > Threads being stuck in btalloc implies a KVA shortage. So: > - What do you mean by "stuck in vmdaemon"? The machine hangs, and my ssh sessions get killed. I can't do anything at the console except break into kdb. When I do, the running thread is always vmdaemon. > - Which commits specifically cause problems? Does reverting r329882 fix the > hang? I'll try reverting it and report back. Thankfully I can buildkernel successfully on the machine before it hangs. Can't do more than that, though. > - Can you break to DDB and get "show page" output when the hang occurs? I'll reproduce and get numbers today, but I do know the free_count was high (6 digits), much higher than the free_min. when I checked yesterday. I'm surprised it's running out of KVA, I've never had the problem before with the same workloads, and has ~7.5GB KVA (almost the same size as the total RAM in the machine). > - What is the system doing to cause the hang to occur? Just a simple buildworld with 2 or 3 jobs (tried both). It's 100% reproducible. - Justin