From owner-freebsd-stable@FreeBSD.ORG Sun Jul 20 14:26:12 2014 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 11802485 for ; Sun, 20 Jul 2014 14:26:12 +0000 (UTC) Received: from smtp1.multiplay.co.uk (smtp1.multiplay.co.uk [85.236.96.35]) by mx1.freebsd.org (Postfix) with ESMTP id 9F5FB2839 for ; Sun, 20 Jul 2014 14:26:11 +0000 (UTC) Received: by smtp1.multiplay.co.uk (Postfix, from userid 65534) id 5879020E7088D; Sun, 20 Jul 2014 14:26:09 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.multiplay.co.uk X-Spam-Level: ** X-Spam-Status: No, score=2.3 required=8.0 tests=AWL,BAYES_00,DOS_OE_TO_MX, FSL_HELO_NON_FQDN_1,HELO_NO_DOMAIN,RDNS_DYNAMIC,STOX_REPLY_TYPE autolearn=no version=3.3.1 Received: from r2d2 (82-69-141-170.dsl.in-addr.zen.co.uk [82.69.141.170]) by smtp1.multiplay.co.uk (Postfix) with ESMTPS id 4DF3B20E7088A; Sun, 20 Jul 2014 14:26:04 +0000 (UTC) Message-ID: <97EA8E571E634DBBAA70F7AA7F0DE97C@multiplay.co.uk> From: "Steven Hartland" To: "Jeremy Chadwick" , References: <20140720062413.GA56318@icarus.home.lan> Subject: Re: Consistently "high" CPU load on 10.0-STABLE Date: Sun, 20 Jul 2014 15:26:02 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Jul 2014 14:26:12 -0000 If you add -H -z to your top command does anything stand out? Regards Steve ----- Original Message ----- From: "Jeremy Chadwick" To: Sent: Sunday, July 20, 2014 7:24 AM Subject: Consistently "high" CPU load on 10.0-STABLE > (Please keep me CC'd as I'm not subscribed to freebsd-stable@) > > Today I took the liberty of upgrading my main home server from > 9.3-STABLE (r268785) to 10.0-STABLE (r268894). The upgrade consisted of > doing a fresh install of 10.0-STABLE on a brand new unused SSD. Most > everything went as planned, barring a couple ports-related anomalies, > and I seemed fairly impressed by the fact that buildworld times had > dropped to 27 minutes and buildkernel to 4 minutes with clang (something > I'd been avoiding like the plague for a long while). Kudos. > > But after an hour or so, I noticed a consistent (i.e. reproducible) > trend: the system load average tends to hang around 0.10 to 0.15 all the > time. There are times where the load drops to 0.03 or 0.04 but then > something kicks it back up to 0.15 or 0.20 and then it slowly levels out > again (over the course of a few minutes) then repeats. > > Obviously this is normal behaviour for a system when something is going > on periodically. So I figured it might have been a userland process > behaving differently under 10.x than 9.x. I let top -a -S -s 1 run and > paid very very close attention to it for several minutes. Nothing. It > doesn't appear to be something userland -- it appears to be something > kernel-level, but nothing in top -S shows up as taking up any CPU time > other than "[idle]" so I have no idea what might be doing it. > > The box isn't doing anything like routing network traffic/NAT, it's pure > IPv4 (IPv6 disabled in world and kernel, and my home network does > basically no IPv6) and sits idle most of the time fetching mail. It > does use ZFS, but not for /, swap, /var, /tmp, or /usr. > > vmstat -i doesn't particularly show anything awful. All the cpuX:timer > entries tend to fluctuate in rate, usually 120-200 or so; I'd expect an > interrupt storm to be showing something in the 1000+ range. > > The only thing I can think of is the fact that the SSD being used has no > 4K quirk entry in the kernel (and its ATA IDENTIFY responds with 512 > logical, 512 physical, even though we know it's 4K). The partitions are > all 1MB-aligned regardless. > > This is all bare-metal, by the way -- no virtualisation involved. > > I do have DTrace enabled/built on this box but I have absolutely no clue > how to go about profiling things. For example maybe output of this sort > would be helpful (but I've no idea how to get it): > > http://lists.freebsd.org/pipermail/freebsd-stable/2014-July/079276.html > > I'm certain I didn't see this behaviour in 9.x so I'd be happy to try > and track it down if I had a little bit of hand-holding. > > I've put all the things I can think of that might be relevant to "system > config/tuning bits" up here: > > http://jdc.koitsu.org/freebsd/releng10_perf_issue/ > > I should note my kernel config is slightly inaccurate (I've removed some > stuff from the file in attempt to rebuild, but building world prior to > kernel failed due to r268896 breaking world, but anyone subscribed here > has already seen the Jenkins job of that ;-) ). > > Thanks. > > -- > | Jeremy Chadwick jdc@koitsu.org | > | UNIX Systems Administrator http://jdc.koitsu.org/ | > | Making life hard for others since 1977. PGP 4BD6C0CB | > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" >