From owner-freebsd-stable@freebsd.org Thu Oct 29 11:47:15 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9308AA20F12 for ; Thu, 29 Oct 2015 11:47:15 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 52379131F for ; Thu, 29 Oct 2015 11:47:14 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (localhost [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id 67CF028435; Thu, 29 Oct 2015 12:47:09 +0100 (CET) Received: from illbsd.quip.test (ip-89-177-49-111.net.upcbroadband.cz [89.177.49.111]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id 6BFDA2842F; Thu, 29 Oct 2015 12:47:03 +0100 (CET) Message-ID: <56320737.2060901@quip.cz> Date: Thu, 29 Oct 2015 12:47:03 +0100 From: Miroslav Lachman <000.fbsd@quip.cz> User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:35.0) Gecko/20100101 Firefox/35.0 SeaMonkey/2.32 MIME-Version: 1.0 To: Jeremy Chadwick CC: freebsd-stable@freebsd.org Subject: Re: stable/10: high load average when box is idle References: <20151027050508.GA7612@icarus.home.lan> <5631EE40.3020404@quip.cz> <20151029100910.GA52255@icarus.home.lan> In-Reply-To: <20151029100910.GA52255@icarus.home.lan> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Oct 2015 11:47:15 -0000 Jeremy Chadwick wrote on 10/29/2015 11:09: > On Thu, Oct 29, 2015 at 11:00:32AM +0100, Miroslav Lachman wrote: >> Jeremy Chadwick wrote on 10/27/2015 06:05: >>> (I am not subscribed to the mailing list, please keep me CC'd) >>> >>> Issue: a stable/10 system that has an abnormally high load average (e.g. >>> 0.15, but may be higher depending on other variables which I can't >>> account for) when the machine is definitely idle (i.e. cannot be traced >>> to high interrupt usage per vmstat -i, cannot be traced to a userland >>> process or kernel thread, etc.). >>> >>> This problem has been discussed many times on the FreeBSD mailing lists >>> and the FreeBSD forum (including some folks seeing it on 9.x, but my >>> complaint here is focused on 10.x so please focus there). >>> >>> I'd politely like to request that anyone experiencing this, or who has >>> experienced it (and if you know when it stopped or why, including what >>> you may have done, include that), to chime in on this ticket from 2012 >>> (made for 9.x but style of issue still applies; c#5 is quite valid): >>> >>> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=173541 >>> >>> For those still experiencing it, I'd suggest reading c#8 and seeing if >>> sysctl kern.eventtimer.periodic=1 relieves the problem for you. (At >>> this time I would not suggest leaving that set indefinitely, as it does >>> seem to increase the interrupt rate in cpuX:timer in vmstat -i. But for >>> me kern.eventtimer.periodic=1 "fixes" the issue) >> >> Is it on real HW server or in some kind of virtualization? I am seeing load >> 0.5 - 1.2 on three virtual machines in VMware. The machines are without any >> traffic. Just fresh instalation of FreeBSD 10.1 and some services without >> any public content. > > I've seen it on both bare-metal and VMs. Please see c#8 in the ticket; > there's an itemised list of where I've seen it, but I'm sure it's not > limited to just those. OK, I have read your c#8 and did some tests on our affected VMs. With sysctl kern.eventtimer.periodic=1 it is better. Where previously load were about 0.40 is 0.15 now. One of these three systems is FreeBSD 10.2 and on this machine the positive effect of kern.eventtimer.periodic=1 is more visible - load is now 0.00 - 0.05. I don't know if this is some coincidence or something is different in 10.2. Settings of kern.eventtimer is the same on all VMs kern.eventtimer.et.LAPIC.flags: 7 kern.eventtimer.et.LAPIC.frequency: 35071418 kern.eventtimer.et.LAPIC.quality: 600 kern.eventtimer.et.i8254.flags: 1 kern.eventtimer.et.i8254.frequency: 1193182 kern.eventtimer.et.i8254.quality: 100 kern.eventtimer.et.RTC.flags: 17 kern.eventtimer.et.RTC.frequency: 32768 kern.eventtimer.et.RTC.quality: 0 kern.eventtimer.choice: LAPIC(600) i8254(100) RTC(0) kern.eventtimer.singlemul: 4 kern.eventtimer.idletick: 0 kern.eventtimer.timer: LAPIC kern.eventtimer.periodic: 1 Miroslav Lachman