From owner-freebsd-hackers@freebsd.org Wed Jun 3 20:29:34 2020 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9F0922F28B0 for ; Wed, 3 Jun 2020 20:29:34 +0000 (UTC) (envelope-from gbergling@gmail.com) Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49cgW12MtRz4cjk; Wed, 3 Jun 2020 20:29:33 +0000 (UTC) (envelope-from gbergling@gmail.com) Received: by mail-wr1-x432.google.com with SMTP id r7so3788989wro.1; Wed, 03 Jun 2020 13:29:33 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :references:mime-version:content-disposition:in-reply-to; bh=MeCAR1jSq/2+CsW4OBV/JQveJWpIcRHWkGQzYcklLr4=; b=A9+McFRn6PgoEEQ4ObgsLRbT7SXqEYaf1u09a2JcsvNwtlNeABDAr9pMw7fZy9lGB2 c6MwVqSmDK5BYGHHom1RnSWzN4zzApvslVGPmKNiTc1ueCQg1EGAovuUDtX1EhNzKrcQ vpa3tOZ/1dBUVWoVrNetWenQCUKCK0tOBySMCSohuCl16AFN+aGjQf+eisp9KpC72rSI 5hJN5/aQZOqZ/DjzGD/IE+uUjeZKy5d83MtMcDYS8DnLZwHRaJtkyRL5tuoATbC4B8jM REQfEEHMTcDzAXpXpooMgN2XqF9jvFublyKAvrcTpTNaMS1Ge8P9Vg6L72ly8XE+P1/r KPsw== X-Gm-Message-State: AOAM531q982UK054ddlTTKWEobKb9Oj2lvhYcFkgcKr37zEAHPA9L4uI HTHK1d6QGWyBsOFHOHLvF3M/xj8s X-Google-Smtp-Source: ABdhPJyqXTeLyj3uUjpyhVjqRbUcWzQFEyjbLmgPWJQ4NtHlyhsZxFNDFnycq7Xyuq1aZqotJGUwhQ== X-Received: by 2002:a5d:4948:: with SMTP id r8mr1046330wrs.290.1591216171379; Wed, 03 Jun 2020 13:29:31 -0700 (PDT) Received: from lion.0xfce3.net (p4fd3af72.dip0.t-ipconnect.de. [79.211.175.114]) by smtp.gmail.com with ESMTPSA id l1sm5833144wrb.31.2020.06.03.13.29.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Jun 2020 13:29:30 -0700 (PDT) Sender: Gordon Bergling Date: Wed, 3 Jun 2020 22:29:29 +0200 From: Gordon Bergling To: freebsd-hackers@freebsd.org Cc: Allan Jude Subject: Re: Constant load of 1 on a recent 12-STABLE Message-ID: <20200603202929.GA65032@lion.0xfce3.net> References: <20200603101607.GA80381@lion.0xfce3.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Url: X-Operating-System: FreeBSD 12.1-STABLE amd64 X-Host-Uptime: 10:14PM up 14:57, 4 users, load averages: 1.27, 1.22, 1.17 X-Rspamd-Queue-Id: 49cgW12MtRz4cjk X-Spamd-Bar: / X-Spamd-Result: default: False [0.56 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; NEURAL_HAM_MEDIUM(-0.94)[-0.936]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36:c]; FREEMAIL_FROM(0.00)[googlemail.com]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-0.11)[-0.112]; DMARC_POLICY_QUARANTINE(1.50)[googlemail.com : SPF not aligned (relaxed), DKIM not aligned (relaxed),quarantine]; NEURAL_SPAM_SHORT(0.31)[0.309]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::432:from]; FORGED_SENDER(0.30)[gbergling@googlemail.com,gbergling@gmail.com]; RECEIVED_SPAMHAUS_PBL(0.00)[79.211.175.114:received]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[gbergling@googlemail.com,gbergling@gmail.com]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Jun 2020 20:29:34 -0000 Hi Allan, On Wed, Jun 03, 2020 at 03:13:47PM -0400, Allan Jude wrote: > On 2020-06-03 06:16, Gordon Bergling via freebsd-hackers wrote: > > since a while I am seeing a constant load of 1.00 on 12-STABLE, > > but all CPUs are shown as 100% idle in top. > > > > Has anyone an idea what could caused this? > > > > The load seems to be somewhat real, since the buildtimes on this > > machine for -CURRENT increased from about 2 hours to 3 hours. > > > > This a virtualized system running on Hyper-V, if that matters. > > > > Any hints are more then appreciated. > > > > Kind regards, > > > > Gordon > > Try running 'top -SP' and see if that shows a specific CPU being busy, > or a specific process using CPU time Below is the output of 'top -SP'. The only relevant process / thread that is relatively constant consumes CPU time seams to be 'zfskern'. ----------------------------------------------------------------------------- last pid: 68549; load averages: 1.10, 1.19, 1.16 up 0+14:59:45 22:17:24 67 processes: 2 running, 64 sleeping, 1 waiting CPU 0: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle CPU 1: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle CPU 2: 0.0% user, 0.0% nice, 0.4% system, 0.0% interrupt, 99.6% idle CPU 3: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle Mem: 108M Active, 4160M Inact, 33M Laundry, 3196M Wired, 444M Free ARC: 1858M Total, 855M MFU, 138M MRU, 96K Anon, 24M Header, 840M Other 461M Compressed, 1039M Uncompressed, 2.25:1 Ratio Swap: 2048M Total, 2048M Free PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 11 root 4 155 ki31 0B 64K RUN 0 47.3H 386.10% idle 8 root 65 -8 - 0B 1040K t->zth 0 115:39 12.61% zfskern ------------------------------------------------------------------------------- The only key performance indicator that is relatively high IMHO, for a non-busy system, are the context switches, that vmstat has reported. ------------------------------------------------------------------------------- procs memory page disks faults cpu r b w avm fre flt re pi po fr sr da0 da1 in sy cs us sy id 0 0 0 514G 444M 7877 2 7 0 9595 171 0 0 0 4347 43322 17 2 81 0 0 0 514G 444M 1 0 0 0 0 44 0 0 0 121 40876 0 0 100 0 0 0 514G 444M 0 0 0 0 0 40 0 0 0 133 42520 0 0 100 0 0 0 514G 444M 0 0 0 0 0 40 0 0 0 120 43830 0 0 100 0 0 0 514G 444M 0 0 0 0 0 40 0 0 0 132 42917 0 0 100 -------------------------------------------------------------------------------- Any other ideas what could generate that load? Best regards, Gordon