From owner-freebsd-hackers@freebsd.org Tue Jan 2 19:59:20 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6CC51EB6436 for ; Tue, 2 Jan 2018 19:59:20 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: from asp.reflexion.net (outbound-mail-210-138.reflexion.net [208.70.210.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 31C19701D8 for ; Tue, 2 Jan 2018 19:59:19 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: (qmail 10069 invoked from network); 2 Jan 2018 19:59:13 -0000 Received: from unknown (HELO rtc-sm-01.app.dca.reflexion.local) (10.81.150.1) by 0 (rfx-qmail) with SMTP; 2 Jan 2018 19:59:13 -0000 Received: by rtc-sm-01.app.dca.reflexion.local (Reflexion email security v8.40.3) with SMTP; Tue, 02 Jan 2018 14:59:13 -0500 (EST) Received: (qmail 13481 invoked from network); 2 Jan 2018 19:59:13 -0000 Received: from unknown (HELO iron2.pdx.net) (69.64.224.71) by 0 (rfx-qmail) with (AES256-SHA encrypted) SMTP; 2 Jan 2018 19:59:13 -0000 Received: from [192.168.1.25] (c-76-115-7-162.hsd1.or.comcast.net [76.115.7.162]) by iron2.pdx.net (Postfix) with ESMTPSA id 4E89DEC94C7; Tue, 2 Jan 2018 11:59:12 -0800 (PST) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: Re: rpi2 head -r327485 (e.g.): rpi2 leaves one "CPU n" always idle for some boots Date: Tue, 2 Jan 2018 11:59:11 -0800 References: <3258F809-F179-4EB7-857C-74667BECD360@dsl-only.net> To: Freebsd-arm , FreeBSD Hackers , FreeBSD Current In-Reply-To: <3258F809-F179-4EB7-857C-74667BECD360@dsl-only.net> Message-Id: <422E2742-7170-4D1A-894F-F310EE819E3A@dsl-only.net> X-Mailer: Apple Mail (2.3273) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Jan 2018 19:59:20 -0000 On 2018-Jan-2, at 11:48 AM, Mark Millard wrote: > I've seen this over many versions of head for months > but have never managed to find a way to force it to > happen. It just shows up once and a while. >=20 > Thus, I'm just dumping out some top and kernel information > here for reference. I've used: >=20 > openssl speed 1>/dev/null 2>&1 & > openssl speed 1>/dev/null 2>&1 & > openssl speed 1>/dev/null 2>&1 & > openssl speed 1>/dev/null 2>&1 & >=20 > to give the rpi2 4 active processes. Various outputs > are from different times without a reboot between. >=20 > top -CaePores shows the likes of: >=20 > PID USERNAME THR PRI NICE SIZE RES SWAP STATE C TIME = CPU COMMAND > 614 root 1 20 0 10452K 10480K 0K select 1 0:00 = 0.03% /usr/sbin/ntpd -g -c /etc/ntp.conf -p /var/run/ntpd.pid -f = /var/db/ntpd.drift > 661 root 1 52 0 9984K 6132K 0K select 1 0:00 = 0.00% /usr/sbin/sshd > 751 root 1 101 0 7256K 4276K 0K RUN 1 0:28 = 99.57% openssl speed > 750 root 1 100 0 7256K 4276K 0K CPU0 0 0:32 = 94.83% openssl speed > 753 root 1 86 0 7256K 4276K 0K RUN 3 0:13 = 52.36% openssl speed > 752 root 1 86 0 7256K 4276K 0K CPU3 3 0:14 = 46.54% openssl speed > 363 root 1 20 0 6428K 3840K 0K select 3 0:00 = 0.00% /sbin/devd > . . . >=20 > and: >=20 > last pid: 754; load averages: 3.70, 2.38, 1.58 = = up 0+00:16:50 01:59:37 > 21 processes: 5 running, 16 sleeping > CPU 0: 94.9% user, 0.0% nice, 0.0% system, 5.1% interrupt, 0.0% = idle > CPU 1: 99.6% user, 0.0% nice, 0.0% system, 0.4% interrupt, 0.0% = idle > CPU 2: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% = idle > CPU 3: 100% user, 0.0% nice, 0.0% system, 0.0% interrupt, 0.0% = idle > Mem: 12M Active, 1136K Inact, 56M Wired, 30M Buf, 722M Free > Swap: 1536M Total, 6M Free >=20 > =46rom problem boot to problem boot, the CPU that stays > idle has varied but usually has been CPU 2. I've never > seen 2 or more stuck in idle. >=20 > show allpcpu shows the likes of: >=20 > db> show allpcpu > Current CPU: 0 >=20 > cpuid =3D 0 > dynamic pcpu =3D 0x3d2540 > curthread =3D 0xd8478ae0: pid 2032 tid 100150 "openssl" > curpcb =3D 0xd852ae98 > fpcurthread =3D 0xd8478ae0: pid 2032 "openssl" > idlethread =3D 0xc376fae0: tid 100002 "idle: cpu0" > curpmap =3D 0xd8e43bf4 > curvnet =3D 0 >=20 > cpuid =3D 1 > dynamic pcpu =3D 0x3998540 > curthread =3D 0xd7e5b3a0: pid 2031 tid 100173 "openssl" > curpcb =3D 0xda7e0e98 > fpcurthread =3D 0xd7e5b3a0: pid 2031 "openssl" > idlethread =3D 0xc376f740: tid 100003 "idle: cpu1" > curpmap =3D 0xd8e43ec4 > curvnet =3D 0 >=20 > cpuid =3D 2 > dynamic pcpu =3D 0x3999540 > curthread =3D 0xc376f3a0: pid 10 tid 100004 "idle: cpu2" > curpcb =3D 0xc378ae98 > fpcurthread =3D none > idlethread =3D 0xc376f3a0: tid 100004 "idle: cpu2" > curpmap =3D 0 > curvnet =3D 0 >=20 > cpuid =3D 3 > dynamic pcpu =3D 0x399a540 > curthread =3D 0xd8477000: pid 2034 tid 100167 "openssl" > curpcb =3D 0xd876de98 > fpcurthread =3D 0xd8477000: pid 2034 "openssl" > idlethread =3D 0xc376f000: tid 100005 "idle: cpu3" > curpmap =3D 0xc377ab04 > curvnet =3D 0 >=20 > In other words: it appears that the cpuN (here cpu2) is > left with idle scheduled all the time for some reason. >=20 > ps from db> shows things like: >=20 >=20 > db> ps > pid ppid pgrp uid state wmesg wchan cmd > 2034 714 2034 0 R+ openssl > 2033 714 2033 0 R+ CPU 3 openssl > 2032 714 2032 0 R+ CPU 0 openssl > 2031 714 2031 0 R+ CPU 1 openssl >=20 > (then later:) >=20 > db> ps > pid ppid pgrp uid state wmesg wchan cmd > 2034 714 2034 0 R+ CPU 3 openssl > 2033 714 2033 0 R+ openssl > 2032 714 2032 0 R+ CPU 0 openssl > 2031 714 2031 0 R+ CPU 1 openssl >=20 > There is also: >=20 > 10 0 0 0 RL (threaded) [idle] > 100002 CanRun [idle: cpu0] > 100003 CanRun [idle: cpu1] > 100004 CanRun [idle: cpu2] > 100005 CanRun [idle: cpu3] >=20 >=20 > These are from: >=20 > # uname -apKU > FreeBSD rpi2 12.0-CURRENT FreeBSD 12.0-CURRENT r327485M arm armv7 = 1200054 1200054 I probably should have reported that as I remember the following sort of thing is true for the problem cpuN (here cpu2): 100074 D - 0xd6f5be80 [softirq_0] 100075 D - 0xd6f5be00 [softirq_1] 100076 RunQ [softirq_2] 100077 D - 0xd6f5bd00 [softirq_3] 100078 D - 0xd6f5bc80 [if_io_tqg_0] 100079 D - 0xd6f5bc00 [if_io_tqg_1] 100080 RunQ [if_io_tqg_2] 100081 D - 0xd6f5bb00 [if_io_tqg_3] =3D=3D=3D Mark Millard markmi at dsl-only.net