From owner-freebsd-ppc@freebsd.org Wed Mar 6 21:19:24 2019 Return-Path: Delivered-To: freebsd-ppc@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7A4A81527B87 for ; Wed, 6 Mar 2019 21:19:24 +0000 (UTC) (envelope-from chmeeedalf@gmail.com) Received: from mail-it1-x130.google.com (mail-it1-x130.google.com [IPv6:2607:f8b0:4864:20::130]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5AFB170FD8 for ; Wed, 6 Mar 2019 21:19:23 +0000 (UTC) (envelope-from chmeeedalf@gmail.com) Received: by mail-it1-x130.google.com with SMTP id z124so12484007itc.2 for ; Wed, 06 Mar 2019 13:19:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version; bh=lSs5w0nLyJhPzq7wVd3HFkCHporylXGoqeKvi0Ux2rc=; b=Kllq7l6N9CdjGS0BJfqfTMfR4r+chOIFfy1XlsDp504Yi2jlfjampuEiX6KP3R5aeh SAvz8NVOVtyDevTZWMhxJjpE+4gDdUHJAJKRmNCcOrZac6UDlWFO7OmLhhqqEIHozkpD gFVhQcbn6vGM1PCIpuKa+KZQwvrpCA4j3WpOzeSq3P5HhAUK8gdPjSxzcrCSCDtoFzdR +P9v438p22RnWmatjU7+QaWKyM6PBMHOLt2P1keRKmV0AGTAcSxVzuO/ly8AQiQF1ylP D6Bm6fnMdfKA+174fznI4WefjldAse+2FL20lnYfcfm4x7d8VpFJGQ4mltJ00if6QqAZ 4UpQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version; bh=lSs5w0nLyJhPzq7wVd3HFkCHporylXGoqeKvi0Ux2rc=; b=L7gPKhCdnGyYNzW1W00ukr9241Src8hgqCjny5hm/oi8/y2foIECvg6N8L3Fukhroa MeSmInyDsQdN8ZPgL6kyW2yrmhelT6ywA0aJPcau+Je0iY1jIw/HdiIQHQExJXfQpniJ G2IOdez5yGX4yC0Lo+ZqNllv6AIWJV9mVsJ4eLn4rri/boRZPHqG4157QKHHPJOyO8zZ 8noWuTjvyVYouBGT+Wf0z+p1UOto5RgLaMBVPsob1548fp1pFNAcjXN1zJh19xtwTmNh VgUpOtL2/+FaUUHPN8qYZJI70q0iQYTt9kJj/3KSKybG+kMLoeG35yPEKQCOlXgeQSbb aFPw== X-Gm-Message-State: APjAAAVyn50dj6pnC32rfAFduTJJtk6Go09k8gc7///gsT2Q2vFxknXl TWfaaxScSDOH3dbsaD0E6Ruc7WXN X-Google-Smtp-Source: APXvYqzWdp6Qde6XkOh8TUPO+kWBFSPmyPpIvtBHwkoAjju1XjbKPJX6bxLe+hamGMHnzRB65WrWoA== X-Received: by 2002:a24:7c46:: with SMTP id a67mr3151684itd.171.1551907162293; Wed, 06 Mar 2019 13:19:22 -0800 (PST) Received: from titan.knownspace (173-25-245-129.client.mchsi.com. [173.25.245.129]) by smtp.gmail.com with ESMTPSA id r2sm1266822itk.5.2019.03.06.13.19.21 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 06 Mar 2019 13:19:21 -0800 (PST) Date: Wed, 6 Mar 2019 15:19:14 -0600 From: Justin Hibbits To: freebsd-ppc Subject: Re: head -r344018 powerpc64 variant on Powermac G5 (2 sockets, 2 cores each): [*buffer arena] shows up more . . .? Message-ID: <20190306151914.44ea831c@titan.knownspace> In-Reply-To: References: X-Mailer: Claws Mail 3.17.3 (GTK+ 2.24.32; powerpc64-portbld-freebsd13.0) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="MP_/95kzJ17J+e.fHE.d+h6njCd" X-Rspamd-Queue-Id: 5AFB170FD8 X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=Kllq7l6N; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of chmeeedalf@gmail.com designates 2607:f8b0:4864:20::130 as permitted sender) smtp.mailfrom=chmeeedalf@gmail.com X-Spamd-Result: default: False [-5.36 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; HAS_ATTACHMENT(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-0.88)[-0.881,0]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; SUBJECT_ENDS_QUESTION(1.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; MIME_UNKNOWN(0.10)[text/x-patch]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/mixed,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-ppc@freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[0.3.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_CC(0.00)[yahoo.com]; IP_SCORE(-2.57)[ip: (-8.03), ipnet: 2607:f8b0::/32(-2.70), asn: 15169(-2.04), country: US(-0.07)] X-BeenThere: freebsd-ppc@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Porting FreeBSD to the PowerPC List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Mar 2019 21:19:24 -0000 --MP_/95kzJ17J+e.fHE.d+h6njCd Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Disposition: inline On Mon, 4 Mar 2019 19:43:09 -0800 Mark Millard via freebsd-ppc wrote: > [It is possible that the following is tied to my hack to > avoid threads ending up stuck-sleeping. But I ask about > an alternative that I see in the code.] > > Context: using the modern powerpc64 VM_MAX_KERNEL_ADDRESS > and using usefdt=1 on an old Powermac G5 (2 sockets, 2 cores > each). Hacks are in use to provide fairly reliable booting > and to avoid threads getting stuck sleeping. > > Before the modern VM_MAX_KERNEL_ADDRESS figure there were only > 2 or 3 bufspacedaemon-* threads as I remember. Now there are 8 > (plus bufdaemon and its worker), for example: > > root 23 0.0 0.0 0 288 - DL 15:48 0:00.39 > [bufdaemon/bufdaemon] root 23 0.0 0.0 0 288 - DL > 15:48 0:00.05 [bufdaemon/bufspaced] root 23 0.0 > 0.0 0 288 - DL 15:48 0:00.05 [bufdaemon/bufspaced] > root 23 0.0 0.0 0 288 - DL 15:48 0:00.05 > [bufdaemon/bufspaced] root 23 0.0 0.0 0 288 - DL > 15:48 0:00.05 [bufdaemon/bufspaced] root 23 0.0 > 0.0 0 288 - DL 15:48 0:00.05 [bufdaemon/bufspaced] > root 23 0.0 0.0 0 288 - DL 15:48 0:00.07 > [bufdaemon/bufspaced] root 23 0.0 0.0 0 288 - DL > 15:48 0:00.05 [bufdaemon/bufspaced] root 23 0.0 > 0.0 0 288 - DL 15:48 0:00.56 [bufdaemon// worker] > > I'm sometimes seeing processes showing [*buffer arena] that > seemed to wait for a fairly long time with that status, not > something I'd seen historically for those same types of > processes for a similar overall load (not much). During such > times trying to create processes to look around at what is > going on seems to also wait. (Probably with the same status?) > Hi Mark, Can you try the attached patch? It might be overkill in the synchronization, and I might be using the wrong barriers to be considered correct, but I think this should narrow the race down, and synchronize the timebases to within a very small margin. The real correct fix would be to suspend the timebase on all cores, which is feasible (there's a GPIO for the G4s, and i2c for G5s), but that's a non-trivial extra work. Be warned, I haven't tested it, I've only compiled it (I don't have a G5 to test with anymore). - Justin --MP_/95kzJ17J+e.fHE.d+h6njCd Content-Type: text/x-patch Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=powermac_tb_sync.diff diff --git a/sys/powerpc/powermac/platform_powermac.c b/sys/powerpc/powermac/platform_powermac.c index fe818829dc7..b5d34ef90c3 100644 --- a/sys/powerpc/powermac/platform_powermac.c +++ b/sys/powerpc/powermac/platform_powermac.c @@ -41,6 +41,7 @@ __FBSDID("$FreeBSD$"); #include #include /* For save_vec() */ +#include #include #include #include /* For save_fpu() */ @@ -396,6 +397,19 @@ powermac_smp_start_cpu(platform_t plat, struct pcpu *pc) static void powermac_smp_timebase_sync(platform_t plat, u_long tb, int ap) { + static int cpus; + static int unleash; + + if (ap) { + atomic_add_int(&cpus, 1); + while (!atomic_load_acq_int(&unleash)) + ; + } else { + atomic_add_int(&cpus, 1); + while (atomic_load_int(&cpus) != mp_ncpus) + ; + atomic_store_rel_int(&unleash, 1); + } mttb(tb); } --MP_/95kzJ17J+e.fHE.d+h6njCd--