From owner-freebsd-hackers@freebsd.org Sun Mar 31 13:26:02 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0A6DB155E0E1 for ; Sun, 31 Mar 2019 13:26:02 +0000 (UTC) (envelope-from chuck@tuffli.net) Received: from mail-io1-xd44.google.com (mail-io1-xd44.google.com [IPv6:2607:f8b0:4864:20::d44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7584B81995 for ; Sun, 31 Mar 2019 13:26:00 +0000 (UTC) (envelope-from chuck@tuffli.net) Received: by mail-io1-xd44.google.com with SMTP id d201so5434901iof.7 for ; Sun, 31 Mar 2019 06:26:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tuffli-net.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=Aeptdwul5hqEmJjO1UUdnj2fLR9Me6KWd97xXQhgalc=; b=L0EB8MsxECLNxSqnkdjUEDPpMbsBz8LRAzbZIwPeTjy9gqzoNUtIdtNEqVkkn31FtS qfVS7Y5NUJcw5A/B/A0w1HyhAIqEGthuDwWE7MMQUjFcbEuOHqDrWu7+YUG9clKZCfQ1 qAaMawXteq/bzZhYK0WEpx7Ptz5V8C7YR0weoXwvhpTGUmvoBgEJvXL6d2ZXwqlgdTXQ xSUOFEegYZLlsymH5jzWGPQqPYkCukai00A/Ow9k8uLyeCuc63XqVhPzS9leztxj9mC6 DYDa5LfUtYyand0USd+s/VXRHUXn/EdKHMO8saWrO0Lhg3kplLYK54o4JL+5VN07KEMD 5G5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=Aeptdwul5hqEmJjO1UUdnj2fLR9Me6KWd97xXQhgalc=; b=UbfKEtw2MiO82CwgbpdL022ZZjPfKqyxh7Yt+BmEdfjIuQR890QVUlN8AuQTFbMtoI oxzZ0OVWFm/fxnyGs7qXFSBGLEday9HDrYkGJtZ5V4VRx+Byo9/hbGfsTn6wBEQEBq6D a8Fa2KY/K+A7YI9ahrWwrz3so+kj/qTNs+bvqLXftdgw2Fixw/9aMFMVitd1kgIR5coE IXd1kZX/P9ujD1Kzkrc12OtDV2oJlxSx0PgMjTFPgRoMwfpmfz8zv25bDSc7JZ72V4BR BK1z9Jo+8ez6m/wQqyt9JhNtiN8x3VM2hHfLT45F85DtOD2TX6swz5CF7DoM9UULKART kE5Q== X-Gm-Message-State: APjAAAUC2Xa4wuuuYO5iJx6llR6D6pCZof29ovtH8XMprJMv0a1C3YRd 2IENj6jskaNEQUf2kfCBaPJNuqrr6xYXqjE1KW8qOQ== X-Google-Smtp-Source: APXvYqxwvXbLLYwyWu5i/Ybv0fEvL3KxWiz4FJoUEz8oVwNFBc8bK5ZVpgKP9MDLUMyeV85XYZ5Ej/prddJjHSgkmRY= X-Received: by 2002:a5d:855a:: with SMTP id b26mr25715297ios.151.1554038759414; Sun, 31 Mar 2019 06:25:59 -0700 (PDT) MIME-Version: 1.0 References: <79b6eebd-2320-1888-1162-d3ca5492670c@physik.tu-berlin.de> In-Reply-To: <79b6eebd-2320-1888-1162-d3ca5492670c@physik.tu-berlin.de> From: Chuck Tuffli Date: Sun, 31 Mar 2019 06:25:48 -0700 Message-ID: Subject: Re: core dumps running in bhyve To: Fabian Freyer Cc: freebsd-emulation@freebsd.org, FreeBSD Hackers , freebsd-virtualization@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 7584B81995 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=tuffli-net.20150623.gappssmtp.com header.s=20150623 header.b=L0EB8Msx; spf=permerror (mx1.freebsd.org: domain of chuck@tuffli.net uses mechanism not recognized by this client) smtp.mailfrom=chuck@tuffli.net X-Spamd-Result: default: False [-3.84 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; R_DKIM_ALLOW(-0.20)[tuffli-net.20150623.gappssmtp.com:s=20150623]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; DMARC_NA(0.00)[tuffli.net]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[tuffli-net.20150623.gappssmtp.com:+]; R_SPF_PERMFAIL(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[4.4.d.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; MX_GOOD(-0.01)[alt1.aspmx.l.google.com,aspmx.l.google.com,aspmx2.googlemail.com,aspmx3.googlemail.com,aspmx4.googlemail.com,alt2.aspmx.l.google.com,aspmx5.googlemail.com]; IP_SCORE(-0.70)[ip: (1.60), ipnet: 2607:f8b0::/32(-2.89), asn: 15169(-2.15), country: US(-0.07)]; NEURAL_HAM_SHORT(-0.84)[-0.836,0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 31 Mar 2019 13:26:02 -0000 On Fri, Dec 28, 2018 at 3:53 AM Fabian Freyer wrote: > > CCing freebsd-virtualization@, because they might know more about this. > > Am 25.12.2018 um 02:24 schrieb Chuck Tuffli: > > Using the latest bhyve, I'm seeing core dumps in the guest when running: > > nvmecontrol identify nvme0 > > against the emulated NVMe drive. The location of the core dump changes > > from run to run, but I suspect the root cause is a memory corruption > > caused by the transfer of the Identify data (4KB) back to the guest. > > This transfer of data is actually a memcpy to an address returned from > > vm_map_gpa() based on the physical address provided by the guest. > > > > Based on the signature of one of the core dumps, I modified > > nvmecontrol to always pass a 4KB aligned buffer to the driver instead > > of the (typically) unaligned address of the structure on the stack. > > With this change, nvmecontrol in the guest no longer core dumps. What > > I don't understand is why this changes the behavior. Do the addresses > > passed to vm_map_gpa() need to be page aligned? > > AFAIK vm_map_gpa maps a page, so yes, it needs to be 4k-aligned. > > > Or did moving the > > memory location from the stack to the heap merely mitigate what is > > corrupted? Thanks Fabian for the redirect to a better list. FWIW, the issue is with bhyve's NVMe emulation code and not anything to do with vm_map_gpa() per se. See https://reviews.freebsd.org/D19695 for those who are curious. --chuck From owner-freebsd-hackers@freebsd.org Mon Apr 1 21:11:54 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F386D156F2F8; Mon, 1 Apr 2019 21:11:53 +0000 (UTC) (envelope-from etnapierala@gmail.com) Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com [IPv6:2a00:1450:4864:20::331]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6D8018A080; Mon, 1 Apr 2019 21:11:52 +0000 (UTC) (envelope-from etnapierala@gmail.com) Received: by mail-wm1-x331.google.com with SMTP id n25so989816wmk.4; Mon, 01 Apr 2019 14:11:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:date:from:to:subject:message-id:mail-followup-to :mime-version:content-disposition:user-agent; bh=eHUr+4T3iQ2WSL+LelN9wyBJ2HetQUpnpYMdr1OqsuM=; b=iQBd9ozni4Rv1fUCTl5mYpXeE0zqKrYXMKAMUCjuAY+uFVMxPRyCcsNJr67nMjcn7q +9XGBdAL52/eRcf96QlsLQl/li36YmQLHo7/uPdTjxlrUZahbWxxikGNYVVkiRssKoNO q3WumQ5NHF4/wn0SnSYG5fgklbLv8PjijZ+SW7utTw59NJeKUn/1XqdcOBr7Nl1frL3W vAdmgUTlTlf5NnX6DnkZyXVA2Zta6jsMYwSGDsjA26OcomaqJcKWbIdF+hXdP19yWtqf XezFqKQK6O1wLxyjMgcOaBVn30zo9rorMxf3cRDZY1V56pWOopYcqLO6xm/q62rVQdQL Rnkg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:from:to:subject:message-id :mail-followup-to:mime-version:content-disposition:user-agent; bh=eHUr+4T3iQ2WSL+LelN9wyBJ2HetQUpnpYMdr1OqsuM=; b=sXBcBpBujBhfnQdwZufYtRo23nhAAwdMB3jdoREg+0rOxohP+55UcVkD/+Gy3ZxVbw OkM9amfVbVu5ljwcmCAJfDJXpy6BmR3/b1FK7//B4uU3KARzWNIl7Y7/G+FVF6zFa6U2 3sIbQZRWju2ENEiOi9280qkYJXVKXuaClWZ4idMV8uCMcyHSuUSdLeMctBFj8cCVuwA5 vwTU0U010q10sXoHlSmk9YQD4SkljExYyOPQyyEMrzjARGaQUGA0sSZb3cg42YW9PPfJ aWcNy+jQjeTQUAm6waN8cAMROh3i0b7Tr0b0oIRs6u+045aLvH+26w+tb5lpXG5MTN2U hDgA== X-Gm-Message-State: APjAAAWLychoB6Qu0ldxOqj/sJfdYngk7nlZ+X0Mc5gEB5lGxB4YzarF JxL+546ehvf3RADFT9M0x0mGzffB X-Google-Smtp-Source: APXvYqwf1pclv0BkKDkO22uIPWMrr2sPC/Ne1imYZQPdZlNh3kr7NLCw3I3JjMZq0k7r614NjFs9XQ== X-Received: by 2002:a1c:6a0d:: with SMTP id f13mr1081218wmc.76.1554153110839; Mon, 01 Apr 2019 14:11:50 -0700 (PDT) Received: from v2 (cpc92302-cmbg19-2-0-cust461.5-4.cable.virginm.net. [82.1.209.206]) by smtp.gmail.com with ESMTPSA id y125sm18686697wmc.39.2019.04.01.14.11.49 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 01 Apr 2019 14:11:49 -0700 (PDT) Sender: =?UTF-8?Q?Edward_Tomasz_Napiera=C5=82a?= Date: Mon, 1 Apr 2019 18:49:52 +0100 From: Edward Tomasz =?utf-8?Q?Napiera=C5=82a?= To: freebsd-hackers@freebsd.org, freebsd-current@freebsd.org Subject: Call for 2019Q1 quarterly status reports Message-ID: <20190401174952.GA8844@v2> Mail-Followup-To: freebsd-hackers@freebsd.org, freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="J/dobhs11T7y2rNN" Content-Disposition: inline User-Agent: Mutt/1.11.3 (2019-02-01) X-Rspamd-Queue-Id: 6D8018A080 X-Spamd-Bar: ------- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=iQBd9ozn; spf=pass (mx1.freebsd.org: domain of etnapierala@gmail.com designates 2a00:1450:4864:20::331 as permitted sender) smtp.mailfrom=etnapierala@gmail.com X-Spamd-Result: default: False [-7.55 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; SIGNED_PGP(-2.00)[]; FORGED_SENDER(0.30)[trasz@freebsd.org,etnapierala@gmail.com]; MIME_TRACE(0.00)[0:+,1:+]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[trasz@freebsd.org,etnapierala@gmail.com]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; DMARC_NA(0.00)[freebsd.org]; NEURAL_HAM_SHORT(-0.99)[-0.990,0]; IP_SCORE(-2.75)[ip: (-9.15), ipnet: 2a00:1450::/32(-2.36), asn: 15169(-2.15), country: US(-0.06)]; RCVD_IN_DNSWL_NONE(0.00)[1.3.3.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0]; MID_RHS_NOT_FQDN(0.50)[] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Apr 2019 21:11:54 -0000 --J/dobhs11T7y2rNN Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Dear FreeBSD Community, The deadline for the next FreeBSD Quarterly Status update is April 30, 2019, for work done since the last round of Quarterly Reports: January, 2019 =E2=80=93 March, 2019. Status report submissions do not need to be very long. They may be about anything happening in the FreeBSD project and community, and provide a great way to inform FreeBSD users and developers about work that is underway and completed. Submission of reports is not restricted to committers; anyone doing anything interesting and FreeBSD related can -- and should -- write one! The preferred method is to follow the guidelines at the Quarterly GitHub repository: https://github.com/freebsd/freebsd-quarterly Alternatively you can fetch the Markdown template, fill it in, and email it to quarterly@FreeBSD.org. The template can be found at: https://raw.githubusercontent.com/freebsd/freebsd-quarterly/master/report-s= ample.md The old XML generator and templates are no longer used. We look forward to seeing your 2019Q1 reports! Thanks, Edward (on behalf of quarterly@) --J/dobhs11T7y2rNN Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQGmBAEBCgCQFiEEbvjBe1hu6u1NeinjJCKD+Vwk/7oFAlyiT0BfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDZF RjhDMTdCNTg2RUVBRUQ0RDdBMjlFMzI0MjI4M0Y5NUMyNEZGQkESHHRyYXN6QGZy ZWVic2Qub3JnAAoJECQig/lcJP+6Jx0H/2YmGr1upqgTjc+Uur8uXFovS4V571Zf ddZnWVJ9kJX2matASDp/OwCFEuWSk9dGAUTkzuAGme/wK1ke3PM9tcH3hY+HLphc xhsxUs6wLgwCe28TX4JFKAkZMlJ8sc2i//AXo7u1YolABvgNwEk51ThDPKlnE7pw FbyOQMLGSTdJbrgz78pD42alCMNSmN/Zz5txD6pIwFBWrqbAF4BHw5kvHplQOk38 FQqH/MW6kIbdiPs/KNUbVllPyaBe/nXAKudm0AjCLzzWbrMheWpjlELMkw9sBe46 7Qfw9wdVSMrwzcRMLq8GUXrajDZvhjI7SC04mjbhiyXoZV1xttiCwOk= =E9Un -----END PGP SIGNATURE----- --J/dobhs11T7y2rNN-- From owner-freebsd-hackers@freebsd.org Tue Apr 2 20:10:31 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0EE3D1572060 for ; Tue, 2 Apr 2019 20:10:31 +0000 (UTC) (envelope-from lists@jnielsen.net) Received: from webmail5.jnielsen.net (webmail5.jnielsen.net [69.87.218.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "webmail2.jnielsen.net", Issuer "freebsdsolutions.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id DFDFD75E15 for ; Tue, 2 Apr 2019 20:10:27 +0000 (UTC) (envelope-from lists@jnielsen.net) Received: from [10.3.135.14] (50-207-240-162-static.hfc.comcastbusiness.net [50.207.240.162]) (authenticated bits=0) by webmail5.jnielsen.net (8.15.2/8.15.2) with ESMTPSA id x32JvxNA048529 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Tue, 2 Apr 2019 13:58:02 -0600 (MDT) (envelope-from lists@jnielsen.net) X-Authentication-Warning: webmail5.jnielsen.net: Host 50-207-240-162-static.hfc.comcastbusiness.net [50.207.240.162] claimed to be [10.3.135.14] From: John Nielsen Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 12.2 \(3445.102.3\)) Subject: online resizing NVME drives? Message-Id: <8F7034DD-AA3A-44DF-86D2-34FB6BF60FD7@jnielsen.net> Date: Tue, 2 Apr 2019 13:57:58 -0600 To: freebsd-hackers@freebsd.org X-Mailer: Apple Mail (2.3445.102.3) X-Rspamd-Queue-Id: DFDFD75E15 X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of lists@jnielsen.net designates 69.87.218.172 as permitted sender) smtp.mailfrom=lists@jnielsen.net X-Spamd-Result: default: False [-1.30 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-0.97)[-0.966,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; MV_CASE(0.50)[]; MIME_GOOD(-0.10)[text/plain]; HAS_XAW(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-0.85)[-0.854,0]; MIME_TRACE(0.00)[0:+]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MX_GOOD(-0.01)[mx0.freebsdsolutions.net,mx1.freebsdsolutions.net,mx2.freebsdsolutions.net,mx3.freebsdsolutions.net]; NEURAL_HAM_SHORT(-0.70)[-0.698,0]; TO_DN_NONE(0.00)[]; IP_SCORE(0.03)[asn: 6364(0.19), country: US(-0.06)]; DMARC_NA(0.00)[jnielsen.net]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; SUBJECT_ENDS_QUESTION(1.00)[]; ASN(0.00)[asn:6364, ipnet:69.87.218.0/24, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Apr 2019 20:10:31 -0000 Hi all- Obviously this wouldn't make sense for physical NVME hardware, but when = running on e.g. bhyve or AWS EC2 it's nice to be able to resize volumes = and filesystems without needing to reboot the (virtual) system. I'm = pretty sure that's do-able with SCSI, AHCI SATA and virtio-block devices = but I don't see a way to do it with NVME. Am I missing something or do = we need to implement something along the lines of "nvmecontrol rescan"? I'm experimenting with FreeBSD 12.0-RELEASE on a c5n.large AWS ec2 = instance. Like most (all?) of the c5 instance types the disks are all = presented as NVME devices even if they are actually backed by EBS. I = started one instance with a 10GB root volume but then decided I wanted = to make it bigger. I grew the volume on the AWS side no problem. When I = did so I got this kernel message: nvme0: async event occurred (type 0x2, info 0x00, page 0x00) Thereafter running "nvmecontrol devlist" correctly showed the increased = size of the parent device: # nvmecontrol devlist nvme0: Amazon Elastic Block Store nvme0ns1 (25600MB) However I couldn't see a way to update the size of the nvd0 block device = without rebooting. Pointers appreciated. Thanks! JN From owner-freebsd-hackers@freebsd.org Tue Apr 2 21:40:37 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B4EB5154F2B2 for ; Tue, 2 Apr 2019 21:40:37 +0000 (UTC) (envelope-from chuck@tuffli.net) Received: from mail-it1-x12b.google.com (mail-it1-x12b.google.com [IPv6:2607:f8b0:4864:20::12b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A07B48186C for ; Tue, 2 Apr 2019 21:40:36 +0000 (UTC) (envelope-from chuck@tuffli.net) Received: by mail-it1-x12b.google.com with SMTP id y10so7932603itc.1 for ; Tue, 02 Apr 2019 14:40:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tuffli-net.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=Tynp6owzhcJcNVcNGL1zusRi46uxNk/S78HZ1FOjHNk=; b=Qox1UACRhlguwjwm0TCTpVAC1xVOqA6O0SrSQkos9twLywhxcTXEp7QfCUNk/h+ke8 OX5K3qArT74QleZ6kxJcN4GncjtwKMe4jMx33hCq015oWHZPPmlk++cVzcB+WLSNyN/3 RpyPJsIPm/SsjzgZe9OtsgjN7zwUqTCzMpGLrkWlVLiJIbsQmJtegbq5GcHKb7aZ8hVA NOAN9DGl0E3OQIJJo6oCFsE69QF7azwJBTruVIcNMdirUtqelvjfd/ngJtvm5nc3S4AP NfVOsjt/SJAZwXgzNNfWQMG/XyzzW0o6vmdZ3syQiRJToCbanJn1/fM6Pnvclxs0r0Sp V6lg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=Tynp6owzhcJcNVcNGL1zusRi46uxNk/S78HZ1FOjHNk=; b=GlYns3mM33AXjuSZOjYu+X3qWNPN1Nv4pvQguWFdElvBRI6Lc2oYg8SopLv4oQi0Pm 2MVUTge6tcN/jwConDDuTfPM9C2llucoabP/2a1DOettGltgvQ0D0BFaXtnqJXuUB3VX Hs6v6cnS2BqBU256j3elD3colM5rq8WSR5S0SXNdykr8AnkzxX6gtkFKpe6iZbn2OxnW H/Hw8xLKLATu4vkxsNwXuF8bv22TBIc8vQnZuok9wKLzGGgSCqRfQTEcfvCbZ1/L8Sxr C1hQwi5+hcelMYlK/s/zLkvaXQKjZQYf0/aUuocIMLiD07Z7TqUwH1nTfBgjOjBci4AF l+6w== X-Gm-Message-State: APjAAAVKUN8UFkW6B/1XKWNuUQ2bj+5VzE89NdERdkgtjJ/hRegZMEcL DYcIQcceBmI85c2DRR/3J2Z3QXnLRux1avg07xGcnhSI X-Google-Smtp-Source: APXvYqzod/5kH8508UnF8O6WDwjumHqulv1YuTvkdJD7N0yW90EaG3poSnFa0TtChfRNp6BTPBm5/VFX8bHeKc1lSfE= X-Received: by 2002:a02:1c49:: with SMTP id c70mr45705429jac.92.1554241235659; Tue, 02 Apr 2019 14:40:35 -0700 (PDT) MIME-Version: 1.0 References: <8F7034DD-AA3A-44DF-86D2-34FB6BF60FD7@jnielsen.net> In-Reply-To: <8F7034DD-AA3A-44DF-86D2-34FB6BF60FD7@jnielsen.net> From: Chuck Tuffli Date: Tue, 2 Apr 2019 14:40:24 -0700 Message-ID: Subject: Re: online resizing NVME drives? To: John Nielsen Cc: FreeBSD Hackers Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: A07B48186C X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=tuffli-net.20150623.gappssmtp.com header.s=20150623 header.b=Qox1UACR; spf=permerror (mx1.freebsd.org: domain of chuck@tuffli.net uses mechanism not recognized by this client) smtp.mailfrom=chuck@tuffli.net X-Spamd-Result: default: False [-4.82 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[tuffli-net.20150623.gappssmtp.com:s=20150623]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; DMARC_NA(0.00)[tuffli.net]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[tuffli-net.20150623.gappssmtp.com:+]; R_SPF_PERMFAIL(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[b.2.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; MX_GOOD(-0.01)[alt1.aspmx.l.google.com,aspmx.l.google.com,aspmx3.googlemail.com,aspmx4.googlemail.com,aspmx5.googlemail.com,alt2.aspmx.l.google.com,aspmx2.googlemail.com]; RCPT_COUNT_TWO(0.00)[2]; RCVD_TLS_LAST(0.00)[]; NEURAL_HAM_SHORT(-0.97)[-0.969,0]; FROM_EQ_ENVFROM(0.00)[]; SUBJECT_ENDS_QUESTION(1.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; IP_SCORE(-2.54)[ip: (-7.56), ipnet: 2607:f8b0::/32(-2.93), asn: 15169(-2.17), country: US(-0.06)] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Apr 2019 21:40:38 -0000 On Tue, Apr 2, 2019 at 1:11 PM John Nielsen wrote: > > Hi all- > > Obviously this wouldn't make sense for physical NVME hardware, but when r= unning on e.g. bhyve or AWS EC2 it's nice to be able to resize volumes and = filesystems without needing to reboot the (virtual) system. I'm pretty sure= that's do-able with SCSI, AHCI SATA and virtio-block devices but I don't s= ee a way to do it with NVME. Am I missing something or do we need to implem= ent something along the lines of "nvmecontrol rescan"? > > I'm experimenting with FreeBSD 12.0-RELEASE on a c5n.large AWS ec2 instan= ce. Like most (all?) of the c5 instance types the disks are all presented a= s NVME devices even if they are actually backed by EBS. I started one insta= nce with a 10GB root volume but then decided I wanted to make it bigger. I = grew the volume on the AWS side no problem. When I did so I got this kernel= message: > nvme0: async event occurred (type 0x2, info 0x00, page 0x00) FWIW, this is a "Namespace Attribute Changed" event from the NVMe Controller. Are you using nvd as the disk driver? If so, can you try nda as that appears to support the XPT_SCAN_* events. --chuck From owner-freebsd-hackers@freebsd.org Tue Apr 2 22:27:45 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C541E1550C93; Tue, 2 Apr 2019 22:27:44 +0000 (UTC) (envelope-from tuexen@fh-muenster.de) Received: from drew.franken.de (mail-n.franken.de [193.175.24.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.franken.de", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A27B28362E; Tue, 2 Apr 2019 22:27:43 +0000 (UTC) (envelope-from tuexen@fh-muenster.de) Received: from mbp.fritz.box (p54952123.dip0.t-ipconnect.de [84.149.33.35]) (Authenticated sender: macmic) by mail-n.franken.de (Postfix) with ESMTPSA id E10F1721E2826; Wed, 3 Apr 2019 00:27:32 +0200 (CEST) From: Michael Tuexen Message-Id: Content-Type: multipart/signed; boundary="Apple-Mail=_A26B82D4-1AFC-4AF8-A85C-EB10136E809F"; protocol="application/pkcs7-signature"; micalg=sha-256 Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.8\)) Subject: Re: powerpc64 head -r344018 stuck sleeping problems: th->th_scale * tc_delta(th) overflows unsigned 64 bits sometimes [patched failed] Date: Wed, 3 Apr 2019 00:27:32 +0200 In-Reply-To: <20190324110138.GR1923@kib.kiev.ua> Cc: Bruce Evans , freebsd-hackers Hackers , FreeBSD PowerPC ML To: Konstantin Belousov References: <20190303111931.GI68879@kib.kiev.ua> <20190303223100.B3572@besplex.bde.org> <20190303161635.GJ68879@kib.kiev.ua> <20190304043416.V5640@besplex.bde.org> <20190304114150.GM68879@kib.kiev.ua> <20190305031010.I4610@besplex.bde.org> <20190306172003.GD2492@kib.kiev.ua> <20190308001005.M2756@besplex.bde.org> <20190307222220.GK2492@kib.kiev.ua> <20190309144844.K1166@besplex.bde.org> <20190324110138.GR1923@kib.kiev.ua> X-Mailer: Apple Mail (2.3445.104.8) X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=disabled version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on mail-n.franken.de X-Rspamd-Queue-Id: A27B28362E X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-1.40 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; MV_CASE(0.50)[]; HAS_ATTACHMENT(0.00)[]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[mx3.fh-muenster.de,mx2.fh-muenster.de,mx1.fh-muenster.de,mx4.fh-muenster.de]; FREEMAIL_TO(0.00)[gmail.com]; RCVD_IN_DNSWL_LOW(-0.10)[27.24.175.193.list.dnswl.org : 127.0.5.1]; MIME_TRACE(0.00)[0:+,1:+]; R_DKIM_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:680, ipnet:193.174.0.0/15, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[35.33.149.84.zen.spamhaus.org : 127.0.0.10]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.84)[-0.843,0]; FROM_HAS_DN(0.00)[]; SIGNED_SMIME(-2.00)[]; NEURAL_SPAM_SHORT(0.02)[0.022,0]; RCPT_COUNT_THREE(0.00)[4]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; DMARC_NA(0.00)[fh-muenster.de]; AUTH_NA(1.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_SPAM_LONG(0.15)[0.153,0]; R_SPF_NA(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; IP_SCORE(0.08)[ipnet: 193.174.0.0/15(0.40), asn: 680(0.02), country: DE(-0.01)]; FREEMAIL_CC(0.00)[optusnet.com.au] X-Mailman-Approved-At: Tue, 02 Apr 2019 23:40:38 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Apr 2019 22:27:45 -0000 --Apple-Mail=_A26B82D4-1AFC-4AF8-A85C-EB10136E809F Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii > On 24. Mar 2019, at 12:01, Konstantin Belousov = wrote: >=20 > On Sat, Mar 09, 2019 at 06:00:14PM +1100, Bruce Evans wrote: >> I more strongly disclike (sic) the more complete merge. The central = APIs >> have even more parameters and reduced type safety to describe objects = as >> (offset, size) pairs. > I changed the patch to be type-safe. Now I like it even more. It = provides > 1. internal > 2. concise > 3. type-safe > API to fetch data from timehands. The implementation needs to be read > only once. Hi, I'm a bit lost... I think this started to fix a problem on G5 PowerMacs. Do you think this patch solves the problem. Should this be tested? Or is this still work in progress or a general improvement not necessary fixing the problem on G5 PowerMacs? Best regards Michael >=20 >> Small delicate loops are ideal for duplicating. They are easier to >> understand individually and short enough to compare without using = diff >> to see gratuitous and substantive differences. Multiple instances = are >> only hard to write and maintain. Since these multiple instances are >> already written, they are only harder to maintain. > This is a good argument to have bintime_off and getthmember unmerged > (there are two small but delicate loops). The API is internal, so it = is > only matter for maintainer, which means that the absence of = duplication > is important. More, all that arguments clearly explain why there = should > be not twenty similar loops scattered over the source. >=20 >>=20 >>>> XX void >>>> XX binuptime(struct bintime *bt) >>>> XX { >>>> XX @@ -361,7 +383,7 @@ >>>> XX th =3D timehands; >>>> XX gen =3D atomic_load_acq_int(&th->th_generation); >>>> XX *bt =3D th->th_offset; >>>> XX - bintime_addx(bt, th->th_scale * tc_delta(th)); >>>> XX + bintime_adddelta(bt, th); >>>> XX atomic_thread_fence_acq(); >>>> XX } while (gen =3D=3D 0 || gen !=3D th->th_generation); >>>> XX } >>>>=20 >>>> This is the kind of non-churning change that I like. >>> Ok. I made all cases where timehands are read, more uniform by >>> moving calculations after the generation loop. This makes the >>> atomic part of the functions easier to see, and loop body has lower >>> chance to hit generation reset. >>=20 >> I think this change is slightly worse: >> - it increases register pressure. 'scale' and 'delta' must be read = in a >> alost program program before the loop exit test. The above order = uses >> them and stores the results to memory, so more registers are free = for >> the exit test. i386 certainly runs out of registers. IIRC, i386 = now >> spills 'gen'. It would have to spill something to load 'gen' or = 'th' >> for the test. > Which does not matter on any modern architecture anyway. >=20 >> - it enlarges the window between reading 'scale' and 'delta' and the >> caller seeing the results. Preemption in this window gives results >> that may be far in the past. > My opinion is that quickly exiting the code and avoid retry is more > important (as in performance) than making an impression that we = protect > against preemption. If preemption is important for the caller, then > the calling place must use some measures like interrupt disabling and > re-checking the time after the operation. Preemption can occur after = the > loop exit with the same consequences. >=20 >> The 'get' name is another problem. I would like all the get*time >> functions and not add new names starting with 'get'. The library >> implementation already doesn't bother optimizing the get*time = functions, >> but always uses the hardware timecounter. >>=20 >> getfoo() is a more natural name than foo_get() for the action of = getting >> foo, but the latter is better for consistency, especially in code = that >> puts the subsystem name first in nearby code. >>=20 >> The get*time functions would be better if they were more like >> time_second. Note that time_second is racy if time_t is too larger >> for the arch so that accesses to it are not atomic, as happens on >> 32-bit arches with premature 64-bit time_t. However, in this 32/64 >> case, the race is only run every 136 years, with the next event >> scheduled in 2038, so this race is even less important now than other >> events scheduled in 2038. Bintimes are 96 or 128 bits, so directly >> copying a global like time_second for them would race every 1/2**32 >> second on 2-bit arches or every 1 second on 64-bit arches. Most of >> the loops on the generation count are for fixing these races, but >> perhaps a simpler method would work. On 64-bit arches with atomic >> 64 accesses on 32-bit boundaries, the following would work: >> - set the lower 32 bits of the fraction to 0, or ignore them >> - load the higher 32 bits of the fraction and the lower 32 bits of = the >> seconds >> - race once every 136 years starting in 2038 reading the higher 32 = bits >> of the seconds non-atomically. >> - alternatively, break instead of racing in 2038 by setting the = higher >> 32 bits to 0. This is the same as using sbintimes instead of = bintimes. >> - drop a few more lower bits by storing a right-shifted value. Right >> shifting by just 1 gives a race frequency of once per 272 years, = with >> the next one in 2006. > It would make sense if the functions were written from scratch, but = since > we already have the generation counts, it is not obvious that such = change > is useful. >=20 > But if we decide to go that route (later), my current patch only > requires exactly one location getthmember() to experiment with and to > change after. So you effectively made yet another, and perhaps most > convincing, argument, for me. >=20 >>=20 >>> diff --git a/sys/kern/kern_tc.c b/sys/kern/kern_tc.c >>> index 2656fb4d22f..8d12847f2cd 100644 >>> --- a/sys/kern/kern_tc.c >>> +++ b/sys/kern/kern_tc.c >>> @@ -200,20 +202,56 @@ tc_delta(struct timehands *th) >>> * the comment in for a description of these 12 = functions. >>> */ >>>=20 >>> -#ifdef FFCLOCK >>> -void >>> -fbclock_binuptime(struct bintime *bt) >>> +static __inline void >>> +bintime_off(struct bintime *bt, u_int off) >>> { >>> struct timehands *th; >>> - unsigned int gen; >>> + struct bintime *btp; >>> + uint64_t scale, x; >>> + u_int delta, gen, large_delta; >>>=20 >>> do { >>> th =3D timehands; >>> gen =3D atomic_load_acq_int(&th->th_generation); >>> - *bt =3D th->th_offset; >>> - bintime_addx(bt, th->th_scale * tc_delta(th)); >>=20 >> You didn't fully obfuscate this by combinining this function with >> getthmember() so as to deduplicate the loop. >>=20 >>> + btp =3D (struct bintime *)((vm_offset_t)th + off); >>=20 >> Ugly conversion to share code. This is technically incorrect. = Improving >> the casts gives: >>=20 >> btp =3D (void *)(uintptr_t)((uintptr_t)(void *)th + off); >>=20 >> but this assumes that arithmetic on the intermediate integer does = what >> is espected. uintptr_t is only guaranteed to work when the = intermediate >> representation held in it is not adjusted. > vm_offset_t has the semantic that is needed for the arithmetic. It is > better uintptr_t for kernel, where we know that all object pointers = are > compatible with vm_offset_t (AKA flat tag-less memory model). >=20 >>=20 >> Fixing the API gives >>=20 >> static __inline void >> bintime_off(struct bintime *btp, struct bintime *base_btp) >>=20 >> where base_btp is &th->th_bintime or &th->th_offset. >>=20 >> (th_offset and th_bintime are badly named. th_offset is really a = base >> time and the offset is tc_delta(). th_bintime is also a base time. >> It is the same as th_offset with another actual offset (the = difference >> between UTC and local time) already added to it as an optimization. = In >> old versions, th_bintime didn't exist, but the related struct members >> th_nanotime and th_microtime existed, since these benefit more from >> not converting on every call. > How could it be &th->th_offset, when th is calculated inside the call = ? > But I did modified the API in this spirit, indeed. It takes the = member > name directly as an argument. >=20 >>=20 >> My old version even documents the struct members, while -current = still >> has no comments. The comments were lost to staticization. My = version >> mostly adds "duh" to the banal comments after recovering them: >>=20 >> XX /* >> XX * XXX rotted comment cloned from . >> XX * >> XX * th_counter is undocumented (duh). >> XX * >> XX * th_adjustment [PPM << 16] which means that the smallest unit of = correction >> XX * you can apply amounts to 481.5 usec/year. >> XX * >> XX * th_scale is undocumented (duh). >> XX * >> XX * th_offset_count is the contents of the counter which = corresponds to the >> XX * >> XX * rest of the offset_* values. >> XX * >> XX * th_offset is undocumented (duh). >> XX * >> XX * th_microtime is undocumented (duh). >> XX * >> XX * th_nanotime is undocumented (duh). >> XX * >> XX * XXX especially massive bitrot here. "three" is now "many"... >> XX * Each timecounter must supply an array of three timecounters. = This is needed >> XX * to guarantee atomicity in the code. Index zero is used to = transport >> XX * modifications, for instance done with sysctl, into the = timecounter being >> XX * used in a safe way. Such changes may be adopted with a delay = of up to 1/HZ. >> XX * Index one and two are used alternately for the actual = timekeeping. >> XX * >> XX * th_generation is undocumented (duh). >> XX * >> XX * th_next is undocumented (duh). >> XX */ >>=20 >>> + *bt =3D *btp; >>> + scale =3D th->th_scale; >>> + delta =3D tc_delta(th); >>> + large_delta =3D th->th_large_delta; >>=20 >> I had forgotten that th_scale is so volatile (it may be adjusted on >> every windup). th_large_delta is equally volatile. So moving the >> calculation outside of the loop gives even more register pressure >> than I noticed above. >>=20 >>> atomic_thread_fence_acq(); >>> } while (gen =3D=3D 0 || gen !=3D th->th_generation); >>> + >>> + if (__predict_false(delta < large_delta)) { >>> + /* Avoid overflow for scale * delta. */ >>> + x =3D (scale >> 32) * delta; >>> + bt->sec +=3D x >> 32; >>> + bintime_addx(bt, x << 32); >>> + bintime_addx(bt, (scale & 0xffffffff) * delta); >>> + } else { >>> + bintime_addx(bt, scale * delta); >>> + } >>> +} >>> + >>> +static __inline void >>> +getthmember(void *out, size_t out_size, u_int off) >>> +{ >>> + struct timehands *th; >>> + u_int gen; >>> + >>> + do { >>> + th =3D timehands; >>> + gen =3D atomic_load_acq_int(&th->th_generation); >>> + memcpy(out, (char *)th + off, out_size); >>=20 >> This isn't so ugly or technically incorrect. Now the object is = generic, >> so the reference to it should be passed as (void *objp, size_t = objsize) >> instead of the type-safe (struct bintime *base_bpt). > _Generic is what gave me a hint how to make the implementation = type-safe. > diff --git a/sys/kern/kern_tc.c b/sys/kern/kern_tc.c > index 2656fb4d22f..4e94f762026 100644 > --- a/sys/kern/kern_tc.c > +++ b/sys/kern/kern_tc.c > @@ -72,6 +72,7 @@ struct timehands { > struct timecounter *th_counter; > int64_t th_adjustment; > uint64_t th_scale; > + u_int th_large_delta; > u_int th_offset_count; > struct bintime th_offset; > struct bintime th_bintime; > @@ -90,6 +91,7 @@ static struct timehands th1 =3D { > static struct timehands th0 =3D { > .th_counter =3D &dummy_timecounter, > .th_scale =3D (uint64_t)-1 / 1000000, > + .th_large_delta =3D 1000000, > .th_offset =3D { .sec =3D 1 }, > .th_generation =3D 1, > .th_next =3D &th1 > @@ -200,20 +202,72 @@ tc_delta(struct timehands *th) > * the comment in for a description of these 12 = functions. > */ >=20 > -#ifdef FFCLOCK > -void > -fbclock_binuptime(struct bintime *bt) > +static __inline void > +bintime_off(struct bintime *bt, u_int off) > { > struct timehands *th; > - unsigned int gen; > + struct bintime *btp; > + uint64_t scale, x; > + u_int delta, gen, large_delta; >=20 > do { > th =3D timehands; > gen =3D atomic_load_acq_int(&th->th_generation); > - *bt =3D th->th_offset; > - bintime_addx(bt, th->th_scale * tc_delta(th)); > + btp =3D (struct bintime *)((vm_offset_t)th + off); > + *bt =3D *btp; > + scale =3D th->th_scale; > + delta =3D tc_delta(th); > + large_delta =3D th->th_large_delta; > atomic_thread_fence_acq(); > } while (gen =3D=3D 0 || gen !=3D th->th_generation); > + > + if (__predict_false(delta >=3D large_delta)) { > + /* Avoid overflow for scale * delta. */ > + x =3D (scale >> 32) * delta; > + bt->sec +=3D x >> 32; > + bintime_addx(bt, x << 32); > + bintime_addx(bt, (scale & 0xffffffff) * delta); > + } else { > + bintime_addx(bt, scale * delta); > + } > +} > +#define GETTHBINTIME(dst, member) = \ > +do { = \ > + _Static_assert(_Generic(((struct timehands *)NULL)->member, = \ > + struct bintime: 1, default: 0) =3D=3D 1, = \ > + "struct timehands member is not of struct bintime type"); = \ > + bintime_off(dst, __offsetof(struct timehands, member)); = \ > +} while (0) > + > +static __inline void > +getthmember(void *out, size_t out_size, u_int off) > +{ > + struct timehands *th; > + u_int gen; > + > + do { > + th =3D timehands; > + gen =3D atomic_load_acq_int(&th->th_generation); > + memcpy(out, (char *)th + off, out_size); > + atomic_thread_fence_acq(); > + } while (gen =3D=3D 0 || gen !=3D th->th_generation); > +} > +#define GETTHMEMBER(dst, member) = \ > +do { = \ > + _Static_assert(_Generic(*dst, = \ > + __typeof(((struct timehands *)NULL)->member): 1, = \ > + default: 0) =3D=3D 1, = \ > + "*dst and struct timehands member have different types"); = \ > + getthmember(dst, sizeof(*dst), __offsetof(struct timehands, = \ > + member)); = \ > +} while (0) > + > +#ifdef FFCLOCK > +void > +fbclock_binuptime(struct bintime *bt) > +{ > + > + GETTHBINTIME(bt, th_offset); > } >=20 > void > @@ -237,16 +291,8 @@ fbclock_microuptime(struct timeval *tvp) > void > fbclock_bintime(struct bintime *bt) > { > - struct timehands *th; > - unsigned int gen; >=20 > - do { > - th =3D timehands; > - gen =3D atomic_load_acq_int(&th->th_generation); > - *bt =3D th->th_bintime; > - bintime_addx(bt, th->th_scale * tc_delta(th)); > - atomic_thread_fence_acq(); > - } while (gen =3D=3D 0 || gen !=3D th->th_generation); > + GETTHBINTIME(bt, th_bintime); > } >=20 > void > @@ -270,100 +316,55 @@ fbclock_microtime(struct timeval *tvp) > void > fbclock_getbinuptime(struct bintime *bt) > { > - struct timehands *th; > - unsigned int gen; >=20 > - do { > - th =3D timehands; > - gen =3D atomic_load_acq_int(&th->th_generation); > - *bt =3D th->th_offset; > - atomic_thread_fence_acq(); > - } while (gen =3D=3D 0 || gen !=3D th->th_generation); > + GETTHMEMBER(bt, th_offset); > } >=20 > void > fbclock_getnanouptime(struct timespec *tsp) > { > - struct timehands *th; > - unsigned int gen; > + struct bintime bt; >=20 > - do { > - th =3D timehands; > - gen =3D atomic_load_acq_int(&th->th_generation); > - bintime2timespec(&th->th_offset, tsp); > - atomic_thread_fence_acq(); > - } while (gen =3D=3D 0 || gen !=3D th->th_generation); > + GETTHMEMBER(&bt, th_offset); > + bintime2timespec(&bt, tsp); > } >=20 > void > fbclock_getmicrouptime(struct timeval *tvp) > { > - struct timehands *th; > - unsigned int gen; > + struct bintime bt; >=20 > - do { > - th =3D timehands; > - gen =3D atomic_load_acq_int(&th->th_generation); > - bintime2timeval(&th->th_offset, tvp); > - atomic_thread_fence_acq(); > - } while (gen =3D=3D 0 || gen !=3D th->th_generation); > + GETTHMEMBER(&bt, th_offset); > + bintime2timeval(&bt, tvp); > } >=20 > void > fbclock_getbintime(struct bintime *bt) > { > - struct timehands *th; > - unsigned int gen; >=20 > - do { > - th =3D timehands; > - gen =3D atomic_load_acq_int(&th->th_generation); > - *bt =3D th->th_bintime; > - atomic_thread_fence_acq(); > - } while (gen =3D=3D 0 || gen !=3D th->th_generation); > + GETTHMEMBER(bt, th_bintime); > } >=20 > void > fbclock_getnanotime(struct timespec *tsp) > { > - struct timehands *th; > - unsigned int gen; >=20 > - do { > - th =3D timehands; > - gen =3D atomic_load_acq_int(&th->th_generation); > - *tsp =3D th->th_nanotime; > - atomic_thread_fence_acq(); > - } while (gen =3D=3D 0 || gen !=3D th->th_generation); > + GETTHMEMBER(tsp, th_nanotime); > } >=20 > void > fbclock_getmicrotime(struct timeval *tvp) > { > - struct timehands *th; > - unsigned int gen; >=20 > - do { > - th =3D timehands; > - gen =3D atomic_load_acq_int(&th->th_generation); > - *tvp =3D th->th_microtime; > - atomic_thread_fence_acq(); > - } while (gen =3D=3D 0 || gen !=3D th->th_generation); > + GETTHMEMBER(tvp, th_microtime); > } > #else /* !FFCLOCK */ > + > void > binuptime(struct bintime *bt) > { > - struct timehands *th; > - u_int gen; >=20 > - do { > - th =3D timehands; > - gen =3D atomic_load_acq_int(&th->th_generation); > - *bt =3D th->th_offset; > - bintime_addx(bt, th->th_scale * tc_delta(th)); > - atomic_thread_fence_acq(); > - } while (gen =3D=3D 0 || gen !=3D th->th_generation); > + GETTHBINTIME(bt, th_offset); > } >=20 > void > @@ -387,16 +388,8 @@ microuptime(struct timeval *tvp) > void > bintime(struct bintime *bt) > { > - struct timehands *th; > - u_int gen; >=20 > - do { > - th =3D timehands; > - gen =3D atomic_load_acq_int(&th->th_generation); > - *bt =3D th->th_bintime; > - bintime_addx(bt, th->th_scale * tc_delta(th)); > - atomic_thread_fence_acq(); > - } while (gen =3D=3D 0 || gen !=3D th->th_generation); > + GETTHBINTIME(bt, th_bintime); > } >=20 > void > @@ -420,85 +413,47 @@ microtime(struct timeval *tvp) > void > getbinuptime(struct bintime *bt) > { > - struct timehands *th; > - u_int gen; >=20 > - do { > - th =3D timehands; > - gen =3D atomic_load_acq_int(&th->th_generation); > - *bt =3D th->th_offset; > - atomic_thread_fence_acq(); > - } while (gen =3D=3D 0 || gen !=3D th->th_generation); > + GETTHMEMBER(bt, th_offset); > } >=20 > void > getnanouptime(struct timespec *tsp) > { > - struct timehands *th; > - u_int gen; > + struct bintime bt; >=20 > - do { > - th =3D timehands; > - gen =3D atomic_load_acq_int(&th->th_generation); > - bintime2timespec(&th->th_offset, tsp); > - atomic_thread_fence_acq(); > - } while (gen =3D=3D 0 || gen !=3D th->th_generation); > + GETTHMEMBER(&bt, th_offset); > + bintime2timespec(&bt, tsp); > } >=20 > void > getmicrouptime(struct timeval *tvp) > { > - struct timehands *th; > - u_int gen; > + struct bintime bt; >=20 > - do { > - th =3D timehands; > - gen =3D atomic_load_acq_int(&th->th_generation); > - bintime2timeval(&th->th_offset, tvp); > - atomic_thread_fence_acq(); > - } while (gen =3D=3D 0 || gen !=3D th->th_generation); > + GETTHMEMBER(&bt, th_offset); > + bintime2timeval(&bt, tvp); > } >=20 > void > getbintime(struct bintime *bt) > { > - struct timehands *th; > - u_int gen; >=20 > - do { > - th =3D timehands; > - gen =3D atomic_load_acq_int(&th->th_generation); > - *bt =3D th->th_bintime; > - atomic_thread_fence_acq(); > - } while (gen =3D=3D 0 || gen !=3D th->th_generation); > + GETTHMEMBER(bt, th_bintime); > } >=20 > void > getnanotime(struct timespec *tsp) > { > - struct timehands *th; > - u_int gen; >=20 > - do { > - th =3D timehands; > - gen =3D atomic_load_acq_int(&th->th_generation); > - *tsp =3D th->th_nanotime; > - atomic_thread_fence_acq(); > - } while (gen =3D=3D 0 || gen !=3D th->th_generation); > + GETTHMEMBER(tsp, th_nanotime); > } >=20 > void > getmicrotime(struct timeval *tvp) > { > - struct timehands *th; > - u_int gen; >=20 > - do { > - th =3D timehands; > - gen =3D atomic_load_acq_int(&th->th_generation); > - *tvp =3D th->th_microtime; > - atomic_thread_fence_acq(); > - } while (gen =3D=3D 0 || gen !=3D th->th_generation); > + GETTHMEMBER(tvp, th_microtime); > } > #endif /* FFCLOCK */ >=20 > @@ -514,15 +469,8 @@ getboottime(struct timeval *boottime) > void > getboottimebin(struct bintime *boottimebin) > { > - struct timehands *th; > - u_int gen; >=20 > - do { > - th =3D timehands; > - gen =3D atomic_load_acq_int(&th->th_generation); > - *boottimebin =3D th->th_boottime; > - atomic_thread_fence_acq(); > - } while (gen =3D=3D 0 || gen !=3D th->th_generation); > + GETTHMEMBER(boottimebin, th_boottime); > } >=20 > #ifdef FFCLOCK > @@ -1038,15 +986,8 @@ getmicrotime(struct timeval *tvp) > void > dtrace_getnanotime(struct timespec *tsp) > { > - struct timehands *th; > - u_int gen; >=20 > - do { > - th =3D timehands; > - gen =3D atomic_load_acq_int(&th->th_generation); > - *tsp =3D th->th_nanotime; > - atomic_thread_fence_acq(); > - } while (gen =3D=3D 0 || gen !=3D th->th_generation); > + GETTHMEMBER(tsp, th_nanotime); > } >=20 > /* > @@ -1464,6 +1405,7 @@ tc_windup(struct bintime *new_boottimebin) > scale +=3D (th->th_adjustment / 1024) * 2199; > scale /=3D th->th_counter->tc_frequency; > th->th_scale =3D scale * 2; > + th->th_large_delta =3D MIN(((uint64_t)1 << 63) / scale, = UINT_MAX); >=20 > /* > * Now that the struct timehands is again consistent, set the = new > _______________________________________________ > freebsd-ppc@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-ppc > To unsubscribe, send any mail to "freebsd-ppc-unsubscribe@freebsd.org" --Apple-Mail=_A26B82D4-1AFC-4AF8-A85C-EB10136E809F Content-Disposition: attachment; filename=smime.p7s Content-Type: application/pkcs7-signature; name=smime.p7s Content-Transfer-Encoding: base64 MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgEFADCABgkqhkiG9w0BBwEAAKCCEJAw ggTVMIIDvaADAgECAghQTsb1PRG0ZDANBgkqhkiG9w0BAQsFADBxMQswCQYDVQQGEwJERTEcMBoG A1UEChMTRGV1dHNjaGUgVGVsZWtvbSBBRzEfMB0GA1UECxMWVC1UZWxlU2VjIFRydXN0IENlbnRl cjEjMCEGA1UEAxMaRGV1dHNjaGUgVGVsZWtvbSBSb290IENBIDIwHhcNMTQwNzIyMTIwODI2WhcN MTkwNzA5MjM1OTAwWjBaMQswCQYDVQQGEwJERTETMBEGA1UEChMKREZOLVZlcmVpbjEQMA4GA1UE CxMHREZOLVBLSTEkMCIGA1UEAxMbREZOLVZlcmVpbiBQQ0EgR2xvYmFsIC0gRzAxMIIBIjANBgkq hkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA6ZvDZ4X5Da71jVTDllA1PWLpbkztlNcAW5UidNQg6zSP 1uzAMQQLmYHiphTSUqAoI4SLdIkEXlvg4njBeMsWyyg1OXstkEXQ7aAAeny/Sg4bAMOG6VwrMRF7 DPOCJEOMHDiLamgAmu7cT3ir0sYTm3at7t4m6O8Br3QPwQmi9mvOvdPNFDBP9eXjpMhim4IaAycw DQJlYE3t0QkjKpY1WCfTdsZxtpAdxO3/NYZ9bzOz2w/FEcKKg6GUXUFr2NIQ9Uz9ylGs2b3vkoO7 2uuLFlZWQ8/h1RM9ph8nMM1JVNvJEzSacXXFbOqnC5j5IZ0nrz6jOTlIaoytyZn7wxLyvQIDAQAB o4IBhjCCAYIwDgYDVR0PAQH/BAQDAgEGMB0GA1UdDgQWBBRJt8bP6D0ff+pEexMp9/EKcD7eZDAf BgNVHSMEGDAWgBQxw3kbuvVT1xfgiXotF2wKsyudMzASBgNVHRMBAf8ECDAGAQH/AgECMGIGA1Ud IARbMFkwEQYPKwYBBAGBrSGCLAEBBAICMBEGDysGAQQBga0hgiwBAQQDADARBg8rBgEEAYGtIYIs AQEEAwEwDwYNKwYBBAGBrSGCLAEBBDANBgsrBgEEAYGtIYIsHjA+BgNVHR8ENzA1MDOgMaAvhi1o dHRwOi8vcGtpMDMzNi50ZWxlc2VjLmRlL3JsL0RUX1JPT1RfQ0FfMi5jcmwweAYIKwYBBQUHAQEE bDBqMCwGCCsGAQUFBzABhiBodHRwOi8vb2NzcDAzMzYudGVsZXNlYy5kZS9vY3NwcjA6BggrBgEF BQcwAoYuaHR0cDovL3BraTAzMzYudGVsZXNlYy5kZS9jcnQvRFRfUk9PVF9DQV8yLmNlcjANBgkq hkiG9w0BAQsFAAOCAQEAYyAo/ZwhhnK+OUZZOTIlvKkBmw3Myn1BnIZtCm4ssxNZdbEzkhthJxb/ w7LVNYL7hCoBSb1mu2YvssIGXW4/buMBWlvKQ2NclbbhMacf1QdfTeZlgk4y+cN8ekvNTVx07iHy dQLsUj7SyWrTkCNuSWc1vn9NVqTszC/Pt6GXqHI+ybxA1lqkCD3WvILDt7cyjrEsjmpttzUCGc/1 OURYY6ckABCwu/xOr24vOLulV0k/2G5QbyyXltwdRpplic+uzPLl2Z9Tsz6hL5Kp2AvGhB8Exuse 6J99tXulAvEkxSRjETTMWpMgKnmIOiVCkKllO3yG0xIVIyn8LNrMOVtUFzCCBaIwggSKoAMCAQIC BxekJKEJSDMwDQYJKoZIhvcNAQELBQAwWjELMAkGA1UEBhMCREUxEzARBgNVBAoTCkRGTi1WZXJl aW4xEDAOBgNVBAsTB0RGTi1QS0kxJDAiBgNVBAMTG0RGTi1WZXJlaW4gUENBIEdsb2JhbCAtIEcw MTAeFw0xNDA1MjcxNDU0MDlaFw0xOTA3MDkyMzU5MDBaMIHGMQswCQYDVQQGEwJERTEcMBoGA1UE CBMTTm9yZHJoZWluLVdlc3RmYWxlbjERMA8GA1UEBxMITXVlbnN0ZXIxIDAeBgNVBAoTF0ZhY2ho b2Noc2NodWxlIE11ZW5zdGVyMSMwIQYDVQQLExpEYXRlbnZlcmFyYmVpdHVuZ3N6ZW50cmFsZTEd MBsGA1UEAxMURkggTXVlbnN0ZXIgQ0EgLSBHMDExIDAeBgkqhkiG9w0BCQEWEWNhQGZoLW11ZW5z dGVyLmRlMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAuHlsrvBs7CL9IqMH9r//QU9E pghTV/3skHuQZ3DpNY+lyJWOW5zbtUubgXt7lYHpIE4d4CclTZWqCHwoAI6gqzSSGjUKuX6/0ui/ LhXmlDvCBfwuER+T+3/R59hlLnhI5iYYPQiNywQIa3wJhBLTZrlXw8nDdjI54MAzcVDUX7l21sbo ZIA6idM7SXmshxoRQ6xsfPHskrceNMcvtHNDhVnVscwRUJQUR55fs0X7Y93PasugWPv3xmgNr1da Cq94eV+nslNU/GJaT9TQ3uG8pagLXl9NbDNkHIrvFAD5zXO0m/d00I4QhUVQyEtwnTegDqcM+WFh JXensgnZhWe6bwIDAQABo4IB/jCCAfowEgYDVR0TAQH/BAgwBgEB/wIBATAOBgNVHQ8BAf8EBAMC AQYwEQYDVR0gBAowCDAGBgRVHSAAMB0GA1UdDgQWBBQK81u85DGA1jVCiabTw8833tHf1zAfBgNV HSMEGDAWgBRJt8bP6D0ff+pEexMp9/EKcD7eZDAcBgNVHREEFTATgRFjYUBmaC1tdWVuc3Rlci5k ZTCBiAYDVR0fBIGAMH4wPaA7oDmGN2h0dHA6Ly9jZHAxLnBjYS5kZm4uZGUvZ2xvYmFsLXJvb3Qt Y2EvcHViL2NybC9jYWNybC5jcmwwPaA7oDmGN2h0dHA6Ly9jZHAyLnBjYS5kZm4uZGUvZ2xvYmFs LXJvb3QtY2EvcHViL2NybC9jYWNybC5jcmwwgdcGCCsGAQUFBwEBBIHKMIHHMDMGCCsGAQUFBzAB hidodHRwOi8vb2NzcC5wY2EuZGZuLmRlL09DU1AtU2VydmVyL09DU1AwRwYIKwYBBQUHMAKGO2h0 dHA6Ly9jZHAxLnBjYS5kZm4uZGUvZ2xvYmFsLXJvb3QtY2EvcHViL2NhY2VydC9jYWNlcnQuY3J0 MEcGCCsGAQUFBzAChjtodHRwOi8vY2RwMi5wY2EuZGZuLmRlL2dsb2JhbC1yb290LWNhL3B1Yi9j YWNlcnQvY2FjZXJ0LmNydDANBgkqhkiG9w0BAQsFAAOCAQEA3kcDNdZKb7kSD7s1ly2qa/2QbQe+ ld3LhZeOcfysdLtN8oweBmgT3MYoZ+D9c+SoUWJAwTKPB15DoGy+fWhelXTpQrqxIGb4ISr1JCjg slnmMUva0xjwZGxojZ9gE1bi18xfKw3+dMpwCLt6LbLTjr/tyH6otacwr2tZzuuJIUAORnefwTcr vmB21n/BEQH/ZXruWu8lSO3L9YAmQB6ViaZFCpn2sMmOLACdoWxmUQb3QAjsa327jHUjsz53k9q5 Zrx/g+zOg5s1Wmy2JOlLQMUIZXXf0/6rB5Fr2llx7dBG/Uk7NhZdNy7OzNzci0C4Wnkd8rDVEWHG hH2gfpcTfjCCBg0wggT1oAMCAQICBxuZiHQ3saMwDQYJKoZIhvcNAQELBQAwgcYxCzAJBgNVBAYT AkRFMRwwGgYDVQQIExNOb3JkcmhlaW4tV2VzdGZhbGVuMREwDwYDVQQHEwhNdWVuc3RlcjEgMB4G A1UEChMXRmFjaGhvY2hzY2h1bGUgTXVlbnN0ZXIxIzAhBgNVBAsTGkRhdGVudmVyYXJiZWl0dW5n c3plbnRyYWxlMR0wGwYDVQQDExRGSCBNdWVuc3RlciBDQSAtIEcwMTEgMB4GCSqGSIb3DQEJARYR Y2FAZmgtbXVlbnN0ZXIuZGUwHhcNMTYwNzA0MDcwNjEzWhcNMTkwNzA0MDcwNjEzWjB8MQswCQYD VQQGEwJERTEgMB4GA1UECgwXRmFjaGhvY2hzY2h1bGUgTXVlbnN0ZXIxMjAwBgNVBAsMKUZhY2hi ZXJlaWNoIEVsZWt0cm90ZWNobmlrIHVuZCBJbmZvcm1hdGlrMRcwFQYDVQQDDA5NaWNoYWVsIFR1 ZXhlbjCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMyaGlBt2ZtuF8QP8zYNrGxXC+es PMajIPl+hu1LGHnN2BJ3J5ZMN44BOZw3n6LO1FaAgO8D4xU4/AELecX6VxJZ2zOOSD8uTYO4OnUu 24hkjFUQAj13tT644AKUQMMBpgj7wC52V5Jij+mZX/t1S38/WFiCGnirt4xTNi5OmN4K+VNZfG4x 0msDqFjJX70rF1y09/Mylu1M/Y0tu/I9DqhwDQT4LBOvyyaAlhSJ8Jb8m8YTt5xlOzrXlBmj4pKs 74y7C2IKRw4tFozGX1cf1LVEs2eBCb5iUwXrlcMipwm62sJ38GD00EOlRNTpAM5rDAcgWxMCffek bRv/01whtOkCAwEAAaOCAkcwggJDMEAGA1UdIAQ5MDcwEQYPKwYBBAGBrSGCLAEBBAMFMBEGDysG AQQBga0hgiwCAQQDATAPBg0rBgEEAYGtIYIsAQEEMAkGA1UdEwQCMAAwDgYDVR0PAQH/BAQDAgXg MB0GA1UdJQQWMBQGCCsGAQUFBwMCBggrBgEFBQcDBDAdBgNVHQ4EFgQU0B2vaoSoEmYAggD04WZF 2hGif3UwHwYDVR0jBBgwFoAUCvNbvOQxgNY1Qomm08PPN97R39cwIAYDVR0RBBkwF4EVdHVleGVu QGZoLW11ZW5zdGVyLmRlMIGIBgNVHR8EgYAwfjA9oDugOYY3aHR0cDovL2NkcDEucGNhLmRmbi5k ZS9maC1tdWVuc3Rlci1jYS9wdWIvY3JsL2NhY3JsLmNybDA9oDugOYY3aHR0cDovL2NkcDIucGNh LmRmbi5kZS9maC1tdWVuc3Rlci1jYS9wdWIvY3JsL2NhY3JsLmNybDCB1wYIKwYBBQUHAQEEgcow gccwMwYIKwYBBQUHMAGGJ2h0dHA6Ly9vY3NwLnBjYS5kZm4uZGUvT0NTUC1TZXJ2ZXIvT0NTUDBH BggrBgEFBQcwAoY7aHR0cDovL2NkcDEucGNhLmRmbi5kZS9maC1tdWVuc3Rlci1jYS9wdWIvY2Fj ZXJ0L2NhY2VydC5jcnQwRwYIKwYBBQUHMAKGO2h0dHA6Ly9jZHAyLnBjYS5kZm4uZGUvZmgtbXVl bnN0ZXItY2EvcHViL2NhY2VydC9jYWNlcnQuY3J0MA0GCSqGSIb3DQEBCwUAA4IBAQBI9v+seJM6 AlSIrmmpopz6zh8QAsqGLJkkY2D0KYFucUY/xZaJTtZxvmWddbKk2903Qhg+vZKOf87PHhip7/4t FSwhxYNSS36WsRJTeUa0f3KkSa28yrIRfWlJATgxfL5X/QQnopjCt34n4221kcsR7LHxBAn37ow+ /2L7WjWDDuOkaM9/ZSCtrN+yFRat1eUVs1Hk7sKT/bfJTsYqzovXitjmCP3YdB40dkuQ6/ZzEdXT bpa4c45RcRnPqKXnxknK0UfRHNHqk15W7dUPVMzSGFUvjhmWPP2wW6a8F1U5sEqfHcoBFC5CGjGy 7Gk2luk3obi/KLrDyZC+dkjhDYEpMYIEOTCCBDUCAQEwgdIwgcYxCzAJBgNVBAYTAkRFMRwwGgYD VQQIExNOb3JkcmhlaW4tV2VzdGZhbGVuMREwDwYDVQQHEwhNdWVuc3RlcjEgMB4GA1UEChMXRmFj aGhvY2hzY2h1bGUgTXVlbnN0ZXIxIzAhBgNVBAsTGkRhdGVudmVyYXJiZWl0dW5nc3plbnRyYWxl MR0wGwYDVQQDExRGSCBNdWVuc3RlciBDQSAtIEcwMTEgMB4GCSqGSIb3DQEJARYRY2FAZmgtbXVl bnN0ZXIuZGUCBxuZiHQ3saMwDQYJYIZIAWUDBAIBBQCgggI3MBgGCSqGSIb3DQEJAzELBgkqhkiG 9w0BBwEwHAYJKoZIhvcNAQkFMQ8XDTE5MDQwMjIyMjczMlowLwYJKoZIhvcNAQkEMSIEIGSIIzmA QlMYcCWNiwVDItNMbp2U1LppxYebTqP1GV1zMIHjBgkrBgEEAYI3EAQxgdUwgdIwgcYxCzAJBgNV BAYTAkRFMRwwGgYDVQQIExNOb3JkcmhlaW4tV2VzdGZhbGVuMREwDwYDVQQHEwhNdWVuc3RlcjEg MB4GA1UEChMXRmFjaGhvY2hzY2h1bGUgTXVlbnN0ZXIxIzAhBgNVBAsTGkRhdGVudmVyYXJiZWl0 dW5nc3plbnRyYWxlMR0wGwYDVQQDExRGSCBNdWVuc3RlciBDQSAtIEcwMTEgMB4GCSqGSIb3DQEJ ARYRY2FAZmgtbXVlbnN0ZXIuZGUCBxuZiHQ3saMwgeUGCyqGSIb3DQEJEAILMYHVoIHSMIHGMQsw CQYDVQQGEwJERTEcMBoGA1UECBMTTm9yZHJoZWluLVdlc3RmYWxlbjERMA8GA1UEBxMITXVlbnN0 ZXIxIDAeBgNVBAoTF0ZhY2hob2Noc2NodWxlIE11ZW5zdGVyMSMwIQYDVQQLExpEYXRlbnZlcmFy YmVpdHVuZ3N6ZW50cmFsZTEdMBsGA1UEAxMURkggTXVlbnN0ZXIgQ0EgLSBHMDExIDAeBgkqhkiG 9w0BCQEWEWNhQGZoLW11ZW5zdGVyLmRlAgcbmYh0N7GjMA0GCSqGSIb3DQEBAQUABIIBAIZaQcal 2J/msfopGijjkS2+107hVANwLAQ6CiD93XD/ijJOkTZQLjqzRmSFAFMQGw451uj11sla6SlLLFYI QR3GJEVfyykRs31RS3HAbU8b96QiB4AYCWStAbulwWbJFHTdKZJsZEc4oHCA3OboIw7gjd7c89PF 3siE2nHFJWUlUevrMqg+xSaqkXV1eOhC8l0foNqzlSYyhYw3wdB3mpZS2kDQ6d5VVO0qskqy5Ua9 RW457EF9AXcW3sCJ6k/B6p78uUUkEkNgBZbRkIykqu/VACIffur+2qF49sKEV4yjhSxFNtJKtqVM RZNPcwhMe446prrHZbWU3ThPedtIv8QAAAAAAAA= --Apple-Mail=_A26B82D4-1AFC-4AF8-A85C-EB10136E809F-- From owner-freebsd-hackers@freebsd.org Wed Apr 3 00:26:10 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AA08E15544C1 for ; Wed, 3 Apr 2019 00:26:10 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic314-20.consmr.mail.gq1.yahoo.com (sonic314-20.consmr.mail.gq1.yahoo.com [98.137.69.83]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BBB7887744 for ; Wed, 3 Apr 2019 00:26:09 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: 7LTWA5QVM1kW3Lb6rCmlym1zAVjEOf5kpfujlRwpU98uzMbUR0dXozL.eCkTYb2 Hg52w2GaMFi_5gRcaQmEEjYYttGg5p1nML0LLGf_EYr64Zasv8oJ.9rI_XiV89RMxYuJ3.bldXxg 5ftaW1fdUkwr36p2Xbbd7_ajVNDzkM7ilrfkfHEHSeuBDWXLgJnpwq3nZ0J_v3tNHES425yfCx6j m2FOrCX3e.pcd9wY0PUEGz9CnAeVTOwuy7rXUUhQ5uMmw5aP7ueOd7ds7qGFLV06F3KiOnle9dgo mz2AGPeF_Co70RlQ5MU8MFGm0eLlCLtqwaZaZVUzAmEOg5F2MzRjlwsde6y8ZnKyIqMMsgYgIAOj uBaOOPWykAuyTzoMzvs3J1AtGwSWMI0zaLZAYynUt_6tbUomxg9zFZPzxwUwibcJdaDr.6w485td CfZmYSHMtOTUIO9DuoeZKzfgwKhwuuA7gOiIMtZ_tWg4ECbun7GAXCPiUUP8L2HTNNSBhlM8O3g8 PuULbzP9UOCMT0MOnwKs4p7ZyHeKVOWFwgoIY051_XwDC6TIA1vczy3m6RdfAK3EOP0KYEgHFGor yMqkAn4HK_5sUoxodMWR1Ut1bZOhBRXXLeM5iKfhj1moqMP.b52TsU4w80VeLZpbtIIObcn1rxVT smhSGpu1erFRgh80sqAYMRXTarEBxJJsJLU36kue9ZmdiM.x73RE13LR5jd9aZ02QgFKGu38AgFO RGAKjf6fxg0cpg4cfF8Ug6Ig8cSV3fCzGEnL.qfRPffVwcrm1FbMSk3NMQ9ibGTt4byQliZfQaJA i.ABdYU7ofx_njt01iaVl3Fbm6SQTCSzc9Z_rETVZplXkDoqdGwtGZnJmOo6P97D5Vr9vcYqH1eX YIu1XlAvj3uuiubavTH67sPvb37yW1fqnrJdvnsHUC36Z417pAQ7F3J66yppNHtm5.ZKjBvBIMvP nmz43X9qfeZr.3h7W0oBxX0LEyjHnGBJRHAQGSnzBE.MJihHobBYZqZhblYGD37ZbgcFjLk6OJM_ 1p9MyyBPrlEHMlGBi7UgOelX7GzuZ8oBZlHLX2NYuAlfsJEMsAL3qnh9riJukBnBrYOF.SVK3N3d A Received: from sonic.gate.mail.ne1.yahoo.com by sonic314.consmr.mail.gq1.yahoo.com with HTTP; Wed, 3 Apr 2019 00:26:07 +0000 Received: from c-67-170-167-181.hsd1.or.comcast.net (EHLO [192.168.1.113]) ([67.170.167.181]) by smtp424.mail.gq1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID 935369d1782ce17aa08491c035fe8ef3; Wed, 03 Apr 2019 00:15:57 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.8\)) Subject: Re: powerpc64 head -r344018 stuck sleeping problems: th->th_scale * tc_delta(th) overflows unsigned 64 bits sometimes [patched failed] From: Mark Millard In-Reply-To: Date: Tue, 2 Apr 2019 17:15:56 -0700 Cc: Konstantin Belousov , freebsd-hackers Hackers , Bruce Evans , FreeBSD PowerPC ML Content-Transfer-Encoding: quoted-printable Message-Id: <70B2E795-1FCC-41A3-8F4D-6E6C97379116@yahoo.com> References: <20190303111931.GI68879@kib.kiev.ua> <20190303223100.B3572@besplex.bde.org> <20190303161635.GJ68879@kib.kiev.ua> <20190304043416.V5640@besplex.bde.org> <20190304114150.GM68879@kib.kiev.ua> <20190305031010.I4610@besplex.bde.org> <20190306172003.GD2492@kib.kiev.ua> <20190308001005.M2756@besplex.bde.org> <20190307222220.GK2492@kib.kiev.ua> <20190309144844.K1166@besplex.bde.org> <20190324110138.GR1923@kib.kiev.ua> To: Michael Tuexen X-Mailer: Apple Mail (2.3445.104.8) X-Rspamd-Queue-Id: BBB7887744 X-Spamd-Bar: +++ X-Spamd-Result: default: False [3.30 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MV_CASE(0.50)[]; FREEMAIL_FROM(0.00)[yahoo.com]; RCPT_COUNT_FIVE(0.00)[5]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[cached: mta6.am0.yahoodns.net]; DKIM_TRACE(0.00)[yahoo.com:+]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/21, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; NEURAL_SPAM_SHORT(0.68)[0.677,0]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE(1.90)[ip: (7.88), ipnet: 98.137.64.0/21(0.95), asn: 36647(0.76), country: US(-0.06)]; NEURAL_SPAM_MEDIUM(0.53)[0.529,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_SPAM_LONG(0.70)[0.700,0]; RCVD_IN_DNSWL_NONE(0.00)[83.69.137.98.list.dnswl.org : 127.0.5.0]; RWL_MAILSPIKE_POSSIBLE(0.00)[83.69.137.98.rep.mailspike.net : 127.0.0.17]; FREEMAIL_CC(0.00)[gmail.com] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Apr 2019 00:26:10 -0000 On 2019-Apr-2, at 15:27, Michael Tuexen = wrote: >> On 24. Mar 2019, at 12:01, Konstantin Belousov wrote: >>=20 >> On Sat, Mar 09, 2019 at 06:00:14PM +1100, Bruce Evans wrote: >>> I more strongly disclike (sic) the more complete merge. The central = APIs >>> have even more parameters and reduced type safety to describe = objects as >>> (offset, size) pairs. >> I changed the patch to be type-safe. Now I like it even more. It = provides >> 1. internal >> 2. concise >> 3. type-safe >> API to fetch data from timehands. The implementation needs to be = read >> only once. > Hi, >=20 > I'm a bit lost... I think this started to fix a problem on G5 = PowerMacs. > Do you think this patch solves the problem. Should this be tested? > Or is this still work in progress or a general improvement not = necessary > fixing the problem on G5 PowerMacs? >=20 > Best regards > Michael > . . . A much earlier version of the patch made things much worse on the PowerMc G5 (massively long hangups, inability to effectively enter commands). I had to cut power, repair the file system, and undo the separately (not booted from that media). I've not tried again since then. I suspect that while my report prompted the looking into the code, the issues noticed are likely not fixes to the behavior observed on the old PowerMac G5's. I've my own hack/workaround that has my context operational, avoiding the original issue. Definitely not what FreeBSD should be doing. It even involves constants figured out by observation of the failing conditions in the specific context. =46rom what I've seen, my detailed hack-text will not apply to the new code. I'll likely have to work out a variant until the original problem and a solution is identified and provided. Note: The only PowerPC contexts that I've access to are old PowerMacs. I've no clue if other contexts have the original issue or not. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-hackers@freebsd.org Wed Apr 3 07:01:00 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4E58D1560354; Wed, 3 Apr 2019 07:01:00 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6BEA09621A; Wed, 3 Apr 2019 07:00:59 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id x3370lKJ077099 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Wed, 3 Apr 2019 10:00:50 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua x3370lKJ077099 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id x3370j6a077098; Wed, 3 Apr 2019 10:00:45 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 3 Apr 2019 10:00:45 +0300 From: Konstantin Belousov To: Michael Tuexen Cc: Bruce Evans , freebsd-hackers Hackers , FreeBSD PowerPC ML Subject: Re: powerpc64 head -r344018 stuck sleeping problems: th->th_scale * tc_delta(th) overflows unsigned 64 bits sometimes [patched failed] Message-ID: <20190403070045.GW1923@kib.kiev.ua> References: <20190303161635.GJ68879@kib.kiev.ua> <20190304043416.V5640@besplex.bde.org> <20190304114150.GM68879@kib.kiev.ua> <20190305031010.I4610@besplex.bde.org> <20190306172003.GD2492@kib.kiev.ua> <20190308001005.M2756@besplex.bde.org> <20190307222220.GK2492@kib.kiev.ua> <20190309144844.K1166@besplex.bde.org> <20190324110138.GR1923@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.11.4 (2019-03-13) X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on tom.home X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Apr 2019 07:01:00 -0000 On Wed, Apr 03, 2019 at 12:27:32AM +0200, Michael Tuexen wrote: > > On 24. Mar 2019, at 12:01, Konstantin Belousov wrote: > > > > On Sat, Mar 09, 2019 at 06:00:14PM +1100, Bruce Evans wrote: > >> I more strongly disclike (sic) the more complete merge. The central APIs > >> have even more parameters and reduced type safety to describe objects as > >> (offset, size) pairs. > > I changed the patch to be type-safe. Now I like it even more. It provides > > 1. internal > > 2. concise > > 3. type-safe > > API to fetch data from timehands. The implementation needs to be read > > only once. > Hi, > > I'm a bit lost... I think this started to fix a problem on G5 PowerMacs. > Do you think this patch solves the problem. Should this be tested? > Or is this still work in progress or a general improvement not necessary > fixing the problem on G5 PowerMacs? It started from a report of issues on G5. The specific issues are bugs on G5, and the posted patches do not fix them. From what I see, the timecounter values were wrapped. This is genuine ppc or G5 issue. The patch fixes time keeping subsystem reaction to the already bad situation, by correctly handling overflow in calculations. This overflow can occur in more reasonable setups as well, e.g. if ddb was activated and interrupts were stopped for prolonged period, even on x86. In addition, my version of the patch reorganizes the code and removes excessive copies of the most delicate loops in lock-less readers. This chunk can be split from the overflow part, but it is not completely trivial. From owner-freebsd-hackers@freebsd.org Wed Apr 3 07:05:38 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1A4481560ADB; Wed, 3 Apr 2019 07:05:38 +0000 (UTC) (envelope-from lists@yamagi.org) Received: from mail1.yamagi.org (mail1.yamagi.org [IPv6:2001:19f0:5001:17bd::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6791296856; Wed, 3 Apr 2019 07:05:37 +0000 (UTC) (envelope-from lists@yamagi.org) Received: from [212.48.125.108] (helo=aka) by mail1.yamagi.org with esmtpsa (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.91 (FreeBSD)) (envelope-from ) id 1hBZxm-000AKq-Pe; Wed, 03 Apr 2019 09:05:36 +0200 Date: Wed, 3 Apr 2019 09:05:22 +0200 From: Yamagi Burmeister To: rozhuk.im@gmail.com Cc: dariusmihaim@gmail.com, freebsd-hackers@freebsd.org, freebsd-current@freebsd.org Subject: Re: Boot hang on ryzen based notebook Message-Id: <20190403090522.548ffbdaad84c2c001a6926d@yamagi.org> In-Reply-To: <20190330211736.05b39193@rimwks> References: <20190330164628.138a2bad@rimwks> <20190330204138.18fe8443@rimwks> <20190330211736.05b39193@rimwks> X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.32; amd64-portbld-freebsd12.0) Mime-Version: 1.0 Content-Type: multipart/signed; protocol="application/pgp-signature"; micalg="PGP-SHA256"; boundary="Signature=_Wed__3_Apr_2019_09_05_22_+0200_GXlVhFLCDLN3VP/8" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Apr 2019 07:05:38 -0000 --Signature=_Wed__3_Apr_2019_09_05_22_+0200_GXlVhFLCDLN3VP/8 Content-Type: text/plain; charset=US-ASCII Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, 30 Mar 2019 21:17:36 +0300 Rozhuk Ivan wrote: > > You wrote "apic.0.disabled", not "acpi.0.disabled". Are you sure you > > wrote the correct command? You do not want to disable the APIC > > (interrupt controller), but the ACPI (control and power interface). >=20 > hint.acpi.0.disabled=3D"1" - 11.13.1. > Then set: > panic: running without device atpic requires a local APIC Most modern hardware won't boot without ACPI. But I can get my Lenovo E485 to boot with Linux and FreeBSD if I disable IOMMU aupport in the BIOS / UEFI. --=20 Homepage: https://www.yamagi.org Github: https://github.com/yamagi GPG: 0x1D502515 --Signature=_Wed__3_Apr_2019_09_05_22_+0200_GXlVhFLCDLN3VP/8 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEOXu/lxyufwz0gC5x6xRy5x1QJRUFAlykWzIACgkQ6xRy5x1Q JRWccQ//Qn5SL++7sKEiTzwKVQPbIJURjxmEMPvXrsHBVDVTR5MeSS8zakZ1bDR/ nWjsR9EC1nx0CJtmNB92xalaH8I44yoTApZlntc6gzJP4fA1gzLrnMP+GUCYbFBF EhXS+f04ti0Vl+VZ5jtP26X0VOrqE10TUwqlaj/yEOlMPq24hPpXggT1iFFHL8SW cxQdWPpPxeU+VfhYhkxYulBXtOPx72EYToaOcvSB41dHQO4/oSn/w2fM1lhWeeVo fcP2s8aK0aeoQSUzaI6gFltBTyU8qVh3zXRap8Gvt9BRhQJn9xuQdayhJlAnqfzi VbA6SF0fXTghV9g0zkBmzov/6OazoXC7w4BOEXqekOlDRCMwWT243iLIayJ6KnTo 2HLTOpU2Kz9XgGSGwGu+shbt6GlK/b28rxnb0thWzISDkaYC4ght67urq1sqO94y DHBu5fxUSKPggPL6AKjroNMFu+6nRFlF0/GjWeo40QSz5dYxmc1qRW7pytljyJcQ 2Ykm/E5lnBj7bFJ1JmBw2xT67bH2GdrvvPF32e+nLlH4UZHFwojoQ45f0RZp0W+7 1RscZAdVOfIKrBlah4YlCFIvlVdZYt5Yx4FESEU4nuTD675bqQan59DoTUUQhJRT d5PoMKp6f8/uFQlVU+EXDCphQO1hv5bk3fPCpzzOfLE50ZnC4Iw= =8EnE -----END PGP SIGNATURE----- --Signature=_Wed__3_Apr_2019_09_05_22_+0200_GXlVhFLCDLN3VP/8-- From owner-freebsd-hackers@freebsd.org Wed Apr 3 09:11:57 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 651D31565734; Wed, 3 Apr 2019 09:11:57 +0000 (UTC) (envelope-from rozhuk.im@gmail.com) Received: from mail-ed1-x52c.google.com (mail-ed1-x52c.google.com [IPv6:2a00:1450:4864:20::52c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D15DF6E25A; Wed, 3 Apr 2019 09:11:56 +0000 (UTC) (envelope-from rozhuk.im@gmail.com) Received: by mail-ed1-x52c.google.com with SMTP id d26so14184030ede.10; Wed, 03 Apr 2019 02:11:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:date:to:cc:subject:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=IIoUkuK0FRxcICaoZO0Y3iJkVSOLmp72E9abMx6heq4=; b=VQNO4f3RRd2LlrGPYvHccPS4FqmRhwFtPN9rXhyXpV8zC31mS+KSZuiRD2lbDhS7Mr +DtqBDRJB0CagMAvSyr5dl/DUMlSdKKeePmKTrmtUy5CjYsKdX0jdHOTzA4bNxevRMjT dKQM0Pzh3+wphArASSSPa5O7PpJxb+7gWADLE1jzKJ1OLHf/d9P6gnqvdnwZEz77CU3Y rgqA06A1uAuTmxKw4s22LSVWye0+9fJytf5lM9i4heTNRZH7/QVP/qFoe2EKJS9mx+Yq Xsqd/RA9zzvWr6xMHxYzX2CEHOdBxK2mRauoM9ujnCJXOmTd3m4HexwPVl0yS/zAzOzN 0sfg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:date:to:cc:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=IIoUkuK0FRxcICaoZO0Y3iJkVSOLmp72E9abMx6heq4=; b=qNBzFWPK9nrIh3e+KqW7x5wi/iy2nlwdzzUpW2zsw3q2SwFvsb8ObOdUHUZLbiy/T0 fpcsfUzarvV4IjD8Ws6UbDDLg5Lu3fS5ebxR8poBLP7oVkixh5h10bS7lQJ7BiUKRGRP TQOVQxm6ANMA47KYHydCHBCR0Seey+pcv/mH1JoX347wq1GygtmS2Vhqa6e5rbO28T4b 5onbyxEKu59MCaDaaMTz2qfnbAFOf8cy+bhyL9H2wTPUUnqI3Xse5TTSl3rJzwI2fD5q +cAZY2wjemW/1vGqu0+WbopWPN2h478/w2//gujrodgNiACg5EedMqe4AHQBe5L6r4zN s2fg== X-Gm-Message-State: APjAAAWSVFzkbHlNchh3+0nSq8wXhTgLmxqea5Wk8EtWA/YXryr5oU+v CPtzhgNP/SqTZu6wh1XlDh0= X-Google-Smtp-Source: APXvYqx9dgnX0aO/u5Vlwt5nFy3xESE1I+pT5XQ4QhTcck/NVe9nQPJm4c2Aq1mZHRA+lXzALQkbGQ== X-Received: by 2002:a50:b79d:: with SMTP id h29mr47876554ede.233.1554282715552; Wed, 03 Apr 2019 02:11:55 -0700 (PDT) Received: from rimwks ([2001:470:1f15:3d8:692:26ff:fec0:61d6]) by smtp.gmail.com with ESMTPSA id f58sm4734916edd.42.2019.04.03.02.11.53 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Wed, 03 Apr 2019 02:11:54 -0700 (PDT) From: Rozhuk Ivan X-Google-Original-From: Rozhuk Ivan Date: Wed, 3 Apr 2019 12:11:52 +0300 To: Yamagi Burmeister Cc: dariusmihaim@gmail.com, freebsd-hackers@freebsd.org, freebsd-current@freebsd.org Subject: Re: Boot hang on ryzen based notebook Message-ID: <20190403121152.7f7f915c@rimwks> In-Reply-To: <20190403090522.548ffbdaad84c2c001a6926d@yamagi.org> References: <20190330164628.138a2bad@rimwks> <20190330204138.18fe8443@rimwks> <20190330211736.05b39193@rimwks> <20190403090522.548ffbdaad84c2c001a6926d@yamagi.org> X-Mailer: Claws Mail 3.17.3 (GTK+ 2.24.32; amd64-portbld-freebsd12.0) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: D15DF6E25A X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.98 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.98)[-0.979,0]; REPLY(-4.00)[]; TAGGED_FROM(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Apr 2019 09:11:57 -0000 On Wed, 3 Apr 2019 09:05:22 +0200 Yamagi Burmeister wrote: > > > You wrote "apic.0.disabled", not "acpi.0.disabled". Are you sure > > > you wrote the correct command? You do not want to disable the APIC > > > (interrupt controller), but the ACPI (control and power > > > interface). > > > > hint.acpi.0.disabled="1" - 11.13.1. > > Then set: > > panic: running without device atpic requires a local APIC > > Most modern hardware won't boot without ACPI. But I can get my Lenovo > E485 to boot with Linux and FreeBSD if I disable IOMMU aupport in the > BIOS / UEFI. > With hw.pci.mcfg=0 - boot ok. From owner-freebsd-hackers@freebsd.org Wed Apr 3 13:50:25 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D1BCA156EDBE for ; Wed, 3 Apr 2019 13:50:24 +0000 (UTC) (envelope-from mahesh86701997@gmail.com) Received: from mail-lf1-x131.google.com (mail-lf1-x131.google.com [IPv6:2a00:1450:4864:20::131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 946CA822CE for ; Wed, 3 Apr 2019 13:50:23 +0000 (UTC) (envelope-from mahesh86701997@gmail.com) Received: by mail-lf1-x131.google.com with SMTP id y18so11785962lfe.1 for ; Wed, 03 Apr 2019 06:50:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=l2hrrEX280+/3acq9FPxy/re41/0ZE5JemzKgBFKPUw=; b=NIEaWYCAg/dnS1vwlxto0P6IeDq+IDXuxEaigltV+D+pdLFNVXAd0TvPO/goobSknG MsCflFE+p1n9w/JsXi3kOAXjKO6wpzHsF5s18CHPQiL1hSotjdPRGrsdRzRm+2coyCUq e4mm2Ibt70M+vZkDCC7OQxsGZADZLhCp7GH+33ABdouHeCzscdjB99Atyn37ZG8r9vmm JNbaTBvrALhmCR2L0dShV8jTsnuk+4g1OA5yucAG2zHpNhEtPWsIIbVxdj76f938MtZY gJYyj2JS2NeLyfYLO/u38vm4YVhPF8d3u3/Im8pg/t0gUWmQvt+EsmGnWLqsmlfYDWs5 wIUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=l2hrrEX280+/3acq9FPxy/re41/0ZE5JemzKgBFKPUw=; b=Riz7pAhvGxuxtzAz4rfwbDK0jQ7xLmy4cU3zgpp+PXbv/fOsdfhIAnJTlAfNv4KHoy kz4GLiyA3toKcTslFH4hzxOqgmyl4/Ap0S/0dKcUpxm/OGOC7Qdbb+cQ2RP9H8hwOl5R jKPohQ8Xzdx/Uk1lWkG6l2lDGCbiAZqzNrdy05oJKj3nyGJ8sH3bF9RrFHZsCdrayqL2 25iPFLdlvQ6Z/SxbS6UN5xSq/Zc5UdAFlqfPeCOPnJsTzfeTOZ4n7gZKlBOIe9Apk26W GmCKcm5zGfJGb2bVEPyxSnpEdbf6ehqbawRvsZlzOIAM9ZFDzjQdS9QajVAfokMUrHT6 KUkw== X-Gm-Message-State: APjAAAVEx4VX/u7R3V9pqboO6ugVYMFRSyG+XFRRvDTV47Z6coTxTdgg uhe7dhEn90eX/TYA+Xo42aXKaY1+4Wg9nAwx1EoK5AjttfIggA== X-Google-Smtp-Source: APXvYqwhk1fGbQpDiYHnupPLyuJtTeVPd9lmw5XiYbmTZpr+9X4118y+CozL0JKGX76pHSKTwL7Lkv8jAMCRbA/KCPg= X-Received: by 2002:a19:c314:: with SMTP id t20mr39618673lff.114.1554299421974; Wed, 03 Apr 2019 06:50:21 -0700 (PDT) MIME-Version: 1.0 From: Mahesh Tudu Date: Wed, 3 Apr 2019 19:20:10 +0530 Message-ID: Subject: Regarding mentorship for GSOC 2019 To: freebsd-hackers@freebsd.org X-Rspamd-Queue-Id: 946CA822CE X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=NIEaWYCA; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of mahesh86701997@gmail.com designates 2a00:1450:4864:20::131 as permitted sender) smtp.mailfrom=mahesh86701997@gmail.com X-Spamd-Result: default: False [-4.26 / 15.00]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; ZERO_FONT(0.10)[1]; TO_DN_NONE(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; NEURAL_HAM_SHORT(-0.63)[-0.631,0]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; HTML_SHORT_LINK_IMG_1(2.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCPT_COUNT_ONE(0.00)[1]; MANY_INVISIBLE_PARTS(0.05)[1]; IP_SCORE(-2.77)[ip: (-9.23), ipnet: 2a00:1450::/32(-2.37), asn: 15169(-2.17), country: US(-0.06)]; RCVD_IN_DNSWL_NONE(0.00)[1.3.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Apr 2019 13:50:25 -0000 Hello, I am very much interested in the project - *Verification of bhyve's instruction emulation, *but on the FreeBSD idea list for GSOC 2019 there is no mentor allotted for it till now. I have done a lot of research work on the project and also have written tests for some instruction emulation, and so I want it to be reviewed by someone in FreeBSD. I also have written the proposal but I want it to be checked once by a mentor. If you know someone who can assist me on this project, then kindly let me know. I would be grateful for your help. [image: Mailtrack] Sender notified by Mailtrack 04/03/19, 7:20:01 PM From owner-freebsd-hackers@freebsd.org Wed Apr 3 13:40:31 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 86CBF156E545 for ; Wed, 3 Apr 2019 13:40:31 +0000 (UTC) (envelope-from mahesh86701997@gmail.com) Received: from mail-lj1-x22a.google.com (mail-lj1-x22a.google.com [IPv6:2a00:1450:4864:20::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2C8418189B for ; Wed, 3 Apr 2019 13:40:30 +0000 (UTC) (envelope-from mahesh86701997@gmail.com) Received: by mail-lj1-x22a.google.com with SMTP id p14so14907624ljg.5 for ; Wed, 03 Apr 2019 06:40:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=Csdjn4PDJlhyoC3Aa6XAIesJVwM3vdetBipnfbbw1hk=; b=L7YNFXAb7t045KjKOHEpdaBhM7omvKbygwZrtVkAxDe+2VyFBA1C13qhq+IPf7eBNG 1bwTO6ljYlf37/8aObl3OHuWbDPrw4J1mx5VG50eCLfHuEmV0yx4PJCiguRv5mNrlV5F NUwkNRQZ6qwyG59/8s3O+9UVeMV8VCB7crl29Lyev2idC/yOKmI/Qgbaehlwm5N1LZgJ tOyI3uF+qHtM8NA3Izom7nQHw9rntjkA/oJY1rXbqinxCOH7ALQ9Ta0mjw6xybOK+jiY qsj80Qn5P/irQx1kC+cVk0coXeEBGHuqpEYTfoUV3N4OOx2e0xehYOF8IGI9HI/sksB+ WwjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=Csdjn4PDJlhyoC3Aa6XAIesJVwM3vdetBipnfbbw1hk=; b=cGt6KYleRloOLK+H8gwHRLd50S2VzKhGL1HFKML738gUjyfbg/9u/55kldRmAX0euD aXj9vCun29CRpQoalkeemEC0JWOc8b11G/1Q0dGRjYQurBuIo5qzllU6x9j+bLIfXtF8 WWBCr8vGss96/vgkZnmc3VyOvS6V6BrjO5WpNFwA6Gb6LWA7gnDYkjMJ+wIhsfqOsZ/d ++BQFT5efKSn/De76lUcudPvtmo2HrlQsgYP85z0RCtcp45K9C5U29w3D6azyYIWycfJ uM9NWCKZDrCguazlMo0xP+Z7GtbLp0nmbgvwbn8uxhh+onFlLjzS3E5C0/mgE2Uj0qQw b2tA== X-Gm-Message-State: APjAAAUdYljPvZbdipHZ8W85b6HGi2z6afvl2ei5Nyl/QQepcwu7puEd YQ9FFbDAtbEll2zOOHhknCgMJogMUBX9bFZk46/rsX7UVxVi0g== X-Google-Smtp-Source: APXvYqzyG5AP4piVklvbGBk7gFtsovru0re27L/94knJmerFA9rHGfMaDVozqmygfbbn/GajVreeC5KtHlh/2g+bY4Y= X-Received: by 2002:a2e:b01a:: with SMTP id y26mr24368216ljk.38.1554298828355; Wed, 03 Apr 2019 06:40:28 -0700 (PDT) MIME-Version: 1.0 From: Mahesh Tudu Date: Wed, 3 Apr 2019 19:10:17 +0530 Message-ID: Subject: Regarding mentorship for GSOC 2019 To: freebsd-hackers@freebsd.org X-Rspamd-Queue-Id: 2C8418189B X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=L7YNFXAb; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of mahesh86701997@gmail.com designates 2a00:1450:4864:20::22a as permitted sender) smtp.mailfrom=mahesh86701997@gmail.com X-Spamd-Result: default: False [-4.17 / 15.00]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; ZERO_FONT(0.10)[1]; TO_DN_NONE(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; NEURAL_HAM_SHORT(-0.63)[-0.627,0]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; HTML_SHORT_LINK_IMG_1(2.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCPT_COUNT_ONE(0.00)[1]; MANY_INVISIBLE_PARTS(0.05)[1]; IP_SCORE(-2.69)[ip: (-8.82), ipnet: 2a00:1450::/32(-2.37), asn: 15169(-2.17), country: US(-0.06)]; RCVD_IN_DNSWL_NONE(0.00)[a.2.2.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0]; RCVD_COUNT_TWO(0.00)[2] X-Mailman-Approved-At: Wed, 03 Apr 2019 14:55:07 +0000 Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Apr 2019 13:40:31 -0000 Hello, I am very much interested in the project - *Verification of bhyve's instruction emulation, *but on the FreeBSD idea list for GSOC 2019 there is no mentor allotted for it till now. I have done a lot of research work on the project and also have written tests for some instruction emulation, and so I want it to be reviewed by someone in FreeBSD. I also have written the proposal but I want it to be checked once by a mentor. If you know someone who can assist me on this project, then kindly let me know. I would be grateful for your help. [image: Mailtrack] Sender notified by Mailtrack 04/03/19, 7:09:52 PM From owner-freebsd-hackers@freebsd.org Wed Apr 3 15:47:41 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E55281571B57; Wed, 3 Apr 2019 15:47:40 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail104.syd.optusnet.com.au (mail104.syd.optusnet.com.au [211.29.132.246]) by mx1.freebsd.org (Postfix) with ESMTP id 2AEBD87E7C; Wed, 3 Apr 2019 15:47:39 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from [192.168.0.102] (c110-21-101-228.carlnfd1.nsw.optusnet.com.au [110.21.101.228]) by mail104.syd.optusnet.com.au (Postfix) with ESMTPS id A9BCF4387D6; Thu, 4 Apr 2019 02:47:35 +1100 (AEDT) Date: Thu, 4 Apr 2019 02:47:34 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Konstantin Belousov cc: Michael Tuexen , Bruce Evans , freebsd-hackers Hackers , FreeBSD PowerPC ML Subject: Re: powerpc64 head -r344018 stuck sleeping problems: th->th_scale * tc_delta(th) overflows unsigned 64 bits sometimes [patched failed] In-Reply-To: <20190403070045.GW1923@kib.kiev.ua> Message-ID: <20190404011802.E2390@besplex.bde.org> References: <20190303161635.GJ68879@kib.kiev.ua> <20190304043416.V5640@besplex.bde.org> <20190304114150.GM68879@kib.kiev.ua> <20190305031010.I4610@besplex.bde.org> <20190306172003.GD2492@kib.kiev.ua> <20190308001005.M2756@besplex.bde.org> <20190307222220.GK2492@kib.kiev.ua> <20190309144844.K1166@besplex.bde.org> <20190324110138.GR1923@kib.kiev.ua> <20190403070045.GW1923@kib.kiev.ua> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.2 cv=P6RKvmIu c=1 sm=1 tr=0 cx=a_idp_d a=PalzARQSbocsUSjMRkwAPg==:117 a=PalzARQSbocsUSjMRkwAPg==:17 a=jpOVt7BSZ2e4Z31A5e1TngXxSK0=:19 a=kj9zAlcOel0A:10 a=pGLkceISAAAA:8 a=82MzWkLtsjH0Vdtkr0wA:9 a=bndD0-KsWL0R_FZ2:21 a=4Wq7EeSnkzghBKyU:21 a=CjuIK1q_8ugA:10 X-Rspamd-Queue-Id: 2AEBD87E7C X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.96 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.96)[-0.960,0]; REPLY(-4.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-Mailman-Approved-At: Wed, 03 Apr 2019 16:05:25 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Apr 2019 15:47:41 -0000 On Wed, 3 Apr 2019, Konstantin Belousov wrote: > On Wed, Apr 03, 2019 at 12:27:32AM +0200, Michael Tuexen wrote: >>> On 24. Mar 2019, at 12:01, Konstantin Belousov wrote: >>> >>> On Sat, Mar 09, 2019 at 06:00:14PM +1100, Bruce Evans wrote: >>>> I more strongly disclike (sic) the more complete merge. The central APIs >>>> have even more parameters and reduced type safety to describe objects as >>>> (offset, size) pairs. >>> I changed the patch to be type-safe. Now I like it even more. It provides >>> 1. internal >>> 2. concise >>> 3. type-safe >>> API to fetch data from timehands. The implementation needs to be read >>> only once. >> Hi, >> >> I'm a bit lost... I think this started to fix a problem on G5 PowerMacs. >> Do you think this patch solves the problem. Should this be tested? >> Or is this still work in progress or a general improvement not necessary >> fixing the problem on G5 PowerMacs? > > It started from a report of issues on G5. The specific issues are > bugs on G5, and the posted patches do not fix them. From what I see, > the timecounter values were wrapped. This is genuine ppc or G5 issue. I am still unhappy with the patch (but haven't got around to replying to the latest version). It is overengineered and doesn't fix the original problem. > The patch fixes time keeping subsystem reaction to the already > bad situation, by correctly handling overflow in calculations. This > overflow can occur in more reasonable setups as well, e.g. if ddb was > activated and interrupts were stopped for prolonged period, even on x86. I have noticed that it mostly fixes this. > In addition, my version of the patch reorganizes the code and removes > excessive copies of the most delicate loops in lock-less readers. This > chunk can be split from the overflow part, but it is not completely > trivial. This is the part that I don't like. I noticed (or better realized) a general problem with multiple timehands. ntpd can slew the clock at up to 500 ppm, and at least an old version of it uses a rate of 50 ppm to fix up fairly small drifts in the milliseconds range. 500 ppm is enormous in CPU cycles -- it is 500 thousand nsec or 2 million cycles at 4GHz. Winding up the timecounter every 1 msec reduces this to only 2000 cycles. More details of ordering and timing for 1 thread: - thread N calls binuptime() and it loads timehands - another or even the same thread runs tc_windup(). This modifies timehands. - thread N is preempted for a long time, but less than the time for updates - thread N checks the generation count. Since this is for the timehands contents and not for the timehands pointer, it hasn't changed, so the old timehands is used - and instant later, the same thread calls binuptime again() and uses the new timehands - now suppose only 2 timehands (as in -current) the worst (?) case of a slew of 500 ppm for the old timehands and -500 ppm for the new timehands and almost the worst case of 10 msec for the oldness of the old timehands relative to the new timehands, with the new timehands about to be updated after 10 msec (assume perfectly periodiodic updates every 10 msec). The calculated times are: old bintime = old_base + (20 msec) * (1 + 500e-6) new base = old_base + 10 msec * (1 + 500e-6) # calc by tc_windup() new bintime = new_base + (10 msec) * (1 - 500e-6) + epsilon error = epsilon - (20 msec) * 500e-6 = epsilon - 10000 nsec Errors in the negative direction are most harmful. ntpd normally doesn't change the skew as much as that in one step, but it is easy for adjtime(2) to change the skew like that and there are no reasonable microadjustments that would accidentally work around this kernel bug (it seems unreasonable to limit the skew to 1 ppm and that still gives an error of epsilon + 20 nsec. phk didn't want to slow down timecounters using extra code to make them them monotonic and coherent (getbinuptime() is incoherent with binuptime() since it former lags the latter by the update interval), but this wouldn't be very slow within a thread. Monotonicity across threads is a larger problem and not helped by using a faked forced monotonic time within threads. So it seems best to fix the above problem by moving the generation count from the timehands contents to the timehands pointer, and maybe also reduce the number of timehands to 1. With 2 timehands, this gives a shorter race: - thread N loads timehands - tc_windup() - thread N preempted - thread N uses old timehands - case tc_windup() completes first: no problem -- thread N checks the generation count on the pointer and loops - case binuptime() completes first: lost a race -- binuptime() is off by approx * . The main point of having multiple timehands (after introducing the per- timehands generation count) is to avoid blocking thread N during the update, but this doesn't actually work, even for only 2 timehands and a global generation count. Bruce From owner-freebsd-hackers@freebsd.org Wed Apr 3 16:11:24 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DE485157276C; Wed, 3 Apr 2019 16:11:24 +0000 (UTC) (envelope-from tuexen@freebsd.org) Received: from drew.franken.de (mail-n.franken.de [193.175.24.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.franken.de", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 78BDF88ED6; Wed, 3 Apr 2019 16:11:24 +0000 (UTC) (envelope-from tuexen@freebsd.org) Received: from [192.168.1.3] (p57BB4B1C.dip0.t-ipconnect.de [87.187.75.28]) (Authenticated sender: macmic) by mail-n.franken.de (Postfix) with ESMTPSA id 6E8E671B63002; Wed, 3 Apr 2019 18:11:20 +0200 (CEST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.8\)) Subject: Re: powerpc64 head -r344018 stuck sleeping problems: th->th_scale * tc_delta(th) overflows unsigned 64 bits sometimes [patched failed] From: Michael Tuexen In-Reply-To: <20190403070045.GW1923@kib.kiev.ua> Date: Wed, 3 Apr 2019 18:11:19 +0200 Cc: Bruce Evans , freebsd-hackers Hackers , FreeBSD PowerPC ML Content-Transfer-Encoding: quoted-printable Message-Id: <8184C845-7721-4F88-974E-3F63F1980FDE@freebsd.org> References: <20190303161635.GJ68879@kib.kiev.ua> <20190304043416.V5640@besplex.bde.org> <20190304114150.GM68879@kib.kiev.ua> <20190305031010.I4610@besplex.bde.org> <20190306172003.GD2492@kib.kiev.ua> <20190308001005.M2756@besplex.bde.org> <20190307222220.GK2492@kib.kiev.ua> <20190309144844.K1166@besplex.bde.org> <20190324110138.GR1923@kib.kiev.ua> <20190403070045.GW1923@kib.kiev.ua> To: Konstantin Belousov X-Mailer: Apple Mail (2.3445.104.8) X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=disabled version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on mail-n.franken.de X-Rspamd-Queue-Id: 78BDF88ED6 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.96 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.96)[-0.957,0]; REPLY(-4.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Apr 2019 16:11:25 -0000 > On 3. Apr 2019, at 09:00, Konstantin Belousov = wrote: >=20 > On Wed, Apr 03, 2019 at 12:27:32AM +0200, Michael Tuexen wrote: >>> On 24. Mar 2019, at 12:01, Konstantin Belousov = wrote: >>>=20 >>> On Sat, Mar 09, 2019 at 06:00:14PM +1100, Bruce Evans wrote: >>>> I more strongly disclike (sic) the more complete merge. The = central APIs >>>> have even more parameters and reduced type safety to describe = objects as >>>> (offset, size) pairs. >>> I changed the patch to be type-safe. Now I like it even more. It = provides >>> 1. internal >>> 2. concise >>> 3. type-safe >>> API to fetch data from timehands. The implementation needs to be = read >>> only once. >> Hi, >>=20 >> I'm a bit lost... I think this started to fix a problem on G5 = PowerMacs. >> Do you think this patch solves the problem. Should this be tested? >> Or is this still work in progress or a general improvement not = necessary >> fixing the problem on G5 PowerMacs? >=20 > It started from a report of issues on G5. The specific issues are > bugs on G5, and the posted patches do not fix them. =46rom what I = see, > the timecounter values were wrapped. This is genuine ppc or G5 issue. Thanks a lot for the clarification. Best regards Michael >=20 > The patch fixes time keeping subsystem reaction to the already > bad situation, by correctly handling overflow in calculations. This > overflow can occur in more reasonable setups as well, e.g. if ddb was > activated and interrupts were stopped for prolonged period, even on = x86. >=20 > In addition, my version of the patch reorganizes the code and removes > excessive copies of the most delicate loops in lock-less readers. This > chunk can be split from the overflow part, but it is not completely > trivial. From owner-freebsd-hackers@freebsd.org Wed Apr 3 18:41:50 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2F957154F3B5 for ; Wed, 3 Apr 2019 18:41:50 +0000 (UTC) (envelope-from brooks@spindle.one-eyed-alien.net) Received: from spindle.one-eyed-alien.net (spindle.one-eyed-alien.net [199.48.129.229]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 25DE18E6BB for ; Wed, 3 Apr 2019 18:41:48 +0000 (UTC) (envelope-from brooks@spindle.one-eyed-alien.net) Received: by spindle.one-eyed-alien.net (Postfix, from userid 3001) id 3E9BA3C475F; Wed, 3 Apr 2019 18:41:42 +0000 (UTC) Date: Wed, 3 Apr 2019 18:41:42 +0000 From: Brooks Davis To: Mahesh Tudu Cc: freebsd-hackers@freebsd.org Subject: Re: Regarding mentorship for GSOC 2019 Message-ID: <20190403184142.GA79445@spindle.one-eyed-alien.net> References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="J/dobhs11T7y2rNN" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.9.4 (2018-02-28) X-Rspamd-Queue-Id: 25DE18E6BB X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-3.86 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.98)[-0.983,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[freebsd.org]; AUTH_NA(1.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[spindle.one-eyed-alien.net]; MX_MISSING(3.50)[requested record is not found]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-0.91)[-0.906,0]; R_SPF_NA(0.00)[]; FREEMAIL_TO(0.00)[gmail.com]; FORGED_SENDER(0.30)[brooks@freebsd.org,brooks@spindle.one-eyed-alien.net]; SIGNED_PGP(-2.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; ASN(0.00)[asn:36236, ipnet:199.48.128.0/22, country:US]; FROM_NEQ_ENVFROM(0.00)[brooks@freebsd.org,brooks@spindle.one-eyed-alien.net]; IP_SCORE(-3.57)[ip: (-9.27), ipnet: 199.48.128.0/22(-4.60), asn: 36236(-3.89), country: US(-0.06)]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Apr 2019 18:41:50 -0000 --J/dobhs11T7y2rNN Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Wed, Apr 03, 2019 at 07:10:17PM +0530, Mahesh Tudu wrote: > Hello, > I am very much interested in the project - *Verification of bhyve's > instruction emulation, *but on the FreeBSD idea list for GSOC 2019 there is > no mentor allotted for it till now. I have done a lot of research work on > the project and also have written tests for some instruction emulation, and > so I want it to be reviewed by someone in FreeBSD. I also have written the > proposal but I want it to be checked once by a mentor. > If you know someone who can assist me on this project, then kindly let me > know. I would be grateful for your help. Thank you for your interest. It sounds like you've made a good start. I'll ask around a bit for possible mentors. -- Brooks --J/dobhs11T7y2rNN Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEcBAEBAgAGBQJcpP5kAAoJEKzQXbSebgfAzUIIAJQrV/eoSOddBLl17o5Y16/3 LRKssBo3Lbm4yQgS2C9Amsh+kJUfG+0DfS5J8V/LvAAnMLSiZ3LId9EI4AiWadw2 eWNUBE1u/PMiBABVgn4XNsE/1IU5p5k+QC0BcMLom9vl5g4ZycseGazSGAsgrR9f qLyBnuESKjjSGGgNB7spCv7Je+3fS3r0LiBHIFZQapJjExlY58lmOG070yeLGs/r f9394kTkeWiD0ZAbztshDEUR5e4i2ypmqA1WhpRil2foUJ0FJw8D2R+yriMPncn6 Aoq1ivg29YWo9kYWSZUqpzCG2agbTa4YLePLBNy6JalNUBXxRjR0KSeIKPsg75M= =N32I -----END PGP SIGNATURE----- --J/dobhs11T7y2rNN-- From owner-freebsd-hackers@freebsd.org Wed Apr 3 20:31:08 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4BA1F155271E for ; Wed, 3 Apr 2019 20:31:08 +0000 (UTC) (envelope-from mahesh86701997@gmail.com) Received: from mail-lf1-x133.google.com (mail-lf1-x133.google.com [IPv6:2a00:1450:4864:20::133]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 54F356C57D; Wed, 3 Apr 2019 20:31:07 +0000 (UTC) (envelope-from mahesh86701997@gmail.com) Received: by mail-lf1-x133.google.com with SMTP id 5so75887lft.12; Wed, 03 Apr 2019 13:31:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=Tf0GqowD3AMicgHScnU8RjCur/ugZhjdyoSgsdPtTyY=; b=lZWl7UhnqYUoxq+RvTNLgVxX8u76bNn+WxLdahPEkMiuaygBevaOu/FUfmwSGs+nwE /qLrLq7PbcieS2UMgECeb/TJf6BGakSx+i48f2gJxYNY2ZznpUKiE+y4JZTyVwaOLgX+ 5juX4KN5a5kA3bUZeWUgrxvsCYlf4Zk3f/1f9zfl6lCNygMeMtc+s1wuzUeePToSU/hY Sr7h/OHwbyW0yn1P/H6Z9CJiDKRUPH9d1EQtrbJ7pHZgjk54mnBjwecTc/4tBTYQjRFa GtRYtnSPPMYD1vuvL51+nzrp88wZk7TcXLsL+/kfcZSL2dbPeWxvUPO0jB2JV/Le10+g 1VzQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=Tf0GqowD3AMicgHScnU8RjCur/ugZhjdyoSgsdPtTyY=; b=gMJLTZFyl+lh55LhSrdvvv+yigvi2LiKQzKS8+XG6nG2mbPw7r4J57oDkogrUe6w3z qRbpRRshK/Ut12XQr5EkwFQ1JmDWJF9ovKWWwjA4WIoh8HmzWrVSeehlU5i70URFxajQ odNQTU8Z3b9TWUzob1aFdLHcxUbeSEvtUqnNElAGlqc3Tp7JfBLweuhe+DaxPRAUbKUf PqiWc+NRH8pFiJn7uM3pzJI/zsahd+Zed7pkgrxgm4YMIFkYd9aR2rxOf+0wgGVdpPU0 5sRhkHx93ECwAiJX6W9mBNi1uhrM0bzPu0gwJHl00fpMlFL9GngirT2NLGt0wkWG4KUI sQmw== X-Gm-Message-State: APjAAAWpavgHpYAB1ydIPn9PZTZwom5tmxeZzS2sMEIxDxY+lepdpq7m UJihybOItzQMG55g9NjTg1Y/gmVhxP9CZ/fY5Z7JnsYGPqI= X-Google-Smtp-Source: APXvYqx5pCzy+BzmqmkCO2rFCXW1kI29y6RMuDt3zIxWQK9UBJtpyLkq/pBevZz9juhIkIbaLb1KGlcdK+lXKHElofY= X-Received: by 2002:a19:c314:: with SMTP id t20mr939539lff.114.1554323465564; Wed, 03 Apr 2019 13:31:05 -0700 (PDT) MIME-Version: 1.0 References: <20190403184142.GA79445@spindle.one-eyed-alien.net> In-Reply-To: <20190403184142.GA79445@spindle.one-eyed-alien.net> From: Mahesh Tudu Date: Thu, 4 Apr 2019 02:00:54 +0530 Message-ID: Subject: Re: Regarding mentorship for GSOC 2019 To: Brooks Davis Cc: freebsd-hackers@freebsd.org X-Rspamd-Queue-Id: 54F356C57D X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=lZWl7Uhn; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of mahesh86701997@gmail.com designates 2a00:1450:4864:20::133 as permitted sender) smtp.mailfrom=mahesh86701997@gmail.com X-Spamd-Result: default: False [-4.45 / 15.00]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; ZERO_FONT(0.10)[1]; FREEMAIL_FROM(0.00)[gmail.com]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; NEURAL_HAM_SHORT(-0.82)[-0.819,0]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.996,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; HTML_SHORT_LINK_IMG_1(2.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; IP_SCORE(-2.78)[ip: (-9.28), ipnet: 2a00:1450::/32(-2.37), asn: 15169(-2.16), country: US(-0.06)]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MANY_INVISIBLE_PARTS(0.05)[1]; RCVD_IN_DNSWL_NONE(0.00)[3.3.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Apr 2019 20:31:08 -0000 Thank you for your response. I would be very pleased if I could get any help from your side. [image: Mailtrack] Sender notified by Mailtrack 04/04/19, 1:59:23 AM On Thu, Apr 4, 2019 at 12:11 AM Brooks Davis wrote: > On Wed, Apr 03, 2019 at 07:10:17PM +0530, Mahesh Tudu wrote: > > Hello, > > I am very much interested in the project - *Verification of bhyve's > > instruction emulation, *but on the FreeBSD idea list for GSOC 2019 there > is > > no mentor allotted for it till now. I have done a lot of research work on > > the project and also have written tests for some instruction emulation, > and > > so I want it to be reviewed by someone in FreeBSD. I also have written > the > > proposal but I want it to be checked once by a mentor. > > If you know someone who can assist me on this project, then kindly let me > > know. I would be grateful for your help. > > Thank you for your interest. It sounds like you've made a good > start. I'll ask around a bit for possible mentors. > > -- Brooks > From owner-freebsd-hackers@freebsd.org Thu Apr 4 20:16:05 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C7EDF1556353 for ; Thu, 4 Apr 2019 20:16:04 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic310-21.consmr.mail.gq1.yahoo.com (sonic310-21.consmr.mail.gq1.yahoo.com [98.137.69.147]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8AA168AC24 for ; Thu, 4 Apr 2019 20:16:03 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: tUl2yRAVM1mWa9EZN.Spbbd98seZjY_PZeD2.x5vKu1m3eempaq_3JfWWx7FY_s fvBY3fOt9Fhg5ve0M.CrRi1sXpQrxrXHOcSce9A4IxZMcNbIpicsSM5DTSaj7wd1maxyMuvOMtgU 0AnY1xYulmHaMv4219_1R0SWjhqtUcP9.31K3zc60yJKeeyFhFMwdNGyjGxo5YGC2Uy9tvr2E7eM itLgBgk11F2J3JqdeOEbk9TT0SdMK52qqfEWO1MUachqz8kTISq5FRHXCSpIPzWJpWF2xXynI1Vw 39j_bGs5luP4YX9bQ2Z0RFd45gt5h7QhPBWGXfso2G3s0c_NwAMZGxPlF59GUuCCKSGmyYUPXnRa blQozlJvxTi9vZ3GZkGkERWYUCfsT88o1P7x2N3m8kOANxfumKc2ESY5WZJjB03GVkhoHNmG4Af4 YaZkZaL.aLp_q.IeV3qeEFbyhk.ZbI2.4ZffDpsj4v4sup_JEAU58mPc94gZsUaX3k9zXPqKfuiR M7ThWGybRThlbxPSUv96VZARbP1MafRTWYu6tolvGMCOZQo314oqK6eAak_VpJMkXqE29NyoYawm LE9mcj42jxE2pmTBG2z6F9.Q8NeNLt8JiXOsPZ.qNb1e2csLotagwgjo1JOQqYLfI.nc5I0xxjKd Fu7LTwjvfIz70zOGH6wzHUP914MoEIL02S.boLroVAXg4tDJHav1J_Kce9HtrKBroioGaNI36jSJ vu9J4G80CyOfvwa.2qvO9P4OL9xvKnDBhiOQ3Ya4D2GMzcgLfQT6BIA0J5I6iJYwRtiNKbLI4RbY P_AntPHcrfIIoFsjahVbSZGRp5k_EIyfyCX_A1pob4ateBe3SBrCzM8ZhcTnBKfX1XVu97vdbE7p Y2puXXEYJ8u2JyzNJh_lSyqCSgRdsyMLCNmNsygAPCiFv1fp6BATXbQV0wYOPMbMrV5RKudE2dXh 6ZusHZ1n6eevT7.bHTQ5qUGqJEZ6bkgeecvlXjtWwWTmXGzhLNQug1Ur8yE_2IzMZd1.v2DYNS6V bjyQH7NX7N2JMb4eJu1cH0rYt9UoVxTYVyBBJrCuoF90sy54_EIRMlZFbGAZB11ZUm14ayC5gxoS CZw-- Received: from sonic.gate.mail.ne1.yahoo.com by sonic310.consmr.mail.gq1.yahoo.com with HTTP; Thu, 4 Apr 2019 20:16:01 +0000 Received: from c-67-170-167-181.hsd1.or.comcast.net (EHLO [192.168.1.113]) ([67.170.167.181]) by smtp420.mail.gq1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID 6a5cfc0e336cb9a55c16cba217ec176c; Thu, 04 Apr 2019 20:15:59 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.8\)) Subject: Re: powerpc64 head -r344018 stuck sleeping problems: th->th_scale * tc_delta(th) overflows unsigned 64 bits sometimes [patched failed] From: Mark Millard In-Reply-To: <20190404011802.E2390@besplex.bde.org> Date: Thu, 4 Apr 2019 13:15:58 -0700 Cc: Konstantin Belousov , freebsd-hackers Hackers , FreeBSD PowerPC ML Content-Transfer-Encoding: quoted-printable Message-Id: References: <20190303161635.GJ68879@kib.kiev.ua> <20190304043416.V5640@besplex.bde.org> <20190304114150.GM68879@kib.kiev.ua> <20190305031010.I4610@besplex.bde.org> <20190306172003.GD2492@kib.kiev.ua> <20190308001005.M2756@besplex.bde.org> <20190307222220.GK2492@kib.kiev.ua> <20190309144844.K1166@besplex.bde.org> <20190324110138.GR1923@kib.kiev.ua> <20190403070045.GW1923@kib.kiev.ua> <20190404011802.E2390@besplex.bde.org> To: Bruce Evans X-Mailer: Apple Mail (2.3445.104.8) X-Rspamd-Queue-Id: 8AA168AC24 X-Spamd-Bar: / X-Spamd-Result: default: False [0.97 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MV_CASE(0.50)[]; FREEMAIL_FROM(0.00)[yahoo.com]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; MX_GOOD(-0.01)[cached: mta6.am0.yahoodns.net]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; FREEMAIL_TO(0.00)[optusnet.com.au]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/21, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.21)[-0.212,0]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; NEURAL_SPAM_SHORT(0.70)[0.698,0]; NEURAL_HAM_LONG(-0.10)[-0.102,0]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE(1.09)[ip: (3.81), ipnet: 98.137.64.0/21(0.96), asn: 36647(0.76), country: US(-0.06)]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[147.69.137.98.list.dnswl.org : 127.0.5.0]; FREEMAIL_CC(0.00)[gmail.com] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Apr 2019 20:16:05 -0000 On 2019-Apr-3, at 08:47, Bruce Evans wrote: > . . . >=20 > I noticed (or better realized) a general problem with multiple > timehands. ntpd can slew the clock at up to 500 ppm, and at least an > old version of it uses a rate of 50 ppm to fix up fairly small drifts > in the milliseconds range. 500 ppm is enormous in CPU cycles -- it is > 500 thousand nsec or 2 million cycles at 4GHz. Winding up the = timecounter > every 1 msec reduces this to only 2000 cycles. >=20 > More details of ordering and timing for 1 thread: > - thread N calls binuptime() and it loads timehands > - another or even the same thread runs tc_windup(). This modifies = timehands. > - thread N is preempted for a long time, but less than the time for > updates > - thread N checks the generation count. Since this is for the = timehands > contents and not for the timehands pointer, it hasn't changed, so the > old timehands is used > - and instant later, the same thread calls binuptime again() and uses = the > new timehands - now suppose only 2 timehands (as in -current) the = worst (?) case of a > slew of 500 ppm for the old timehands and -500 ppm for the new = timehands > and almost the worst case of 10 msec for the oldness of the old = timehands > relative to the new timehands, with the new timehands about to be = updated > after 10 msec (assume perfectly periodiodic updates every 10 msec). = The > calculated times are: >=20 > old bintime =3D old_base + (20 msec) * (1 + 500e-6) > new base =3D old_base + 10 msec * (1 + 500e-6) # calc by = tc_windup() > new bintime =3D new_base + (10 msec) * (1 - 500e-6) + epsilon >=20 > error =3D epsilon - (20 msec) * 500e-6 =3D epsilon - 10000 nsec >=20 > Errors in the negative direction are most harmful. ntpd normally = doesn't > change the skew as much as that in one step, but it is easy for = adjtime(2) > to change the skew like that and there are no reasonable = microadjustments > that would accidentally work around this kernel bug (it seems = unreasonable > to limit the skew to 1 ppm and that still gives an error of epsilon + = 20 > nsec. >=20 > phk didn't want to slow down timecounters using extra code to make > them them monotonic and coherent (getbinuptime() is incoherent with > binuptime() since it former lags the latter by the update interval), > but this wouldn't be very slow within a thread. >=20 > Monotonicity across threads is a larger problem and not helped by = using > a faked forced monotonic time within threads. >=20 > So it seems best to fix the above problem by moving the generation = count > from the timehands contents to the timehands pointer, and maybe also > reduce the number of timehands to 1. With 2 timehands, this gives a > shorter race: >=20 > - thread N loads timehands > - tc_windup() > - thread N preempted > - thread N uses old timehands > - case tc_windup() completes first: no problem -- thread N checks the > generation count on the pointer and loops > - case binuptime() completes first: lost a race -- binuptime() is off > by approx * . >=20 > The main point of having multiple timehands (after introducing the = per- > timehands generation count) is to avoid blocking thread N during the > update, but this doesn't actually work, even for only 2 timehands and = a global generation count. >=20 Thanks for the description of an example way that sbinuptime and the like might not give weakly increasing results. Unfortunately, all the multi-socket contexts that I sometimes have access to are old PowerMacs. And, currently, the only such context is the G5 with 2 sockets, 2 cores per socket (powerpc64). So I've not been able to set up other types of examples to see if problems repeat. I do not have access to a single-socket powerpc64 for contrast in that direction. One oddity is that the eventtimer's decrementer and timecounter may change (nearly) together: both change at 33,333,333 Hz, as if they are tied to the same clock (at least on one socket). In case it helps with knowing how analogous your investigations are to the original problem context, I report the following. If you do not care for such information, stop reading here. # grep ntpd /etc/rc.conf ntpd_enable=3D"YES" ntpd_sync_on_start=3D"YES" # sysctl kern.eventtimer kern.eventtimer.periodic: 0 kern.eventtimer.timer: decrementer kern.eventtimer.idletick: 0 kern.eventtimer.singlemul: 2 kern.eventtimer.choice: decrementer(1000) kern.eventtimer.et.decrementer.quality: 1000 kern.eventtimer.et.decrementer.frequency: 33333333 kern.eventtimer.et.decrementer.flags: 7 # vmstat -ai | grep decrementer cpu0:decrementer 4451007 35 cpu3:decrementer 1466010 11 cpu2:decrementer 1481722 12 cpu1:decrementer 1478618 12 (That last is from a basically-idle timeframe.) # sysctl -a | grep hz kern.clockrate: { hz =3D 1000, tick =3D 1000, profhz =3D 8128, stathz =3D = 127 } kern.hz: 1000 # sysctl kern.timecounter kern.timecounter.fast_gettime: 1 kern.timecounter.tick: 1 kern.timecounter.choice: timebase(0) dummy(-1000000) kern.timecounter.hardware: timebase kern.timecounter.alloweddeviation: 5 kern.timecounter.stepwarnings: 0 kern.timecounter.tc.timebase.quality: 0 kern.timecounter.tc.timebase.frequency: 33333333 kern.timecounter.tc.timebase.counter: 1144662532 kern.timecounter.tc.timebase.mask: 4294967295 (The actual Time Base Register (tbr) i s 64 bits and freebsd truncates it down.) # sysctl -a | grep 'cpu.*freq' device cpufreq debug.cpufreq.verbose: 0 debug.cpufreq.lowest: 0 dev.cpufreq.0.%parent: cpu3 dev.cpufreq.0.%pnpinfo:=20 dev.cpufreq.0.%location:=20 dev.cpufreq.0.%driver: cpufreq dev.cpufreq.0.%desc:=20 dev.cpufreq.%parent:=20 dev.cpu.3.freq_levels: 2500/-1 1250/-1 dev.cpu.3.freq: 2500 So 2500 MHz / 33333333 Hz is very near 75 clock periods per timebase counter value. I do sometimes have access to a Ryzen Threadripper 1950X based system: FreeBSD/SMP: Multiprocessor System Detected: 32 CPUs FreeBSD/SMP: 1 package(s) x 2 groups x 2 cache groups x 4 core(s) x 2 = hardware threads but it is single=3Dsocket. It has . . . [It turns out that I've accidentally been running it without ntpd being enabled in /etc.rc.conf . Just fixed for next boot.] # sysctl kern.eventtimer = = kern.eventtimer.periodic: = 0 kern.eventtimer.timer: LAPIC kern.eventtimer.idletick: 0 kern.eventtimer.singlemul: 2 kern.eventtimer.choice: LAPIC(600) HPET(350) HPET1(350) HPET2(350) = i8254(100) RTC(0) kern.eventtimer.et.RTC.quality: 0 kern.eventtimer.et.RTC.frequency: 32768 kern.eventtimer.et.RTC.flags: 17 kern.eventtimer.et.i8254.quality: 100 kern.eventtimer.et.i8254.frequency: 1193182 kern.eventtimer.et.i8254.flags: 1 kern.eventtimer.et.HPET2.quality: 350 kern.eventtimer.et.HPET2.frequency: 14318180 kern.eventtimer.et.HPET2.flags: 3 kern.eventtimer.et.HPET1.quality: 350 kern.eventtimer.et.HPET1.frequency: 14318180 kern.eventtimer.et.HPET1.flags: 3 kern.eventtimer.et.HPET.quality: 350 kern.eventtimer.et.HPET.frequency: 14318180 kern.eventtimer.et.HPET.flags: 3 kern.eventtimer.et.LAPIC.quality: 600 kern.eventtimer.et.LAPIC.frequency: 49907650 kern.eventtimer.et.LAPIC.flags: 7 # vmstat -ai | grep timer irq0: attimer0 0 0 cpu0:timer 609974 14 cpu1:timer 160546 4 cpu2:timer 99803 2 cpu3:timer 158073 4 cpu4:timer 102870 2 cpu5:timer 168433 4 cpu6:timer 163004 4 cpu7:timer 162947 4 cpu8:timer 163501 4 cpu9:timer 160595 4 cpu10:timer 169100 4 cpu11:timer 163888 4 cpu12:timer 144278 3 cpu13:timer 162706 4 cpu14:timer 164161 4 cpu15:timer 167101 4 cpu16:timer 187732 4 cpu17:timer 189228 4 cpu18:timer 179423 4 cpu19:timer 182701 4 cpu20:timer 139658 3 cpu21:timer 186046 4 cpu22:timer 201248 5 cpu23:timer 184823 4 cpu24:timer 186839 4 cpu25:timer 186008 4 cpu26:timer 191473 4 cpu27:timer 182573 4 cpu28:timer 181213 4 cpu29:timer 197659 5 cpu30:timer 188190 4 cpu31:timer 189092 4 (Again: from a basically-idle timeframe.) # sysctl -a | grep hz kern.clockrate: { hz =3D 1000, tick =3D 1000, profhz =3D 8128, stathz =3D = 127 } kern.hz: 1000 debug.psm.hz: 20 # sysctl -a | grep 'cpu.*freq' device cpufreq debug.cpufreq.verbose: 0 debug.cpufreq.lowest: 0 dev.cpufreq.0.%parent: cpu0 dev.cpufreq.0.%pnpinfo:=20 dev.cpufreq.0.%location:=20 dev.cpufreq.0.%driver: cpufreq dev.cpufreq.0.%desc:=20 dev.cpufreq.%parent:=20 dev.cpu.0.freq_levels: 3400/-1 2800/-1 2200/-1 dev.cpu.0.freq: 3400 # sysctl kern.timecounter kern.timecounter.tsc_shift: 1 kern.timecounter.smp_tsc_adjust: 0 kern.timecounter.smp_tsc: 1 kern.timecounter.invariant_tsc: 1 kern.timecounter.fast_gettime: 1 kern.timecounter.tick: 1 kern.timecounter.choice: ACPI-fast(900) i8254(0) HPET(950) TSC-low(1000) = dummy(-1000000) kern.timecounter.hardware: TSC-low kern.timecounter.alloweddeviation: 5 kern.timecounter.stepwarnings: 0 kern.timecounter.tc.ACPI-fast.quality: 900 kern.timecounter.tc.ACPI-fast.frequency: 3579545 kern.timecounter.tc.ACPI-fast.counter: 3941257163 kern.timecounter.tc.ACPI-fast.mask: 4294967295 kern.timecounter.tc.i8254.quality: 0 kern.timecounter.tc.i8254.frequency: 1193182 kern.timecounter.tc.i8254.counter: 54249 kern.timecounter.tc.i8254.mask: 65535 kern.timecounter.tc.HPET.quality: 950 kern.timecounter.tc.HPET.frequency: 14318180 kern.timecounter.tc.HPET.counter: 796737466 kern.timecounter.tc.HPET.mask: 4294967295 kern.timecounter.tc.TSC-low.quality: 1000 kern.timecounter.tc.TSC-low.frequency: 1696860321 kern.timecounter.tc.TSC-low.counter: 3613142695 kern.timecounter.tc.TSC-low.mask: 4294967295 =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-hackers@freebsd.org Thu Apr 4 20:43:47 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 17F1E15589CC for ; Thu, 4 Apr 2019 20:43:47 +0000 (UTC) (envelope-from pkaipila@gmail.com) Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com [IPv6:2a00:1450:4864:20::436]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 105CD8C4C4 for ; Thu, 4 Apr 2019 20:43:46 +0000 (UTC) (envelope-from pkaipila@gmail.com) Received: by mail-wr1-x436.google.com with SMTP id w10so5459815wrm.4 for ; Thu, 04 Apr 2019 13:43:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=to:from:subject:message-id:date:user-agent:mime-version :content-language:content-transfer-encoding; bh=rmE2Av6NivIWR2DcDqaRcyud7ygxUwNytzmKdlT4RA4=; b=MK47FjUJyBy+QXQyyvEJhS+y1QwFZtV3hJgH8nTdH7OnpSlPu4Fe+KI5gADheC/c4L eZZhA6RDl6xJc5n03kvokEciYD1SuXIFAss04uJaaO4osy5xYXKzGYjC0WgRX4cwXoMg Aw3NVUZ6TeySYAPdVAmnI0o2dWLSIJO3nYjNnwREORQgG7FFt1+BU1RcQOfmSoSTiDkr jKs/KFIv4rNEhRvWj5FUWDZqc+bVb4Qt5RA/084yTjgGy5Vn59MwSRztzpvUYzI9tnI7 kRLsrIcEFzhde2GIA8BC6Slsu74ygWWbtr8f3GbRR3CLR1vjjLGAQDTXQIzSP/Hpbxzs L8gw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:to:from:subject:message-id:date:user-agent :mime-version:content-language:content-transfer-encoding; bh=rmE2Av6NivIWR2DcDqaRcyud7ygxUwNytzmKdlT4RA4=; b=J1toKhwlIlkgnBy+Hqb1xF/HPOKZh2mVwtquC4fbStO0QuyLYv52VYqgW3mbP3asFi yV8tFkChjvT6ZiGDWuqXdNAv5KZ5RauzCL8qsYSKBc9w3np8KRtU282+KPv2Wknku4eO xaZ/Cnv1FH1fBmIk2WhLj1UCQCXvY71FvcYX6wlj9BW+PHdjx6Lyd34LsqGtoaXfKBsM M6lI5OVVxZw/NsHxM0TL34RhDVMcBm64U6okkF2qXe3bp4NrtwyPCCxMCVMDvAwFaT+t Qg5gJO/vO/6na6+zjJ2RGcGbhkskbuU9VtyXad+X/fh29NCxgYCxIxqCFCuR5vnzQc5R 0X0g== X-Gm-Message-State: APjAAAUAjtTeUge/GIpe6YvO6XECRNPdb1W/1H4Te/LAKHQ/60hzfeVE Rabhlm3pj1jGb7UjRU7D76/vyZ2T X-Google-Smtp-Source: APXvYqzuJyljxbvxrXYahyOGrsX4imieR+lbkRM4Qu16Rde5YqurV0vGqd3ZZn2PKg1hFMFQEtmQ/Q== X-Received: by 2002:adf:e50d:: with SMTP id j13mr5859876wrm.165.1554410623790; Thu, 04 Apr 2019 13:43:43 -0700 (PDT) Received: from [153.1.251.192] (roam-fi-251-192.uta.fi. [153.1.251.192]) by smtp.gmail.com with ESMTPSA id e9sm32364335wrp.35.2019.04.04.13.43.42 for (version=TLS1_3 cipher=AEAD-AES128-GCM-SHA256 bits=128/128); Thu, 04 Apr 2019 13:43:42 -0700 (PDT) To: freebsd-hackers@freebsd.org From: Paavo-Einari Kaipila Subject: Virtual memory compression [GSoC proposal] Message-ID: Date: Thu, 4 Apr 2019 23:43:40 +0300 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:60.0) Gecko/20100101 Thunderbird/60.6.1 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 105CD8C4C4 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=MK47FjUJ; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of pkaipila@gmail.com designates 2a00:1450:4864:20::436 as permitted sender) smtp.mailfrom=pkaipila@gmail.com X-Spamd-Result: default: False [-6.84 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-0.99)[-0.986,0]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; IP_SCORE(-2.84)[ip: (-9.59), ipnet: 2a00:1450::/32(-2.37), asn: 15169(-2.17), country: US(-0.06)]; RCVD_IN_DNSWL_NONE(0.00)[6.3.4.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Apr 2019 20:43:47 -0000 Hey I am a student from Tampere University and I've been thinking about virtual memory compression for FreeBSD for few years. I am now going to propose it as a GSoC project. Any comments are highly appreciated. And of course, suggestions for potential mentor too. The the proposal draft: https://docs.google.com/document/d/1QijMGCWq69vWTm-noOn0K4Qmh7wYKBAoqRZvrgewR70/edit?usp=sharing regards, Paavo-Einari Kaipila From owner-freebsd-hackers@freebsd.org Fri Apr 5 08:40:19 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9F4AE156EB7D for ; Fri, 5 Apr 2019 08:40:19 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic316-8.consmr.mail.gq1.yahoo.com (sonic316-8.consmr.mail.gq1.yahoo.com [98.137.69.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 41FDB850A9 for ; Fri, 5 Apr 2019 08:40:18 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: VXMvz5oVM1lVot.ZVGNj0UQSOyj6Ni7RoNkSCdjgRVXFBnGiPKOjpE5Zi6a5y.X qN9LPcDwHp_bK4aECSY19vbASlQ0YO1klqDnu_6De9akLMgvCY4nNbZ6unYeC_wflLQIEx5yVUfm W4EDvxzfaYetPY1SdR8LbXte7Z4wXLVQD3ESae70sBu3mxGJUx.RIa20UZv6zveRgRPzHMrUeUZ3 oknukAca4A9fXK8nCmLU35OCV_5MM4WZdi03iY_aEZSI4FpmsvShRoaNL5R5eknvLVVpKLkrfWuL 6j81MDZfSHZpJG_clu5cwlAgU6HJd0u04N5GbLVTuAMXUiE3KuWLBUORsGNU1NJLCBe2AL0Z.NtJ mbwsmxmvvS1T25yWiYRicNbseNcBQfZErelkOeX1yDkKIwwRAr6bE_V2E7Nd5Qxo9MuBrJcc7v6m aL0lkqyehIsJXiQ152wrlQW2idWPHVvo1uGS.SkzZBE1E14e547hGtZnkgzwVZYkDZeZ_zZdkEL9 anTz7ZNwo0U7zlbKV2X8tCdEyGk3gfYcT6.6u9a2813jx.79t4Hprc3xN6U8iliqcD5cQfwWos_p C_ZagEBWfau8GFgDmsURlaW_tt17u7eHad7dUunI2nICXaLRQPXgnFQ52QkuK.IT2_Vyx.mQgJAN cEv.LxZ34lI4jb8jghgcAzSy401vtLOkKc9rsbEeDssJ7g4PESmejxmoIh8XL4Yo12u5PY_r4VJu gkbJdhsj1A7R15S7sF56wAVRK2dBGuSKBiGCPY5scR2.CJBwUezUyTjFGARm9JEgz3xibvqERgLh tR2QwVa79V8ZuD7LLQBaOIZ3yYKgGwkGq53FZMkpKymnhxEaU5RcPIgIMv4CO3uqR5ahJzLV5D1X WNryDviRgCHNAvcZikXZQEA3jzRxZmRbqk4puAhPMrCp7RDIgGcpqRrJF4AvH711D84xHT6SLYwG DLBB.BZ0vFB1o4x64_GgxRARxik1LtuhaSIgD_RKXRihR1TV3DN.CQ7GtG4p4hICocTswXgZDnPL A2dUyvaH8_VLQ5Pn.vLaiO5QkjU6d.Qp7B0vvY7TrCG8FOKVo6ow5Gszm7QBr4SmkyVKwYoq.3o9 b8g-- Received: from sonic.gate.mail.ne1.yahoo.com by sonic316.consmr.mail.gq1.yahoo.com with HTTP; Fri, 5 Apr 2019 08:40:10 +0000 Received: from c-67-170-167-181.hsd1.or.comcast.net (EHLO [192.168.1.113]) ([67.170.167.181]) by smtp428.mail.gq1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID 4ebaa5d02eedc5f0d26af1d554561570; Fri, 05 Apr 2019 08:40:08 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.8\)) Subject: Re: powerpc64 head -r344018 stuck sleeping problems: th->th_scale * tc_delta(th) overflows unsigned 64 bits sometimes [patched failed] From: Mark Millard In-Reply-To: <20190405150236.A959@besplex.bde.org> Date: Fri, 5 Apr 2019 01:40:07 -0700 Cc: Konstantin Belousov , freebsd-hackers Hackers , FreeBSD PowerPC ML Content-Transfer-Encoding: quoted-printable Message-Id: References: <20190303161635.GJ68879@kib.kiev.ua> <20190304043416.V5640@besplex.bde.org> <20190304114150.GM68879@kib.kiev.ua> <20190305031010.I4610@besplex.bde.org> <20190306172003.GD2492@kib.kiev.ua> <20190308001005.M2756@besplex.bde.org> <20190307222220.GK2492@kib.kiev.ua> <20190309144844.K1166@besplex.bde.org> <20190324110138.GR1923@kib.kiev.ua> <20190403070045.GW1923@kib.kiev.ua> <20190404011802.E2390@besplex.bde.org> <20190405150236.A959@besplex.bde.org> To: Bruce Evans X-Mailer: Apple Mail (2.3445.104.8) X-Rspamd-Queue-Id: 41FDB850A9 X-Spamd-Bar: + X-Spamd-Result: default: False [1.83 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MV_CASE(0.50)[]; FREEMAIL_FROM(0.00)[yahoo.com]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; MX_GOOD(-0.01)[cached: mta6.am0.yahoodns.net]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; FREEMAIL_TO(0.00)[optusnet.com.au]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/21, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; NEURAL_SPAM_SHORT(0.81)[0.808,0]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE(1.28)[ip: (4.75), ipnet: 98.137.64.0/21(0.95), asn: 36647(0.76), country: US(-0.06)]; NEURAL_SPAM_MEDIUM(0.03)[0.028,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_SPAM_LONG(0.23)[0.226,0]; RCVD_IN_DNSWL_NONE(0.00)[32.69.137.98.list.dnswl.org : 127.0.5.0]; FREEMAIL_CC(0.00)[gmail.com] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Apr 2019 08:40:19 -0000 On 2019-Apr-4, at 21:38, Bruce Evans wrote: > On Thu, 4 Apr 2019, Mark Millard wrote: >=20 >> On 2019-Apr-3, at 08:47, Bruce Evans wrote: >>> . . . >>>=20 >>> I noticed (or better realized) a general problem with multiple >>> timehands. ntpd can slew the clock at up to 500 ppm, and at least = an >>> old version of it uses a rate of 50 ppm to fix up fairly small = drifts >>> in the milliseconds range. 500 ppm is enormous in CPU cycles -- it = is >>> 500 thousand nsec or 2 million cycles at 4GHz. Winding up the = timecounter >>> every 1 msec reduces this to only 2000 cycles. >>>=20 >>> More details of ordering and timing for 1 thread: >>> ... >> Thanks for the description of an example way that sbinuptime and >> the like might not give weakly increasing results. >>=20 >> Unfortunately, all the multi-socket contexts that I sometimes have >> access to are old PowerMacs. And, currently, the only such context >> is the G5 with 2 sockets, 2 cores per socket (powerpc64). So I've >> not been able to set up other types of examples to see if problems >> repeat. >>=20 >> I do not have access to a single-socket powerpc64 for contrast in >> that direction. >=20 > Testing 1 socket is time-consuming enough. Do these old systems > use the equivalent of an x86 TSC for the timecounter? With multiple > sockets, it isn't clear how even a hardware timer independent of the > CPUs can be distributed so as to appear to be monotonic on all cors. "The DEC frequency is based on the same implementation-dependent frequency that drives the time base." The frequency may well be fixed by the PowerMac G5 model implementation but is not fixed by the powerpc64 architecture. The Time Base Register (TBR) in a powerpc64 core (cpu in FreeBSD terms) increments at 33,333,333 Hz (nominal) and is 64 bits wide. Its value can be set (mttb instruction) and the boot sequence in FreeBSD does attempt to adjust as the FreeBSD CPU is brought-up/started. mftb is used to read the 64-bit value. FreeBSD masks it down to 32-bits to contribute to the time-counter. (Is that description sufficient for what you were after? I've never seen documentation of how the 33,333,333 MHz is produced.) As FreeBSD supports multi-socket, what are its criteria for a sufficient context for it to work with for supporting sbinuptime and the like? Is FreeBSD supposed to then make it appear that sbinputime and the like are weakly increasing, even as threads migrate between CPUs (cores, hw-threads)? >> One oddity is that the eventtimer's decrementer and timecounter >> may change (nearly) together: both change at 33,333,333 Hz, as if >> they are tied to the same clock (at least on one socket). >=20 > I think this is from a normal hardware implementation. On all of > my x86 systems with a TSC, the TSC frequency is an exact fractional > multiple of the i8254, the ACPI timer (if present) and the HPET (if > present). Only the RTC has an independent frequency. The fraction is > changed by changing the nominal TSC frequency in the BIOS, but is not > changed by temperature variations. This must be because most clocks = are > derived from a common clock using a PLL. I use this to calibrate all > clocks (except the RTC) by calibrating only 1. I'm not aware of the OpenFirmware having any control over the TBR-change frequency behavior. I've no evidence about any variability based on temperature. >> In case it helps with knowing how analogous your investigations >> are to the original problem context, I report the following. >>=20 >> If you do not care for such information, stop reading here. >>=20 >> # grep ntpd /etc/rc.conf >> ntpd_enable=3D"YES" >> ntpd_sync_on_start=3D"YES" >>=20 >> # sysctl kern.eventtimer >> kern.eventtimer.periodic: 0 >> kern.eventtimer.timer: decrementer >> kern.eventtimer.idletick: 0 >> kern.eventtimer.singlemul: 2 >> kern.eventtimer.choice: decrementer(1000) >> kern.eventtimer.et.decrementer.quality: 1000 >> kern.eventtimer.et.decrementer.frequency: 33333333 >> kern.eventtimer.et.decrementer.flags: 7 >>=20 >> # vmstat -ai | grep decrementer >> cpu0:decrementer 4451007 35 >> cpu3:decrementer 1466010 11 >> cpu2:decrementer 1481722 12 >> cpu1:decrementer 1478618 12 >=20 > Powerpc seems to have a PLL in software too. Event timers don't need = to > be very precise or accurate. powerpc64 sets the value to count down from (in the 32-bit DEC regsiter) via the mtdec instruction. I'm not ware of being=20 able to change the frequency that the countdown occurs at on the old PowerMac G5's. (mtdec is a form of mtspr SPRNUM,rs .) [Accessing the DEC's value is via a privileged instruction on powerpc64 vs. a non-privileged one on power, different instruction encodings but the same mnemonic unless mfspr is used directly as the mnemonic. Just a difference in one bit position in the SPUNUM in teh encoding.] >> (That last is from a basically-idle timeframe.) >>=20 >> # sysctl -a | grep hz >> kern.clockrate: { hz =3D 1000, tick =3D 1000, profhz =3D 8128, stathz = =3D 127 } >> kern.hz: 1000 >=20 > x86 is similar. I think synchronization from using PLLs still gives > unfair scheduling, but with multiple CPUs and often more cycles than = can > be used, no one cares about accidental synchronization or bothers to = steal > cycles using intentional synchronization. Good to know. >> # sysctl kern.timecounter >> kern.timecounter.fast_gettime: 1 >> kern.timecounter.tick: 1 >> kern.timecounter.choice: timebase(0) dummy(-1000000) >> kern.timecounter.hardware: timebase >> kern.timecounter.alloweddeviation: 5 >> kern.timecounter.stepwarnings: 0 >> kern.timecounter.tc.timebase.quality: 0 >> kern.timecounter.tc.timebase.frequency: 33333333 >> kern.timecounter.tc.timebase.counter: 1144662532 >> kern.timecounter.tc.timebase.mask: 4294967295 >>=20 >> (The actual Time Base Register (tbr) i s 64 bits >> and freebsd truncates it down.) >>=20 >> # sysctl -a | grep 'cpu.*freq' >> device cpufreq >> debug.cpufreq.verbose: 0 >> debug.cpufreq.lowest: 0 >> dev.cpufreq.0.%parent: cpu3 >> dev.cpufreq.0.%pnpinfo: >> dev.cpufreq.0.%location: >> dev.cpufreq.0.%driver: cpufreq >> dev.cpufreq.0.%desc: >> dev.cpufreq.%parent: >> dev.cpu.3.freq_levels: 2500/-1 1250/-1 >> dev.cpu.3.freq: 2500 >>=20 >> So 2500 MHz / 33333333 Hz is very near 75 clock periods per >> timebase counter value. >=20 > Looks like it is exactly 75. Fractions are especially easy to guess = and > verify when they are integral. I'm not sure what happens for DEC and TBR change frequencies if the 2500 MHz cpu frequency is dropped down to 1250 MHz. But as I understand my context, 1250 MHz is not in use at all, just 2500 MHz. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-hackers@freebsd.org Fri Apr 5 04:38:16 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AED1715695DB; Fri, 5 Apr 2019 04:38:16 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail106.syd.optusnet.com.au (mail106.syd.optusnet.com.au [211.29.132.42]) by mx1.freebsd.org (Postfix) with ESMTP id 8DD5674FED; Fri, 5 Apr 2019 04:38:06 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from [192.168.0.102] (c110-21-101-228.carlnfd1.nsw.optusnet.com.au [110.21.101.228]) by mail106.syd.optusnet.com.au (Postfix) with ESMTPS id C8F303DBE4B; Fri, 5 Apr 2019 15:38:02 +1100 (AEDT) Date: Fri, 5 Apr 2019 15:38:02 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Mark Millard cc: Bruce Evans , Konstantin Belousov , freebsd-hackers Hackers , FreeBSD PowerPC ML Subject: Re: powerpc64 head -r344018 stuck sleeping problems: th->th_scale * tc_delta(th) overflows unsigned 64 bits sometimes [patched failed] In-Reply-To: Message-ID: <20190405150236.A959@besplex.bde.org> References: <20190303161635.GJ68879@kib.kiev.ua> <20190304043416.V5640@besplex.bde.org> <20190304114150.GM68879@kib.kiev.ua> <20190305031010.I4610@besplex.bde.org> <20190306172003.GD2492@kib.kiev.ua> <20190308001005.M2756@besplex.bde.org> <20190307222220.GK2492@kib.kiev.ua> <20190309144844.K1166@besplex.bde.org> <20190324110138.GR1923@kib.kiev.ua> <20190403070045.GW1923@kib.kiev.ua> <20190404011802.E2390@besplex.bde.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.2 cv=FNpr/6gs c=1 sm=1 tr=0 cx=a_idp_d a=PalzARQSbocsUSjMRkwAPg==:117 a=PalzARQSbocsUSjMRkwAPg==:17 a=jpOVt7BSZ2e4Z31A5e1TngXxSK0=:19 a=kj9zAlcOel0A:10 a=PO7r1zJSAAAA:8 a=jA9InAqbobDorSXSjysA:9 a=nJ01U1HSc7Yp0U_T:21 a=jiLLkcfR9Aiq0lNV:21 a=CjuIK1q_8ugA:10 X-Rspamd-Queue-Id: 8DD5674FED X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of brde@optusnet.com.au designates 211.29.132.42 as permitted sender) smtp.mailfrom=brde@optusnet.com.au X-Spamd-Result: default: False [-4.87 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; RCVD_IN_DNSWL_LOW(-0.10)[42.132.29.211.list.dnswl.org : 127.0.5.1]; FROM_HAS_DN(0.00)[]; FREEMAIL_FROM(0.00)[optusnet.com.au]; R_SPF_ALLOW(-0.20)[+ip4:211.29.132.0/23]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; DMARC_NA(0.00)[optusnet.com.au]; RCPT_COUNT_FIVE(0.00)[5]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[extmail.optusnet.com.au]; NEURAL_SPAM_SHORT(0.23)[0.232,0]; IP_SCORE(-2.79)[ip: (-7.79), ipnet: 211.28.0.0/14(-3.40), asn: 4804(-2.71), country: AU(-0.04)]; FREEMAIL_TO(0.00)[yahoo.com]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[optusnet.com.au]; ASN(0.00)[asn:4804, ipnet:211.28.0.0/14, country:AU]; FREEMAIL_CC(0.00)[optusnet.com.au]; RCVD_COUNT_TWO(0.00)[2] X-Mailman-Approved-At: Fri, 05 Apr 2019 10:35:37 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Apr 2019 04:38:16 -0000 On Thu, 4 Apr 2019, Mark Millard wrote: > On 2019-Apr-3, at 08:47, Bruce Evans wrote: >> . . . >> >> I noticed (or better realized) a general problem with multiple >> timehands. ntpd can slew the clock at up to 500 ppm, and at least an >> old version of it uses a rate of 50 ppm to fix up fairly small drifts >> in the milliseconds range. 500 ppm is enormous in CPU cycles -- it is >> 500 thousand nsec or 2 million cycles at 4GHz. Winding up the timecounter >> every 1 msec reduces this to only 2000 cycles. >> >> More details of ordering and timing for 1 thread: >> ... > Thanks for the description of an example way that sbinuptime and > the like might not give weakly increasing results. > > Unfortunately, all the multi-socket contexts that I sometimes have > access to are old PowerMacs. And, currently, the only such context > is the G5 with 2 sockets, 2 cores per socket (powerpc64). So I've > not been able to set up other types of examples to see if problems > repeat. > > I do not have access to a single-socket powerpc64 for contrast in > that direction. Testing 1 socket is time-consuming enough. Do these old systems use the equivalent of an x86 TSC for the timecounter? With multiple sockets, it isn't clear how even a hardware timer independent of the CPUs can be distributed so as to appear to be monotonic on all cors. > One oddity is that the eventtimer's decrementer and timecounter > may change (nearly) together: both change at 33,333,333 Hz, as if > they are tied to the same clock (at least on one socket). I think this is from a normal hardware implementation. On all of my x86 systems with a TSC, the TSC frequency is an exact fractional multiple of the i8254, the ACPI timer (if present) and the HPET (if present). Only the RTC has an independent frequency. The fraction is changed by changing the nominal TSC frequency in the BIOS, but is not changed by temperature variations. This must be because most clocks are derived from a common clock using a PLL. I use this to calibrate all clocks (except the RTC) by calibrating only 1. > In case it helps with knowing how analogous your investigations > are to the original problem context, I report the following. > > If you do not care for such information, stop reading here. > > # grep ntpd /etc/rc.conf > ntpd_enable="YES" > ntpd_sync_on_start="YES" > > # sysctl kern.eventtimer > kern.eventtimer.periodic: 0 > kern.eventtimer.timer: decrementer > kern.eventtimer.idletick: 0 > kern.eventtimer.singlemul: 2 > kern.eventtimer.choice: decrementer(1000) > kern.eventtimer.et.decrementer.quality: 1000 > kern.eventtimer.et.decrementer.frequency: 33333333 > kern.eventtimer.et.decrementer.flags: 7 > > # vmstat -ai | grep decrementer > cpu0:decrementer 4451007 35 > cpu3:decrementer 1466010 11 > cpu2:decrementer 1481722 12 > cpu1:decrementer 1478618 12 Powerpc seems to have a PLL in software too. Event timers don't need to be very precise or accurate. > (That last is from a basically-idle timeframe.) > > # sysctl -a | grep hz > kern.clockrate: { hz = 1000, tick = 1000, profhz = 8128, stathz = 127 } > kern.hz: 1000 x86 is similar. I think synchronization from using PLLs still gives unfair scheduling, but with multiple CPUs and often more cycles than can be used, no one cares about accidental synchronization or bothers to steal cycles using intentional synchronization. > # sysctl kern.timecounter > kern.timecounter.fast_gettime: 1 > kern.timecounter.tick: 1 > kern.timecounter.choice: timebase(0) dummy(-1000000) > kern.timecounter.hardware: timebase > kern.timecounter.alloweddeviation: 5 > kern.timecounter.stepwarnings: 0 > kern.timecounter.tc.timebase.quality: 0 > kern.timecounter.tc.timebase.frequency: 33333333 > kern.timecounter.tc.timebase.counter: 1144662532 > kern.timecounter.tc.timebase.mask: 4294967295 > > (The actual Time Base Register (tbr) i s 64 bits > and freebsd truncates it down.) > > # sysctl -a | grep 'cpu.*freq' > device cpufreq > debug.cpufreq.verbose: 0 > debug.cpufreq.lowest: 0 > dev.cpufreq.0.%parent: cpu3 > dev.cpufreq.0.%pnpinfo: > dev.cpufreq.0.%location: > dev.cpufreq.0.%driver: cpufreq > dev.cpufreq.0.%desc: > dev.cpufreq.%parent: > dev.cpu.3.freq_levels: 2500/-1 1250/-1 > dev.cpu.3.freq: 2500 > > So 2500 MHz / 33333333 Hz is very near 75 clock periods per > timebase counter value. Looks like it is exactly 75. Fractions are especially easy to guess and verify when they are integral. > I do sometimes have access to a Ryzen Threadripper 1950X based system: > > FreeBSD/SMP: Multiprocessor System Detected: 32 CPUs > FreeBSD/SMP: 1 package(s) x 2 groups x 2 cache groups x 4 core(s) x 2 hardware threads > > but it is single=socket. It has . . . > ... Bruce From owner-freebsd-hackers@freebsd.org Fri Apr 5 10:13:58 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D115A1570B54; Fri, 5 Apr 2019 10:13:57 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail104.syd.optusnet.com.au (mail104.syd.optusnet.com.au [211.29.132.246]) by mx1.freebsd.org (Postfix) with ESMTP id 3646587EDA; Fri, 5 Apr 2019 10:13:54 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from [192.168.0.102] (c110-21-101-228.carlnfd1.nsw.optusnet.com.au [110.21.101.228]) by mail104.syd.optusnet.com.au (Postfix) with ESMTPS id 0D37343A785; Fri, 5 Apr 2019 21:13:51 +1100 (AEDT) Date: Fri, 5 Apr 2019 21:13:50 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Mark Millard cc: Bruce Evans , Konstantin Belousov , freebsd-hackers Hackers , FreeBSD PowerPC ML Subject: Re: powerpc64 head -r344018 stuck sleeping problems: th->th_scale * tc_delta(th) overflows unsigned 64 bits sometimes [patched failed] In-Reply-To: Message-ID: <20190405201055.I2396@besplex.bde.org> References: <20190303161635.GJ68879@kib.kiev.ua> <20190304043416.V5640@besplex.bde.org> <20190304114150.GM68879@kib.kiev.ua> <20190305031010.I4610@besplex.bde.org> <20190306172003.GD2492@kib.kiev.ua> <20190308001005.M2756@besplex.bde.org> <20190307222220.GK2492@kib.kiev.ua> <20190309144844.K1166@besplex.bde.org> <20190324110138.GR1923@kib.kiev.ua> <20190403070045.GW1923@kib.kiev.ua> <20190404011802.E2390@besplex.bde.org> <20190405150236.A959@besplex.bde.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.2 cv=P6RKvmIu c=1 sm=1 tr=0 cx=a_idp_d a=PalzARQSbocsUSjMRkwAPg==:117 a=PalzARQSbocsUSjMRkwAPg==:17 a=jpOVt7BSZ2e4Z31A5e1TngXxSK0=:19 a=kj9zAlcOel0A:10 a=PO7r1zJSAAAA:8 a=E-FLDMUcsM50LrUROKkA:9 a=CjuIK1q_8ugA:10 X-Rspamd-Queue-Id: 3646587EDA X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of brde@optusnet.com.au designates 211.29.132.246 as permitted sender) smtp.mailfrom=brde@optusnet.com.au X-Spamd-Result: default: False [-6.01 / 15.00]; R_SPF_ALLOW(-0.20)[+ip4:211.29.132.0/23]; FREEMAIL_FROM(0.00)[optusnet.com.au]; RCPT_COUNT_FIVE(0.00)[5]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[cached: extmail.optusnet.com.au]; NEURAL_HAM_SHORT(-0.98)[-0.983,0]; FREEMAIL_TO(0.00)[yahoo.com]; RCVD_NO_TLS_LAST(0.10)[]; RCVD_IN_DNSWL_LOW(-0.10)[246.132.29.211.list.dnswl.org : 127.0.5.1]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[optusnet.com.au]; ASN(0.00)[asn:4804, ipnet:211.28.0.0/14, country:AU]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[optusnet.com.au]; TO_MATCH_ENVRCPT_SOME(0.00)[]; IP_SCORE(-2.72)[ip: (-7.39), ipnet: 211.28.0.0/14(-3.42), asn: 4804(-2.72), country: AU(-0.04)]; FREEMAIL_CC(0.00)[optusnet.com.au]; RCVD_COUNT_TWO(0.00)[2] X-Mailman-Approved-At: Fri, 05 Apr 2019 10:35:54 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Apr 2019 10:13:58 -0000 On Fri, 5 Apr 2019, Mark Millard wrote: > On 2019-Apr-4, at 21:38, Bruce Evans wrote: > >> On Thu, 4 Apr 2019, Mark Millard wrote: >>> ... >>> Unfortunately, all the multi-socket contexts that I sometimes have >>> access to are old PowerMacs. And, currently, the only such context >>> is the G5 with 2 sockets, 2 cores per socket (powerpc64). So I've >>> not been able to set up other types of examples to see if problems >>> repeat. >>> >>> I do not have access to a single-socket powerpc64 for contrast in >>> that direction. >> >> Testing 1 socket is time-consuming enough. Do these old systems >> use the equivalent of an x86 TSC for the timecounter? With multiple >> sockets, it isn't clear how even a hardware timer independent of the >> CPUs can be distributed so as to appear to be monotonic on all cors. > > "The DEC frequency is based on the same implementation-dependent > frequency that drives the time base." The frequency may well be > fixed by the PowerMac G5 model implementation but is not fixed > by the powerpc64 architecture. > > The Time Base Register (TBR) in a powerpc64 core (cpu in FreeBSD terms) > increments at 33,333,333 Hz (nominal) and is 64 bits wide. Its value > can be set (mttb instruction) and the boot sequence in FreeBSD does > attempt to adjust as the FreeBSD CPU is brought-up/started. mftb is > used to read the 64-bit value. FreeBSD masks it down to 32-bits to > contribute to the time-counter. > > (Is that description sufficient for what you were after? I've never > seen documentation of how the 33,333,333 MHz is produced.) This seems to be equivalent to an x86 TSC. x86 P-state-invariant TSCs apparently use a 100 MHz clock which corresponds even more closely to this, except for historical reasons this clock is scaled and interpolated to a clock resembling the CPU cycle count at a nominal frequency. > As FreeBSD supports multi-socket, what are its criteria for a sufficient > context for it to work with for supporting sbinuptime and the like? Is > FreeBSD supposed to then make it appear that sbinputime and the like are > weakly increasing, even as threads migrate between CPUs (cores, > hw-threads)? CLOCK_MONOTONIC has to be monotonic, but it is only clear what this means within a single thread: sequential clock_gettime() calls must occur in program order and the results must be consistent with that order. Across threads, I think it should mean that the results must be consistent with any order established using any supported ordering methods. >... >>> One oddity is that the eventtimer's decrementer and timecounter >>> may change (nearly) together: both change at 33,333,333 Hz, as if >>> they are tied to the same clock (at least on one socket). >> >> I think this is from a normal hardware implementation. On all of >> my x86 systems with a TSC, the TSC frequency is an exact fractional >> multiple of the i8254, the ACPI timer (if present) and the HPET (if >> present). Only the RTC has an independent frequency. The fraction is >> changed by changing the nominal TSC frequency in the BIOS, but is not >> changed by temperature variations. This must be because most clocks are >> derived from a common clock using a PLL. I use this to calibrate all >> clocks (except the RTC) by calibrating only 1. > > I'm not aware of the OpenFirmware having any control over the > TBR-change frequency behavior. I've no evidence about any variability > based on temperature. Temperature changes usually affect the actual frequency but not the nominal frequency. Bruce From owner-freebsd-hackers@freebsd.org Fri Apr 5 11:35:17 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A17221573ACD; Fri, 5 Apr 2019 11:35:17 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C87618CFE7; Fri, 5 Apr 2019 11:35:16 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id x35BZ6jL002604 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 5 Apr 2019 14:35:09 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua x35BZ6jL002604 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id x35BZ4Vo002581; Fri, 5 Apr 2019 14:35:04 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 5 Apr 2019 14:35:04 +0300 From: Konstantin Belousov To: Bruce Evans Cc: Mark Millard , freebsd-hackers Hackers , FreeBSD PowerPC ML Subject: Re: powerpc64 head -r344018 stuck sleeping problems: th->th_scale * tc_delta(th) overflows unsigned 64 bits sometimes [patched failed] Message-ID: <20190405113504.GA1923@kib.kiev.ua> References: <20190307222220.GK2492@kib.kiev.ua> <20190309144844.K1166@besplex.bde.org> <20190324110138.GR1923@kib.kiev.ua> <20190403070045.GW1923@kib.kiev.ua> <20190404011802.E2390@besplex.bde.org> <20190405150236.A959@besplex.bde.org> <20190405201055.I2396@besplex.bde.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20190405201055.I2396@besplex.bde.org> User-Agent: Mutt/1.11.4 (2019-03-13) X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on tom.home X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Apr 2019 11:35:18 -0000 On Fri, Apr 05, 2019 at 09:13:50PM +1100, Bruce Evans wrote: > On Fri, 5 Apr 2019, Mark Millard wrote: > > > On 2019-Apr-4, at 21:38, Bruce Evans wrote: > > > >> On Thu, 4 Apr 2019, Mark Millard wrote: > >>> ... > >>> Unfortunately, all the multi-socket contexts that I sometimes have > >>> access to are old PowerMacs. And, currently, the only such context > >>> is the G5 with 2 sockets, 2 cores per socket (powerpc64). So I've > >>> not been able to set up other types of examples to see if problems > >>> repeat. > >>> > >>> I do not have access to a single-socket powerpc64 for contrast in > >>> that direction. > >> > >> Testing 1 socket is time-consuming enough. Do these old systems > >> use the equivalent of an x86 TSC for the timecounter? With multiple > >> sockets, it isn't clear how even a hardware timer independent of the > >> CPUs can be distributed so as to appear to be monotonic on all cors. > > > > "The DEC frequency is based on the same implementation-dependent > > frequency that drives the time base." The frequency may well be > > fixed by the PowerMac G5 model implementation but is not fixed > > by the powerpc64 architecture. > > > > The Time Base Register (TBR) in a powerpc64 core (cpu in FreeBSD terms) > > increments at 33,333,333 Hz (nominal) and is 64 bits wide. Its value > > can be set (mttb instruction) and the boot sequence in FreeBSD does > > attempt to adjust as the FreeBSD CPU is brought-up/started. mftb is > > used to read the 64-bit value. FreeBSD masks it down to 32-bits to > > contribute to the time-counter. > > > > (Is that description sufficient for what you were after? I've never > > seen documentation of how the 33,333,333 MHz is produced.) > > This seems to be equivalent to an x86 TSC. It is equivalent from the interface PoV. I saw some references that Powers do have per-core PLLs, the best I can find now is https://stackoverflow.com/questions/43844438/are-multiple-clock-domains-common-in-modern-processors For Intel there is no much information, the best guess is that TSC is in uncore and effectively shared by all cores. See also https://software.intel.com/en-us/articles/best-timing-function-for-measuring-ipp-api-timing/ > > x86 P-state-invariant TSCs apparently use a 100 MHz clock which corresponds > even more closely to this, except for historical reasons this clock is > scaled and interpolated to a clock resembling the CPU cycle count at a > nominal frequency. For Intel designs, there is indeed a single-source 100MHz signal which is distributed over all consumers using clock fan-out buffers like DB1900Z. > > > As FreeBSD supports multi-socket, what are its criteria for a sufficient > > context for it to work with for supporting sbinuptime and the like? Is > > FreeBSD supposed to then make it appear that sbinputime and the like are > > weakly increasing, even as threads migrate between CPUs (cores, > > hw-threads)? > > CLOCK_MONOTONIC has to be monotonic, but it is only clear what this means > within a single thread: sequential clock_gettime() calls must occur in > program order and the results must be consistent with that order. Across > threads, I think it should mean that the results must be consistent with > any order established using any supported ordering methods. > > >... > >>> One oddity is that the eventtimer's decrementer and timecounter > >>> may change (nearly) together: both change at 33,333,333 Hz, as if > >>> they are tied to the same clock (at least on one socket). > >> > >> I think this is from a normal hardware implementation. On all of > >> my x86 systems with a TSC, the TSC frequency is an exact fractional > >> multiple of the i8254, the ACPI timer (if present) and the HPET (if > >> present). Only the RTC has an independent frequency. The fraction is > >> changed by changing the nominal TSC frequency in the BIOS, but is not > >> changed by temperature variations. This must be because most clocks are > >> derived from a common clock using a PLL. I use this to calibrate all > >> clocks (except the RTC) by calibrating only 1. > > > > I'm not aware of the OpenFirmware having any control over the > > TBR-change frequency behavior. I've no evidence about any variability > > based on temperature. > > Temperature changes usually affect the actual frequency but not the > nominal frequency. > > Bruce From owner-freebsd-hackers@freebsd.org Fri Apr 5 11:39:23 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 60FC01573DEB; Fri, 5 Apr 2019 11:39:23 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id AC98A8D528; Fri, 5 Apr 2019 11:39:22 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id x35BdCH1003695 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 5 Apr 2019 14:39:15 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua x35BdCH1003695 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id x35BdCH7003692; Fri, 5 Apr 2019 14:39:12 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 5 Apr 2019 14:39:12 +0300 From: Konstantin Belousov To: Bruce Evans Cc: Michael Tuexen , freebsd-hackers Hackers , FreeBSD PowerPC ML Subject: Re: powerpc64 head -r344018 stuck sleeping problems: th->th_scale * tc_delta(th) overflows unsigned 64 bits sometimes [patched failed] Message-ID: <20190405113912.GB1923@kib.kiev.ua> References: <20190304114150.GM68879@kib.kiev.ua> <20190305031010.I4610@besplex.bde.org> <20190306172003.GD2492@kib.kiev.ua> <20190308001005.M2756@besplex.bde.org> <20190307222220.GK2492@kib.kiev.ua> <20190309144844.K1166@besplex.bde.org> <20190324110138.GR1923@kib.kiev.ua> <20190403070045.GW1923@kib.kiev.ua> <20190404011802.E2390@besplex.bde.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20190404011802.E2390@besplex.bde.org> User-Agent: Mutt/1.11.4 (2019-03-13) X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on tom.home X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Apr 2019 11:39:23 -0000 On Thu, Apr 04, 2019 at 02:47:34AM +1100, Bruce Evans wrote: > I noticed (or better realized) a general problem with multiple > timehands. ntpd can slew the clock at up to 500 ppm, and at least an > old version of it uses a rate of 50 ppm to fix up fairly small drifts > in the milliseconds range. 500 ppm is enormous in CPU cycles -- it is > 500 thousand nsec or 2 million cycles at 4GHz. Winding up the timecounter > every 1 msec reduces this to only 2000 cycles. > > More details of ordering and timing for 1 thread: > - thread N calls binuptime() and it loads timehands > - another or even the same thread runs tc_windup(). This modifies timehands. > - thread N is preempted for a long time, but less than the time for > updates > - thread N checks the generation count. Since this is for the timehands > contents and not for the timehands pointer, it hasn't changed, so the > old timehands is used > - and instant later, the same thread calls binuptime again() and uses the > new timehands > - now suppose only 2 timehands (as in -current) the worst (?) case of a > slew of 500 ppm for the old timehands and -500 ppm for the new timehands > and almost the worst case of 10 msec for the oldness of the old timehands > relative to the new timehands, with the new timehands about to be updated > after 10 msec (assume perfectly periodiodic updates every 10 msec). The > calculated times are: > > old bintime = old_base + (20 msec) * (1 + 500e-6) > new base = old_base + 10 msec * (1 + 500e-6) # calc by tc_windup() > new bintime = new_base + (10 msec) * (1 - 500e-6) + epsilon > > error = epsilon - (20 msec) * 500e-6 = epsilon - 10000 nsec > > Errors in the negative direction are most harmful. ntpd normally doesn't > change the skew as much as that in one step, but it is easy for adjtime(2) > to change the skew like that and there are no reasonable microadjustments > that would accidentally work around this kernel bug (it seems unreasonable > to limit the skew to 1 ppm and that still gives an error of epsilon + 20 > nsec. > > phk didn't want to slow down timecounters using extra code to make > them them monotonic and coherent (getbinuptime() is incoherent with > binuptime() since it former lags the latter by the update interval), > but this wouldn't be very slow within a thread. > > Monotonicity across threads is a larger problem and not helped by using > a faked forced monotonic time within threads. > > So it seems best to fix the above problem by moving the generation count > from the timehands contents to the timehands pointer, and maybe also > reduce the number of timehands to 1. With 2 timehands, this gives a > shorter race: > > - thread N loads timehands > - tc_windup() > - thread N preempted > - thread N uses old timehands > - case tc_windup() completes first: no problem -- thread N checks the > generation count on the pointer and loops > - case binuptime() completes first: lost a race -- binuptime() is off > by approx * . > > The main point of having multiple timehands (after introducing the per- > timehands generation count) is to avoid blocking thread N during the > update, but this doesn't actually work, even for only 2 timehands and > a global generation count. You are describing the generic race between reader and writer. The same race would exist even with one timehand (and/or one global generation counter), where ntp adjustment might come earlier or later of some consumer accessing the timehands. If timehand instance was read before tc_windup() run but code consumed the result after the windup, it might appear as if time went backward, and this cannot be fixed without either re-reading the time after time-depended calculations were done and restarting, or some globabl lock ensuring serialization. From owner-freebsd-hackers@freebsd.org Fri Apr 5 12:52:40 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8572F157664F; Fri, 5 Apr 2019 12:52:40 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail106.syd.optusnet.com.au (mail106.syd.optusnet.com.au [211.29.132.42]) by mx1.freebsd.org (Postfix) with ESMTP id E92F190085; Fri, 5 Apr 2019 12:52:39 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from [192.168.0.102] (c110-21-101-228.carlnfd1.nsw.optusnet.com.au [110.21.101.228]) by mail106.syd.optusnet.com.au (Postfix) with ESMTPS id 2CA103DC95D; Fri, 5 Apr 2019 23:52:29 +1100 (AEDT) Date: Fri, 5 Apr 2019 23:52:27 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Konstantin Belousov cc: Bruce Evans , Michael Tuexen , freebsd-hackers Hackers , FreeBSD PowerPC ML Subject: Re: powerpc64 head -r344018 stuck sleeping problems: th->th_scale * tc_delta(th) overflows unsigned 64 bits sometimes [patched failed] In-Reply-To: <20190405113912.GB1923@kib.kiev.ua> Message-ID: <20190405230717.D3383@besplex.bde.org> References: <20190304114150.GM68879@kib.kiev.ua> <20190305031010.I4610@besplex.bde.org> <20190306172003.GD2492@kib.kiev.ua> <20190308001005.M2756@besplex.bde.org> <20190307222220.GK2492@kib.kiev.ua> <20190309144844.K1166@besplex.bde.org> <20190324110138.GR1923@kib.kiev.ua> <20190403070045.GW1923@kib.kiev.ua> <20190404011802.E2390@besplex.bde.org> <20190405113912.GB1923@kib.kiev.ua> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.2 cv=P6RKvmIu c=1 sm=1 tr=0 cx=a_idp_d a=PalzARQSbocsUSjMRkwAPg==:117 a=PalzARQSbocsUSjMRkwAPg==:17 a=jpOVt7BSZ2e4Z31A5e1TngXxSK0=:19 a=kj9zAlcOel0A:10 a=MfOOJuOeLnfdOtmGB0YA:9 a=-f6ZOhMsL1iUIJkq:21 a=gaGQGk8RQ06_Sp8i:21 a=CjuIK1q_8ugA:10 X-Rspamd-Queue-Id: E92F190085 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.98 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.98)[-0.982,0]; REPLY(-4.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-Mailman-Approved-At: Fri, 05 Apr 2019 13:09:48 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Apr 2019 12:52:40 -0000 On Fri, 5 Apr 2019, Konstantin Belousov wrote: > On Thu, Apr 04, 2019 at 02:47:34AM +1100, Bruce Evans wrote: >> I noticed (or better realized) a general problem with multiple >> timehands. ntpd can slew the clock at up to 500 ppm, and at least an >> old version of it uses a rate of 50 ppm to fix up fairly small drifts >> in the milliseconds range. 500 ppm is enormous in CPU cycles -- it is >> 500 thousand nsec or 2 million cycles at 4GHz. Winding up the timecounter >> every 1 msec reduces this to only 2000 cycles. >> ... >> The main point of having multiple timehands (after introducing the per- >> timehands generation count) is to avoid blocking thread N during the >> update, but this doesn't actually work, even for only 2 timehands and >> a global generation count. > > You are describing the generic race between reader and writer. The same > race would exist even with one timehand (and/or one global generation > counter), where ntp adjustment might come earlier or later of some > consumer accessing the timehands. If timehand instance was read before > tc_windup() run but code consumed the result after the windup, it might > appear as if time went backward, and this cannot be fixed without either > re-reading the time after time-depended calculations were done and > restarting, or some globabl lock ensuring serialization. With 1 timehand, its generation count would be global. I think its ordering is strong enough to ensure serialization. I think the fix in the kernel to use a global generation count (with > 1 timehands) is simply s/th->th_generation/tc_generation/g. Oops, that makes multiple timehands useless and gives some blocking. The critical case is when a new timehands is under construction. The old timehands becomes unsafe to use when the writer (tc_windup()) updates the offset. tc_windup() currently sets th_generation to 0 to indicate that the new timehands is unsafe to use. Doing the same with a global tc_generation would give serialization at the cost of busy-waiting for tc_generation to become nonzero. It would indicate that all timehands are unsafe to use. In the library, does it just work to put the global generation count in the shared page? Bruce From owner-freebsd-hackers@freebsd.org Fri Apr 5 13:21:38 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AAA831576E5C; Fri, 5 Apr 2019 13:21:38 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1F96490C20; Fri, 5 Apr 2019 13:21:38 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id x35DLSfb011961 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 5 Apr 2019 16:21:31 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua x35DLSfb011961 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id x35DLSX2011960; Fri, 5 Apr 2019 16:21:28 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 5 Apr 2019 16:21:28 +0300 From: Konstantin Belousov To: Bruce Evans Cc: Michael Tuexen , freebsd-hackers Hackers , FreeBSD PowerPC ML Subject: Re: powerpc64 head -r344018 stuck sleeping problems: th->th_scale * tc_delta(th) overflows unsigned 64 bits sometimes [patched failed] Message-ID: <20190405132128.GD1923@kib.kiev.ua> References: <20190306172003.GD2492@kib.kiev.ua> <20190308001005.M2756@besplex.bde.org> <20190307222220.GK2492@kib.kiev.ua> <20190309144844.K1166@besplex.bde.org> <20190324110138.GR1923@kib.kiev.ua> <20190403070045.GW1923@kib.kiev.ua> <20190404011802.E2390@besplex.bde.org> <20190405113912.GB1923@kib.kiev.ua> <20190405230717.D3383@besplex.bde.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20190405230717.D3383@besplex.bde.org> User-Agent: Mutt/1.11.4 (2019-03-13) X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on tom.home X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Apr 2019 13:21:38 -0000 On Fri, Apr 05, 2019 at 11:52:27PM +1100, Bruce Evans wrote: > On Fri, 5 Apr 2019, Konstantin Belousov wrote: > > > On Thu, Apr 04, 2019 at 02:47:34AM +1100, Bruce Evans wrote: > >> I noticed (or better realized) a general problem with multiple > >> timehands. ntpd can slew the clock at up to 500 ppm, and at least an > >> old version of it uses a rate of 50 ppm to fix up fairly small drifts > >> in the milliseconds range. 500 ppm is enormous in CPU cycles -- it is > >> 500 thousand nsec or 2 million cycles at 4GHz. Winding up the timecounter > >> every 1 msec reduces this to only 2000 cycles. > >> ... > >> The main point of having multiple timehands (after introducing the per- > >> timehands generation count) is to avoid blocking thread N during the > >> update, but this doesn't actually work, even for only 2 timehands and > >> a global generation count. > > > > You are describing the generic race between reader and writer. The same > > race would exist even with one timehand (and/or one global generation > > counter), where ntp adjustment might come earlier or later of some > > consumer accessing the timehands. If timehand instance was read before > > tc_windup() run but code consumed the result after the windup, it might > > appear as if time went backward, and this cannot be fixed without either > > re-reading the time after time-depended calculations were done and > > restarting, or some globabl lock ensuring serialization. > > With 1 timehand, its generation count would be global. I think its ordering > is strong enough to ensure serialization. Yes, single timehands result in global generation. But it would not solve the same race appearing in slightly different manner, as I described above. If reader finished while generation number in th was not yet reset, but caller uses the result after tc_windup(), the effect is same as if we have two th's and reader used the outdated one. > > I think the fix in the kernel to use a global generation count (with > 1 > timehands) is simply s/th->th_generation/tc_generation/g. Oops, that > makes multiple timehands useless and gives some blocking. The critical > case is when a new timehands is under construction. The old timehands > becomes unsafe to use when the writer (tc_windup()) updates the offset. > tc_windup() currently sets th_generation to 0 to indicate that the new > timehands is unsafe to use. Doing the same with a global tc_generation > would give serialization at the cost of busy-waiting for tc_generation > to become nonzero. It would indicate that all timehands are unsafe > to use. > > In the library, does it just work to put the global generation count in > the shared page? libc always reload tk_current in the loop, so it works with any number of vdso timehands greater or equal to one. From owner-freebsd-hackers@freebsd.org Fri Apr 5 14:36:29 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 06123157890C; Fri, 5 Apr 2019 14:36:29 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6A7CB94089; Fri, 5 Apr 2019 14:36:28 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id x35EaJdK029276 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 5 Apr 2019 17:36:22 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua x35EaJdK029276 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id x35EaJKI029275; Fri, 5 Apr 2019 17:36:19 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 5 Apr 2019 17:36:19 +0300 From: Konstantin Belousov To: Bruce Evans Cc: Michael Tuexen , freebsd-hackers Hackers , FreeBSD PowerPC ML Subject: Re: powerpc64 head -r344018 stuck sleeping problems: th->th_scale * tc_delta(th) overflows unsigned 64 bits sometimes [patched failed] Message-ID: <20190405143619.GF1923@kib.kiev.ua> References: <20190307222220.GK2492@kib.kiev.ua> <20190309144844.K1166@besplex.bde.org> <20190324110138.GR1923@kib.kiev.ua> <20190403070045.GW1923@kib.kiev.ua> <20190404011802.E2390@besplex.bde.org> <20190405113912.GB1923@kib.kiev.ua> <20190405230717.D3383@besplex.bde.org> <20190405132128.GD1923@kib.kiev.ua> <20190406003907.C3872@besplex.bde.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20190406003907.C3872@besplex.bde.org> User-Agent: Mutt/1.11.4 (2019-03-13) X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on tom.home X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Apr 2019 14:36:29 -0000 On Sat, Apr 06, 2019 at 01:01:19AM +1100, Bruce Evans wrote: > On Fri, 5 Apr 2019, Konstantin Belousov wrote: > > > On Fri, Apr 05, 2019 at 11:52:27PM +1100, Bruce Evans wrote: > >> On Fri, 5 Apr 2019, Konstantin Belousov wrote: > >> > >>> On Thu, Apr 04, 2019 at 02:47:34AM +1100, Bruce Evans wrote: > >>>> I noticed (or better realized) a general problem with multiple > >>>> timehands. ntpd can slew the clock at up to 500 ppm, and at least an > >>>> old version of it uses a rate of 50 ppm to fix up fairly small drifts > >>>> in the milliseconds range. 500 ppm is enormous in CPU cycles -- it is > >>>> 500 thousand nsec or 2 million cycles at 4GHz. Winding up the timecounter > >>>> every 1 msec reduces this to only 2000 cycles. > >>>> ... > >>>> The main point of having multiple timehands (after introducing the per- > >>>> timehands generation count) is to avoid blocking thread N during the > >>>> update, but this doesn't actually work, even for only 2 timehands and > >>>> a global generation count. > >>> > >>> You are describing the generic race between reader and writer. The same > >>> race would exist even with one timehand (and/or one global generation > >>> counter), where ntp adjustment might come earlier or later of some > >>> consumer accessing the timehands. If timehand instance was read before > >>> tc_windup() run but code consumed the result after the windup, it might > >>> appear as if time went backward, and this cannot be fixed without either > >>> re-reading the time after time-depended calculations were done and > >>> restarting, or some globabl lock ensuring serialization. > >> > >> With 1 timehand, its generation count would be global. I think its ordering > >> is strong enough to ensure serialization. > > Yes, single timehands result in global generation. But it would not solve > > the same race appearing in slightly different manner, as I described above. > > If reader finished while generation number in th was not yet reset, but > > caller uses the result after tc_windup(), the effect is same as if we > > have two th's and reader used the outdated one. > > You described it too concisely for me to understand :-). > > I now see that a single generation count doesn't give serialization. I > thought that setting the generation to 0 at the start of tc_windup() > serialized the reader and writer. But all it does is prevent use of the > results of the windup while only some of them are visible. If the > setting the generation count to 0 doesn't become before tc_windup() reads > the hardware timecounter, then this read may be before other reads using > the old timehands, but it needs to be after. If we have either single th or global gen counter, current code would become serialized, but this is not what I am about. Lets assume, for the sake of the discussion only, that all readers take the same spinlock as tc_windup (i.e. tc_setclock_mtx). It is still possible that reader unlocked the mutex, tc_windup() kicked in, locked the mutex and moved timehands (as you noted, this might even happen on the same CPU), and only then the reader continues. For consumer of bintime() or any other function' result, it looks exactly the same as if we did not serialized with writer but used outdated timehands. Let me formulate this diffeently: as far as consumer of the bintime() result does not serialize itself with tc_windup(), serializing bintime() itself against tc_windup() does not close the race, but it is not obvious that the race matters. Either we should just accept the race as we currently do, or readers must take the spinlock where the exact value of the current time is important, or readers must re-read the time after doing something important, and redo if the new measuremedtime is outside the acceptable range. > > A not so good fix for this is to wait a bit after setting the generation > count to 0, so that the change becomes visible on all CPUs. > > Bruce From owner-freebsd-hackers@freebsd.org Fri Apr 5 14:01:25 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6DEF21577F9E; Fri, 5 Apr 2019 14:01:25 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail104.syd.optusnet.com.au (mail104.syd.optusnet.com.au [211.29.132.246]) by mx1.freebsd.org (Postfix) with ESMTP id B4F33928FD; Fri, 5 Apr 2019 14:01:24 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from [192.168.0.102] (c110-21-101-228.carlnfd1.nsw.optusnet.com.au [110.21.101.228]) by mail104.syd.optusnet.com.au (Postfix) with ESMTPS id AD7E5438988; Sat, 6 Apr 2019 01:01:20 +1100 (AEDT) Date: Sat, 6 Apr 2019 01:01:19 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Konstantin Belousov cc: Michael Tuexen , freebsd-hackers Hackers , FreeBSD PowerPC ML Subject: Re: powerpc64 head -r344018 stuck sleeping problems: th->th_scale * tc_delta(th) overflows unsigned 64 bits sometimes [patched failed] In-Reply-To: <20190405132128.GD1923@kib.kiev.ua> Message-ID: <20190406003907.C3872@besplex.bde.org> References: <20190306172003.GD2492@kib.kiev.ua> <20190308001005.M2756@besplex.bde.org> <20190307222220.GK2492@kib.kiev.ua> <20190309144844.K1166@besplex.bde.org> <20190324110138.GR1923@kib.kiev.ua> <20190403070045.GW1923@kib.kiev.ua> <20190404011802.E2390@besplex.bde.org> <20190405113912.GB1923@kib.kiev.ua> <20190405230717.D3383@besplex.bde.org> <20190405132128.GD1923@kib.kiev.ua> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.2 cv=P6RKvmIu c=1 sm=1 tr=0 cx=a_idp_d a=PalzARQSbocsUSjMRkwAPg==:117 a=PalzARQSbocsUSjMRkwAPg==:17 a=jpOVt7BSZ2e4Z31A5e1TngXxSK0=:19 a=kj9zAlcOel0A:10 a=xRf97b5gqAnxw9LQlxwA:9 a=q9dmEwd7LJ99Beko:21 a=KBmFWjL1gV5mJfMI:21 a=CjuIK1q_8ugA:10 X-Rspamd-Queue-Id: B4F33928FD X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.94 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.94)[-0.943,0]; REPLY(-4.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-Mailman-Approved-At: Fri, 05 Apr 2019 15:40:39 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Apr 2019 14:01:25 -0000 On Fri, 5 Apr 2019, Konstantin Belousov wrote: > On Fri, Apr 05, 2019 at 11:52:27PM +1100, Bruce Evans wrote: >> On Fri, 5 Apr 2019, Konstantin Belousov wrote: >> >>> On Thu, Apr 04, 2019 at 02:47:34AM +1100, Bruce Evans wrote: >>>> I noticed (or better realized) a general problem with multiple >>>> timehands. ntpd can slew the clock at up to 500 ppm, and at least an >>>> old version of it uses a rate of 50 ppm to fix up fairly small drifts >>>> in the milliseconds range. 500 ppm is enormous in CPU cycles -- it is >>>> 500 thousand nsec or 2 million cycles at 4GHz. Winding up the timecounter >>>> every 1 msec reduces this to only 2000 cycles. >>>> ... >>>> The main point of having multiple timehands (after introducing the per- >>>> timehands generation count) is to avoid blocking thread N during the >>>> update, but this doesn't actually work, even for only 2 timehands and >>>> a global generation count. >>> >>> You are describing the generic race between reader and writer. The same >>> race would exist even with one timehand (and/or one global generation >>> counter), where ntp adjustment might come earlier or later of some >>> consumer accessing the timehands. If timehand instance was read before >>> tc_windup() run but code consumed the result after the windup, it might >>> appear as if time went backward, and this cannot be fixed without either >>> re-reading the time after time-depended calculations were done and >>> restarting, or some globabl lock ensuring serialization. >> >> With 1 timehand, its generation count would be global. I think its ordering >> is strong enough to ensure serialization. > Yes, single timehands result in global generation. But it would not solve > the same race appearing in slightly different manner, as I described above. > If reader finished while generation number in th was not yet reset, but > caller uses the result after tc_windup(), the effect is same as if we > have two th's and reader used the outdated one. You described it too concisely for me to understand :-). I now see that a single generation count doesn't give serialization. I thought that setting the generation to 0 at the start of tc_windup() serialized the reader and writer. But all it does is prevent use of the results of the windup while only some of them are visible. If the setting the generation count to 0 doesn't become before tc_windup() reads the hardware timecounter, then this read may be before other reads using the old timehands, but it needs to be after. A not so good fix for this is to wait a bit after setting the generation count to 0, so that the change becomes visible on all CPUs. Bruce From owner-freebsd-hackers@freebsd.org Fri Apr 5 15:26:17 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0774B157A302; Fri, 5 Apr 2019 15:26:17 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail105.syd.optusnet.com.au (mail105.syd.optusnet.com.au [211.29.132.249]) by mx1.freebsd.org (Postfix) with ESMTP id 4AA62965F1; Fri, 5 Apr 2019 15:26:15 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from [192.168.0.102] (c110-21-101-228.carlnfd1.nsw.optusnet.com.au [110.21.101.228]) by mail105.syd.optusnet.com.au (Postfix) with ESMTPS id AC183105BAD5; Sat, 6 Apr 2019 02:26:11 +1100 (AEDT) Date: Sat, 6 Apr 2019 02:26:11 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Konstantin Belousov cc: Michael Tuexen , freebsd-hackers Hackers , FreeBSD PowerPC ML Subject: Re: powerpc64 head -r344018 stuck sleeping problems: th->th_scale * tc_delta(th) overflows unsigned 64 bits sometimes [patched failed] In-Reply-To: <20190405143619.GF1923@kib.kiev.ua> Message-ID: <20190406014724.X4174@besplex.bde.org> References: <20190307222220.GK2492@kib.kiev.ua> <20190309144844.K1166@besplex.bde.org> <20190324110138.GR1923@kib.kiev.ua> <20190403070045.GW1923@kib.kiev.ua> <20190404011802.E2390@besplex.bde.org> <20190405113912.GB1923@kib.kiev.ua> <20190405230717.D3383@besplex.bde.org> <20190405132128.GD1923@kib.kiev.ua> <20190406003907.C3872@besplex.bde.org> <20190405143619.GF1923@kib.kiev.ua> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.2 cv=FNpr/6gs c=1 sm=1 tr=0 cx=a_idp_d a=PalzARQSbocsUSjMRkwAPg==:117 a=PalzARQSbocsUSjMRkwAPg==:17 a=jpOVt7BSZ2e4Z31A5e1TngXxSK0=:19 a=kj9zAlcOel0A:10 a=WpN8qSVOMLHOXD53BCgA:9 a=_yIFZNw7G-84PWPD:21 a=tYTChskUWkClw5JK:21 a=CjuIK1q_8ugA:10 X-Rspamd-Queue-Id: 4AA62965F1 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.93 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[]; NEURAL_HAM_SHORT(-0.93)[-0.928,0] X-Mailman-Approved-At: Fri, 05 Apr 2019 16:08:40 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Apr 2019 15:26:17 -0000 On Fri, 5 Apr 2019, Konstantin Belousov wrote: > On Sat, Apr 06, 2019 at 01:01:19AM +1100, Bruce Evans wrote: >> On Fri, 5 Apr 2019, Konstantin Belousov wrote: >> >>> On Fri, Apr 05, 2019 at 11:52:27PM +1100, Bruce Evans wrote: >>>> On Fri, 5 Apr 2019, Konstantin Belousov wrote: >>>> >>>>> On Thu, Apr 04, 2019 at 02:47:34AM +1100, Bruce Evans wrote: >>>>>> I noticed (or better realized) a general problem with multiple >>>>>> timehands. ntpd can slew the clock at up to 500 ppm, and at least an >>>>>> old version of it uses a rate of 50 ppm to fix up fairly small drifts >>>>>> in the milliseconds range. 500 ppm is enormous in CPU cycles -- it is >>>>>> 500 thousand nsec or 2 million cycles at 4GHz. Winding up the timecounter >>>>>> every 1 msec reduces this to only 2000 cycles. >>>>>> ... >>>>>> The main point of having multiple timehands (after introducing the per- >>>>>> timehands generation count) is to avoid blocking thread N during the >>>>>> update, but this doesn't actually work, even for only 2 timehands and >>>>>> a global generation count. >>>>> >>>>> You are describing the generic race between reader and writer. The same >>>>> race would exist even with one timehand (and/or one global generation >>>>> counter), where ntp adjustment might come earlier or later of some >>>>> consumer accessing the timehands. If timehand instance was read before >>>>> tc_windup() run but code consumed the result after the windup, it might >>>>> appear as if time went backward, and this cannot be fixed without either >>>>> re-reading the time after time-depended calculations were done and >>>>> restarting, or some globabl lock ensuring serialization. >>>> >>>> With 1 timehand, its generation count would be global. I think its ordering >>>> is strong enough to ensure serialization. >>> Yes, single timehands result in global generation. But it would not solve >>> the same race appearing in slightly different manner, as I described above. >>> If reader finished while generation number in th was not yet reset, but >>> caller uses the result after tc_windup(), the effect is same as if we >>> have two th's and reader used the outdated one. >> >> You described it too concisely for me to understand :-). >> >> I now see that a single generation count doesn't give serialization. I >> thought that setting the generation to 0 at the start of tc_windup() >> serialized the reader and writer. But all it does is prevent use of the >> results of the windup while only some of them are visible. If the >> setting the generation count to 0 doesn't become before tc_windup() reads >> the hardware timecounter, then this read may be before other reads using >> the old timehands, but it needs to be after. > If we have either single th or global gen counter, current code would > become serialized, but this is not what I am about. Lets assume, for No, generation counts used like they are now (or in any way that I can see) can't give serialization. > the sake of the discussion only, that all readers take the same spinlock > as tc_windup (i.e. tc_setclock_mtx). Spinlocks are far too heavyweight. Most of the complications in timecounter locking are to avoid using them. But OK for the discussion. > It is still possible that reader unlocked the mutex, tc_windup() kicked > in, locked the mutex and moved timehands (as you noted, this might > even happen on the same CPU), and only then the reader continues. For > consumer of bintime() or any other function' result, it looks exactly > the same as if we did not serialized with writer but used outdated > timehands. Not with full serialization. The writer tc_windup() is also a reader, and serializing its read gives the necessary monotonicity (for a single thread): - normal reader locks the mutex, reads the timecounter and unlocks. The mutex makes visible all previous writes, so the reader doesn't use a stale timehands. Consumers of bintime(), etc., use this time in the past. - tc_windup() locks the mutex, reads the timecounter hardware and writes the timecounter soft state. The new offset is after all previous times read, since this is serialized. - normal reader as above sees the new state, so it reads times after the time of the windup, so also after the time of previous normal reads. > Let me formulate this diffeently: as far as consumer of the bintime() > result does not serialize itself with tc_windup(), serializing bintime() > itself against tc_windup() does not close the race, but it is not > obvious that the race matters. Readers can easily see times far in the past, but the times increase in program order. > Either we should just accept the race as > we currently do, or readers must take the spinlock where the exact value > of the current time is important, Disabling interrupts should be enough. In my version of 5.2, spinlocks don't disable hardware interrupts and may be preempted by fast interrupt handlers which may be not so fast and take hundreds of usec. Actually, even disabling interrupts might not be enough. A single isa bus read can take at least 138 usec (when it is behind a DMA queue or something). There are also NMI's and SMI's. > or readers must re-read the time after > doing something important, and redo if the new measuremedtime is outside > the acceptable range. This method seems to be essential for robustness. But I don't see any race (for a single thread and no timecounter skew across CPUs). Sloppy readers just see times an unknown but usually small time in the past. Non-sloppy readers can also defend against timecounter skew by binding to 1 CPU. Mutex locking of the timecounter doesn't give monotonic times across threads. It gives some order, but you don't know which. Another mutex or rendezvous is needed to control the order. Bruce From owner-freebsd-hackers@freebsd.org Fri Apr 5 19:35:09 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 27099155BB9B for ; Fri, 5 Apr 2019 19:35:09 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic314-20.consmr.mail.gq1.yahoo.com (sonic314-20.consmr.mail.gq1.yahoo.com [98.137.69.83]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id F3CB8712BD for ; Fri, 5 Apr 2019 19:35:07 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: uUzurf0VM1lxFYMoT_VBwfIKKg_s0eLih6LSYHMSNeXWXxiBkoP2rvVrWL0OxJD d8iJGhL1jake0_oyOkaNVOZmCy27aLpZYcM7gofxA3CuOfsqVndeXhVlcWxqhNkdNM1Yj2Cz6dPw wybRMPE2BjOlPvh3l10jl9ZdUuSJBCf4_MZhvroD6pKBdxavkTzMzWSVijZM8xfJ44v73EzoclQ0 h4PJVrGvn9H4aH1WR8DCKi6L1AQcTXiTTSxvSpqrwWx_aypqsI7SOs6VZzly.exRnX8B4v0BLUuy _WLEECk3rs4xPadTY12V32_ljrmWrGBMGcyDBhcFEowc7rY5kGlDZEZCEA1gNqmaBPq3VsSuJsvV qlqUJGPCMiGMvCgI.FUltTTlK4hq_FXG9N.eL6hFmHRz0ILtfUCdUDM3.DXbD8izpDIoYPwfziS2 WYWbyO8S4oTErJbZwXnHwZkDIghitYAWGhOfXXMVQ0r9SGn6iH9KWI14eucdXQxwtZ.had2oHv.L b83htDwmBZXc_XFul3qrF3lIYWBc2URPcFVdCFx1l0WBo.Zpj8IX3vdcpSS5JrNvsPobpp7y9pU. vS5RQplaBuKiRi5VaSmdvTPWieY3pILVoMlud70OKbbXvwpOFApWaPP0r36g2Ar3tDE3.t2SXwfP bvkBTSxPTZ4H1ZKC8sNr8p2aC9Rm6SuvdCvzvoGxVsQRlnu2tQRgOVGBh8xKCFuOLKpOkh.iEviJ uTQbaN92zku1Ca578H5Zqt4R8mNzedj6utmcE7wjoFi9o7XIUx45DNaZ4MDauMWr7UPPD2MdZacC 8qouyPItMtGT0Ad3n4vxBgPp7q0UdLgx2v64bd2pyttqJqMHiyVqOJNzCd3zznyfLDdIIQac9oXf Pc5pDZujnGMyPEjKCezUBD0HjRdHLHCnzXXW1GaCIbUQxTvkmomk1Of_KStVIzRZTzFzlC1RWhLm MG_d33ruDYKLaox2h9wQc_f5RFssNJSSNYBiJ_AoMEdNwxwOPNpIxWcKBePqJTFFEbSzzsHVjlcl Ql30S_V860plIFcyCYqsFMP78XVSMIBQLsKDOBZlFEewacGzYDYlGvALX_YrWwICidLO88SpLZQA 8zA-- Received: from sonic.gate.mail.ne1.yahoo.com by sonic314.consmr.mail.gq1.yahoo.com with HTTP; Fri, 5 Apr 2019 19:35:01 +0000 Received: from c-67-170-167-181.hsd1.or.comcast.net (EHLO [192.168.1.113]) ([67.170.167.181]) by smtp431.mail.gq1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID 86af237d517259e6241beadd5b873703; Fri, 05 Apr 2019 19:35:00 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.8\)) Subject: Re: powerpc64 head -r344018 stuck sleeping problems: th->th_scale * tc_delta(th) overflows unsigned 64 bits sometimes [patched failed] From: Mark Millard In-Reply-To: <20190406014724.X4174@besplex.bde.org> Date: Fri, 5 Apr 2019 12:34:59 -0700 Cc: Konstantin Belousov , freebsd-hackers Hackers , Michael Tuexen , FreeBSD PowerPC ML Content-Transfer-Encoding: quoted-printable Message-Id: <48DB5A87-1681-47DE-969F-FA569EBF6DF5@yahoo.com> References: <20190307222220.GK2492@kib.kiev.ua> <20190309144844.K1166@besplex.bde.org> <20190324110138.GR1923@kib.kiev.ua> <20190403070045.GW1923@kib.kiev.ua> <20190404011802.E2390@besplex.bde.org> <20190405113912.GB1923@kib.kiev.ua> <20190405230717.D3383@besplex.bde.org> <20190405132128.GD1923@kib.kiev.ua> <20190406003907.C3872@besplex.bde.org> <20190405143619.GF1923@kib.kiev.ua> <20190406014724.X4174@besplex.bde.org> To: Bruce Evans X-Mailer: Apple Mail (2.3445.104.8) X-Rspamd-Queue-Id: F3CB8712BD X-Spamd-Bar: +++ X-Spamd-Result: default: False [3.14 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MV_CASE(0.50)[]; FREEMAIL_FROM(0.00)[yahoo.com]; RCPT_COUNT_FIVE(0.00)[5]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; MX_GOOD(-0.01)[cached: mta6.am0.yahoodns.net]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; FREEMAIL_TO(0.00)[optusnet.com.au]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/21, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; NEURAL_SPAM_SHORT(0.27)[0.270,0]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE(1.90)[ip: (7.87), ipnet: 98.137.64.0/21(0.95), asn: 36647(0.76), country: US(-0.06)]; NEURAL_SPAM_MEDIUM(0.95)[0.948,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_SPAM_LONG(0.53)[0.533,0]; RCVD_IN_DNSWL_NONE(0.00)[83.69.137.98.list.dnswl.org : 127.0.5.0]; RWL_MAILSPIKE_POSSIBLE(0.00)[83.69.137.98.rep.mailspike.net : 127.0.0.17]; FREEMAIL_CC(0.00)[gmail.com] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Apr 2019 19:35:09 -0000 On 2019-Apr-5, at 08:26, Bruce Evans wrote: > On Fri, 5 Apr 2019, Konstantin Belousov wrote: >=20 >> On Sat, Apr 06, 2019 at 01:01:19AM +1100, Bruce Evans wrote: >>> On Fri, 5 Apr 2019, Konstantin Belousov wrote: >>>=20 >>>> On Fri, Apr 05, 2019 at 11:52:27PM +1100, Bruce Evans wrote: >>>>> On Fri, 5 Apr 2019, Konstantin Belousov wrote: >>>>>=20 >>>>>> On Thu, Apr 04, 2019 at 02:47:34AM +1100, Bruce Evans wrote: >>>>>>> I noticed (or better realized) a general problem with multiple >>>>>>> timehands. ntpd can slew the clock at up to 500 ppm, and at = least an >>>>>>> old version of it uses a rate of 50 ppm to fix up fairly small = drifts >>>>>>> in the milliseconds range. 500 ppm is enormous in CPU cycles -- = it is >>>>>>> 500 thousand nsec or 2 million cycles at 4GHz. Winding up the = timecounter >>>>>>> every 1 msec reduces this to only 2000 cycles. >>>>>>> ... >>>>>>> The main point of having multiple timehands (after introducing = the per- >>>>>>> timehands generation count) is to avoid blocking thread N during = the >>>>>>> update, but this doesn't actually work, even for only 2 = timehands and >>>>>>> a global generation count. >>>>>>=20 >>>>>> You are describing the generic race between reader and writer. = The same >>>>>> race would exist even with one timehand (and/or one global = generation >>>>>> counter), where ntp adjustment might come earlier or later of = some >>>>>> consumer accessing the timehands. If timehand instance was read = before >>>>>> tc_windup() run but code consumed the result after the windup, it = might >>>>>> appear as if time went backward, and this cannot be fixed without = either >>>>>> re-reading the time after time-depended calculations were done = and >>>>>> restarting, or some globabl lock ensuring serialization. >>>>>=20 >>>>> With 1 timehand, its generation count would be global. I think = its ordering >>>>> is strong enough to ensure serialization. >>>> Yes, single timehands result in global generation. But it would = not solve >>>> the same race appearing in slightly different manner, as I = described above. >>>> If reader finished while generation number in th was not yet reset, = but >>>> caller uses the result after tc_windup(), the effect is same as if = we >>>> have two th's and reader used the outdated one. >>>=20 >>> You described it too concisely for me to understand :-). >>>=20 >>> I now see that a single generation count doesn't give serialization. = I >>> thought that setting the generation to 0 at the start of tc_windup() >>> serialized the reader and writer. But all it does is prevent use of = the >>> results of the windup while only some of them are visible. If the >>> setting the generation count to 0 doesn't become before tc_windup() = reads >>> the hardware timecounter, then this read may be before other reads = using >>> the old timehands, but it needs to be after. >> If we have either single th or global gen counter, current code would >> become serialized, but this is not what I am about. Lets assume, for >=20 > No, generation counts used like they are now (or in any way that I can > see) can't give serialization. >=20 >> the sake of the discussion only, that all readers take the same = spinlock >> as tc_windup (i.e. tc_setclock_mtx). >=20 > Spinlocks are far too heavyweight. Most of the complications in = timecounter > locking are to avoid using them. But OK for the discussion. >=20 >> It is still possible that reader unlocked the mutex, tc_windup() = kicked >> in, locked the mutex and moved timehands (as you noted, this might >> even happen on the same CPU), and only then the reader continues. For >> consumer of bintime() or any other function' result, it looks exactly >> the same as if we did not serialized with writer but used outdated >> timehands. >=20 > Not with full serialization. The writer tc_windup() is also a reader, = and > serializing its read gives the necessary monotonicity (for a single = thread): > - normal reader locks the mutex, reads the timecounter and unlocks. = The > mutex makes visible all previous writes, so the reader doesn't use a > stale timehands. Consumers of bintime(), etc., use this time in the = past. > - tc_windup() locks the mutex, reads the timecounter hardware and = writes the > timecounter soft state. The new offset is after all previous times = read, > since this is serialized. > - normal reader as above sees the new state, so it reads times after = the > time of the windup, so also after the time of previous normal reads. >=20 >> Let me formulate this diffeently: as far as consumer of the bintime() >> result does not serialize itself with tc_windup(), serializing = bintime() >> itself against tc_windup() does not close the race, but it is not >> obvious that the race matters. >=20 > Readers can easily see times far in the past, but the times increase = in > program order. >=20 >> Either we should just accept the race as >> we currently do, or readers must take the spinlock where the exact = value >> of the current time is important, >=20 > Disabling interrupts should be enough. In my version of 5.2, = spinlocks > don't disable hardware interrupts and may be preempted by fast = interrupt > handlers which may be not so fast and take hundreds of usec. = Actually, > even disabling interrupts might not be enough. A single isa bus read > can take at least 138 usec (when it is behind a DMA queue or = something). > There are also NMI's and SMI's. >=20 >> or readers must re-read the time after >> doing something important, and redo if the new measuremedtime is = outside >> the acceptable range. >=20 > This method seems to be essential for robustness. >=20 > But I don't see any race (for a single thread and no timecounter skew > across CPUs). Sloppy readers just see times an unknown but usually = small > time in the past. Non-sloppy readers can also defend against = timecounter > skew by binding to 1 CPU. >=20 > Mutex locking of the timecounter doesn't give monotonic times across = threads. > It gives some order, but you don't know which. Another mutex or = rendezvous > is needed to control the order. >=20 Just for context for the original problem, in case it helps: The sleepq_timeout went into the case: if (td->td_sleeptimo > sbinuptime() || td->td_sleeptimo =3D=3D 0) = { /* * The thread does not want a timeout (yet). */ and after that the specific sleep did not try again (deleted?), thus the hangup for the sleeping thread. This was with a call backtrace looking like the below at the time: 0xe00000009af7c730: at sleepq_timeout+0x148 0xe00000009af7c7d0: at softclock_call_cc+0x234 0xe00000009af7c910: at callout_process+0x2e0 0xe00000009af7c9f0: at handleevents+0x22c 0xe00000009af7caa0: at timercb+0x340 0xe00000009af7cba0: at decr_intr+0x140 0xe00000009af7cbd0: at powerpc_interrupt+0x268 (I added a call to cause the backtrace to be reported.) For this call chain: timercb gets a "now" value that is passsed along and into callout_process but not to softclock_call_cc or sleepq_timeout . The callout_process is doing CALLOUT_DIRECT handling when it directly calls softclock_call_cc: . . . /* Iterate callwheel from firstb to nowb and then up to lastb. = */ do { sc =3D &cc->cc_callwheel[firstb & callwheelmask]; tmp =3D LIST_FIRST(sc); while (tmp !=3D NULL) { /* Run the callout if present time within = allowed. */ if (tmp->c_time <=3D now) { /* * Consumer told us the callout may be = run * directly from hardware interrupt = context. */ if (tmp->c_iflags & CALLOUT_DIRECT) { #ifdef CALLOUT_PROFILING ++depth_dir; #endif cc_exec_next(cc) =3D LIST_NEXT(tmp, c_links.le); cc->cc_bucket =3D firstb & = callwheelmask; LIST_REMOVE(tmp, c_links.le); softclock_call_cc(tmp, cc, #ifdef CALLOUT_PROFILING &mpcalls_dir, = &lockcalls_dir, NULL, #endif 1); tmp =3D cc_exec_next(cc); cc_exec_next(cc) =3D NULL; } else { . . . =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-hackers@freebsd.org Fri Apr 5 21:25:55 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 80311155EFA2 for ; Fri, 5 Apr 2019 21:25:55 +0000 (UTC) (envelope-from dmonopatis@gmail.com) Received: from mail-pf1-x436.google.com (mail-pf1-x436.google.com [IPv6:2607:f8b0:4864:20::436]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E938C76055 for ; Fri, 5 Apr 2019 21:25:53 +0000 (UTC) (envelope-from dmonopatis@gmail.com) Received: by mail-pf1-x436.google.com with SMTP id 8so3952925pfr.4 for ; Fri, 05 Apr 2019 14:25:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=othBklqNzXn6fbAC+PRw/VwCJKsFxO/YJC9kei6rxXM=; b=KHJMymTrRpKeYim9C7XVsLqWRoPnpBf5Te9zqoqJItWP86fMC8q6K9CGmiOMSZHqks +0Aa9yKxnDay+B/OjWayteFrO+62+aBMmvtlz6pgpHXESn1fDHwbKS/AR6sWYXNcj5N7 e6HrxTCioBK94aqZPAimajbaC9nJ0/LvYbg51eEKXjXmaP4eMN/nsDQZQNZuXDxAYEeK TKN/fKuDJFzKLVzJKk8U27TvEKLrwb62uWlv0dTRwAku5MLjCwZVDsrzxnBngzfrqvOK GJXKR9gc89mmeK4ijwQ0/k5zXFEWZuQcDj97m3WLA7HKSspSkbrpLfPT3mhmlh/SeWBd 1Dtw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=othBklqNzXn6fbAC+PRw/VwCJKsFxO/YJC9kei6rxXM=; b=Xx7eA8Xuafo5TCNrHbz4oNrJSDXrctB7akJSIL/t8PSYx5Ky2p2UUFsJOvSZksAoNo oTwDVMCBW8FbNze/Vt+LtI2BWSyRxweB22wPwz6A9b7Lwi6tZgdayw5PIg9ZEZCLPU8J 9pAwV2NFbYmawgWH5sE2/HyQ5XIIHWazQlalOEWuB4n19FSIQeGxsmxxY2maZyRj6Sos b+etKccChTE61Fdzc70k6PHj8j7WgDrYVjFv3JDdRwq33ek/zu4liYLm/0cgPeG1rc7t qr04yiZzdRFBjW0WjFxKYQVQLGoK4U/rQ5rxXuWz1YC52Hvhewa9isYjRC8T+J8XN6Jl 9iaQ== X-Gm-Message-State: APjAAAX1kFpXc/xbXA1qalDZNrEyysHO59uWhBRE3UNkyIuclD5tFoW9 6ALjUbLWHarzsz7ALEUJDVSgiGAYJQ9huSThNOg4Yg== X-Google-Smtp-Source: APXvYqwMc1Mp56O1kfQVkVdgZrnETooxgF8f5H+v6wW6CM6oHaZ+lBJhsoZxH+EFy/+RIZQb2gA+3NeKw1+HJxU3DOQ= X-Received: by 2002:a63:6cc7:: with SMTP id h190mr14610334pgc.350.1554499552401; Fri, 05 Apr 2019 14:25:52 -0700 (PDT) MIME-Version: 1.0 From: Dimitris Monopatis Date: Sat, 6 Apr 2019 00:25:41 +0300 Message-ID: Subject: SoC: Dual-stack ping(1) command To: freebsd-hackers@freebsd.org X-Rspamd-Queue-Id: E938C76055 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=KHJMymTr; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of dmonopatis@gmail.com designates 2607:f8b0:4864:20::436 as permitted sender) smtp.mailfrom=dmonopatis@gmail.com X-Spamd-Result: default: False [-6.86 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_TLS_LAST(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[6.3.4.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; IP_SCORE(-2.95)[ip: (-9.59), ipnet: 2607:f8b0::/32(-2.93), asn: 15169(-2.17), country: US(-0.06)]; NEURAL_HAM_SHORT(-0.90)[-0.899,0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Apr 2019 21:25:55 -0000 Hello all, I'm looking for a mentor for merging ping with ping6 as part of Summer of Code. You can call me Dimitri Monopati, I almost finish my integrated master in the Computer Engineering and Informatics Department of University of Patras. From owner-freebsd-hackers@freebsd.org Fri Apr 5 21:31:52 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 06CE5155F397 for ; Fri, 5 Apr 2019 21:31:52 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic302-3.consmr.mail.bf2.yahoo.com (sonic302-3.consmr.mail.bf2.yahoo.com [74.6.135.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9443F762FD for ; Fri, 5 Apr 2019 21:31:51 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: m3n.7wIVM1kc6473I5483ksjM9s0yR8uGzlSWg3L9xpBxZIAEq2e2TeMKoLHYKk 7NKnJVbk8QF3DI37BqEt5tSU9MMVpK3jaRGQDEFAshAa.2WUSG2D3aTuIcC99upH.KxqzdIlbG23 ngKmuMl0UsK2IphWhUuK0IGKPrfHIYZkNspo7WWi0QonLAGn3WkiEaBFH6k5wIKyr5Zbu7JOaqr_ 15OhnGjVKgxms6hxZK5_q3Q4M1i1TwiQ2RBhDx2oLr6e8roMYvz9WOlnTAo.pyr5EIKYzK7ijbL0 F7rKwsZiANSgwsRoEeu16nbrcFsjLbWxUYV35.zbSs6gVPKxDcn49P_o7udmpczqX2zjd.8YgDj4 a2RVOQtWBvxmF8Nxz8yqLuE5rzuPkitj9lxuE5VafuuG6YMOPN546RnmzzA_g2rS6ouDyH9bq1xU _PEJadq1zBmuJtIs0mX1ejPl9mUiwEKKPBy9cyS9aUaYOS6YLv_WCFK26_SW6Rj2yLo9zSugxj1c mQONqVhHD_k6G4K3k0gIbtLhYrTT91CrvhNBQSv6kT_cJDMyTdQrmLRCe6dXKV5.tNTF8x7jB7t. ItXNVgrgBKwn0zr.N_YXPJs140tsiHmBAu1yJhYRxDq7J6tfrT6dJRTMGg0pGcNu_p.SaVTeuerx C.7GhqK5xv9lVhmDjLAacK5rYIh69wcTAkzl3071VzjNPXh939c2KZiFbWaoBkTJfdJRwzXpJQD3 tmWsCjZb1LNhC8NzLbQevSs7r3xwoM19FBJRIEvuaUgDJ05WXpV6WpwocPkYjp18cQ7gb5QMcA62 r7WoOaB7kXabWjJWfw.L.KYvApEPQGKsDLf.O5WcIwwJibt6.cE6XTzWIxL4AYXSC.P3shpiZc0D 702ZzBc9OvDp5GnIkq0KbK7y2nY2QnwNERYxAetyLcRED0vINsu.qJ5WXaYkUo3ge_lpbCWGKq5G pjD6SI4OmU7ayibz2A._z4Go4LqX5ImS3cM.HpdK7yAqj8McxPfhvmawukXAuuOXwUAIxaSmtSkT hustoDi2P_tLfQLDVG6XuaeNweK9IEDmjc7h5E3xUGhtYWd8ATl_FT3coIalTQc.BlaWIXbPp Received: from sonic.gate.mail.ne1.yahoo.com by sonic302.consmr.mail.bf2.yahoo.com with HTTP; Fri, 5 Apr 2019 21:31:50 +0000 Received: from c-67-170-167-181.hsd1.or.comcast.net (EHLO [192.168.1.113]) ([67.170.167.181]) by smtp401.mail.bf1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID 7418801b5e1fee9bf1fae3b3fb8f9af6; Fri, 05 Apr 2019 21:31:46 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.8\)) Subject: Re: powerpc64 head -r344018 stuck sleeping problems: th->th_scale * tc_delta(th) overflows unsigned 64 bits sometimes [patched failed] From: Mark Millard In-Reply-To: <20190405113504.GA1923@kib.kiev.ua> Date: Fri, 5 Apr 2019 14:31:43 -0700 Cc: freebsd-hackers Hackers , FreeBSD PowerPC ML Content-Transfer-Encoding: quoted-printable Message-Id: <55604EF6-CB81-4300-8E9E-E3A94504D0B5@yahoo.com> References: <20190307222220.GK2492@kib.kiev.ua> <20190309144844.K1166@besplex.bde.org> <20190324110138.GR1923@kib.kiev.ua> <20190403070045.GW1923@kib.kiev.ua> <20190404011802.E2390@besplex.bde.org> <20190405150236.A959@besplex.bde.org> <20190405201055.I2396@besplex.bde.org> <20190405113504.GA1923@kib.kiev.ua> To: Konstantin Belousov , Bruce Evans X-Mailer: Apple Mail (2.3445.104.8) X-Rspamd-Queue-Id: 9443F762FD X-Spamd-Bar: ------ X-Spamd-Result: default: False [-6.98 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.98)[-0.983,0]; REPLY(-4.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Apr 2019 21:31:52 -0000 [I found a document covering PLLs for the 970MP's in the old PowerMac G5 involved.] > On 2019-Apr-5, at 04:35, Konstantin Belousov = wrote: >=20 > On Fri, Apr 05, 2019 at 09:13:50PM +1100, Bruce Evans wrote: >> On Fri, 5 Apr 2019, Mark Millard wrote: >>=20 >>> On 2019-Apr-4, at 21:38, Bruce Evans = wrote: >>>=20 >>>> On Thu, 4 Apr 2019, Mark Millard wrote: >>>>> ... >>>>> Unfortunately, all the multi-socket contexts that I sometimes have >>>>> access to are old PowerMacs. And, currently, the only such context >>>>> is the G5 with 2 sockets, 2 cores per socket (powerpc64). So I've >>>>> not been able to set up other types of examples to see if problems >>>>> repeat. >>>>>=20 >>>>> I do not have access to a single-socket powerpc64 for contrast in >>>>> that direction. >>>>=20 >>>> Testing 1 socket is time-consuming enough. Do these old systems >>>> use the equivalent of an x86 TSC for the timecounter? With = multiple >>>> sockets, it isn't clear how even a hardware timer independent of = the >>>> CPUs can be distributed so as to appear to be monotonic on all = cors. >>>=20 >>> "The DEC frequency is based on the same implementation-dependent >>> frequency that drives the time base." The frequency may well be >>> fixed by the PowerMac G5 model implementation but is not fixed >>> by the powerpc64 architecture. >>>=20 >>> The Time Base Register (TBR) in a powerpc64 core (cpu in FreeBSD = terms) >>> increments at 33,333,333 Hz (nominal) and is 64 bits wide. Its value >>> can be set (mttb instruction) and the boot sequence in FreeBSD does >>> attempt to adjust as the FreeBSD CPU is brought-up/started. mftb is >>> used to read the 64-bit value. FreeBSD masks it down to 32-bits to >>> contribute to the time-counter. >>>=20 >>> (Is that description sufficient for what you were after? I've never >>> seen documentation of how the 33,333,333 MHz is produced.) >>=20 >> This seems to be equivalent to an x86 TSC. > It is equivalent from the interface PoV. I saw some references that > Powers do have per-core PLLs, the best I can find now is > = https://stackoverflow.com/questions/43844438/are-multiple-clock-domains-co= mmon-in-modern-processors I finally found a document covering the 970MP in the old PowerMac G5 in question: http://datasheets.chipdb.org/IBM/PowerPC/970/PowerPC-970MP.pdf It has material about the PLL's, pinout, etc. : "This section will help in configuring the PLL and determining SYSCLK = input frequency for PowerPC 970MP systems." Also: 100 MHz to 700 MHz SYSCLK. "SYSCLK minimum frequency is for PLL mode. In PLL bypass mode (BYPASS = low), SYSCLK frequency may be as low as 10MHz." BUS_CFG[0:2], PLL_MULT, and PLL_RANGE[1:0] are involved for PLL mode. Ratio from BUS_CFG: 3:1, 6:1, 12:1 normally use PLL_MULT=3D0 for a PLL = multiplier of 12. Ratio from BUS_CFG: 2:1, 4:1, 8:1 normally use PLL_MULT=3D1 for a PLL = multiplier of 8. (It describes the PLL power supply filtering circuit as well.) So far I've not seen anything that is directly for the rate that the TBR and DEC registers change. > For Intel there is no much information, the best guess is that TSC is = in > uncore and effectively shared by all cores. See also > = https://software.intel.com/en-us/articles/best-timing-function-for-measuri= ng-ipp-api-timing/ >=20 >>=20 >> x86 P-state-invariant TSCs apparently use a 100 MHz clock which = corresponds >> even more closely to this, except for historical reasons this clock = is >> scaled and interpolated to a clock resembling the CPU cycle count at = a >> nominal frequency. > For Intel designs, there is indeed a single-source 100MHz signal which > is distributed over all consumers using clock fan-out buffers like = DB1900Z. >=20 >>=20 >>> As FreeBSD supports multi-socket, what are its criteria for a = sufficient >>> context for it to work with for supporting sbinuptime and the like? = Is >>> FreeBSD supposed to then make it appear that sbinputime and the like = are >>> weakly increasing, even as threads migrate between CPUs (cores, >>> hw-threads)? >>=20 >> CLOCK_MONOTONIC has to be monotonic, but it is only clear what this = means >> within a single thread: sequential clock_gettime() calls must occur = in >> program order and the results must be consistent with that order. = Across >> threads, I think it should mean that the results must be consistent = with >> any order established using any supported ordering methods. >>=20 >>> ... >>>>> One oddity is that the eventtimer's decrementer and timecounter >>>>> may change (nearly) together: both change at 33,333,333 Hz, as if >>>>> they are tied to the same clock (at least on one socket). >>>>=20 >>>> I think this is from a normal hardware implementation. On all of >>>> my x86 systems with a TSC, the TSC frequency is an exact fractional >>>> multiple of the i8254, the ACPI timer (if present) and the HPET (if >>>> present). Only the RTC has an independent frequency. The fraction = is >>>> changed by changing the nominal TSC frequency in the BIOS, but is = not >>>> changed by temperature variations. This must be because most = clocks are >>>> derived from a common clock using a PLL. I use this to calibrate = all >>>> clocks (except the RTC) by calibrating only 1. >>>=20 >>> I'm not aware of the OpenFirmware having any control over the >>> TBR-change frequency behavior. I've no evidence about any = variability >>> based on temperature. >>=20 >> Temperature changes usually affect the actual frequency but not the >> nominal frequency. >>=20 =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-hackers@freebsd.org Sat Apr 6 07:54:15 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 701251579035 for ; Sat, 6 Apr 2019 07:54:15 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic313-10.consmr.mail.ne1.yahoo.com (sonic313-10.consmr.mail.ne1.yahoo.com [66.163.185.33]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BEBF06D305 for ; Sat, 6 Apr 2019 07:54:13 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: krgR4qkVM1lp_il.ec8qPQ_FqWpoc9GCtFRDRv.tGcazLsOHleHFCe9kZhtuhMD PIgmzklwHihOlY7Ypad.R8_9H2oRCfy6H8DXhdA_tC5FoGFq0nyQFZnAx4EaArLiQDxSvBBiC6TK hkr8F6sO_iHyHlvt0IgkZ9DPY3cD3Ghnb.WIFxIAJNM9IpjAZWq8LwXhknnrOho1X6_QK61Rj5G7 dxmxcS4dfp1jxTE2OaarF8khua6b8qHy7qCLD1jJujOaUzMH_E5D4R.bB_YdXQP4ToklgO.7FclY oCEDjBT5DYFJIQMP7SBVRgfGll.7jmOHoY_zaUnTDSMO5AWs_C6QyPWsFm5VPiK0e0F2Pa4r9Lmb uyN3lifTuHgwSPUKSAlzCIRRz2sr5fe8ZefU3RdFS4G6KX0OwMEhg8oxKEt3kuM.ypNQIG7wIV1L FfI3mD93WBKaqGPstIBdTSu_xBdvlY2SNxdmfSVPPdd0Zn5A7JgO_CizfIIzL8CChWwbEs_lcNJA 9yLZY5sivju16ORgRtKY4RxiOTAjzkyRswU9MVZ1xKkuLz6sD9WTjTxVfknMIzSAsR9o8nWlt7gZ b9U60miS2dYAEf32lKfEEb5KQ45nJG2Td0yWxjTqWEgZnTeCPjXMPQsc9ZR8h2eJuDN__PZRki7Z RAjwdOd4yse299CwuLnIEoXiFHRvs5PiH7_7y7rlRa9Wy7cYPGTdLTnZG6lNnoNrvv5yP9.ckQLv G9tjg4CgBkviLYZ3YEvQLNoalrDy2KaVIe3RQlaSL9TRS4d7EeT8KmhZQPH7W0wSkF1ivADFlW5K fylhXVOQHV4v5oygYAjhR8FW1F2WxWUqHZJ.IwglVBIBb4.eTsZ9Gr1lRy6hOieMV2PQlRzDhdAe pXtLcwN9eop0iU6PeNLMjShHkbHMwKVWTuBZpVWEdu8O46hxpNe3XV6aHn1ljUNTenG4EoBwUA7c g_sFjAKXuqZlYc7GDLjJcfRMQIdUGhflRdky5bShGdw1zRYRadnOCHwUlcH8cTRFhFjf9zK9STw3 vBuppWsPdKZgtj3RlcRUaSmJiFDtVEtkMNw8xTSgYK5lILMBHtOs0B.lxGuGLoyitl._W7V7z Received: from sonic.gate.mail.ne1.yahoo.com by sonic313.consmr.mail.ne1.yahoo.com with HTTP; Sat, 6 Apr 2019 07:54:05 +0000 Received: from c-67-170-167-181.hsd1.or.comcast.net (EHLO [192.168.1.113]) ([67.170.167.181]) by smtp417.mail.ne1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID 71c50158df587c5d8e6f5cfcbcf89622; Sat, 06 Apr 2019 07:54:02 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.8\)) Subject: Re: powerpc64 head -r344018 stuck sleeping problems: th->th_scale * tc_delta(th) overflows unsigned 64 bits sometimes [patched failed] From: Mark Millard In-Reply-To: <55604EF6-CB81-4300-8E9E-E3A94504D0B5@yahoo.com> Date: Sat, 6 Apr 2019 00:54:00 -0700 Cc: freebsd-hackers Hackers , FreeBSD PowerPC ML Content-Transfer-Encoding: quoted-printable Message-Id: <3592A7F1-9EAE-4A33-B51A-678BE104E18C@yahoo.com> References: <20190307222220.GK2492@kib.kiev.ua> <20190309144844.K1166@besplex.bde.org> <20190324110138.GR1923@kib.kiev.ua> <20190403070045.GW1923@kib.kiev.ua> <20190404011802.E2390@besplex.bde.org> <20190405150236.A959@besplex.bde.org> <20190405201055.I2396@besplex.bde.org> <20190405113504.GA1923@kib.kiev.ua> <55604EF6-CB81-4300-8E9E-E3A94504D0B5@yahoo.com> To: Konstantin Belousov , Bruce Evans X-Mailer: Apple Mail (2.3445.104.8) X-Rspamd-Queue-Id: BEBF06D305 X-Spamd-Bar: +++ X-Spamd-Result: default: False [3.42 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MV_CASE(0.50)[]; FREEMAIL_FROM(0.00)[yahoo.com]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[cached: mta6.am0.yahoodns.net]; DKIM_TRACE(0.00)[yahoo.com:+]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36646, ipnet:66.163.184.0/21, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; NEURAL_SPAM_SHORT(0.95)[0.951,0]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE(1.50)[ip: (4.94), ipnet: 66.163.184.0/21(1.46), asn: 36646(1.17), country: US(-0.06)]; NEURAL_SPAM_MEDIUM(0.96)[0.960,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_SPAM_LONG(0.52)[0.516,0]; RCVD_IN_DNSWL_NONE(0.00)[33.185.163.66.list.dnswl.org : 127.0.5.0]; RWL_MAILSPIKE_POSSIBLE(0.00)[33.185.163.66.rep.mailspike.net : 127.0.0.17] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Apr 2019 07:54:15 -0000 On 2019-Apr-5, at 14:31, Mark Millard wrote: [I found a document covering PLLs for the 970MP's in the old PowerMac G5 involved.] > On 2019-Apr-5, at 04:35, Konstantin Belousov = wrote: >>=20 >> On Fri, Apr 05, 2019 at 09:13:50PM +1100, Bruce Evans wrote: >>> On Fri, 5 Apr 2019, Mark Millard wrote: >>>=20 >>>> On 2019-Apr-4, at 21:38, Bruce Evans = wrote: >>>>=20 >>>>> On Thu, 4 Apr 2019, Mark Millard wrote: >>>>>> ... >>>>>> Unfortunately, all the multi-socket contexts that I sometimes = have >>>>>> access to are old PowerMacs. And, currently, the only such = context >>>>>> is the G5 with 2 sockets, 2 cores per socket (powerpc64). So I've >>>>>> not been able to set up other types of examples to see if = problems >>>>>> repeat. >>>>>>=20 >>>>>> I do not have access to a single-socket powerpc64 for contrast in >>>>>> that direction. >>>>>=20 >>>>> Testing 1 socket is time-consuming enough. Do these old systems >>>>> use the equivalent of an x86 TSC for the timecounter? With = multiple >>>>> sockets, it isn't clear how even a hardware timer independent of = the >>>>> CPUs can be distributed so as to appear to be monotonic on all = cors. >>>>=20 >>>> "The DEC frequency is based on the same implementation-dependent >>>> frequency that drives the time base." The frequency may well be >>>> fixed by the PowerMac G5 model implementation but is not fixed >>>> by the powerpc64 architecture. >>>>=20 >>>> The Time Base Register (TBR) in a powerpc64 core (cpu in FreeBSD = terms) >>>> increments at 33,333,333 Hz (nominal) and is 64 bits wide. Its = value >>>> can be set (mttb instruction) and the boot sequence in FreeBSD does >>>> attempt to adjust as the FreeBSD CPU is brought-up/started. mftb is >>>> used to read the 64-bit value. FreeBSD masks it down to 32-bits to >>>> contribute to the time-counter. >>>>=20 >>>> (Is that description sufficient for what you were after? I've never >>>> seen documentation of how the 33,333,333 MHz is produced.) >>>=20 >>> This seems to be equivalent to an x86 TSC. >> It is equivalent from the interface PoV. I saw some references that >> Powers do have per-core PLLs, the best I can find now is >> = https://stackoverflow.com/questions/43844438/are-multiple-clock-domains-co= mmon-in-modern-processors >=20 > I finally found a document covering the 970MP in the old > PowerMac G5 in question: >=20 > http://datasheets.chipdb.org/IBM/PowerPC/970/PowerPC-970MP.pdf >=20 > It has material about the PLL's, pinout, etc. : >=20 > "This section will help in configuring the PLL and determining SYSCLK = input > frequency for PowerPC 970MP systems." Also: 100 MHz to 700 MHz SYSCLK. > "SYSCLK minimum frequency is for PLL mode. In PLL bypass mode (BYPASS = low), > SYSCLK frequency may be as low as 10MHz." BUS_CFG[0:2], PLL_MULT, and > PLL_RANGE[1:0] are involved for PLL mode. >=20 > Ratio from BUS_CFG: 3:1, 6:1, 12:1 normally use PLL_MULT=3D0 for a PLL = multiplier of 12. > Ratio from BUS_CFG: 2:1, 4:1, 8:1 normally use PLL_MULT=3D1 for a PLL = multiplier of 8. >=20 > (It describes the PLL power supply filtering circuit as well.) >=20 > So far I've not seen anything that is directly for the rate that the = TBR > and DEC registers change. >=20 >> For Intel there is no much information, the best guess is that TSC is = in >> uncore and effectively shared by all cores. See also >> = https://software.intel.com/en-us/articles/best-timing-function-for-measuri= ng-ipp-api-timing/ >>=20 >>>=20 >>> x86 P-state-invariant TSCs apparently use a 100 MHz clock which = corresponds >>> even more closely to this, except for historical reasons this clock = is >>> scaled and interpolated to a clock resembling the CPU cycle count at = a >>> nominal frequency. >> For Intel designs, there is indeed a single-source 100MHz signal = which >> is distributed over all consumers using clock fan-out buffers like = DB1900Z. >>=20 >>>=20 >>>> As FreeBSD supports multi-socket, what are its criteria for a = sufficient >>>> context for it to work with for supporting sbinuptime and the like? = Is >>>> FreeBSD supposed to then make it appear that sbinputime and the = like are >>>> weakly increasing, even as threads migrate between CPUs (cores, >>>> hw-threads)? >>>=20 >>> CLOCK_MONOTONIC has to be monotonic, but it is only clear what this = means >>> within a single thread: sequential clock_gettime() calls must occur = in >>> program order and the results must be consistent with that order. = Across >>> threads, I think it should mean that the results must be consistent = with >>> any order established using any supported ordering methods. >>>=20 >>>> ... >>>>>> One oddity is that the eventtimer's decrementer and timecounter >>>>>> may change (nearly) together: both change at 33,333,333 Hz, as if >>>>>> they are tied to the same clock (at least on one socket). >>>>>=20 >>>>> I think this is from a normal hardware implementation. On all of >>>>> my x86 systems with a TSC, the TSC frequency is an exact = fractional >>>>> multiple of the i8254, the ACPI timer (if present) and the HPET = (if >>>>> present). Only the RTC has an independent frequency. The = fraction is >>>>> changed by changing the nominal TSC frequency in the BIOS, but is = not >>>>> changed by temperature variations. This must be because most = clocks are >>>>> derived from a common clock using a PLL. I use this to calibrate = all >>>>> clocks (except the RTC) by calibrating only 1. >>>>=20 >>>> I'm not aware of the OpenFirmware having any control over the >>>> TBR-change frequency behavior. I've no evidence about any = variability >>>> based on temperature. >>>=20 >>> Temperature changes usually affect the actual frequency but not the >>> nominal frequency. >>>=20 >=20 I found more about the 970MP's TB/DEC rate: "The TBEN input pin can be used as an enable for the internal timebase/decrementer or as an external clock input." Details: HID0 bit 19 =3D 0: update at 1/16th processor core frequency, but only when TBEN is high. HID0 bit 19 =3D 1: update at rising edge of TBEN (must not exceed 1/16th of the core processor's maximum frequency). So the 33,333,333 Hz TB&DEC update rate vs. 2500 MHz would mean that rising edges of TBEN are in use (HID0 bit 19 =3D 1) in the PowerMac G5 context in question. I've no information about how closely matched the TBEN signals are at the 2 sockets, beyond the nominal frequency. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-hackers@freebsd.org Sat Apr 6 08:25:22 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CA9EB157A43F for ; Sat, 6 Apr 2019 08:25:21 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic302-20.consmr.mail.gq1.yahoo.com (sonic302-20.consmr.mail.gq1.yahoo.com [98.137.68.146]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 49C896F2EE for ; Sat, 6 Apr 2019 08:25:20 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: 4cnKdn8VM1nYoyhy2wjSk5RYKMFVZTzfVtToQCcExGw7cscAKevnD4ij5g3CIU9 viVNlpClSvqK0OXvT2YrjznQdIvkDG7KaC3GknQ2OhUvQMfPwG9.Y9y4o_ZvFuJXalXuP66wv88s lBiFTjGGpMVQjPzDj3TL3FgGd0rP9R6yxRu.plvknkZsPSdn_nmzfz4_OBDD4xh5iT1mvb.xMqzy Mx2.zDXOvlTIKAbqNPYivl2hCq4tpH2oYq0etF3iDA3N9_WRnwRmX2_toCJSdfkSKnoUkKWAITRR Wo_jUDo6y5bnnNUgm1Ti5eshPNQgyqNP9wmNdn4mr8Re187njoQWywv_MMR12B_DagAdyitl5QRX 52eXpa4H65C9kUU4fMXSkLUN9VBaE0zTZsuw5IOm5sy16QKhtNZgTmwtxV.zD84VQiaJkqEhy8Gv P3cVySaEoF4fTwQNAhQzUDaw.tzlZS6eR0Yqm_Ha3__qbfbXvTCffJFvs8wyrw5QRSJWyeFZiFES JEIRw9Q05So3VWcrhPSk7xKbdZgIrgOHvwZs9HUGbejQ1bjOjE4Aw0VqRelhkldu17Ow3ECT09VV kctRcPrPFNnIC_Gl42j7RvBtuYc1gRYjDDBTEGEep0Xgm6PQiv5eynmq3k5TyVATnHdzeo8xv_Jl u0_a8np3rBonMGR5mYRso4Bs6xukNBdB2IlgPQOF7DCApD8a8J3synuuYgp3MIvNbS41uFpvc2Tu DSMVPR1GcyNtr1m6FezGO9ncw9gTl1owzzQfG5fncpZLVH397UukvV.x1FW2TzbRv38KWExnPS8y xUGzY18SvDUFqdwOdQs3GrzTSVI.E3cGzW2g375Wk_IPIBqJQDU5DU.88HikFMNjQriA8Fv_qNVF 2yZRtM7G8XceAyh1iFab_xEpPwnoLuUOUsUx6WaAF_x8FYwHvmSkT4iZsHs6lyXCBE.SlwS41uCx gJma5CzFClBXJY49fqO.1Uao3CJkm.D8OzlFCLsEWXM2JAsL_nsJCFEEAgbmFwVrJq42QoynW0Sx vYHpvudnhVw3c2FWpijRE4dlqtfUW.A2b91YtzTcfph7UC3nBRLYIpvOr1Yu1BLyEb3FNA_c4Xg- - Received: from sonic.gate.mail.ne1.yahoo.com by sonic302.consmr.mail.gq1.yahoo.com with HTTP; Sat, 6 Apr 2019 08:25:18 +0000 Received: from c-67-170-167-181.hsd1.or.comcast.net (EHLO [192.168.1.113]) ([67.170.167.181]) by smtp420.mail.gq1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID 75821d711dc554505c099e91b68f42a6; Sat, 06 Apr 2019 08:25:15 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.8\)) Subject: Re: powerpc64 head -r344018 stuck sleeping problems: th->th_scale * tc_delta(th) overflows unsigned 64 bits sometimes [patched failed] From: Mark Millard In-Reply-To: <3592A7F1-9EAE-4A33-B51A-678BE104E18C@yahoo.com> Date: Sat, 6 Apr 2019 01:25:13 -0700 Cc: freebsd-hackers Hackers , FreeBSD PowerPC ML Content-Transfer-Encoding: quoted-printable Message-Id: <7B096BB4-7DBA-4B95-AC1E-DDD9A7DF3B22@yahoo.com> References: <20190307222220.GK2492@kib.kiev.ua> <20190309144844.K1166@besplex.bde.org> <20190324110138.GR1923@kib.kiev.ua> <20190403070045.GW1923@kib.kiev.ua> <20190404011802.E2390@besplex.bde.org> <20190405150236.A959@besplex.bde.org> <20190405201055.I2396@besplex.bde.org> <20190405113504.GA1923@kib.kiev.ua> <55604EF6-CB81-4300-8E9E-E3A94504D0B5@yahoo.com> <3592A7F1-9EAE-4A33-B51A-678BE104E18C@yahoo.com> To: Konstantin Belousov , Bruce Evans X-Mailer: Apple Mail (2.3445.104.8) X-Rspamd-Queue-Id: 49C896F2EE X-Spamd-Bar: + X-Spamd-Result: default: False [1.45 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MV_CASE(0.50)[]; FREEMAIL_FROM(0.00)[yahoo.com]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; MX_GOOD(-0.01)[cached: mta6.am0.yahoodns.net]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/21, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; NEURAL_SPAM_SHORT(0.89)[0.891,0]; NEURAL_HAM_LONG(-0.52)[-0.521,0]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE(0.86)[ip: (2.66), ipnet: 98.137.64.0/21(0.95), asn: 36647(0.76), country: US(-0.06)]; NEURAL_SPAM_MEDIUM(0.73)[0.727,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[146.68.137.98.list.dnswl.org : 127.0.5.0] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Apr 2019 08:25:22 -0000 [Another document was more explicit about 1/16 mode.] On 2019-Apr-6, at 00:54, Mark Millard wrote: > On 2019-Apr-5, at 14:31, Mark Millard wrote: >=20 > [I found a document covering PLLs for the 970MP's in the old > PowerMac G5 involved.] >=20 >> On 2019-Apr-5, at 04:35, Konstantin Belousov = wrote: >>>=20 >>> On Fri, Apr 05, 2019 at 09:13:50PM +1100, Bruce Evans wrote: >>>> On Fri, 5 Apr 2019, Mark Millard wrote: >>>>=20 >>>>> On 2019-Apr-4, at 21:38, Bruce Evans = wrote: >>>>>=20 >>>>>> On Thu, 4 Apr 2019, Mark Millard wrote: >>>>>>> ... >>>>>>> Unfortunately, all the multi-socket contexts that I sometimes = have >>>>>>> access to are old PowerMacs. And, currently, the only such = context >>>>>>> is the G5 with 2 sockets, 2 cores per socket (powerpc64). So = I've >>>>>>> not been able to set up other types of examples to see if = problems >>>>>>> repeat. >>>>>>>=20 >>>>>>> I do not have access to a single-socket powerpc64 for contrast = in >>>>>>> that direction. >>>>>>=20 >>>>>> Testing 1 socket is time-consuming enough. Do these old systems >>>>>> use the equivalent of an x86 TSC for the timecounter? With = multiple >>>>>> sockets, it isn't clear how even a hardware timer independent of = the >>>>>> CPUs can be distributed so as to appear to be monotonic on all = cors. >>>>>=20 >>>>> "The DEC frequency is based on the same implementation-dependent >>>>> frequency that drives the time base." The frequency may well be >>>>> fixed by the PowerMac G5 model implementation but is not fixed >>>>> by the powerpc64 architecture. >>>>>=20 >>>>> The Time Base Register (TBR) in a powerpc64 core (cpu in FreeBSD = terms) >>>>> increments at 33,333,333 Hz (nominal) and is 64 bits wide. Its = value >>>>> can be set (mttb instruction) and the boot sequence in FreeBSD = does >>>>> attempt to adjust as the FreeBSD CPU is brought-up/started. mftb = is >>>>> used to read the 64-bit value. FreeBSD masks it down to 32-bits to >>>>> contribute to the time-counter. >>>>>=20 >>>>> (Is that description sufficient for what you were after? I've = never >>>>> seen documentation of how the 33,333,333 MHz is produced.) >>>>=20 >>>> This seems to be equivalent to an x86 TSC. >>> It is equivalent from the interface PoV. I saw some references that >>> Powers do have per-core PLLs, the best I can find now is >>> = https://stackoverflow.com/questions/43844438/are-multiple-clock-domains-co= mmon-in-modern-processors >>=20 >> I finally found a document covering the 970MP in the old >> PowerMac G5 in question: >>=20 >> http://datasheets.chipdb.org/IBM/PowerPC/970/PowerPC-970MP.pdf >>=20 >> It has material about the PLL's, pinout, etc. : >>=20 >> "This section will help in configuring the PLL and determining SYSCLK = input >> frequency for PowerPC 970MP systems." Also: 100 MHz to 700 MHz = SYSCLK. >> "SYSCLK minimum frequency is for PLL mode. In PLL bypass mode (BYPASS = low), >> SYSCLK frequency may be as low as 10MHz." BUS_CFG[0:2], PLL_MULT, and >> PLL_RANGE[1:0] are involved for PLL mode. >>=20 >> Ratio from BUS_CFG: 3:1, 6:1, 12:1 normally use PLL_MULT=3D0 for a = PLL multiplier of 12. >> Ratio from BUS_CFG: 2:1, 4:1, 8:1 normally use PLL_MULT=3D1 for a PLL = multiplier of 8. >>=20 >> (It describes the PLL power supply filtering circuit as well.) >>=20 >> So far I've not seen anything that is directly for the rate that the = TBR >> and DEC registers change. >>=20 >>> For Intel there is no much information, the best guess is that TSC = is in >>> uncore and effectively shared by all cores. See also >>> = https://software.intel.com/en-us/articles/best-timing-function-for-measuri= ng-ipp-api-timing/ >>>=20 >>>>=20 >>>> x86 P-state-invariant TSCs apparently use a 100 MHz clock which = corresponds >>>> even more closely to this, except for historical reasons this clock = is >>>> scaled and interpolated to a clock resembling the CPU cycle count = at a >>>> nominal frequency. >>> For Intel designs, there is indeed a single-source 100MHz signal = which >>> is distributed over all consumers using clock fan-out buffers like = DB1900Z. >>>=20 >>>>=20 >>>>> As FreeBSD supports multi-socket, what are its criteria for a = sufficient >>>>> context for it to work with for supporting sbinuptime and the = like? Is >>>>> FreeBSD supposed to then make it appear that sbinputime and the = like are >>>>> weakly increasing, even as threads migrate between CPUs (cores, >>>>> hw-threads)? >>>>=20 >>>> CLOCK_MONOTONIC has to be monotonic, but it is only clear what this = means >>>> within a single thread: sequential clock_gettime() calls must occur = in >>>> program order and the results must be consistent with that order. = Across >>>> threads, I think it should mean that the results must be consistent = with >>>> any order established using any supported ordering methods. >>>>=20 >>>>> ... >>>>>>> One oddity is that the eventtimer's decrementer and timecounter >>>>>>> may change (nearly) together: both change at 33,333,333 Hz, as = if >>>>>>> they are tied to the same clock (at least on one socket). >>>>>>=20 >>>>>> I think this is from a normal hardware implementation. On all of >>>>>> my x86 systems with a TSC, the TSC frequency is an exact = fractional >>>>>> multiple of the i8254, the ACPI timer (if present) and the HPET = (if >>>>>> present). Only the RTC has an independent frequency. The = fraction is >>>>>> changed by changing the nominal TSC frequency in the BIOS, but is = not >>>>>> changed by temperature variations. This must be because most = clocks are >>>>>> derived from a common clock using a PLL. I use this to calibrate = all >>>>>> clocks (except the RTC) by calibrating only 1. >>>>>=20 >>>>> I'm not aware of the OpenFirmware having any control over the >>>>> TBR-change frequency behavior. I've no evidence about any = variability >>>>> based on temperature. >>>>=20 >>>> Temperature changes usually affect the actual frequency but not the >>>> nominal frequency. >>>>=20 >>=20 >=20 > I found more about the 970MP's TB/DEC rate: >=20 > "The TBEN input pin can be used as an enable for the internal > timebase/decrementer or as an external clock input." Details: >=20 > HID0 bit 19 =3D 0: update at 1/16th processor core frequency, > but only when TBEN is high. That 1/16th should be of "the full processor frequency, even when the processor itself is running at a lower frequency". The 970FX has 1/8th instead of 1/16th. "Since the mesh clock frequency can be lowered to 1/64th of the full-speed, the time base/decrementer may be increased/decreased by more than one at a time." The decrementer tests for wrap, not for reaching zero. > HID0 bit 19 =3D 1: update at rising edge of TBEN > (must not exceed 1/16th of the core processor's > maximum frequency). >=20 > So the 33,333,333 Hz TB&DEC update rate vs. 2500 MHz would mean > that rising edges of TBEN are in use (HID0 bit 19 =3D 1) in the > PowerMac G5 context in question. >=20 > I've no information about how closely matched the TBEN signals are > at the 2 sockets, beyond the nominal frequency. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-hackers@freebsd.org Sat Apr 6 08:36:53 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 74D6E157A94C for ; Sat, 6 Apr 2019 08:36:53 +0000 (UTC) (envelope-from dewayne.geraghty@heuristicsystems.com.au) Received: from hermes.heuristicsystems.com.au (hermes.heuristicsystems.com.au [203.41.22.115]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "hermes.heuristicsystems.com.au", Issuer "Heuristic Systems Type 4 Host CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id C620F6F8AA for ; Sat, 6 Apr 2019 08:36:44 +0000 (UTC) (envelope-from dewayne.geraghty@heuristicsystems.com.au) Received: from [10.0.5.3] (noddy.hs [10.0.5.3]) (authenticated bits=0) by hermes.heuristicsystems.com.au (8.15.2/8.15.2) with ESMTPSA id x368KTTI061837 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Sat, 6 Apr 2019 19:20:32 +1100 (AEDT) (envelope-from dewayne.geraghty@heuristicsystems.com.au) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=heuristicsystems.com.au; s=hsa; t=1554538832; x=1555143633; bh=/fIVVhY4iy+gTyYBIbNXFfVAZH8yPNo9yDOWLBguwHo=; h=To:From:Subject:Message-ID:Date; b=BZ2gAoJRk+PJpGaqMZKZ3LoUn16b2QBp4SDAXG1rsax0myxepgsncDfbuQdWB9DnH TLRZVzM+EQ4yfFSmVyOE6sq1XeJGbhBIIpsXr9yFobmmW5dgI2YzwP0/fDWdjGBzPw atzzSZ+l4a+DuHIBvJpEOVOssPxnrgFpvo8LoRYLlwuCDqbEmLEeR X-Authentication-Warning: b3.hs: Host noddy.hs [10.0.5.3] claimed to be [10.0.5.3] To: "freebsd-hackers@freebsd.org" From: Dewayne Geraghty Subject: vmstat -z on FreeBSD11.1S prior to a move to 12 Openpgp: preference=signencrypt Message-ID: <9b7a4cf2-4193-21a0-8680-7ee5b6893e40@heuristicsystems.com.au> Date: Sat, 6 Apr 2019 19:20:28 +1100 User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:60.0) Gecko/20100101 Thunderbird/60.6.1 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Language: en-AU Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: C620F6F8AA X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=heuristicsystems.com.au header.s=hsa header.b=BZ2gAoJR; spf=pass (mx1.freebsd.org: domain of dewayne.geraghty@heuristicsystems.com.au designates 203.41.22.115 as permitted sender) smtp.mailfrom=dewayne.geraghty@heuristicsystems.com.au X-Spamd-Result: default: False [-4.87 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; HAS_XAW(0.00)[]; DKIM_TRACE(0.00)[heuristicsystems.com.au:+]; MX_GOOD(-0.01)[hermes.heuristicsystems.com.au]; NEURAL_HAM_SHORT(-0.55)[-0.553,0]; FROM_EQ_ENVFROM(0.00)[]; IP_SCORE(-0.71)[asn: 1221(-3.49), country: AU(-0.04)]; MIME_TRACE(0.00)[0:+]; RCVD_IN_DNSWL_LOW(-0.10)[115.22.41.203.list.dnswl.org : 127.0.4.1]; ASN(0.00)[asn:1221, ipnet:203.40.0.0/13, country:AU]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[heuristicsystems.com.au:s=hsa]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; DMARC_NA(0.00)[heuristicsystems.com.au]; RCPT_COUNT_ONE(0.00)[1]; DWL_DNSWL_LOW(-1.00)[heuristicsystems.com.au.dwl.dnswl.org : 127.0.4.1]; TO_DN_EQ_ADDR_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Apr 2019 08:36:53 -0000 I'm trying to understand if there is anything that I should do regarding the FAILs froma vmstat -z result ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP UMA Hash: 256, 0 , 5, 10, 13, 0, 0 4 Bucket: 32, 0, 113, 13637,56567851,1016721, 0 6 Bucket: 48, 0, 38, 10420, 5813437,1644722, 0 8 Bucket: 64, 0, 588, 10076,15183107,48522, 0 12 Bucket: 96, 0, 110, 1858, 4103705,87972, 0 16 Bucket: 128, 0, 835, 1583, 6158208, 1, 0 32 Bucket: 256, 0, 574, 2051,12819286, 10, 0 64 Bucket: 512, 0, 14016, 6008,13042857,4594231, 0 128 Bucket: 1024, 0, 1081, 7455,10754938,512246, 0 256 Bucket: 2048, 0, 1669, 2749, 2393011,1381094, 0 vmem btag: 56, 0, 65907, 49752, 5035839, 872, 0 I noticed the FreeBSD12 GENERIC kernel has an option regarding NUMA and thought I should understand the impact to my systems and how I can monitor the system for deviation (good or bad). (Its been a very long time since I've asked for help, but this isn't covered in my FreeBSD Design book - yes, paper! ;) From owner-freebsd-hackers@freebsd.org Sat Apr 6 12:17:04 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C35791558DCD; Sat, 6 Apr 2019 12:17:04 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 14F4A763A7; Sat, 6 Apr 2019 12:17:03 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id x36CGsx5061946 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sat, 6 Apr 2019 15:16:57 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua x36CGsx5061946 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id x36CGoJq061945; Sat, 6 Apr 2019 15:16:50 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 6 Apr 2019 15:16:50 +0300 From: Konstantin Belousov To: Bruce Evans Cc: Michael Tuexen , freebsd-hackers Hackers , FreeBSD PowerPC ML Subject: Re: powerpc64 head -r344018 stuck sleeping problems: th->th_scale * tc_delta(th) overflows unsigned 64 bits sometimes [patched failed] Message-ID: <20190406121650.GJ1923@kib.kiev.ua> References: <20190324110138.GR1923@kib.kiev.ua> <20190403070045.GW1923@kib.kiev.ua> <20190404011802.E2390@besplex.bde.org> <20190405113912.GB1923@kib.kiev.ua> <20190405230717.D3383@besplex.bde.org> <20190405132128.GD1923@kib.kiev.ua> <20190406003907.C3872@besplex.bde.org> <20190405143619.GF1923@kib.kiev.ua> <20190406014724.X4174@besplex.bde.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20190406014724.X4174@besplex.bde.org> User-Agent: Mutt/1.11.4 (2019-03-13) X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on tom.home X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Apr 2019 12:17:05 -0000 On Sat, Apr 06, 2019 at 02:26:11AM +1100, Bruce Evans wrote: > On Fri, 5 Apr 2019, Konstantin Belousov wrote: > > > On Sat, Apr 06, 2019 at 01:01:19AM +1100, Bruce Evans wrote: > >> On Fri, 5 Apr 2019, Konstantin Belousov wrote: > >> > >>> On Fri, Apr 05, 2019 at 11:52:27PM +1100, Bruce Evans wrote: > >>>> On Fri, 5 Apr 2019, Konstantin Belousov wrote: > >>>> > >>>>> On Thu, Apr 04, 2019 at 02:47:34AM +1100, Bruce Evans wrote: > >>>>>> I noticed (or better realized) a general problem with multiple > >>>>>> timehands. ntpd can slew the clock at up to 500 ppm, and at least an > >>>>>> old version of it uses a rate of 50 ppm to fix up fairly small drifts > >>>>>> in the milliseconds range. 500 ppm is enormous in CPU cycles -- it is > >>>>>> 500 thousand nsec or 2 million cycles at 4GHz. Winding up the timecounter > >>>>>> every 1 msec reduces this to only 2000 cycles. > >>>>>> ... > >>>>>> The main point of having multiple timehands (after introducing the per- > >>>>>> timehands generation count) is to avoid blocking thread N during the > >>>>>> update, but this doesn't actually work, even for only 2 timehands and > >>>>>> a global generation count. > >>>>> > >>>>> You are describing the generic race between reader and writer. The same > >>>>> race would exist even with one timehand (and/or one global generation > >>>>> counter), where ntp adjustment might come earlier or later of some > >>>>> consumer accessing the timehands. If timehand instance was read before > >>>>> tc_windup() run but code consumed the result after the windup, it might > >>>>> appear as if time went backward, and this cannot be fixed without either > >>>>> re-reading the time after time-depended calculations were done and > >>>>> restarting, or some globabl lock ensuring serialization. > >>>> > >>>> With 1 timehand, its generation count would be global. I think its ordering > >>>> is strong enough to ensure serialization. > >>> Yes, single timehands result in global generation. But it would not solve > >>> the same race appearing in slightly different manner, as I described above. > >>> If reader finished while generation number in th was not yet reset, but > >>> caller uses the result after tc_windup(), the effect is same as if we > >>> have two th's and reader used the outdated one. > >> > >> You described it too concisely for me to understand :-). > >> > >> I now see that a single generation count doesn't give serialization. I > >> thought that setting the generation to 0 at the start of tc_windup() > >> serialized the reader and writer. But all it does is prevent use of the > >> results of the windup while only some of them are visible. If the > >> setting the generation count to 0 doesn't become before tc_windup() reads > >> the hardware timecounter, then this read may be before other reads using > >> the old timehands, but it needs to be after. > > If we have either single th or global gen counter, current code would > > become serialized, but this is not what I am about. Lets assume, for > > No, generation counts used like they are now (or in any way that I can > see) can't give serialization. > > > the sake of the discussion only, that all readers take the same spinlock > > as tc_windup (i.e. tc_setclock_mtx). > > Spinlocks are far too heavyweight. Most of the complications in timecounter > locking are to avoid using them. But OK for the discussion. > > > It is still possible that reader unlocked the mutex, tc_windup() kicked > > in, locked the mutex and moved timehands (as you noted, this might > > even happen on the same CPU), and only then the reader continues. For > > consumer of bintime() or any other function' result, it looks exactly > > the same as if we did not serialized with writer but used outdated > > timehands. > > Not with full serialization. The writer tc_windup() is also a reader, and > serializing its read gives the necessary monotonicity (for a single thread): > - normal reader locks the mutex, reads the timecounter and unlocks. The > mutex makes visible all previous writes, so the reader doesn't use a > stale timehands. Consumers of bintime(), etc., use this time in the past. > - tc_windup() locks the mutex, reads the timecounter hardware and writes the > timecounter soft state. The new offset is after all previous times read, > since this is serialized. > - normal reader as above sees the new state, so it reads times after the > time of the windup, so also after the time of previous normal reads. > > > Let me formulate this diffeently: as far as consumer of the bintime() > > result does not serialize itself with tc_windup(), serializing bintime() > > itself against tc_windup() does not close the race, but it is not > > obvious that the race matters. > > Readers can easily see times far in the past, but the times increase in > program order. This is true for the current implementation (single-thread monotonic). > > > Either we should just accept the race as > > we currently do, or readers must take the spinlock where the exact value > > of the current time is important, > > Disabling interrupts should be enough. In my version of 5.2, spinlocks > don't disable hardware interrupts and may be preempted by fast interrupt > handlers which may be not so fast and take hundreds of usec. Actually, > even disabling interrupts might not be enough. A single isa bus read > can take at least 138 usec (when it is behind a DMA queue or something). > There are also NMI's and SMI's. > > > or readers must re-read the time after > > doing something important, and redo if the new measuremedtime is outside > > the acceptable range. > > This method seems to be essential for robustness. > > But I don't see any race (for a single thread and no timecounter skew > across CPUs). Sloppy readers just see times an unknown but usually small > time in the past. Non-sloppy readers can also defend against timecounter > skew by binding to 1 CPU. Yes, this is true. > > Mutex locking of the timecounter doesn't give monotonic times across threads. > It gives some order, but you don't know which. Another mutex or rendezvous > is needed to control the order. Define monotonic between threads ? From owner-freebsd-hackers@freebsd.org Sat Apr 6 16:21:04 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3B21E156372B; Sat, 6 Apr 2019 16:21:04 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail104.syd.optusnet.com.au (mail104.syd.optusnet.com.au [211.29.132.246]) by mx1.freebsd.org (Postfix) with ESMTP id 3BAE68741F; Sat, 6 Apr 2019 16:21:02 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from [192.168.0.102] (c110-21-101-228.carlnfd1.nsw.optusnet.com.au [110.21.101.228]) by mail104.syd.optusnet.com.au (Postfix) with ESMTPS id 99BAE4383CA; Sun, 7 Apr 2019 02:20:51 +1000 (AEST) Date: Sun, 7 Apr 2019 02:20:49 +1000 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Konstantin Belousov cc: Bruce Evans , Michael Tuexen , freebsd-hackers Hackers , FreeBSD PowerPC ML Subject: Re: powerpc64 head -r344018 stuck sleeping problems: th->th_scale * tc_delta(th) overflows unsigned 64 bits sometimes [patched failed] In-Reply-To: <20190406121650.GJ1923@kib.kiev.ua> Message-ID: <20190407015735.W2994@besplex.bde.org> References: <20190324110138.GR1923@kib.kiev.ua> <20190403070045.GW1923@kib.kiev.ua> <20190404011802.E2390@besplex.bde.org> <20190405113912.GB1923@kib.kiev.ua> <20190405230717.D3383@besplex.bde.org> <20190405132128.GD1923@kib.kiev.ua> <20190406003907.C3872@besplex.bde.org> <20190405143619.GF1923@kib.kiev.ua> <20190406014724.X4174@besplex.bde.org> <20190406121650.GJ1923@kib.kiev.ua> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.2 cv=P6RKvmIu c=1 sm=1 tr=0 cx=a_idp_d a=PalzARQSbocsUSjMRkwAPg==:117 a=PalzARQSbocsUSjMRkwAPg==:17 a=jpOVt7BSZ2e4Z31A5e1TngXxSK0=:19 a=kj9zAlcOel0A:10 a=lIKfSjjBpJ_XLBo-IhQA:9 a=PV0E4CjrIwHpNLuQ:21 a=bqPXIC2uY7JGT8R7:21 a=CjuIK1q_8ugA:10 X-Rspamd-Queue-Id: 3BAE68741F X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.98 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.98)[-0.980,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[] X-Mailman-Approved-At: Sat, 06 Apr 2019 16:28:39 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Apr 2019 16:21:04 -0000 On Sat, 6 Apr 2019, Konstantin Belousov wrote: > On Sat, Apr 06, 2019 at 02:26:11AM +1100, Bruce Evans wrote: >> On Fri, 5 Apr 2019, Konstantin Belousov wrote: >> >>> On Sat, Apr 06, 2019 at 01:01:19AM +1100, Bruce Evans wrote: >>>> On Fri, 5 Apr 2019, Konstantin Belousov wrote: >>>> >>>>> On Fri, Apr 05, 2019 at 11:52:27PM +1100, Bruce Evans wrote: >>>>>> On Fri, 5 Apr 2019, Konstantin Belousov wrote: >>>>>> >>>>>>> On Thu, Apr 04, 2019 at 02:47:34AM +1100, Bruce Evans wrote: >>>>>>>> I noticed (or better realized) a general problem with multiple >>>>>>>> timehands. ntpd can slew the clock at up to 500 ppm, and at least an >>>>>>>> old version of it uses a rate of 50 ppm to fix up fairly small drifts >>>>>>>> in the milliseconds range. 500 ppm is enormous in CPU cycles -- it is >>>>>>>> 500 thousand nsec or 2 million cycles at 4GHz. Winding up the timecounter >>>>>>>> every 1 msec reduces this to only 2000 cycles. >>>>>>>> ... >>>>>>>> The main point of having multiple timehands (after introducing the per- >>>>>>>> timehands generation count) is to avoid blocking thread N during the >>>>>>>> update, but this doesn't actually work, even for only 2 timehands and >>>>>>>> a global generation count. >>>>>>> >>>>>>> You are describing the generic race between reader and writer. The same >>>>>>> race would exist even with one timehand (and/or one global generation >>>>>>> counter), where ntp adjustment might come earlier or later of some >>>>>>> consumer accessing the timehands. If timehand instance was read before >>>>>>> tc_windup() run but code consumed the result after the windup, it might >>>>>>> appear as if time went backward, and this cannot be fixed without either >>>>>>> re-reading the time after time-depended calculations were done and >>>>>>> restarting, or some globabl lock ensuring serialization. >>>>>> >>>>>> With 1 timehand, its generation count would be global. I think its ordering >>>>>> is strong enough to ensure serialization. >>>>> Yes, single timehands result in global generation. But it would not solve >>>>> the same race appearing in slightly different manner, as I described above. >>>>> If reader finished while generation number in th was not yet reset, but >>>>> caller uses the result after tc_windup(), the effect is same as if we >>>>> have two th's and reader used the outdated one. >>>> >>>> You described it too concisely for me to understand :-). >>>> >>>> I now see that a single generation count doesn't give serialization. I >>>> thought that setting the generation to 0 at the start of tc_windup() >>>> serialized the reader and writer. But all it does is prevent use of the >>>> results of the windup while only some of them are visible. If the >>>> setting the generation count to 0 doesn't become before tc_windup() reads >>>> the hardware timecounter, then this read may be before other reads using >>>> the old timehands, but it needs to be after. >>> If we have either single th or global gen counter, current code would >>> become serialized, but this is not what I am about. Lets assume, for >> >> No, generation counts used like they are now (or in any way that I can >> see) can't give serialization. >> >>> the sake of the discussion only, that all readers take the same spinlock >>> as tc_windup (i.e. tc_setclock_mtx). >> >> Spinlocks are far too heavyweight. Most of the complications in timecounter >> locking are to avoid using them. But OK for the discussion. >> >>> It is still possible that reader unlocked the mutex, tc_windup() kicked >>> in, locked the mutex and moved timehands (as you noted, this might >>> even happen on the same CPU), and only then the reader continues. For >>> consumer of bintime() or any other function' result, it looks exactly >>> the same as if we did not serialized with writer but used outdated >>> timehands. >> >> Not with full serialization. The writer tc_windup() is also a reader, and >> serializing its read gives the necessary monotonicity (for a single thread): >> - normal reader locks the mutex, reads the timecounter and unlocks. The >> mutex makes visible all previous writes, so the reader doesn't use a >> stale timehands. Consumers of bintime(), etc., use this time in the past. >> - tc_windup() locks the mutex, reads the timecounter hardware and writes the >> timecounter soft state. The new offset is after all previous times read, >> since this is serialized. >> - normal reader as above sees the new state, so it reads times after the >> time of the windup, so also after the time of previous normal reads. >> >>> Let me formulate this diffeently: as far as consumer of the bintime() >>> result does not serialize itself with tc_windup(), serializing bintime() >>> itself against tc_windup() does not close the race, but it is not >>> obvious that the race matters. >> >> Readers can easily see times far in the past, but the times increase in >> program order. > This is true for the current implementation (single-thread monotonic). Provided there is no skew across CPUs. I think this is a requirement for any implementation. >> >>> Either we should just accept the race as >>> we currently do, or readers must take the spinlock where the exact value >>> of the current time is important, >> >> Disabling interrupts should be enough. In my version of 5.2, spinlocks >> don't disable hardware interrupts and may be preempted by fast interrupt >> handlers which may be not so fast and take hundreds of usec. Actually, >> even disabling interrupts might not be enough. A single isa bus read >> can take at least 138 usec (when it is behind a DMA queue or something). >> There are also NMI's and SMI's. >> >>> or readers must re-read the time after >>> doing something important, and redo if the new measuremedtime is outside >>> the acceptable range. >> >> This method seems to be essential for robustness. >> >> But I don't see any race (for a single thread and no timecounter skew >> across CPUs). Sloppy readers just see times an unknown but usually small >> time in the past. Non-sloppy readers can also defend against timecounter >> skew by binding to 1 CPU. > Yes, this is true. > >> >> Mutex locking of the timecounter doesn't give monotonic times across threads. >> It gives some order, but you don't know which. Another mutex or rendezvous >> is needed to control the order. > Define monotonic between threads ? The threads establish an order of events using _any_ reasonable method, e.g., memory ordering. Then if the events are a monotonic clock-reading event E0, any event E1, and a monotonic clock-reading event E2, in the order E0 <= E1 <= E2 according to the reasonable method, then E0 must be <= E2 according to the timespec value of the clock. One thread can arrange for E0 <= E1 very easily using only program order. E1 might consist of setting a flag. Another thread watching this flag can't tell when it was changed, but can tell that it was changed after E0, so E2 in the other thread must be after E0, by program order E1 <= E2 in the other thread. More precisely,there are really 2 different events E1. One (E1a) is setting the flag in the first thread and the other (E1b) is observing the change in the other thread. The full order is E0 <= E1a <= E1b <= E2, where E0 <= E1a and E1b <= E2 by program order in each thread and E1a <= E2b by the arrow of time (this doesn't take any memory ordering magic, except in practice you would have to do more to not miss changes when reusing the flag). Establishing an order using external events doesn't work so well, since the events become visible in different threads at different times. If you have a clock good enough for measuring and compensating for the different times, then you should be using it for the timecounter and can't use it for implementing the timecounter :-). Ticks on a system-wide timer is an example of an external event. Another is arrival of packets at an interface. If the packets are read from the interface by separate threads and put into a buffer using any fast method, then they are likely to become out of order. Hardware timestamps on them would allow fixing up the order, but software timestamps made by separate readers don't work at all without using a necessarily slow method to communicate the order. I think the multiple timehands method works perfectly if the frequency or phase is never adjusted. It is just an optimization of the calculation 'result = base + timecounter_value * inverse_frequency', except for negligible rounding errors. We want to do the multiplication using only 64-bit integers, so we replace the RHS by '(base + offset * inverse_freqency) + (timecountere_value - base) * inverse_frequency'. When inverse_frequency is constant, It doesn't matter when the (base, offset) pair is changed provided the change is atomic and not too far in the past for overflow to occur with 64-bit integers. The method gives atomicity, and a bug apparently gives overflow. Races from decreases in the frequency are harmless (for monotonicity) since they increase the inverse frequency so increase time differences so never give negative differences. So only races from increases in the frequency are a problem. I think changes in the frequency are rare (ntpd normally only makes them every 64 seconds) so it wouldn't hurt to wait for them. The problem is know when to wait. Bruce From owner-freebsd-hackers@freebsd.org Sat Apr 6 21:10:49 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D686A156D405 for ; Sat, 6 Apr 2019 21:10:48 +0000 (UTC) (envelope-from mahesh86701997@gmail.com) Received: from mail-lj1-x22d.google.com (mail-lj1-x22d.google.com [IPv6:2a00:1450:4864:20::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4F24D6C6C7; Sat, 6 Apr 2019 21:10:47 +0000 (UTC) (envelope-from mahesh86701997@gmail.com) Received: by mail-lj1-x22d.google.com with SMTP id y6so8012043ljd.12; Sat, 06 Apr 2019 14:10:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=bIHZuRbJ37GdGRKoS8bm5YWEivjL5lZnfea7Fna6r7A=; b=JbeDFUMveqXMKmms52RA+eHZWEj/LpFKLuruWs8AckOIeQvCm3ypKeM428RU+R12Dl RK29Yv+eo5a3ec4qMzH8NJo3aYMXmuUM2SPT8C+1t42Ly6Fk3n8H805G3cWW8EdTaUTT ph3eZnvYgY90u7Mv+YEGdnJDnXoYu3upBoeJn+4QKulNo3JMXM55byje6cszhfR8RLit /8oPWIJpZwzw8S1VPGRtoAHHfT3b13On4OVFA8tnyzr1+wewo4uAfPaCO/mE8azYFphh cDjhoH/uYbE0TOpGVtcAvb+LLgL/qcXmEq3GEFheYh6Gh+oy/AwqKzR9iza0B70+vEGM AbFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=bIHZuRbJ37GdGRKoS8bm5YWEivjL5lZnfea7Fna6r7A=; b=tdIzwHxEVEEUTfHrX8Qeual0L7SWi6uY9VwrKKKQD6WndOP2QDLYIHKTuuWDjXCoA7 UeealyQyrDGf1Rutc8ViiyevnXSPz3mkyxUU+CWoi4yk4pt9kKAKGxW3QY53dw9XxUxP mNV0J8FK5nBxbF12BGQmxS4NCvgC3j0V789HBgUrlugUbwwmOKiQKRFUMFPZYDcZzmVE jocLmNE3/vzfTdfngvKUxkfJZPl8DAV6cE9J9vPPG3C3Sxbo670oJ1ZpgUQ9aLhBa4sA W5WDuB4Lg6c8gtb3Afv8I9c3S+ZF72v1deGFuXHJEZLfMW0a0Zid3UOCgGq1N0lcvYii syLA== X-Gm-Message-State: APjAAAWRDNehnoTgi+BBOAik8tejGo58vSaVUc/A9z6xW0MBGruf2qt9 vxGpeMDTZRePNoj0SwepzI8KGDBveoyDN9ZVKcyyrOBx X-Google-Smtp-Source: APXvYqwUoczw36JXDiEo+NurBCEUOGXZQw+kfSy6IkVI+MgtSduCoU1ocDinZpEex2IEpzwXGpOjj4ZRWEbIec8ntDY= X-Received: by 2002:a2e:2b16:: with SMTP id q22mr11780242lje.20.1554585045738; Sat, 06 Apr 2019 14:10:45 -0700 (PDT) MIME-Version: 1.0 References: <1C4595D2-B150-460F-9161-89A1A8865830@freebsdfoundation.org> In-Reply-To: From: Mahesh Tudu Date: Sun, 7 Apr 2019 02:40:33 +0530 Message-ID: Subject: Re: Regarding mentorship for GSOC 2019 To: Anne Dickison , soc-admins@freebsd.org, freebsd-hackers@freebsd.org X-Rspamd-Queue-Id: 4F24D6C6C7 X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=JbeDFUMv; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of mahesh86701997@gmail.com designates 2a00:1450:4864:20::22d as permitted sender) smtp.mailfrom=mahesh86701997@gmail.com X-Spamd-Result: default: False [-5.28 / 15.00]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; ZERO_FONT(0.20)[3]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; NEURAL_HAM_SHORT(-0.98)[-0.985,0]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; HTML_SHORT_LINK_IMG_2(1.00)[]; MANY_INVISIBLE_PARTS(0.20)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[d.2.2.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0]; IP_SCORE(-2.69)[ip: (-8.82), ipnet: 2a00:1450::/32(-2.37), asn: 15169(-2.17), country: US(-0.06)]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Apr 2019 21:10:49 -0000 Hello, I have sent a message earlier regarding mentorship for the project *Verification of bhyve's instruction emulation. *Since there are only 3 days left for the final submission, I am really in great need of a mentor to review my proposal before final submission. I know that you are surely working hard on solving my query, but still it would be very kind of you to let me know about the progress. Thanking You [image: Mailtrack] Sender notified by Mailtrack 04/07/19, 2:34:44 AM On Thu, Apr 4, 2019 at 10:07 PM Mahesh Tudu wrote: > Thank you for your response. > > > > [image: Mailtrack] > Sender > notified by > Mailtrack > 04/04/19, > 10:07:28 PM > > On Thu, Apr 4, 2019 at 9:32 PM Anne Dickison > wrote: > >> Hi Mahesh, >> We=E2=80=99re working on the matter and will be in touch as soon as we h= ave more >> information. Thank you for your patience. >> >> Anne >> On behalf of FreeBSD GSoC admins >> >> On Apr 4, 2019, at 10:45 AM, Mahesh Tudu >> wrote: >> >> Hello, >> I have already sent an email regarding mentorship for the project *Verif= ication >> of bhyve's instruction emulation. *But I have still got no response from >> your side. I have done lot of research work on this project and I am sur= e >> that I will complete it. >> I would be very pleased if you look into my matter and send me a respons= e >> soon. >> Thank you. >> >> >> >> [image: Mailtrack] >> Sender >> notified by >> Mailtrack >> 04/04/19, >> 9:12:16 PM >> >> From owner-freebsd-hackers@freebsd.org Sat Apr 6 21:39:16 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0975C156E0B0 for ; Sat, 6 Apr 2019 21:39:16 +0000 (UTC) (envelope-from mahesh86701997@gmail.com) Received: from mail-lj1-x235.google.com (mail-lj1-x235.google.com [IPv6:2a00:1450:4864:20::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C8CB26D837; Sat, 6 Apr 2019 21:39:14 +0000 (UTC) (envelope-from mahesh86701997@gmail.com) Received: by mail-lj1-x235.google.com with SMTP id v13so8051633ljk.4; Sat, 06 Apr 2019 14:39:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=qCtZcoafSj67ahChhSTOLJaRL7fRhdiMkc/kaE2nMT0=; b=VS2G3bpABHxfrkyEU+slPIv7B65wUO5gczO+yGE4Nw0O9plceLFp4WTfmki+lA0/Bx Eq3CbYrR4Ffb4sQCezq8A8Dvqv5vnAD92tDjAARwa5gXaI95ye6pt1m6WB9lxYbegEOQ toRPENGZarXAKRaG6mcBNInTpJo7P1u5LisP/EeOdOZJJqJ2EDOLgnO5wmE/aUiD4UL0 ncoxwtUcpHVnAbdJGCi8mhj8kkKjyvpBGlbNtmd7hLSI//zEoMecUZRk3rsk5YRY7oje wnNhGDIB4GlXui5DgCX+U2iT1IScbSaNOkSWmYeUvpkFRENN3Q2RqknJa0iJOvwgCOpj PfdA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=qCtZcoafSj67ahChhSTOLJaRL7fRhdiMkc/kaE2nMT0=; b=kMmnTvrRbbs8WvTNn5AyJiQom2zJDm/kKxD6MAKqAvg0NVCOWhHHlxv8jQQJevgb/i DipdUJkUv7Y7ZVVQu4j38zFmpGyb9mVXfUHZoB6Pkyge2tvKqPfTN+KLTnhnJ5qe21Mg 9F4MW/zFIGTyS2V6l1yuYcKEU6m6yM1LGBgpSgvhnXLWeA9UnyNtk4fJPo3d4HxeTK84 rQCohyFNNrqeJs3UnEgHlnHbXBrTiXoeI+9e8DUIXc0OQRfSaocPzoOXRANQXk6bpbTV Sp9mw1wlSflsfU0Jx3o0xrn64SwqAhp8MHji2wt0wUBoBbKWFtMsyAWPkYa5sWQ89tnG kJ9A== X-Gm-Message-State: APjAAAW1Mq5SR9wbt2j7pyCgLXr36PQy1Roc5ytbjbYUNzDJ3KWkq2Nk z1WPAyOdgGoEfiLIWh0S+zV5z7zDxfGjfc213oNKZ1Y1 X-Google-Smtp-Source: APXvYqwr0KQYBjkJ155h5oKCtcTMTY8qTf1f4v23Z7S7Upv3lIo/Xw71XEPb5QVY/a3ZUx6wVhncC+DBhpUFg0wMonI= X-Received: by 2002:a2e:8156:: with SMTP id t22mr10794314ljg.77.1554586752305; Sat, 06 Apr 2019 14:39:12 -0700 (PDT) MIME-Version: 1.0 References: <1C4595D2-B150-460F-9161-89A1A8865830@freebsdfoundation.org> In-Reply-To: From: Mahesh Tudu Date: Sun, 7 Apr 2019 03:09:01 +0530 Message-ID: Subject: Re: Regarding mentorship for GSOC 2019 To: Anne Dickison , soc-admins@freebsd.org, freebsd-hackers@freebsd.org X-Rspamd-Queue-Id: C8CB26D837 X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=VS2G3bpA; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of mahesh86701997@gmail.com designates 2a00:1450:4864:20::235 as permitted sender) smtp.mailfrom=mahesh86701997@gmail.com X-Spamd-Result: default: False [-5.26 / 15.00]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; ZERO_FONT(0.30)[4]; FREEMAIL_FROM(0.00)[gmail.com]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; NEURAL_HAM_SHORT(-0.98)[-0.983,0]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; HTML_SHORT_LINK_IMG_2(1.00)[]; MANY_INVISIBLE_PARTS(0.30)[4]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[5.3.2.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0]; IP_SCORE(-2.86)[ip: (-9.71), ipnet: 2a00:1450::/32(-2.37), asn: 15169(-2.17), country: US(-0.06)]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Apr 2019 21:39:16 -0000 Hello, Kindly respond. [image: Mailtrack] Sender notified by Mailtrack 04/07/19, 3:08:44 AM On Sun, Apr 7, 2019 at 2:40 AM Mahesh Tudu wrote= : > Hello, > I have sent a message earlier regarding mentorship for the project *Verif= ication > of bhyve's instruction emulation. *Since there are only 3 days left for > the final submission, I am really in great need of a mentor to review my > proposal before final submission. > I know that you are surely working hard on solving my query, but still it > would be very kind of you to let me know about the progress. > Thanking You > [image: Mailtrack] > Sender > notified by > Mailtrack > 04/07/19, > 2:34:44 AM > > On Thu, Apr 4, 2019 at 10:07 PM Mahesh Tudu > wrote: > >> Thank you for your response. >> >> >> >> [image: Mailtrack] >> Sender >> notified by >> Mailtrack >> 04/04/19, >> 10:07:28 PM >> >> On Thu, Apr 4, 2019 at 9:32 PM Anne Dickison >> wrote: >> >>> Hi Mahesh, >>> We=E2=80=99re working on the matter and will be in touch as soon as we = have more >>> information. Thank you for your patience. >>> >>> Anne >>> On behalf of FreeBSD GSoC admins >>> >>> On Apr 4, 2019, at 10:45 AM, Mahesh Tudu >>> wrote: >>> >>> Hello, >>> I have already sent an email regarding mentorship for the project *Veri= fication >>> of bhyve's instruction emulation. *But I have still got no response >>> from your side. I have done lot of research work on this project and I = am >>> sure that I will complete it. >>> I would be very pleased if you look into my matter and send me a >>> response soon. >>> Thank you. >>> >>> >>> >>> [image: Mailtrack] >>> Sender >>> notified by >>> Mailtrack >>> 04/04/19, >>> 9:12:16 PM >>> >>>