From owner-freebsd-arm@freebsd.org Sun Oct 18 16:30:09 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9F520436A2D for ; Sun, 18 Oct 2020 16:30:09 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: from mail-io1-f68.google.com (mail-io1-f68.google.com [209.85.166.68]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CDljX5NRBz4h7G for ; Sun, 18 Oct 2020 16:30:08 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: by mail-io1-f68.google.com with SMTP id k6so10207968ior.2 for ; Sun, 18 Oct 2020 09:30:08 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=Klnmf2LCqTEeM1QvD92yb+POuOz0hYISM2a/IesFE/k=; b=TodxvIsq7Zd4V57Vt+b8UHrSDY+O8ljyqir4hekdc6CWyvVFAuBS9DlTS/jx5CmLjf 6Oo3WJlqp5HzRvLFhHtX4QBrtV5ZYBwmqxo+EY//xsCWWC9NQkN8wyXJq0ctTCPK44tx JY1qJyQC56+5Sx7yRf2tdctFoe3x+vtU7Cj29aOYbMtkxSo/DO8xCpOh2IQYpW6ORp6e giNd+/PDUsBqQj/T/J7tLjnpPeH3ujlxeDYRbNf2C4KTBdklq8PFrM0Dw0vH9yLzMa6T e0mtsO76sAdJXsSITSQMdx1O9enJaIjEzY9+T6DP30iGJ52AygighWdmF+Igk7UKgmt5 bPog== X-Gm-Message-State: AOAM532g+U9FWF8BpA8gnLrS7WF11r0Q5M0VnnSTGpmzhsXaEU82hQSA cLqCybqdT9D0OhtYZETTVvF3dFJUiu8wUkHA294= X-Google-Smtp-Source: ABdhPJzNyG8sNnleMBQ6RW9iUzFNTfQ7VGVUzur1ZiCzOR8XWRGFgqVQclyXjbdHLyHx7PI/FvQaZUuOySOz12GifkM= X-Received: by 2002:a6b:d80d:: with SMTP id y13mr8414575iob.15.1603038607712; Sun, 18 Oct 2020 09:30:07 -0700 (PDT) MIME-Version: 1.0 References: <472221209.496244.1602086031265@mail.yahoo.com> In-Reply-To: <472221209.496244.1602086031265@mail.yahoo.com> From: Ed Maste Date: Sun, 18 Oct 2020 12:29:55 -0400 Message-ID: Subject: Re: BBB boot failure between r366365 and r366386 To: Oskar Holmlund Cc: freebsd-arm Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4CDljX5NRBz4h7G X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of carpeddiem@gmail.com designates 209.85.166.68 as permitted sender) smtp.mailfrom=carpeddiem@gmail.com X-Spamd-Result: default: False [-2.76 / 15.00]; ARC_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; FREEFALL_USER(0.00)[carpeddiem]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17:c]; RCVD_TLS_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-arm@freebsd.org]; DMARC_NA(0.00)[freebsd.org]; NEURAL_HAM_LONG(-0.94)[-0.943]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; NEURAL_HAM_SHORT(-0.94)[-0.936]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[209.85.166.68:from]; NEURAL_HAM_MEDIUM(-0.88)[-0.877]; FREEMAIL_TO(0.00)[yahoo.com]; FORGED_SENDER(0.30)[emaste@freebsd.org,carpeddiem@gmail.com]; RWL_MAILSPIKE_POSSIBLE(0.00)[209.85.166.68:from]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[emaste@freebsd.org,carpeddiem@gmail.com]; MAILMAN_DEST(0.00)[freebsd-arm]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Oct 2020 16:30:09 -0000 On Wed, 7 Oct 2020 at 11:54, Oskar Holmlund wrote: > > I can not reproduce the problem - I have tested r366366 - r366386 and as of today r366515 and all works fine. U-boot is same as yours 2020.07. But i have not tested to netboot the BBB, i have only tested from SD card and from eMMC. > I can try the kernel and base from the CI builds later. It turns out the issue was dependent on the order of allocations etc., so it may be that some seemingly unrelated difference (such as whether or not netbooting) would result in success or failure. Mark has a set of reviews open for the fix, and I hope to see CI green again next week. From owner-freebsd-arm@freebsd.org Mon Oct 19 06:19:32 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 90B0A44633F for ; Mon, 19 Oct 2020 06:19:32 +0000 (UTC) (envelope-from freebsd-arm@darkain.com) Received: from MTA-07-4.privateemail.com (mta-07-4.privateemail.com [68.65.122.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4CF66W6hpQz48Zq for ; Mon, 19 Oct 2020 06:19:31 +0000 (UTC) (envelope-from freebsd-arm@darkain.com) Received: from MTA-07.privateemail.com (localhost [127.0.0.1]) by MTA-07.privateemail.com (Postfix) with ESMTP id 102AA60045 for ; Mon, 19 Oct 2020 02:19:29 -0400 (EDT) Received: from mail-io1-f42.google.com (unknown [10.20.151.249]) by MTA-07.privateemail.com (Postfix) with ESMTPA id C7FBF60047 for ; Mon, 19 Oct 2020 06:19:28 +0000 (UTC) Received: by mail-io1-f42.google.com with SMTP id q9so11661067iow.6 for ; Sun, 18 Oct 2020 23:19:28 -0700 (PDT) X-Gm-Message-State: AOAM533gohMt2B9Zf9Ch/iPhOmIYZwJR47gD1wc8vIjQPgQFR0CEgg1m 3/Dfj5rhtLU6v4eGrgtny3+4+HRBtVUhgZB9tV0= X-Google-Smtp-Source: ABdhPJyLN854e4W/DBQgPcw4bBhvXCpHd0UgdZaXNl59/rVCdjYyHG8fwoVuKDmFDixtitwF8/cHIUqNef6kVvrYXfY= X-Received: by 2002:a5d:815a:: with SMTP id f26mr10045347ioo.111.1603088366807; Sun, 18 Oct 2020 23:19:26 -0700 (PDT) MIME-Version: 1.0 From: Vincent Milum Jr Date: Sun, 18 Oct 2020 23:19:15 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: FreeBSD under VMware ESXi ARM Fling To: freebsd-arm@freebsd.org X-Virus-Scanned: ClamAV using ClamSMTP X-Rspamd-Queue-Id: 4CF66W6hpQz48Zq X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of freebsd-arm@darkain.com designates 68.65.122.27 as permitted sender) smtp.mailfrom=freebsd-arm@darkain.com X-Spamd-Result: default: False [-2.95 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.06)[-1.063]; FROM_HAS_DN(0.00)[]; RWL_MAILSPIKE_VERYGOOD(0.00)[68.65.122.27:from]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:68.65.122.0/27]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; TO_DN_NONE(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-arm@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-0.99)[-0.989]; RCVD_COUNT_THREE(0.00)[4]; DMARC_NA(0.00)[darkain.com]; NEURAL_HAM_SHORT(-0.60)[-0.600]; RCVD_IN_DNSWL_NONE(0.00)[68.65.122.27:from]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:22612, ipnet:68.65.122.0/24, country:US]; RCVD_TLS_LAST(0.00)[]; MAILMAN_DEST(0.00)[freebsd-arm] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Oct 2020 06:19:32 -0000 Recently, VMware released what they're calling "ESXi ARM Fling" - an ARM64/Aarch64 version of VMware ESXi server. Details of the release along with the download can be found here: https://flings.vmware.com/esxi-arm-edition They support a number of ARM platforms already, most notably is the Raspberry Pi 4 (only 4GB and 8GB models). They have instructions on their site on how to install and configure the UEFI firmware on the Pi for getting the ESXi installer and then ESXi itself up and running. UEFI builds of FreeBSD ARM64/Aarch64 mostly work "out of the box" under ESXi ARM, whereas ARM 32-bit kernels are not supported by any guest OS under ESXi ARM. There are, however, some areas that could use some improvement, or need some massive work to get working. The largest issue currently is that SMP will cause the FreeBSD kernel to panic during boot. A VM with only one single vCPU however will boot and run just fine. Prior to the 13.0-CURRENT snapshot 2020-10-15, the CD-Rom driver was not included with the ARM64 builds, so the ISO would fail to mount itself on boot. This means the easiest way to try earlier versions, such as 12.1-RELEASE, you'll need to either convert the pre-built vmdk file from Workstation to ESXi format, or convert the raw disk image to ESXi vmdk format. This is how I initially got FreeBSD up and running prior to the 2020-10-15 snapshot. Every version of FreeBSD I've tested on ARM is lacking several drivers by default. uhci for USB 2.0 is not enabled by default. Ensure your VM is initially set up for USB 3.1 so the virtual keyboard and mouse will function. This is easy enough to fix post install with: uhci_load="YES" ums is the same as above. ESXi ARM uses USB instead of PS2 for the virtual mouse, so add this post install if you want mouse support: ums_load="YES" vmxnet and pvscsi are not pre-compiled. However, they compile and operate perfectly as expected without any modification on ARM. vmci has some x86 assembly that I've not dug into yet, so that is currently not working. I've documented in detail, and continue to update as I test more drivers over on my blog: https://vincerants.com/freebsd-on-esxi-arm-fling-fixing-virtual-hardware/ The ESXi UI has a known bug in that it'll list some virtual hardware that doesn't actually exist inside of the VM. These are all legacy devices, and should be easy enough to figure out. There are no virtual IDE/ATA or PS2 ports. Keyboard and mouse now use the virtual USB controller instead. open-vm-tools has required a bit of work to make functional on FreeBSD under ARM. There is currently a patch open with VMware with my changes to the codebase to make it functional. To make things easy, I've uploaded some pre-compiled binaries to my blog, as well as included steps to compile this customized package yourself. The blog will also be updated as development progresses upstream with VMware. https://vincerants.com/open-vm-tools-on-freebsd-under-vmware-esxi-arm-fling/ Xorg desktop runs, and is mostly functional. I've yet to get the xf86-input-vmmouse or xf86-video-vmware working. Xorg can use sysmouse and the console framebuffer for rendering, and this is working okay for basic testing right now. I also have an early dmesg, but this was from before getting most drivers working. https://dmesgd.nycbug.org/index.cgi?do=view&id=5693 I welcome any additional feedback on getting FreeBSD to 100% under ESXi ARM! - Vince (darkain) From owner-freebsd-arm@freebsd.org Mon Oct 19 18:49:15 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 17E41431730 for ; Mon, 19 Oct 2020 18:49:15 +0000 (UTC) (envelope-from crowston@protonmail.com) Received: from mail-40134.protonmail.ch (mail-40134.protonmail.ch [185.70.40.134]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "protonmail.com", Issuer "SwissSign Server Gold CA 2014 - G22" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CFQlZ12wTz41dr for ; Mon, 19 Oct 2020 18:49:13 +0000 (UTC) (envelope-from crowston@protonmail.com) Date: Mon, 19 Oct 2020 18:49:04 +0000 To: Vincent Milum Jr , freebsd-arm@freebsd.org From: Robert Crowston Reply-To: Robert Crowston Subject: Re: FreeBSD under VMware ESXi ARM Fling Message-ID: In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-1.2 required=10.0 tests=ALL_TRUSTED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM,HTML_MESSAGE shortcircuit=no autolearn=disabled version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on mailout.protonmail.ch X-Rspamd-Queue-Id: 4CFQlZ12wTz41dr X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.61 / 15.00]; HAS_REPLYTO(0.00)[crowston@protonmail.com]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[protonmail.com]; R_SPF_ALLOW(-0.20)[+ip4:185.70.40.0/24]; DKIM_TRACE(0.00)[protonmail.com:+]; MIME_BASE64_TEXT(0.10)[]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[protonmail.com,quarantine]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[protonmail.com]; ASN(0.00)[asn:62371, ipnet:185.70.40.0/24, country:CH]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.988]; R_DKIM_ALLOW(-0.20)[protonmail.com:s=protonmail]; REPLYTO_EQ_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; NEURAL_SPAM_SHORT(0.27)[0.268]; NEURAL_HAM_LONG(-0.99)[-0.988]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; FREEMAIL_REPLYTO(0.00)[protonmail.com]; HAS_PHPMAILER_SIG(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[185.70.40.134:from]; MAILMAN_DEST(0.00)[freebsd-arm] Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: base64 X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Oct 2020 18:49:15 -0000 PiBUaGUgbGFyZ2VzdCBpc3N1ZSBjdXJyZW50bHkgaXMgdGhhdCBTTVAgd2lsbCBjYXVzZSB0aGUg RnJlZUJTRCBrZXJuZWwgdG9wYW5pYyBkdXJpbmcgYm9vdC4KCkNvdWxkIHlvdSBzZW5kIHRoZSBw YW5pYz8KCknigJltIGd1ZXNzaW5nIGxhY2sgb2Ygc3BpbnRhYmxlIHN1cHBvcnQuCgrigJQgUkhD LgoKT24gTW9uLCBPY3QgMTksIDIwMjAgYXQgMDc6MTksIFZpbmNlbnQgTWlsdW0gSnIgPGZyZWVi c2QtYXJtQGRhcmthaW4uY29tPiB3cm90ZToKCj4gUmVjZW50bHksIFZNd2FyZSByZWxlYXNlZCB3 aGF0IHRoZXkncmUgY2FsbGluZyAiRVNYaSBBUk0gRmxpbmciIC0gYW4KPiBBUk02NC9BYXJjaDY0 IHZlcnNpb24gb2YgVk13YXJlIEVTWGkgc2VydmVyLgo+Cj4gRGV0YWlscyBvZiB0aGUgcmVsZWFz ZSBhbG9uZyB3aXRoIHRoZSBkb3dubG9hZCBjYW4gYmUgZm91bmQgaGVyZToKPiBodHRwczovL2Zs aW5ncy52bXdhcmUuY29tL2VzeGktYXJtLWVkaXRpb24KPgo+IFRoZXkgc3VwcG9ydCBhIG51bWJl ciBvZiBBUk0gcGxhdGZvcm1zIGFscmVhZHksIG1vc3Qgbm90YWJseSBpcyB0aGUKPiBSYXNwYmVy cnkgUGkKPiA0IChvbmx5IDRHQiBhbmQgOEdCIG1vZGVscykuIFRoZXkgaGF2ZSBpbnN0cnVjdGlv bnMgb24gdGhlaXIgc2l0ZSBvbiBob3cgdG8KPiBpbnN0YWxsIGFuZCBjb25maWd1cmUgdGhlIFVF RkkgZmlybXdhcmUgb24gdGhlIFBpIGZvciBnZXR0aW5nIHRoZSBFU1hpCj4gaW5zdGFsbGVyCj4g YW5kIHRoZW4gRVNYaSBpdHNlbGYgdXAgYW5kIHJ1bm5pbmcuCj4KPiBVRUZJIGJ1aWxkcyBvZiBG cmVlQlNEIEFSTTY0L0FhcmNoNjQgbW9zdGx5IHdvcmsgIm91dCBvZiB0aGUgYm94IiB1bmRlciBF U1hpCj4gQVJNLCB3aGVyZWFzIEFSTSAzMi1iaXQga2VybmVscyBhcmUgbm90IHN1cHBvcnRlZCBi eSBhbnkgZ3Vlc3QgT1MgdW5kZXIgRVNYaQo+IEFSTS4KPgo+IFRoZXJlIGFyZSwgaG93ZXZlciwg c29tZSBhcmVhcyB0aGF0IGNvdWxkIHVzZSBzb21lIGltcHJvdmVtZW50LCBvciBuZWVkIHNvbWUK PiBtYXNzaXZlIHdvcmsgdG8gZ2V0IHdvcmtpbmcuCj4KPiBUaGUgbGFyZ2VzdCBpc3N1ZSBjdXJy ZW50bHkgaXMgdGhhdCBTTVAgd2lsbCBjYXVzZSB0aGUgRnJlZUJTRCBrZXJuZWwgdG8KPiBwYW5p Ywo+IGR1cmluZyBib290LiBBIFZNIHdpdGggb25seSBvbmUgc2luZ2xlIHZDUFUgaG93ZXZlciB3 aWxsIGJvb3QgYW5kIHJ1biBqdXN0Cj4gZmluZS4KPgo+IFByaW9yIHRvIHRoZSAxMy4wLUNVUlJF TlQgc25hcHNob3QgMjAyMC0xMC0xNSwgdGhlIENELVJvbSBkcml2ZXIgd2FzIG5vdAo+IGluY2x1 ZGVkIHdpdGggdGhlIEFSTTY0IGJ1aWxkcywgc28gdGhlIElTTyB3b3VsZCBmYWlsIHRvIG1vdW50 IGl0c2VsZiBvbgo+IGJvb3QuIFRoaXMKPiBtZWFucyB0aGUgZWFzaWVzdCB3YXkgdG8gdHJ5IGVh cmxpZXIgdmVyc2lvbnMsIHN1Y2ggYXMgMTIuMS1SRUxFQVNFLCB5b3UnbGwKPiBuZWVkCj4gdG8g ZWl0aGVyIGNvbnZlcnQgdGhlIHByZS1idWlsdCB2bWRrIGZpbGUgZnJvbSBXb3Jrc3RhdGlvbiB0 byBFU1hpIGZvcm1hdCwKPiBvcgo+IGNvbnZlcnQgdGhlIHJhdyBkaXNrIGltYWdlIHRvIEVTWGkg dm1kayBmb3JtYXQuIFRoaXMgaXMgaG93IEkgaW5pdGlhbGx5IGdvdAo+IEZyZWVCU0QgdXAgYW5k IHJ1bm5pbmcgcHJpb3IgdG8gdGhlIDIwMjAtMTAtMTUgc25hcHNob3QuCj4KPiBFdmVyeSB2ZXJz aW9uIG9mIEZyZWVCU0QgSSd2ZSB0ZXN0ZWQgb24gQVJNIGlzIGxhY2tpbmcgc2V2ZXJhbCBkcml2 ZXJzIGJ5Cj4gZGVmYXVsdC4KPgo+IHVoY2kgZm9yIFVTQiAyLjAgaXMgbm90IGVuYWJsZWQgYnkg ZGVmYXVsdC4gRW5zdXJlIHlvdXIgVk0gaXMgaW5pdGlhbGx5IHNldAo+IHVwCj4gZm9yIFVTQiAz LjEgc28gdGhlIHZpcnR1YWwga2V5Ym9hcmQgYW5kIG1vdXNlIHdpbGwgZnVuY3Rpb24uIFRoaXMg aXMgZWFzeQo+IGVub3VnaAo+IHRvIGZpeCBwb3N0IGluc3RhbGwgd2l0aDogdWhjaV9sb2FkPSJZ RVMiCj4KPiB1bXMgaXMgdGhlIHNhbWUgYXMgYWJvdmUuIEVTWGkgQVJNIHVzZXMgVVNCIGluc3Rl YWQgb2YgUFMyIGZvciB0aGUgdmlydHVhbAo+IG1vdXNlLCBzbyBhZGQgdGhpcyBwb3N0IGluc3Rh bGwgaWYgeW91IHdhbnQgbW91c2Ugc3VwcG9ydDogdW1zX2xvYWQ9IllFUyIKPgo+IHZteG5ldCBh bmQgcHZzY3NpIGFyZSBub3QgcHJlLWNvbXBpbGVkLiBIb3dldmVyLCB0aGV5IGNvbXBpbGUgYW5k IG9wZXJhdGUKPiBwZXJmZWN0bHkgYXMgZXhwZWN0ZWQgd2l0aG91dCBhbnkgbW9kaWZpY2F0aW9u IG9uIEFSTS4KPgo+IHZtY2kgaGFzIHNvbWUgeDg2IGFzc2VtYmx5IHRoYXQgSSd2ZSBub3QgZHVn IGludG8geWV0LCBzbyB0aGF0IGlzIGN1cnJlbnRseQo+IG5vdAo+IHdvcmtpbmcuCj4KPiBJJ3Zl IGRvY3VtZW50ZWQgaW4gZGV0YWlsLCBhbmQgY29udGludWUgdG8gdXBkYXRlIGFzIEkgdGVzdCBt b3JlIGRyaXZlcnMKPiBvdmVyIG9uCj4gbXkgYmxvZzoKPiBodHRwczovL3ZpbmNlcmFudHMuY29t L2ZyZWVic2Qtb24tZXN4aS1hcm0tZmxpbmctZml4aW5nLXZpcnR1YWwtaGFyZHdhcmUvCj4KPiBU aGUgRVNYaSBVSSBoYXMgYSBrbm93biBidWcgaW4gdGhhdCBpdCdsbCBsaXN0IHNvbWUgdmlydHVh bCBoYXJkd2FyZSB0aGF0Cj4gZG9lc24ndCBhY3R1YWxseSBleGlzdCBpbnNpZGUgb2YgdGhlIFZN LiBUaGVzZSBhcmUgYWxsIGxlZ2FjeSBkZXZpY2VzLCBhbmQKPiBzaG91bGQgYmUgZWFzeSBlbm91 Z2ggdG8gZmlndXJlIG91dC4gVGhlcmUgYXJlIG5vIHZpcnR1YWwgSURFL0FUQSBvciBQUzIKPiBw b3J0cy4KPiBLZXlib2FyZCBhbmQgbW91c2Ugbm93IHVzZSB0aGUgdmlydHVhbCBVU0IgY29udHJv bGxlciBpbnN0ZWFkLgo+Cj4gb3Blbi12bS10b29scyBoYXMgcmVxdWlyZWQgYSBiaXQgb2Ygd29y ayB0byBtYWtlIGZ1bmN0aW9uYWwgb24gRnJlZUJTRCB1bmRlcgo+IEFSTS4gVGhlcmUgaXMgY3Vy cmVudGx5IGEgcGF0Y2ggb3BlbiB3aXRoIFZNd2FyZSB3aXRoIG15IGNoYW5nZXMgdG8gdGhlCj4g Y29kZWJhc2UKPiB0byBtYWtlIGl0IGZ1bmN0aW9uYWwuIFRvIG1ha2UgdGhpbmdzIGVhc3ksIEkn dmUgdXBsb2FkZWQgc29tZSBwcmUtY29tcGlsZWQKPiBiaW5hcmllcyB0byBteSBibG9nLCBhcyB3 ZWxsIGFzIGluY2x1ZGVkIHN0ZXBzIHRvIGNvbXBpbGUgdGhpcyBjdXN0b21pemVkCj4gcGFja2Fn ZSB5b3Vyc2VsZi4gVGhlIGJsb2cgd2lsbCBhbHNvIGJlIHVwZGF0ZWQgYXMgZGV2ZWxvcG1lbnQg cHJvZ3Jlc3Nlcwo+IHVwc3RyZWFtIHdpdGggVk13YXJlLgo+IGh0dHBzOi8vdmluY2VyYW50cy5j b20vb3Blbi12bS10b29scy1vbi1mcmVlYnNkLXVuZGVyLXZtd2FyZS1lc3hpLWFybS1mbGluZy8K Pgo+IFhvcmcgZGVza3RvcCBydW5zLCBhbmQgaXMgbW9zdGx5IGZ1bmN0aW9uYWwuIEkndmUgeWV0 IHRvIGdldCB0aGUKPiB4Zjg2LWlucHV0LXZtbW91c2Ugb3IgeGY4Ni12aWRlby12bXdhcmUgd29y a2luZy4gWG9yZyBjYW4gdXNlIHN5c21vdXNlIGFuZAo+IHRoZQo+IGNvbnNvbGUgZnJhbWVidWZm ZXIgZm9yIHJlbmRlcmluZywgYW5kIHRoaXMgaXMgd29ya2luZyBva2F5IGZvciBiYXNpYwo+IHRl c3RpbmcKPiByaWdodCBub3cuCj4KPiBJIGFsc28gaGF2ZSBhbiBlYXJseSBkbWVzZywgYnV0IHRo aXMgd2FzIGZyb20gYmVmb3JlIGdldHRpbmcgbW9zdCBkcml2ZXJzCj4gd29ya2luZy4KPiBodHRw czovL2RtZXNnZC5ueWNidWcub3JnL2luZGV4LmNnaT9kbz12aWV3JmlkPTU2OTMKPgo+IEkgd2Vs Y29tZSBhbnkgYWRkaXRpb25hbCBmZWVkYmFjayBvbiBnZXR0aW5nIEZyZWVCU0QgdG8gMTAwJSB1 bmRlciBFU1hpIEFSTSEKPgo+IC0gVmluY2UgKGRhcmthaW4pCj4gX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX18KPiBmcmVlYnNkLWFybUBmcmVlYnNkLm9yZyBt YWlsaW5nIGxpc3QKPiBodHRwczovL2xpc3RzLmZyZWVic2Qub3JnL21haWxtYW4vbGlzdGluZm8v ZnJlZWJzZC1hcm0KPiBUbyB1bnN1YnNjcmliZSwgc2VuZCBhbnkgbWFpbCB0byAiZnJlZWJzZC1h cm0tdW5zdWJzY3JpYmVAZnJlZWJzZC5vcmci From owner-freebsd-arm@freebsd.org Mon Oct 19 19:19:51 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 59511432212 for ; Mon, 19 Oct 2020 19:19:51 +0000 (UTC) (envelope-from freebsd-arm@darkain.com) Received: from MTA-08-4.privateemail.com (mta-08-4.privateemail.com [198.54.122.58]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4CFRQt1CxBz43Cy for ; Mon, 19 Oct 2020 19:19:49 +0000 (UTC) (envelope-from freebsd-arm@darkain.com) Received: from MTA-08.privateemail.com (localhost [127.0.0.1]) by MTA-08.privateemail.com (Postfix) with ESMTP id C834C6003E for ; Mon, 19 Oct 2020 15:19:42 -0400 (EDT) Received: from mail-il1-f170.google.com (unknown [10.20.151.217]) by MTA-08.privateemail.com (Postfix) with ESMTPA id 8E63860033 for ; Mon, 19 Oct 2020 19:19:42 +0000 (UTC) Received: by mail-il1-f170.google.com with SMTP id p9so1458374ilr.1 for ; Mon, 19 Oct 2020 12:19:42 -0700 (PDT) X-Gm-Message-State: AOAM530dkNq7lgnI2RocIM9sXNFrkthSy8Ng0h7du/OP+OcZk2OQHZ79 NNrqUWQw8WZbmW6JMFUEbEl4htp1lejkY9O0AYM= X-Google-Smtp-Source: ABdhPJyWcANMQZyo6HLocqJF/zOYHZc8s3ZNAJ55U9LrmIw+ZZSnI+x0D1kSANUTqpZVBVxUNo7wOO4Natae8qes7kw= X-Received: by 2002:a92:c843:: with SMTP id b3mr1234559ilq.18.1603135181940; Mon, 19 Oct 2020 12:19:41 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Vincent Milum Jr Date: Mon, 19 Oct 2020 12:19:28 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: FreeBSD under VMware ESXi ARM Fling To: Robert Crowston Cc: freebsd-arm@freebsd.org X-Virus-Scanned: ClamAV using ClamSMTP X-Rspamd-Queue-Id: 4CFRQt1CxBz43Cy X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of freebsd-arm@darkain.com designates 198.54.122.58 as permitted sender) smtp.mailfrom=freebsd-arm@darkain.com X-Spamd-Result: default: False [-2.73 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.991]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:198.54.122.32/27]; NEURAL_HAM_LONG(-1.00)[-1.001]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-arm@freebsd.org]; DMARC_NA(0.00)[darkain.com]; RCVD_COUNT_THREE(0.00)[4]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RWL_MAILSPIKE_VERYGOOD(0.00)[198.54.122.58:from]; NEURAL_HAM_SHORT(-0.44)[-0.437]; RCPT_COUNT_TWO(0.00)[2]; FREEMAIL_TO(0.00)[protonmail.com]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:22612, ipnet:198.54.122.0/24, country:US]; RCVD_TLS_LAST(0.00)[]; MAILMAN_DEST(0.00)[freebsd-arm] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Oct 2020 19:19:51 -0000 I'm not too familiar with the FreeBSD kernel itself. Here is a recording of booting the VM from the 13.0 2020-10-15 Snapshot ISO. There are a few different outcomes that seem to randomly happen when booting. https://www.youtube.com/watch?v=3DLs9gkQ14Z-A On Mon, Oct 19, 2020 at 11:49 AM Robert Crowston wrote: > > The largest issue currently is that SMP will cause the FreeBSD kernel t= o > panic during boot. > > Could you send the panic? > > I=E2=80=99m guessing lack of spintable support. > > =E2=80=94 RHC. > > On Mon, Oct 19, 2020 at 07:19, Vincent Milum Jr > wrote: > > Recently, VMware released what they're calling "ESXi ARM Fling" - an > ARM64/Aarch64 version of VMware ESXi server. > > Details of the release along with the download can be found here: > https://flings.vmware.com/esxi-arm-edition > > > They support a number of ARM platforms already, most notably is the > Raspberry Pi > 4 (only 4GB and 8GB models). They have instructions on their site on how = to > install and configure the UEFI firmware on the Pi for getting the ESXi > installer > and then ESXi itself up and running. > > UEFI builds of FreeBSD ARM64/Aarch64 mostly work "out of the box" under > ESXi > ARM, whereas ARM 32-bit kernels are not supported by any guest OS under > ESXi > ARM. > > There are, however, some areas that could use some improvement, or need > some > massive work to get working. > > The largest issue currently is that SMP will cause the FreeBSD kernel to > panic > during boot. A VM with only one single vCPU however will boot and run jus= t > fine. > > Prior to the 13.0-CURRENT snapshot 2020-10-15, the CD-Rom driver was not > included with the ARM64 builds, so the ISO would fail to mount itself on > boot. This > means the easiest way to try earlier versions, such as 12.1-RELEASE, you'= ll > need > to either convert the pre-built vmdk file from Workstation to ESXi format= , > or > convert the raw disk image to ESXi vmdk format. This is how I initially g= ot > FreeBSD up and running prior to the 2020-10-15 snapshot. > > Every version of FreeBSD I've tested on ARM is lacking several drivers by > default. > > uhci for USB 2.0 is not enabled by default. Ensure your VM is initially s= et > up > for USB 3.1 so the virtual keyboard and mouse will function. This is easy > enough > to fix post install with: uhci_load=3D"YES" > > ums is the same as above. ESXi ARM uses USB instead of PS2 for the virtua= l > mouse, so add this post install if you want mouse support: ums_load=3D"YE= S" > > vmxnet and pvscsi are not pre-compiled. However, they compile and operate > perfectly as expected without any modification on ARM. > > vmci has some x86 assembly that I've not dug into yet, so that is current= ly > not > working. > > I've documented in detail, and continue to update as I test more drivers > over on > my blog: > https://vincerants.com/freebsd-on-esxi-arm-fling-fixing-virtual-hardware/ > > > The ESXi UI has a known bug in that it'll list some virtual hardware that > doesn't actually exist inside of the VM. These are all legacy devices, an= d > should be easy enough to figure out. There are no virtual IDE/ATA or PS2 > ports. > Keyboard and mouse now use the virtual USB controller instead. > > open-vm-tools has required a bit of work to make functional on FreeBSD > under > ARM. There is currently a patch open with VMware with my changes to the > codebase > to make it functional. To make things easy, I've uploaded some pre-compil= ed > binaries to my blog, as well as included steps to compile this customized > package yourself. The blog will also be updated as development progresses > upstream with VMware. > > https://vincerants.com/open-vm-tools-on-freebsd-under-vmware-esxi-arm-fli= ng/ > > Xorg desktop runs, and is mostly functional. I've yet to get the > xf86-input-vmmouse or xf86-video-vmware working. Xorg can use sysmouse an= d > the > console framebuffer for rendering, and this is working okay for basic > testing > right now. > > > I also have an early dmesg, but this was from before getting most drivers > working. > https://dmesgd.nycbug.org/index.cgi?do=3Dview&id=3D5693 > > > I welcome any additional feedback on getting FreeBSD to 100% under ESXi > ARM! > > - Vince (darkain) > _______________________________________________ > freebsd-arm@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-arm > To unsubscribe, send any mail to "freebsd-arm-unsubscribe@freebsd.org" > > > > From owner-freebsd-arm@freebsd.org Mon Oct 19 20:39:59 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id DC499434481; Mon, 19 Oct 2020 20:39:59 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-io1-xd41.google.com (mail-io1-xd41.google.com [IPv6:2607:f8b0:4864:20::d41]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CFTCL3Ff9z48j6; Mon, 19 Oct 2020 20:39:58 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: by mail-io1-xd41.google.com with SMTP id q25so1457485ioh.4; Mon, 19 Oct 2020 13:39:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=CmoJc3Se4kYJ4D0w5hM+bPiL1uZ+22dXmJZbpjxT7vQ=; b=M5nQdLXTH1MZMjtX6R9eSt1lI8vN9gIqxc98XVIHLP0/GT1EQOGtOkNj8JyNQK26nl 9GXHF2ZOUtjvhK9ECdeAYhaoZ7p8L/qyWUdvgoS27XiTrUoiCafnWOQ3O+Mz3oAZ5A5p H7Dd+gjyVdEAUEelLG5/YRSafeH0kwPaVQj/s4q3Sa6Peo2a4OavKmyTUpy7i7Uj0Xi0 77XM0gf/C+syywAcRWTcU58ovbEeglMzjbXLlwFOux94zPPlL+Z6NGFifwwHX2ahOwr8 NMjGoY87H0S5C0Tg1OIn/jgNBwxo4ypkcNhRgd2SZQot2tQoHa2+u8oMeVzPUN25V99U e1Gg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :references:mime-version:content-disposition:in-reply-to; bh=CmoJc3Se4kYJ4D0w5hM+bPiL1uZ+22dXmJZbpjxT7vQ=; b=OXs8+RCGtOxL6n/WkVwq+3dFEm11mniP+nIDtdarmI0oKMzgxvw27ijbhfwG0J84tS sGfy9D6f0P+pBdHct5OKsfyOAMzifNGFrP2iE6n2GY1H+kBRLcf8hPyJnIBYeNAdkiwJ ce/1G18kNjvDTtyrSQ/hJOw7E1v75W+WUK1ej60FmPHuVYsJgtn/p8c8jcA1OE/9AQ2L nTR+FlqNtdKzi2ng+LqUm5A7n0xtAdE8vCny46vQdsv3+JiaHabZJnA+/GoPnePXt8SZ 9p6UdZRDTRaqBnBqIZCyGcqjw3DSVVZI6N2LcmDQyMqbQHPGPrpI0K5vc/QDCz8V1i85 b87Q== X-Gm-Message-State: AOAM530cDf5fdgyUJFzgG9YUP5/CFJgF1kakrEwcxEogHdHRbm4nfMwc CqZqo/LRjNtslPEerPcyWmQHrold4PM= X-Google-Smtp-Source: ABdhPJxj18hsbo193UotBMD0F8QtF4mwEBjjIO6XZobfvfqfP17Z8P67HgDEGKXfs5BEYZpZrzGbcQ== X-Received: by 2002:a02:a798:: with SMTP id e24mr1531280jaj.105.1603139997051; Mon, 19 Oct 2020 13:39:57 -0700 (PDT) Received: from raichu (toroon0560w-lp130-01-174-88-77-103.dsl.bell.ca. [174.88.77.103]) by smtp.gmail.com with ESMTPSA id h14sm766399ilc.38.2020.10.19.13.39.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 19 Oct 2020 13:39:56 -0700 (PDT) Sender: Mark Johnston Date: Mon, 19 Oct 2020 16:39:54 -0400 From: Mark Johnston To: mmel@freebsd.org Cc: bob prohaska , freebsd-current@freebsd.org, freebsd-arm@freebsd.org Subject: Re: panic: non-current pmap 0xffffa00020eab8f0 on Rpi3 Message-ID: <20201019203954.GC46122@raichu> References: <20201006021029.GA13260@www.zefox.net> <20201006133743.GA96285@raichu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Queue-Id: 4CFTCL3Ff9z48j6 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=M5nQdLXT; dmarc=none; spf=pass (mx1.freebsd.org: domain of markjdb@gmail.com designates 2607:f8b0:4864:20::d41 as permitted sender) smtp.mailfrom=markjdb@gmail.com X-Spamd-Result: default: False [-2.62 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; NEURAL_HAM_LONG(-1.02)[-1.020]; MIME_GOOD(-0.10)[text/plain]; MID_RHS_NOT_FQDN(0.50)[]; DMARC_NA(0.00)[freebsd.org]; TO_DN_SOME(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; NEURAL_HAM_SHORT(-0.89)[-0.893]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::d41:from]; NEURAL_HAM_MEDIUM(-1.01)[-1.009]; FORGED_SENDER(0.30)[markj@freebsd.org,markjdb@gmail.com]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[markj@freebsd.org,markjdb@gmail.com]; MAILMAN_DEST(0.00)[freebsd-current,freebsd-arm] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Oct 2020 20:39:59 -0000 On Fri, Oct 16, 2020 at 11:53:56AM +0200, Michal Meloun wrote: > > > On 06.10.2020 15:37, Mark Johnston wrote: > > On Mon, Oct 05, 2020 at 07:10:29PM -0700, bob prohaska wrote: > >> Still seeing non-current pmap panics on the Pi3, this time a B+ running > >> 13.0-CURRENT (GENERIC-MMCCAM) #0 71e02448ffb-c271826(master) > >> during a -j4 buildworld. The backtrace reports > >> > >> panic: non-current pmap 0xffffa00020eab8f0 > > > > Could you show the output of "show procvm" from the debugger? > > I see same panic too, in my case its very rare - typical scenario is > rebuild of kf5 ports (~250, 2 days of full load). Any idea how to debug > this? > Michal I suspect that there is some race involving the pmap switching in vmspace_exit(), but I can't see it. In the example below, presumably process 22604 on CPU 0 is also exiting? Could you show the backtrace? It would also be useful to see the value of PCPU_GET(curpmap) at the time of the panic. I'm not sure if there's a way to get that from DDB, but I suspect it should be equal to &vmspace0->vm_pmap. I think vmspace_exit() should issue a release fence with the cmpset and an acquire fence when handling the refcnt == 1 case, but I don't see why that would make a difference here. So, if you can test a debug patch, this one will yield a bit more debug info. If you can provide access to a vmcore and kernel debug symbols, that'd be even better. diff --git a/sys/arm64/arm64/pmap.c b/sys/arm64/arm64/pmap.c index 284f00b3cc0d..3c53ae3b4c1e 100644 --- a/sys/arm64/arm64/pmap.c +++ b/sys/arm64/arm64/pmap.c @@ -4838,7 +4838,8 @@ pmap_remove_pages(pmap_t pmap) int allfree, field, freed, idx, lvl; vm_paddr_t pa; - KASSERT(pmap == PCPU_GET(curpmap), ("non-current pmap %p", pmap)); + KASSERT(pmap == PCPU_GET(curpmap), + ("non-current pmap %p %p", pmap, PCPU_GET(curpmap))); lock = NULL; diff --git a/sys/vm/vm_map.c b/sys/vm/vm_map.c index c20005ae64cf..0ad415e3b88c 100644 --- a/sys/vm/vm_map.c +++ b/sys/vm/vm_map.c @@ -358,7 +358,10 @@ vmspace_exit(struct thread *td) p = td->td_proc; vm = p->p_vmspace; atomic_add_int(&vmspace0.vm_refcnt, 1); - refcnt = vm->vm_refcnt; + refcnt = atomic_load_int(&vm->vm_refcnt); + + KASSERT(vmspace_pmap(vm) == PCPU_GET(curpmap), + ("non-current pmap %p %p", pmap, PCPU_GET(curpmap))); do { if (refcnt > 1 && p->p_vmspace != &vmspace0) { /* Switch now since other proc might free vmspace */ From owner-freebsd-arm@freebsd.org Mon Oct 19 20:52:24 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 2604A434A1B for ; Mon, 19 Oct 2020 20:52:24 +0000 (UTC) (envelope-from crowston@protonmail.com) Received: from mail-40134.protonmail.ch (mail-40134.protonmail.ch [185.70.40.134]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "protonmail.com", Issuer "SwissSign Server Gold CA 2014 - G22" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CFTTf2gmLz4Bst for ; Mon, 19 Oct 2020 20:52:21 +0000 (UTC) (envelope-from crowston@protonmail.com) Date: Mon, 19 Oct 2020 20:52:10 +0000 To: Vincent Milum Jr From: Robert Crowston Cc: "freebsd-arm@freebsd.org" Reply-To: Robert Crowston Subject: Re: FreeBSD under VMware ESXi ARM Fling Message-ID: In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Status: No, score=-1.2 required=10.0 tests=ALL_TRUSTED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM shortcircuit=no autolearn=disabled version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on mailout.protonmail.ch X-Rspamd-Queue-Id: 4CFTTf2gmLz4Bst X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.58 / 15.00]; HAS_REPLYTO(0.00)[crowston@protonmail.com]; TO_DN_EQ_ADDR_SOME(0.00)[]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[protonmail.com]; R_SPF_ALLOW(-0.20)[+ip4:185.70.40.0/24]; DKIM_TRACE(0.00)[protonmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[protonmail.com,quarantine]; NEURAL_HAM_SHORT(-0.58)[-0.577]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[protonmail.com]; ASN(0.00)[asn:62371, ipnet:185.70.40.0/24, country:CH]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.985]; R_DKIM_ALLOW(-0.20)[protonmail.com:s=protonmail]; REPLYTO_EQ_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.01)[-1.012]; MIME_GOOD(-0.10)[text/plain]; FREEMAIL_REPLYTO(0.00)[protonmail.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[185.70.40.134:from]; MAILMAN_DEST(0.00)[freebsd-arm] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Oct 2020 20:52:24 -0000 My initial hypothesis was wrong. I had meant, can you provide a text copy of the dmesg (collected from a ser= ial port!)! But, from the video, it seems that as soon as we get to the Release APs pha= se, there is some race condition that is causing text to be concurrently wr= itten to the screen by multiple processors--you can see it becomes garbled.= KDB is entered shortly before the "Release APs..." is printed to the scree= n, but text continues to be written to the console. [We don't really know i= n what order these events happen, though.] A panic() is triggered within a = few moments. At this point, we enter termcn_cnputc(), and violate a KASSERT= in __mtx_lock_spin_flags. This causes a recursive panic. For me the next logical step would be to attach a debugger to the guest pro= cess. Any idea if ESXi ARM allows debug or single stepping of guests? -- RHC. On Mon, Oct 19, 2020 at 20:19, Vincent Milum Jr w= rote: > I'm not too familiar with the FreeBSD kernel itself. Here is a recording = of booting the VM from the 13.0 2020-10-15 Snapshot ISO. There are a few di= fferent outcomes that seem to randomly happen when booting. > > https://www.youtube.com/watch?v=3DLs9gkQ14Z-A > > On Mon, Oct 19, 2020 at 11:49 AM Robert Crowston wrote: > > > >=C2=A0The largest issue currently is that SMP will cause the FreeBSD k= ernel to > > panic=C2=A0during boot. > > > > Could you send the panic? > > > > I=E2=80=99m guessing lack of spintable support. > > > > =C2=A0 =C2=A0 =E2=80=94 RHC. > > > > On Mon, Oct 19, 2020 at 07:19, Vincent Milum Jr wrote: > > > > > Recently, VMware released what they're calling "ESXi ARM Fling" - an > > > ARM64/Aarch64 version of VMware ESXi server. > > > > > > Details of the release along with the download can be found here: > > > https://flings.vmware.com/esxi-arm-edition > > > > > > They support a number of ARM platforms already, most notably is the > > > Raspberry Pi > > > 4 (only 4GB and 8GB models). They have instructions on their site on = how to > > > install and configure the UEFI firmware on the Pi for getting the ESX= i > > > installer > > > and then ESXi itself up and running. > > > > > > UEFI builds of FreeBSD ARM64/Aarch64 mostly work "out of the box" und= er ESXi > > > ARM, whereas ARM 32-bit kernels are not supported by any guest OS und= er ESXi > > > ARM. > > > > > > There are, however, some areas that could use some improvement, or ne= ed some > > > massive work to get working. > > > > > > The largest issue currently is that SMP will cause the FreeBSD kernel= to > > > panic > > > during boot. A VM with only one single vCPU however will boot and run= just > > > fine. > > > > > > Prior to the 13.0-CURRENT snapshot 2020-10-15, the CD-Rom driver was = not > > > included with the ARM64 builds, so the ISO would fail to mount itself= on > > > boot. This > > > means the easiest way to try earlier versions, such as 12.1-RELEASE, = you'll > > > need > > > to either convert the pre-built vmdk file from Workstation to ESXi fo= rmat, > > > or > > > convert the raw disk image to ESXi vmdk format. This is how I initial= ly got > > > FreeBSD up and running prior to the 2020-10-15 snapshot. > > > > > > Every version of FreeBSD I've tested on ARM is lacking several driver= s by > > > default. > > > > > > uhci for USB 2.0 is not enabled by default. Ensure your VM is initial= ly set > > > up > > > for USB 3.1 so the virtual keyboard and mouse will function. This is = easy > > > enough > > > to fix post install with: uhci_load=3D"YES" > > > > > > ums is the same as above. ESXi ARM uses USB instead of PS2 for the vi= rtual > > > mouse, so add this post install if you want mouse support: ums_load= =3D"YES" > > > > > > vmxnet and pvscsi are not pre-compiled. However, they compile and ope= rate > > > perfectly as expected without any modification on ARM. > > > > > > vmci has some x86 assembly that I've not dug into yet, so that is cur= rently > > > not > > > working. > > > > > > I've documented in detail, and continue to update as I test more driv= ers > > > over on > > > my blog: > > > https://vincerants.com/freebsd-on-esxi-arm-fling-fixing-virtual-hardw= are/ > > > > > > The ESXi UI has a known bug in that it'll list some virtual hardware = that > > > doesn't actually exist inside of the VM. These are all legacy devices= , and > > > should be easy enough to figure out. There are no virtual IDE/ATA or = PS2 > > > ports. > > > Keyboard and mouse now use the virtual USB controller instead. > > > > > > open-vm-tools has required a bit of work to make functional on FreeBS= D under > > > ARM. There is currently a patch open with VMware with my changes to t= he > > > codebase > > > to make it functional. To make things easy, I've uploaded some pre-co= mpiled > > > binaries to my blog, as well as included steps to compile this custom= ized > > > package yourself. The blog will also be updated as development progre= sses > > > upstream with VMware. > > > https://vincerants.com/open-vm-tools-on-freebsd-under-vmware-esxi-arm= -fling/ > > > > > > Xorg desktop runs, and is mostly functional. I've yet to get the > > > xf86-input-vmmouse or xf86-video-vmware working. Xorg can use sysmous= e and > > > the > > > console framebuffer for rendering, and this is working okay for basic > > > testing > > > right now. > > > > > > I also have an early dmesg, but this was from before getting most dri= vers > > > working. > > > https://dmesgd.nycbug.org/index.cgi?do=3Dview&id=3D5693 > > > > > > I welcome any additional feedback on getting FreeBSD to 100% under ES= Xi ARM! > > > > > > - Vince (darkain) > > > _______________________________________________ > > > freebsd-arm@freebsd.org mailing list > > > https://lists.freebsd.org/mailman/listinfo/freebsd-arm > > > To unsubscribe, send any mail to "freebsd-arm-unsubscribe@freebsd.org= " From owner-freebsd-arm@freebsd.org Mon Oct 19 23:09:06 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 15B53437741; Mon, 19 Oct 2020 23:09:06 +0000 (UTC) (envelope-from fbsd@www.zefox.net) Received: from www.zefox.net (www.zefox.net [50.1.20.27]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "www.zefox.com", Issuer "www.zefox.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CFXWP4GYCz4KRW; Mon, 19 Oct 2020 23:09:03 +0000 (UTC) (envelope-from fbsd@www.zefox.net) Received: from www.zefox.net (localhost [127.0.0.1]) by www.zefox.net (8.15.2/8.15.2) with ESMTPS id 09JN9AtF066727 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Mon, 19 Oct 2020 16:09:10 -0700 (PDT) (envelope-from fbsd@www.zefox.net) Received: (from fbsd@localhost) by www.zefox.net (8.15.2/8.15.2/Submit) id 09JN9AMo066726; Mon, 19 Oct 2020 16:09:10 -0700 (PDT) (envelope-from fbsd) Date: Mon, 19 Oct 2020 16:09:09 -0700 From: bob prohaska To: Mark Johnston Cc: mmel@freebsd.org, freebsd-current@freebsd.org, freebsd-arm@freebsd.org, bob prohaska Subject: Re: panic: non-current pmap 0xffffa00020eab8f0 on Rpi3 Message-ID: <20201019230909.GA66675@www.zefox.net> References: <20201006021029.GA13260@www.zefox.net> <20201006133743.GA96285@raichu> <20201019203954.GC46122@raichu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20201019203954.GC46122@raichu> X-Rspamd-Queue-Id: 4CFXWP4GYCz4KRW X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; ASN(0.00)[asn:7065, ipnet:50.1.16.0/20, country:US]; REPLY(-4.00)[] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Oct 2020 23:09:06 -0000 On Mon, Oct 19, 2020 at 04:39:54PM -0400, Mark Johnston wrote: > > I think vmspace_exit() should issue a release fence with the cmpset and > an acquire fence when handling the refcnt == 1 case, but I don't see why > that would make a difference here. So, if you can test a debug patch, > this one will yield a bit more debug info. If you can provide access to > a vmcore and kernel debug symbols, that'd be even better. > I haven't seen an invalid pmap panic since the report of October 5th. Your patch applied cleanly on the Pi3 running HEAD at r366780M, the M being due to patches supplied by Kyle Evans applied to M sys/arm/broadcom/bcm2835/bcm2835_mbox.c M sys/arm/broadcom/bcm2835/bcm2835_sdhci.c M sys/arm/broadcom/bcm2835/bcm2835_vcbus.c M sys/arm/broadcom/bcm2835/bcm2835_vcbus.h AIUI, they're something to do with DMA for peripherals. They've caused no obvious trouble, if you anticipate conflicts let me know and I'll remove them I've never seen either a vmcore file or debug symbols on this machine. A sequence of instructions to generate the data needed would be helpful. Thanks for reading! bob prohaska From owner-freebsd-arm@freebsd.org Mon Oct 19 23:49:50 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 884A84382B9 for ; Mon, 19 Oct 2020 23:49:50 +0000 (UTC) (envelope-from freebsd-arm@darkain.com) Received: from MTA-14-4.privateemail.com (mta-14-4.privateemail.com [198.54.118.206]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4CFYQP3PFrz4M5X for ; Mon, 19 Oct 2020 23:49:49 +0000 (UTC) (envelope-from freebsd-arm@darkain.com) Received: from mta-14.privateemail.com (localhost [127.0.0.1]) by mta-14.privateemail.com (Postfix) with ESMTP id 9098980059 for ; Mon, 19 Oct 2020 19:49:41 -0400 (EDT) Received: from mail-io1-f54.google.com (unknown [10.20.151.230]) by mta-14.privateemail.com (Postfix) with ESMTPA id 3DCCE80056 for ; Mon, 19 Oct 2020 23:49:41 +0000 (UTC) Received: by mail-io1-f54.google.com with SMTP id q25so214733ioh.4 for ; Mon, 19 Oct 2020 16:49:41 -0700 (PDT) X-Gm-Message-State: AOAM530saf3xRhz/wBmD20bUu/Usi3PKN2GPYtILGEHceCX142OBK+Iu TSvPpzxyaoqW/pQLcIkZgxXipqkobptNmSKtt8E= X-Google-Smtp-Source: ABdhPJzqzSNjgNX2ek/OJ9lGcZ55xGivDY2Pz+i0MssYH0oC55orxZeZk2QAr+bwnzUH7HAEhIqzUe3Wb/nH4BXbQ4o= X-Received: by 2002:a02:b709:: with SMTP id g9mr168892jam.90.1603151376506; Mon, 19 Oct 2020 16:49:36 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Vincent Milum Jr Date: Mon, 19 Oct 2020 16:49:26 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: FreeBSD under VMware ESXi ARM Fling To: Robert Crowston Cc: "freebsd-arm@freebsd.org" X-Virus-Scanned: ClamAV using ClamSMTP X-Rspamd-Queue-Id: 4CFYQP3PFrz4M5X X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of freebsd-arm@darkain.com designates 198.54.118.206 as permitted sender) smtp.mailfrom=freebsd-arm@darkain.com X-Spamd-Result: default: False [-2.92 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:198.54.118.192/27]; NEURAL_HAM_LONG(-1.01)[-1.006]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-arm@freebsd.org]; DMARC_NA(0.00)[darkain.com]; RCVD_COUNT_THREE(0.00)[4]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RWL_MAILSPIKE_VERYGOOD(0.00)[198.54.118.206:from]; NEURAL_HAM_SHORT(-0.64)[-0.636]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_MEDIUM(-0.98)[-0.978]; FREEMAIL_TO(0.00)[protonmail.com]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:22612, ipnet:198.54.118.0/24, country:US]; RCVD_TLS_LAST(0.00)[]; MAILMAN_DEST(0.00)[freebsd-arm] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Oct 2020 23:49:50 -0000 According to the ESXi devs, there is no SIO controller currently, and I believe this is required for the virtual serial ports to work https://flings.vmware.com/esxi-arm-edition/bugs/1104 I'm not sure about debugging through ESXi. I'll forward these questions to them and see what they say. On Mon, Oct 19, 2020 at 1:52 PM Robert Crowston wrote: > My initial hypothesis was wrong. > > I had meant, can you provide a text copy of the dmesg (collected from a > serial port!)! > > But, from the video, it seems that as soon as we get to the Release APs > phase, there is some race condition that is causing text to be concurrent= ly > written to the screen by multiple processors--you can see it becomes > garbled. KDB is entered shortly before the "Release APs..." is printed to > the screen, but text continues to be written to the console. [We don't > really know in what order these events happen, though.] A panic() is > triggered within a few moments. At this point, we enter termcn_cnputc(), > and violate a KASSERT in __mtx_lock_spin_flags. This causes a recursive > panic. > > For me the next logical step would be to attach a debugger to the guest > process. Any idea if ESXi ARM allows debug or single stepping of guests? > > -- RHC. > > On Mon, Oct 19, 2020 at 20:19, Vincent Milum Jr > wrote: > > > I'm not too familiar with the FreeBSD kernel itself. Here is a recordin= g > of booting the VM from the 13.0 2020-10-15 Snapshot ISO. There are a few > different outcomes that seem to randomly happen when booting. > > > > https://www.youtube.com/watch?v=3DLs9gkQ14Z-A > > > > On Mon, Oct 19, 2020 at 11:49 AM Robert Crowston < > crowston@protonmail.com> wrote: > > > > > > The largest issue currently is that SMP will cause the FreeBSD > kernel to > > > panic during boot. > > > > > > Could you send the panic? > > > > > > I=E2=80=99m guessing lack of spintable support. > > > > > > =E2=80=94 RHC. > > > > > > On Mon, Oct 19, 2020 at 07:19, Vincent Milum Jr < > freebsd-arm@darkain.com> wrote: > > > > > > > Recently, VMware released what they're calling "ESXi ARM Fling" - a= n > > > > ARM64/Aarch64 version of VMware ESXi server. > > > > > > > > Details of the release along with the download can be found here: > > > > https://flings.vmware.com/esxi-arm-edition > > > > > > > > They support a number of ARM platforms already, most notably is the > > > > Raspberry Pi > > > > 4 (only 4GB and 8GB models). They have instructions on their site o= n > how to > > > > install and configure the UEFI firmware on the Pi for getting the > ESXi > > > > installer > > > > and then ESXi itself up and running. > > > > > > > > UEFI builds of FreeBSD ARM64/Aarch64 mostly work "out of the box" > under ESXi > > > > ARM, whereas ARM 32-bit kernels are not supported by any guest OS > under ESXi > > > > ARM. > > > > > > > > There are, however, some areas that could use some improvement, or > need some > > > > massive work to get working. > > > > > > > > The largest issue currently is that SMP will cause the FreeBSD > kernel to > > > > panic > > > > during boot. A VM with only one single vCPU however will boot and > run just > > > > fine. > > > > > > > > Prior to the 13.0-CURRENT snapshot 2020-10-15, the CD-Rom driver wa= s > not > > > > included with the ARM64 builds, so the ISO would fail to mount > itself on > > > > boot. This > > > > means the easiest way to try earlier versions, such as 12.1-RELEASE= , > you'll > > > > need > > > > to either convert the pre-built vmdk file from Workstation to ESXi > format, > > > > or > > > > convert the raw disk image to ESXi vmdk format. This is how I > initially got > > > > FreeBSD up and running prior to the 2020-10-15 snapshot. > > > > > > > > Every version of FreeBSD I've tested on ARM is lacking several > drivers by > > > > default. > > > > > > > > uhci for USB 2.0 is not enabled by default. Ensure your VM is > initially set > > > > up > > > > for USB 3.1 so the virtual keyboard and mouse will function. This i= s > easy > > > > enough > > > > to fix post install with: uhci_load=3D"YES" > > > > > > > > ums is the same as above. ESXi ARM uses USB instead of PS2 for the > virtual > > > > mouse, so add this post install if you want mouse support: > ums_load=3D"YES" > > > > > > > > vmxnet and pvscsi are not pre-compiled. However, they compile and > operate > > > > perfectly as expected without any modification on ARM. > > > > > > > > vmci has some x86 assembly that I've not dug into yet, so that is > currently > > > > not > > > > working. > > > > > > > > I've documented in detail, and continue to update as I test more > drivers > > > > over on > > > > my blog: > > > > > https://vincerants.com/freebsd-on-esxi-arm-fling-fixing-virtual-hardware/ > > > > > > > > The ESXi UI has a known bug in that it'll list some virtual hardwar= e > that > > > > doesn't actually exist inside of the VM. These are all legacy > devices, and > > > > should be easy enough to figure out. There are no virtual IDE/ATA o= r > PS2 > > > > ports. > > > > Keyboard and mouse now use the virtual USB controller instead. > > > > > > > > open-vm-tools has required a bit of work to make functional on > FreeBSD under > > > > ARM. There is currently a patch open with VMware with my changes to > the > > > > codebase > > > > to make it functional. To make things easy, I've uploaded some > pre-compiled > > > > binaries to my blog, as well as included steps to compile this > customized > > > > package yourself. The blog will also be updated as development > progresses > > > > upstream with VMware. > > > > > https://vincerants.com/open-vm-tools-on-freebsd-under-vmware-esxi-arm-fli= ng/ > > > > > > > > Xorg desktop runs, and is mostly functional. I've yet to get the > > > > xf86-input-vmmouse or xf86-video-vmware working. Xorg can use > sysmouse and > > > > the > > > > console framebuffer for rendering, and this is working okay for bas= ic > > > > testing > > > > right now. > > > > > > > > I also have an early dmesg, but this was from before getting most > drivers > > > > working. > > > > https://dmesgd.nycbug.org/index.cgi?do=3Dview&id=3D5693 > > > > > > > > I welcome any additional feedback on getting FreeBSD to 100% under > ESXi ARM! > > > > > > > > - Vince (darkain) > > > > _______________________________________________ > > > > freebsd-arm@freebsd.org mailing list > > > > https://lists.freebsd.org/mailman/listinfo/freebsd-arm > > > > To unsubscribe, send any mail to " > freebsd-arm-unsubscribe@freebsd.org" > From owner-freebsd-arm@freebsd.org Tue Oct 20 16:30:11 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 29BF44352A7; Tue, 20 Oct 2020 16:30:11 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-io1-xd41.google.com (mail-io1-xd41.google.com [IPv6:2607:f8b0:4864:20::d41]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CFzcf10Vhz4TkS; Tue, 20 Oct 2020 16:30:09 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: by mail-io1-xd41.google.com with SMTP id r9so1933133ioo.7; Tue, 20 Oct 2020 09:30:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=n9/mLNibkRyMFgkmn75fuNTABv/TFuMTIt4AZ2tHrQ4=; b=dhOJ9a3CXqIWXjWpFuZBtZMHg6lbRrKI9BqLE1UiOCO0X65H31TtiHCvGurgTftB+K E5PvRz+ryPewshQaiVjt456UDJDQ3/ET232CDWBo5s8cSj+RtMKa3OZ7ksRgMJSyDDWe gaWFiUXR8ZbAi2MWg0MzDMPxboC58LXUfywPoSGvJwYq67LrbMr7eMRTQxFCzhD/H+xa HClzGq9PGZqZvHIaf9qeuVptXClJDEBfUiF5XExTMwQtIEEbXG13uBtmd1cqtXilrcnz Vxy13HiJvtx/IrV3QpCNyhfylg3UKDtNlCplqtjnog6lM+GZI5HDvi4GIgMJR2Nxp1Y2 exXQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :references:mime-version:content-disposition:in-reply-to; bh=n9/mLNibkRyMFgkmn75fuNTABv/TFuMTIt4AZ2tHrQ4=; b=BotXb+nx8O/EaGte9bD3dFgGc1WDuZ9QLXwc7AQe6i3I0Qj/V0D0MFdyd/b9MskYjz qqBpCuvJ/fnv7zuz2wbTeEbjr2A7eA1wHZQQ6MEJ0LdXKYHfHnurwiBQFPEeEAFbOSVT 6doKdq/KJA8EHluaS2t4dxX7xeq+jWRqheQfVWoFJYZ+6ONGaSxWq/omMeqPwULmhiHt IbkNK6wVC4IPe0MgyLmBeLo2eASyvcrDMs2eh8zF7ppiNoczkuiDYbO4C0ASQmDEcfY1 XW3f4s1jynl4M5ESY8J8SbPP38N97Kgdbu0+J/DM9wpPtLgfhpgtBGdJvhLQ4u+KqjuW rZIQ== X-Gm-Message-State: AOAM532GeowNsd2Hyc7Tx8sJBIPDTIY3xi3IFoWrtUgwLCIeaTbCrv87 Ha2YQwMkvIFj+NAvPULLtx0oCBNZkp0= X-Google-Smtp-Source: ABdhPJzWFds/A+ilvfq2sewINcMpgTHv9wKTe3dKeOrLgNDI5K2OuO3dhJzBMBeKdCt1+QmY4KUpgw== X-Received: by 2002:a02:1349:: with SMTP id 70mr2835627jaz.130.1603211408493; Tue, 20 Oct 2020 09:30:08 -0700 (PDT) Received: from raichu (toroon0560w-lp130-01-174-88-77-103.dsl.bell.ca. [174.88.77.103]) by smtp.gmail.com with ESMTPSA id c9sm1951131iob.14.2020.10.20.09.30.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Oct 2020 09:30:07 -0700 (PDT) Sender: Mark Johnston Date: Tue, 20 Oct 2020 12:30:05 -0400 From: Mark Johnston To: bob prohaska Cc: mmel@freebsd.org, freebsd-current@freebsd.org, freebsd-arm@freebsd.org Subject: Re: panic: non-current pmap 0xffffa00020eab8f0 on Rpi3 Message-ID: <20201020163005.GD46122@raichu> References: <20201006021029.GA13260@www.zefox.net> <20201006133743.GA96285@raichu> <20201019203954.GC46122@raichu> <20201019230909.GA66675@www.zefox.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20201019230909.GA66675@www.zefox.net> X-Rspamd-Queue-Id: 4CFzcf10Vhz4TkS X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=dhOJ9a3C; dmarc=none; spf=pass (mx1.freebsd.org: domain of markjdb@gmail.com designates 2607:f8b0:4864:20::d41 as permitted sender) smtp.mailfrom=markjdb@gmail.com X-Spamd-Result: default: False [-2.31 / 15.00]; RCVD_TLS_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; MIME_GOOD(-0.10)[text/plain]; MID_RHS_NOT_FQDN(0.50)[]; DMARC_NA(0.00)[freebsd.org]; NEURAL_HAM_LONG(-1.02)[-1.018]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; NEURAL_HAM_SHORT(-0.58)[-0.578]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::d41:from]; NEURAL_HAM_MEDIUM(-1.02)[-1.015]; FORGED_SENDER(0.30)[markj@freebsd.org,markjdb@gmail.com]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[markj@freebsd.org,markjdb@gmail.com]; MAILMAN_DEST(0.00)[freebsd-current,freebsd-arm] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Oct 2020 16:30:11 -0000 On Mon, Oct 19, 2020 at 04:09:09PM -0700, bob prohaska wrote: > On Mon, Oct 19, 2020 at 04:39:54PM -0400, Mark Johnston wrote: > > > > I think vmspace_exit() should issue a release fence with the cmpset and > > an acquire fence when handling the refcnt == 1 case, but I don't see why > > that would make a difference here. So, if you can test a debug patch, > > this one will yield a bit more debug info. If you can provide access to > > a vmcore and kernel debug symbols, that'd be even better. > > > > I haven't seen an invalid pmap panic since the report of October 5th. > Your patch applied cleanly on the Pi3 running HEAD at r366780M, > the M being due to patches supplied by Kyle Evans applied to > M sys/arm/broadcom/bcm2835/bcm2835_mbox.c > M sys/arm/broadcom/bcm2835/bcm2835_sdhci.c > M sys/arm/broadcom/bcm2835/bcm2835_vcbus.c > M sys/arm/broadcom/bcm2835/bcm2835_vcbus.h > > AIUI, they're something to do with DMA for peripherals. They've > caused no obvious trouble, if you anticipate conflicts let me know > and I'll remove them There should be no need to remove them. > I've never seen either a vmcore file or debug symbols on this machine. > A sequence of instructions to generate the data needed would be helpful. I set up a RPi3 to try and repro this and have so far managed to trigger it once using Peter Holm's stress2 suite, so I'll keep investigating. I hadn't configured a dump device, but I was able to confirm from DDB that PCPU_GET(curpmap) == &vmspace0->vm_pmap. For future reference, https://www.freebsd.org/doc/en_US.ISO8859-1/books/developers-handbook/kerneldebug.html has some info on configuring a dump device. For the RPi I'm just using a USB stick as the dump device. > Thanks for reading! > > bob prohaska > From owner-freebsd-arm@freebsd.org Tue Oct 20 16:56:28 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id EEA05435BF1 for ; Tue, 20 Oct 2020 16:56:28 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: from mail-il1-f178.google.com (mail-il1-f178.google.com [209.85.166.178]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CG0C005w9z4WgM for ; Tue, 20 Oct 2020 16:56:27 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: by mail-il1-f178.google.com with SMTP id j8so3256741ilk.0 for ; Tue, 20 Oct 2020 09:56:27 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=VjBEGDgzU5CjQlFa+PuD3tTGZfmCyDp4CSUm4fTIFOc=; b=IqXjauyyoT/dByMBbY4qZjjQXN+KTW8TDUrxS8fHrQ4Vv+1pVcrywE9FUTXOJvqbW0 T4KsBVi+u7S9WY/YU2arl9ZWOcC2qRuMsyXLJR3VtY0x3F/3U6/IKgzJOnmVLoZeKFlp u9AnN49rjwBASb8IL6b7tuihvK1+tFgNFjnieXBxK4S72GsJeXzkPBkDAHMeRAJbBanC 4pcI9CVQ/03RjfNpJiIP8HUJrMf3qKCuOASwVSte5h/N0rh0zsqULbY8a04KhF8baqun zrQoU0mDeIIitJK4Pam2ra6gUTFrXMppOqAvzo+q0LJNK77VBCJgjypXFRgpX5bC397T scgw== X-Gm-Message-State: AOAM530/Owe473REyW9NrEVOCTCslEMDHY5AtBiKuhZJqq0f9NwkOud8 /xWCgGj1CbWQZ6Ar+VFyoEjmSMHnaH1ABy7UHFAgZm0Y X-Google-Smtp-Source: ABdhPJx+rDVQ2OSwWYcF37Kl1O+9NULKsYxnnTcwLjZ+abnWcPrM0ZwyxNbPOB6ruWU8wKuTzQhnHRL25QZg9oVrSYE= X-Received: by 2002:a92:6711:: with SMTP id b17mr2896193ilc.100.1603212986734; Tue, 20 Oct 2020 09:56:26 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Ed Maste Date: Tue, 20 Oct 2020 12:56:14 -0400 Message-ID: Subject: Re: FreeBSD under VMware ESXi ARM Fling To: Vincent Milum Jr Cc: "freebsd-arm@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4CG0C005w9z4WgM X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of carpeddiem@gmail.com designates 209.85.166.178 as permitted sender) smtp.mailfrom=carpeddiem@gmail.com X-Spamd-Result: default: False [-2.49 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; ARC_NA(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FREEFALL_USER(0.00)[carpeddiem]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; RCVD_TLS_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-arm@freebsd.org]; DMARC_NA(0.00)[freebsd.org]; NEURAL_HAM_LONG(-1.01)[-1.008]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-0.50)[-0.504]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[209.85.166.178:from]; NEURAL_HAM_MEDIUM(-0.98)[-0.983]; FORGED_SENDER(0.30)[emaste@freebsd.org,carpeddiem@gmail.com]; RWL_MAILSPIKE_POSSIBLE(0.00)[209.85.166.178:from]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; MIME_TRACE(0.00)[0:+]; FROM_NEQ_ENVFROM(0.00)[emaste@freebsd.org,carpeddiem@gmail.com]; MAILMAN_DEST(0.00)[freebsd-arm] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Oct 2020 16:56:29 -0000 On Mon, 19 Oct 2020 at 02:19, Vincent Milum Jr wrote: > > uhci for USB 2.0 is not enabled by default. Ensure your VM is initially set > up > for USB 3.1 so the virtual keyboard and mouse will function. This is easy > enough > to fix post install with: uhci_load="YES" We can definitely add this one to arm64 GENERIC. It looks like we've just faulted in the various *HCI USB interfaces as platforms with support appeared. > ums is the same as above. ESXi ARM uses USB instead of PS2 for the virtual > mouse, so add this post install if you want mouse support: ums_load="YES" ums is not included in amd64 GENERIC; it's presumably being automatically loaded when needed. Does this work here too? > vmxnet and pvscsi are not pre-compiled. However, they compile and operate > perfectly as expected without any modification on ARM. I've committed a change to build these as modules now. We can also add to GENERIC if necessary. From owner-freebsd-arm@freebsd.org Tue Oct 20 18:30:15 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3B59C437E2C for ; Tue, 20 Oct 2020 18:30:15 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qk1-x72c.google.com (mail-qk1-x72c.google.com [IPv6:2607:f8b0:4864:20::72c]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CG2HB0zVSz4cMt for ; Tue, 20 Oct 2020 18:30:13 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qk1-x72c.google.com with SMTP id 188so2485357qkk.12 for ; Tue, 20 Oct 2020 11:30:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=BUM5JXYtqtIvxNdEEFqmAHDcYQDGTvt6E6b1emSQH7g=; b=U2sG9jj452NwGEF2TqpeI1Ew6iy2Lyhr9/XjbHKcfAcsx0e+7L3tmO64uE9jS/ZXJm vrpkaTHV36p12WORC6Eu7Qx4fiOsh7SGUQb/OmuMSHoEXGMXTq2ohKelKoHrj0SQjMXk bReZ97oEMqbc50UvTKORHY30TSbEDHrU1BTnb8/lnmduUzpL0eqpqLhUa2zs6uL8QFji wzVP3REmC7aNhOnLk0flcpeH20b9DpYqdFtpMh32JA6UU7LZ1Q91McY+MKVokpwo+LMU Opxk9gYHh22oyN32cVtS/dQtx1o6Q4rPw6iabsenwqwGZZywezE93Fr9ERy7vVLtRkFn JXrQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=BUM5JXYtqtIvxNdEEFqmAHDcYQDGTvt6E6b1emSQH7g=; b=fNyEjVpZevLvjxp7cbhvIEzxVuynCY7vqzbkfNtCuiQ4eXWNDxwx5ac60CMDrZHXlL nvnXrdudR8p+jHdSB8YC4Qs6U6gbmYZDUYEjGny2WtSBWJkfOGs+3HybYeibR8PKpM0v 9SJ9UpW78HU3qT9uvdhOMj8nkPxCP3TYKZTbu6fG2ZJjVyLZICXznOr3F7fAyFJHgtyw MbX8tk3eQERw+l/LkvyAwQI5JurI2oLyeewmgySAJspLtA6rN9vmrr7IpuvPxVcpmXp0 jfwfGzUMM8LJ8NAzSASe2Ye2wF7Y6zNWoH86Myx4m37Zh6i0ya32gGExNxpHB0pPTfHq BQPA== X-Gm-Message-State: AOAM5306JvZhbI8TI7up937utQ4oz+z1c4+3SPYETgWYRBOllWHSNPac VhT2j2NuTAP5CJEF15+3/UXaalRrHsBUUfqDFaVztg== X-Google-Smtp-Source: ABdhPJx5q5ZcheXwTOn9rEyhpYUn+xNQ+mSZ/ViXdx3MDnSFqtXXruN3RRGwZwKLuKHcAEyhGm6ewjinNY3SrLe367k= X-Received: by 2002:a37:5ca:: with SMTP id 193mr4118565qkf.44.1603218612789; Tue, 20 Oct 2020 11:30:12 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Warner Losh Date: Tue, 20 Oct 2020 12:30:01 -0600 Message-ID: Subject: Re: FreeBSD under VMware ESXi ARM Fling To: Ed Maste Cc: Vincent Milum Jr , "freebsd-arm@freebsd.org" X-Rspamd-Queue-Id: 4CG2HB0zVSz4cMt X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20150623.gappssmtp.com header.s=20150623 header.b=U2sG9jj4; dmarc=none; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2607:f8b0:4864:20::72c) smtp.mailfrom=wlosh@bsdimp.com X-Spamd-Result: default: False [-1.90 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20150623.gappssmtp.com:s=20150623]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-1.02)[-1.017]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-arm@freebsd.org]; DMARC_NA(0.00)[bsdimp.com]; NEURAL_SPAM_SHORT(0.07)[0.066]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20150623.gappssmtp.com:+]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::72c:from]; NEURAL_HAM_MEDIUM(-0.95)[-0.953]; R_SPF_NA(0.00)[no SPF record]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCVD_COUNT_TWO(0.00)[2]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; MAILMAN_DEST(0.00)[freebsd-arm] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Oct 2020 18:30:15 -0000 On Tue, Oct 20, 2020 at 10:56 AM Ed Maste wrote: > On Mon, 19 Oct 2020 at 02:19, Vincent Milum Jr > wrote: > > > > uhci for USB 2.0 is not enabled by default. Ensure your VM is initially > set > > up > > for USB 3.1 so the virtual keyboard and mouse will function. This is easy > > enough > > to fix post install with: uhci_load="YES" > > We can definitely add this one to arm64 GENERIC. It looks like we've > just faulted in the various *HCI USB interfaces as platforms with > support appeared. > > > ums is the same as above. ESXi ARM uses USB instead of PS2 for the > virtual > > mouse, so add this post install if you want mouse support: ums_load="YES" > > ums is not included in amd64 GENERIC; it's presumably being > automatically loaded when needed. Does this work here too? > It's getting loaded for me on my aarch64 server via devmatch. > > vmxnet and pvscsi are not pre-compiled. However, they compile and operate > > perfectly as expected without any modification on ARM. > > I've committed a change to build these as modules now. We can also add > to GENERIC if necessary. > Yea, more modules is preferable, but in GENERIC if that path doesn't work. Warner From owner-freebsd-arm@freebsd.org Tue Oct 20 18:39:44 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id BE0924381A1 for ; Tue, 20 Oct 2020 18:39:44 +0000 (UTC) (envelope-from freebsd-arm@darkain.com) Received: from MTA-13-4.privateemail.com (mta-13-4.privateemail.com [198.54.127.109]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4CG2V81bhCz4d2v; Tue, 20 Oct 2020 18:39:43 +0000 (UTC) (envelope-from freebsd-arm@darkain.com) Received: from mta-13.privateemail.com (localhost [127.0.0.1]) by mta-13.privateemail.com (Postfix) with ESMTP id 9C8A680058; Tue, 20 Oct 2020 14:39:36 -0400 (EDT) Received: from mail-io1-f49.google.com (unknown [10.20.151.239]) by mta-13.privateemail.com (Postfix) with ESMTPA id 669DF80055; Tue, 20 Oct 2020 18:39:36 +0000 (UTC) Received: by mail-io1-f49.google.com with SMTP id y20so5040457iod.5; Tue, 20 Oct 2020 11:39:36 -0700 (PDT) X-Gm-Message-State: AOAM530/3hdVv9fY20MADzeHPMLNV+dS3V5kok5h3ajSBTLYGlt1YbEs nRwY3bYnThNLm5xqjILmMpBm4y6pAPuqD4mf0KU= X-Google-Smtp-Source: ABdhPJznnbbqAOrQxSlrdibp1RWNU1l/I4gFzAsIvc7wZ/JpiaiP8fRE8rbYwZuACrm0RgqosEsO60KBqv7R4FO5MEc= X-Received: by 2002:a6b:920b:: with SMTP id u11mr3224673iod.191.1603219175640; Tue, 20 Oct 2020 11:39:35 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Vincent Milum Jr Date: Tue, 20 Oct 2020 11:39:22 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: FreeBSD under VMware ESXi ARM Fling To: Ed Maste Cc: "freebsd-arm@freebsd.org" X-Virus-Scanned: ClamAV using ClamSMTP X-Rspamd-Queue-Id: 4CG2V81bhCz4d2v X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of freebsd-arm@darkain.com designates 198.54.127.109 as permitted sender) smtp.mailfrom=freebsd-arm@darkain.com X-Spamd-Result: default: False [-3.08 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:198.54.127.96/27]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; DMARC_NA(0.00)[darkain.com]; RWL_MAILSPIKE_VERYGOOD(0.00)[198.54.127.109:from]; NEURAL_HAM_LONG(-1.00)[-1.005]; RCVD_COUNT_THREE(0.00)[4]; NEURAL_HAM_MEDIUM(-0.99)[-0.990]; NEURAL_HAM_SHORT(-0.78)[-0.782]; RCPT_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:22612, ipnet:198.54.127.0/24, country:US]; RCVD_TLS_LAST(0.00)[]; MAILMAN_DEST(0.00)[freebsd-arm] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Oct 2020 18:39:44 -0000 ums isn't being loaded automatically. I'm not sure why. I've had to manually add it to loader.conf to get mouse support working at all. unrelated to vmware, a while back i was testing freebsd on a raspberry pi competitor board (forget which atm) that only had USB storage. it would boot to the point of switching to kernel mounting the file system, and failing to find the USB drive. I now suspect that this is also related to lack of *HCI (possibly specifically uhci), so I want to give that a try there too sometime soon. also, update on serial port. The vmware devs confirmed this is indeed available on ESXi ARM. It was late last night when I tested just a very basic connection, nothing using gdb yet. But this means we should have easy kernel remote debugging access by linking two VM's virtual serial ports together. On Tue, Oct 20, 2020 at 9:56 AM Ed Maste wrote: > On Mon, 19 Oct 2020 at 02:19, Vincent Milum Jr > wrote: > > > > uhci for USB 2.0 is not enabled by default. Ensure your VM is initially > set > > up > > for USB 3.1 so the virtual keyboard and mouse will function. This is easy > > enough > > to fix post install with: uhci_load="YES" > > We can definitely add this one to arm64 GENERIC. It looks like we've > just faulted in the various *HCI USB interfaces as platforms with > support appeared. > > > ums is the same as above. ESXi ARM uses USB instead of PS2 for the > virtual > > mouse, so add this post install if you want mouse support: ums_load="YES" > > ums is not included in amd64 GENERIC; it's presumably being > automatically loaded when needed. Does this work here too? > > > vmxnet and pvscsi are not pre-compiled. However, they compile and operate > > perfectly as expected without any modification on ARM. > > I've committed a change to build these as modules now. We can also add > to GENERIC if necessary. > From owner-freebsd-arm@freebsd.org Tue Oct 20 18:40:48 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 545A343858B for ; Tue, 20 Oct 2020 18:40:48 +0000 (UTC) (envelope-from crowston@protonmail.com) Received: from mail1.protonmail.ch (mail1.protonmail.ch [185.70.40.18]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "protonmail.com", Issuer "SwissSign Server Gold CA 2014 - G22" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CG2WM0SQdz4cmF for ; Tue, 20 Oct 2020 18:40:46 +0000 (UTC) (envelope-from crowston@protonmail.com) Date: Tue, 20 Oct 2020 18:40:29 +0000 To: Vincent Milum Jr From: Robert Crowston Cc: "freebsd-arm@freebsd.org" Reply-To: Robert Crowston Subject: Re: FreeBSD under VMware ESXi ARM Fling Message-ID: In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Status: No, score=-1.2 required=10.0 tests=ALL_TRUSTED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM shortcircuit=no autolearn=disabled version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on mailout.protonmail.ch X-Rspamd-Queue-Id: 4CG2WM0SQdz4cmF X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.95 / 15.00]; HAS_REPLYTO(0.00)[crowston@protonmail.com]; TO_DN_EQ_ADDR_SOME(0.00)[]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[protonmail.com]; R_SPF_ALLOW(-0.20)[+ip4:185.70.40.0/24]; DKIM_TRACE(0.00)[protonmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[protonmail.com,quarantine]; NEURAL_HAM_SHORT(-0.96)[-0.955]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[protonmail.com]; ASN(0.00)[asn:62371, ipnet:185.70.40.0/24, country:CH]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.987]; R_DKIM_ALLOW(-0.20)[protonmail.com:s=protonmail]; REPLYTO_EQ_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.01)[-1.012]; MIME_GOOD(-0.10)[text/plain]; FREEMAIL_REPLYTO(0.00)[protonmail.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[185.70.40.18:from]; MAILMAN_DEST(0.00)[freebsd-arm] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Oct 2020 18:40:48 -0000 > According to the ESXi devs, there is no SIO controller currently, and I b= elieve this is required for the virtual serial ports to work > https://flings.vmware.com/esxi-arm-edition/bugs/1104 I do think this a requirement. Indeed, a UART is detected in your dmesg: uart0: <16550 or compatible> mem 0x3fff03f8-0x3fff03ff irq 5 on ofwbus0 uart0: console (230400,n,8,1) Is it not exposed on the host? If we cannot debug, a verbose boot would at least add some colour. Could yo= u add boot_verbose=3D"YES" to /etc/loader.conf? I took a look at the instructions for getting this up and running on my own= Pi. It is certainly a lot of instructions. It is a shame they do not just = provide a .img I could flash to an sdcard. -- RHC. From owner-freebsd-arm@freebsd.org Tue Oct 20 18:48:02 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 898B44381F0 for ; Tue, 20 Oct 2020 18:48:02 +0000 (UTC) (envelope-from freebsd-arm@darkain.com) Received: from MTA-07-4.privateemail.com (mta-07-4.privateemail.com [68.65.122.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4CG2gj5BgLz4dCV for ; Tue, 20 Oct 2020 18:48:01 +0000 (UTC) (envelope-from freebsd-arm@darkain.com) Received: from MTA-07.privateemail.com (localhost [127.0.0.1]) by MTA-07.privateemail.com (Postfix) with ESMTP id E905F6004E for ; Tue, 20 Oct 2020 14:47:59 -0400 (EDT) Received: from mail-il1-f169.google.com (unknown [10.20.151.238]) by MTA-07.privateemail.com (Postfix) with ESMTPA id BC8CF60047 for ; Tue, 20 Oct 2020 18:47:59 +0000 (UTC) Received: by mail-il1-f169.google.com with SMTP id n5so3669282ile.7 for ; Tue, 20 Oct 2020 11:47:59 -0700 (PDT) X-Gm-Message-State: AOAM533XGMsADy8ucmNPgffF0qrOgRtqvtiZ6PVX/bso5IVb36PxjkQr kDd20L95OvcX/ZBobTHuC7votSqMDqapzP9CxYA= X-Google-Smtp-Source: ABdhPJxX3h0942kWBzPHgdtGmxxwpqa7EfOwfTjQHRfyfisSBW5uh0sAjb+RvvO5OT00MglCSWayeHzJwj1m+WbE1k0= X-Received: by 2002:a05:6e02:541:: with SMTP id i1mr2849826ils.156.1603219679085; Tue, 20 Oct 2020 11:47:59 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Vincent Milum Jr Date: Tue, 20 Oct 2020 11:47:45 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: FreeBSD under VMware ESXi ARM Fling To: Robert Crowston Cc: "freebsd-arm@freebsd.org" X-Virus-Scanned: ClamAV using ClamSMTP X-Rspamd-Queue-Id: 4CG2gj5BgLz4dCV X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of freebsd-arm@darkain.com designates 68.65.122.27 as permitted sender) smtp.mailfrom=freebsd-arm@darkain.com X-Spamd-Result: default: False [-2.95 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:68.65.122.0/27]; NEURAL_HAM_LONG(-1.01)[-1.007]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-arm@freebsd.org]; DMARC_NA(0.00)[darkain.com]; RCVD_COUNT_THREE(0.00)[4]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RWL_MAILSPIKE_VERYGOOD(0.00)[68.65.122.27:from]; NEURAL_HAM_SHORT(-0.64)[-0.638]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[68.65.122.27:from]; NEURAL_HAM_MEDIUM(-1.01)[-1.009]; FREEMAIL_TO(0.00)[protonmail.com]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:22612, ipnet:68.65.122.0/24, country:US]; RCVD_TLS_LAST(0.00)[]; MAILMAN_DEST(0.00)[freebsd-arm] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Oct 2020 18:48:02 -0000 Most of those instructions are just for "update the RPi's firmware" - if you're on a new enough firmware already, you can skip that entire section involving Raspberry Pi OS. After that, they just provide an installer image which can be used to install to either local storage, or remote network storage. It is being treated more like a typical x86 workflow rather than what we're more familiar with for ARM/SBC workflows. On Tue, Oct 20, 2020 at 11:40 AM Robert Crowston wrote: > > According to the ESXi devs, there is no SIO controller currently, and I > believe this is required for the virtual serial ports to work > > https://flings.vmware.com/esxi-arm-edition/bugs/1104 > > I do think this a requirement. Indeed, a UART is detected in your dmesg: > > uart0: <16550 or compatible> mem 0x3fff03f8-0x3fff03ff irq 5 on ofwbus0 > uart0: console (230400,n,8,1) > > Is it not exposed on the host? > > If we cannot debug, a verbose boot would at least add some colour. Could > you add boot_verbose="YES" to /etc/loader.conf? > > I took a look at the instructions for getting this up and running on my > own Pi. It is certainly a lot of instructions. It is a shame they do not > just provide a .img I could flash to an sdcard. > > -- RHC. > > > From owner-freebsd-arm@freebsd.org Tue Oct 20 18:55:20 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id CA40E438989; Tue, 20 Oct 2020 18:55:20 +0000 (UTC) (envelope-from fbsd@www.zefox.net) Received: from www.zefox.net (www.zefox.net [50.1.20.27]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "www.zefox.com", Issuer "www.zefox.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CG2r82FxPz4dbS; Tue, 20 Oct 2020 18:55:20 +0000 (UTC) (envelope-from fbsd@www.zefox.net) Received: from www.zefox.net (localhost [127.0.0.1]) by www.zefox.net (8.15.2/8.15.2) with ESMTPS id 09KItWYO069517 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Tue, 20 Oct 2020 11:55:32 -0700 (PDT) (envelope-from fbsd@www.zefox.net) Received: (from fbsd@localhost) by www.zefox.net (8.15.2/8.15.2/Submit) id 09KItWW4069516; Tue, 20 Oct 2020 11:55:32 -0700 (PDT) (envelope-from fbsd) Date: Tue, 20 Oct 2020 11:55:30 -0700 From: bob prohaska To: Mark Johnston Cc: mmel@freebsd.org, freebsd-current@freebsd.org, freebsd-arm@freebsd.org Subject: Re: panic: non-current pmap 0xffffa00020eab8f0 on Rpi3 Message-ID: <20201020185530.GA69476@www.zefox.net> References: <20201006021029.GA13260@www.zefox.net> <20201006133743.GA96285@raichu> <20201019203954.GC46122@raichu> <20201019230909.GA66675@www.zefox.net> <20201020163005.GD46122@raichu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20201020163005.GD46122@raichu> X-Rspamd-Queue-Id: 4CG2r82FxPz4dbS X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; ASN(0.00)[asn:7065, ipnet:50.1.16.0/20, country:US]; REPLY(-4.00)[] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Oct 2020 18:55:20 -0000 On Tue, Oct 20, 2020 at 12:30:05PM -0400, Mark Johnston wrote: > > I set up a RPi3 to try and repro this and have so far managed to trigger > it once using Peter Holm's stress2 suite, so I'll keep investigating. I > hadn't configured a dump device, but I was able to confirm from DDB that > PCPU_GET(curpmap) == &vmspace0->vm_pmap. > Is the invalid pmap fault related in any way to intensity of swap usage? That's easily adjusted using -j values building things like www/chromium. In the past, when I've reported crashes caused by stress2 I've observed a "that's inevitable" sort of response with some regularity. Panics when doing more normal things like make seem to stimulate greater interest. > For future reference, > https://www.freebsd.org/doc/en_US.ISO8859-1/books/developers-handbook/kerneldebug.html The pieces are all in place, but the machine doesn't seem to find core dumps when coming up after a crash. It does routinely issue "no core dumps found" during reboot, so it's looking. It it necessary to issue a dump command from inside the debugger? Thanks for writing! bob prohaska From owner-freebsd-arm@freebsd.org Wed Oct 21 16:07:55 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id BEEB244F3CA; Wed, 21 Oct 2020 16:07:55 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-qk1-x744.google.com (mail-qk1-x744.google.com [IPv6:2607:f8b0:4864:20::744]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CGb4W1Yvtz4TcP; Wed, 21 Oct 2020 16:07:54 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: by mail-qk1-x744.google.com with SMTP id q199so2485261qke.10; Wed, 21 Oct 2020 09:07:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=QGC+ixlMMLfVL7wRDp7w7Oz3gTx7eWRWfVASP2SOaTo=; b=ewgJ/2NmgYY/66l52FJkcKodb8vS4c0rE7OUpYyobGdiJ94O2PsBbiDgxTjJCJOCnZ pP6SIc6jiqVJiipfr36MWwrHi3RG35g52VCtxnoWxJ+iCcrfRYigVMenTa80wDb4Lriq 5M8E7fzdkpcvC2Ix1dhfWbUz6uNasuGREPOznFqsoCVpEH5SRV0OQ0Sx/vOMVRMy1uw8 6AQb3iMaU01hoLPlL5LzLZfx4Bu2l09sNczkpHWdSMzf05AtxvVxom9fLwXlAkrL6TvX 8cLMSffE4W3fcJBtUWoE0Sln/qg2n743YLtZsMU45V2RxQFGnf6tOCXRIvKIKcZsGLZk UCxQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :references:mime-version:content-disposition:in-reply-to; bh=QGC+ixlMMLfVL7wRDp7w7Oz3gTx7eWRWfVASP2SOaTo=; b=Fm1W+jy0TTI5VK9TpfabBsukWRMhVnXlEbJ5u7/4kjSPsUyyUQESHxiSOKgJWMMxYA X4dY8HHcYCMd3ytT8k3FyRh47g07XWI3LgcpkqssxHSUMJGbwcF14VGYDOoHYig6BZjh /mEid4ayA9/Crnz/xfSofv4HzvHWyI1UafuKihBlA9BPVRxw15KgGMjbMWg+9pZSjayq 5mNgaTm7eo4cUIhDMECpJIJmtF2xQ4XK1K9JwwrbtAyf6bwWNcylZjZDoiDTG9tVUr5f XDRhNqWUJILPdvY0oM9QFn5uloIL43kxhcexGlf504XRDvSA83ph3Eex0gzJUclfXYrI LkgQ== X-Gm-Message-State: AOAM533oxMO7tuZg0SrYR2441+zXzYk6gl2vYw8+rn4mmzYDucYuuyY6 /+2NBH2UX3tWZGWPtgtPPfKglOckf+A= X-Google-Smtp-Source: ABdhPJz53pgJfXX6Zlhg0/vQv45Z8c3LxlwLfhEoQ4q0NsPepGCgIUh19APmmnV6A11GaZo4cCncgg== X-Received: by 2002:a37:988:: with SMTP id 130mr3792711qkj.313.1603296474209; Wed, 21 Oct 2020 09:07:54 -0700 (PDT) Received: from raichu (toroon0560w-lp130-01-174-88-77-103.dsl.bell.ca. [174.88.77.103]) by smtp.gmail.com with ESMTPSA id l19sm1545486qkk.99.2020.10.21.09.07.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Oct 2020 09:07:53 -0700 (PDT) Sender: Mark Johnston Date: Wed, 21 Oct 2020 12:07:48 -0400 From: Mark Johnston To: bob prohaska Cc: mmel@freebsd.org, freebsd-current@freebsd.org, freebsd-arm@freebsd.org Subject: Re: panic: non-current pmap 0xffffa00020eab8f0 on Rpi3 Message-ID: <20201021160748.GA17318@raichu> References: <20201006021029.GA13260@www.zefox.net> <20201006133743.GA96285@raichu> <20201019203954.GC46122@raichu> <20201019230909.GA66675@www.zefox.net> <20201020163005.GD46122@raichu> <20201020185530.GA69476@www.zefox.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20201020185530.GA69476@www.zefox.net> X-Rspamd-Queue-Id: 4CGb4W1Yvtz4TcP X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=ewgJ/2Nm; dmarc=none; spf=pass (mx1.freebsd.org: domain of markjdb@gmail.com designates 2607:f8b0:4864:20::744 as permitted sender) smtp.mailfrom=markjdb@gmail.com X-Spamd-Result: default: False [-2.48 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; NEURAL_HAM_LONG(-0.99)[-0.986]; MIME_GOOD(-0.10)[text/plain]; MID_RHS_NOT_FQDN(0.50)[]; DMARC_NA(0.00)[freebsd.org]; TO_DN_SOME(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; NEURAL_HAM_SHORT(-0.78)[-0.775]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::744:from]; NEURAL_HAM_MEDIUM(-1.01)[-1.014]; FORGED_SENDER(0.30)[markj@freebsd.org,markjdb@gmail.com]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[markj@freebsd.org,markjdb@gmail.com]; MAILMAN_DEST(0.00)[freebsd-current,freebsd-arm] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Oct 2020 16:07:55 -0000 On Tue, Oct 20, 2020 at 11:55:30AM -0700, bob prohaska wrote: > On Tue, Oct 20, 2020 at 12:30:05PM -0400, Mark Johnston wrote: > > > > I set up a RPi3 to try and repro this and have so far managed to trigger > > it once using Peter Holm's stress2 suite, so I'll keep investigating. I > > hadn't configured a dump device, but I was able to confirm from DDB that > > PCPU_GET(curpmap) == &vmspace0->vm_pmap. > > > > Is the invalid pmap fault related in any way to intensity of swap usage? > That's easily adjusted using -j values building things like www/chromium. > In the past, when I've reported crashes caused by stress2 I've observed > a "that's inevitable" sort of response with some regularity. Panics when > doing more normal things like make seem to stimulate greater interest. I don't think there is any direct relation with swap usage. The one time I've reproduced this so far there was no swapping involved. I've noticed that stress2 runs on the RPi3 tend to abort or hang in various ways, but so far haven't seen any panics except the one we're chasing. > > For future reference, > > https://www.freebsd.org/doc/en_US.ISO8859-1/books/developers-handbook/kerneldebug.html > > The pieces are all in place, but the machine doesn't seem to find core > dumps when coming up after a crash. It does routinely issue "no core dumps > found" during reboot, so it's looking. It it necessary to issue a dump > command from inside the debugger? It depends: if debug.debugger_on_panic is set to 1, you will indeed need to issue a "dump" command from DDB before rebooting. Otherwise the system should automatically dump and reboot. You'll need to have "dumpdev" configured in rc.conf in order for vmcores to be recovered automatically. Otherwise savecore(8) can be run manually. > Thanks for writing! > > bob prohaska > From owner-freebsd-arm@freebsd.org Thu Oct 22 01:10:03 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5B08B436083 for ; Thu, 22 Oct 2020 01:10:03 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic311-25.consmr.mail.gq1.yahoo.com (sonic311-25.consmr.mail.gq1.yahoo.com [98.137.65.206]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4CGq623DlMz44Bs for ; Thu, 22 Oct 2020 01:10:01 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1603329000; bh=hi7dGKC2V8N+yI5LL9K6/9bg1+A11guTXvZo7/1hHqS=; h=From:Subject:Date:To; b=o+2e42Z+DRZhIif414Xtho3Ybuamw05bF5gD8vl9Km3KahmyKIMZ+fppJiW0l6rIsv9sJVLCD+ISOC7QzFqif2s+WOdBBTadP4IwsTcdJCZmnsZlU1eYARAU31HdpJYyrKJgrpfmGPAKv+pcMraH9e+ntyoaGek0KOGxZpZM2hifHLsg973qTaTyR4BSZ0+1ujel05WybZ+WYf8YFS7GJI+SD6B6esq+Ep5O4iC6n4CVvenXllNlOq8T45OlSJGHR2p6KIHjEwnjEEDQS6Eab1SOz3/HiyQ+DczuXnJAlO66BZ2sk4Pl+nPGYwkNsC6ZUFlESkYhpTIwSopFZJcSwQ== X-YMail-OSG: iut5b70VM1k3yDMuUb4iI73VkH7JT880LOprWr__eLwpfuzD20ktcLF_vn0s.Nk F20.VQEq9.8TokVo4kKqYYwXsnTL5Xf7i6g5CgkzE6OcbK9uWAOJYOlfwPnvAAxYdryu5pwewK1c TaRcPwUc5t6vTkLwcZHg0ytLpXceB8X0eCEIiOWqBfhOgZhtoBACTvhncgiXuhWFH1yhB90jgU0t XIbAl7B1J8UUkgUwSkz2qcMZTrAjflEBgqk1UrPuXSzwQPTvJsbE_XiY8ARWuxsqihHzJ1MCtvaL oX23Cy2NuD7rSD62.YM6zWkEqteSHUMJNXOIWxSxoKwndKT1tvD656DamL0oa0iR0D_T3QI9TF96 nrUDpGBO6O5xOD9OzajT48gook44k8rljHZIsWFdc9yzvw0o78267DxTQWelG2hrxY8f0VpbMORq OP6V27B.45tIGxbOmQ8pwV9nwS6fMVUDES7cHgvywuUwVGUmAZJmAqA3fCn8YQk9XRax8vOX5dkf a1r.ijFOsfG5pYYmpIOuh2ZXD0ZXYYIXR73HuQlbsIrBpkG2Q7DCzgkN692X8qH1dIFIBI33A3MG PMxNs7yWOg82PcY7xOYLqQRgd0UMySDg9m3TcuwfX9Xd.KaSV9qHDlVB6HE1SMfDyfmAA9braZN0 atDiv5NYvO_3izIxCPobErQuo08ONihEdP.pFxel93u0M2X4.OwKJ4Eah8nEkso6i0q6_aOi5NQ6 ASN2gedJ7.XHHlWD4gjD0A6Z__6tCQfNoNBW1ecGNfnPD9bxfSqRzaRpb00M.CH_bIVb8djzNfGY uyRERUQxSwVXN.lAsHbBjACFmkWnUsMTahsZz81VxtAme.246J9jpSBql0V1R2SJ09e3mBcn2Xxy jEqxp1etSoKz3nEfAj46AJh5hr9N3ignp.RVkFqskCwMXhDiYwHPM.dCnSz4hoRmgH4lbb7G5GKu Q6CZdmZtfm2uDraFV.QcX2o7R620TY6bw6ypu15psCxz8j1Lga4s6yHQpabxhjuj_mRqRRVp.30w HQUGN1IgYAmzc9p9oKebfASF9J11hQJ8paitsYoCi5Cy25wgrB6YVeX34v1GjFd1cYs_2_.z2byJ cdxiMddcdI5WAEj.WABVAgxToFyCTej3nTv31isGeaqGLh6z991dg_Ooa6cNX9w_yFTQJ42QKcKq xAyafa9m.vKKq8aQ53bzzRFsYOzz9eBOdL6qD.6za3cRREIsc7oXqaIdPWocYvfd2sp9uM9J6RK. HLZlXgabzgmsEGO_By82fqQBwUD9ULWgBiR9uWAbZqD1DoKeakjhKHK9TPiyB_DOWqCJfpGvd5TK YsLEIEFrZVTZIZvmrtP71bQRef_PpMcjeR7c9UvUTKhC4C3Vr9aRdHrl3NvLeKWj.npNrqiXwOJJ fBa_Dw5FjlPyT6IxPig6mhVpbAY2dMuLvS1MxFQmG7Mp5OO36PzRBdUmRyyt8xKK22bN8gZzCLoz GBcnbBaz_MQYqsWjbBWkyA6m4OKQBTSCtREi83ZKE6yTnc5NNsdzYwf8kuneg8Yu3VaDvhqtLOTY i5q7R3HCRGOYinA-- Received: from sonic.gate.mail.ne1.yahoo.com by sonic311.consmr.mail.gq1.yahoo.com with HTTP; Thu, 22 Oct 2020 01:09:59 +0000 Received: by smtp422.mail.ne1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID 6275f4bd04afd11e229daf75e054c294; Thu, 22 Oct 2020 01:09:57 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.4\)) Subject: FYI: -x assembler-with-cpp -m32 use during buildkernel vs. gcc9 based cross-builds, at least for targeting aarch64 via use of some armv6 notation; more Message-Id: <4252FFCA-21C4-4C42-B3D4-8C213031D689@yahoo.com> Date: Wed, 21 Oct 2020 18:09:56 -0700 To: freebsd-arm X-Mailer: Apple Mail (2.3608.120.23.2.4) References: <4252FFCA-21C4-4C42-B3D4-8C213031D689.ref@yahoo.com> X-Rspamd-Queue-Id: 4CGq623DlMz44Bs X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.55 / 15.00]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_SHORT(-0.03)[-0.034]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.01)[-1.009]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.01)[-1.007]; MIME_GOOD(-0.10)[text/plain]; RCPT_COUNT_ONE(0.00)[1]; RCVD_IN_DNSWL_NONE(0.00)[98.137.65.206:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.65.206:from]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-arm] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Oct 2020 01:10:03 -0000 I've not tried to build via a gcc* toochain in a long time so I decided = to try again, picking aarch64 (tailored to -mcpu=3Dcortext-a72 targeting in my = context) as a way to learn the status. (I did not expect it to finish, even with = some effort to adjust simple things.) After taking care of some source code notational issues that the attempt ran into, it ran into the following . . . aarch64-unknown-freebsd13.0-gcc9: error: unrecognized command line = option '-m32' . . . _ERROR_CMD=3D'/usr/local/bin/aarch64-unknown-freebsd13.0-gcc9 = -mcpu=3Dcortex-a72 = --sysroot=3D/usr/obj/cortexA72_xtoolchain-gcc/arm64.aarch64/usr/src/arm64.= aarch64/tmp -B/usr/local/aarch64-unknown-freebsd13.0/bin/ -x = assembler-with-cpp -m32 -shared -nostdinc -nostdlib = -Wl,-T/usr/src/sys/compat/cloudabi/cloudabi_vdso.lds = /usr/src/sys/contrib/cloudabi/cloudabi_vdso_armv6_on_64bit.S -o = cloudabi32_vdso.o; In other words, it appears that the build environment for buildkernel = has not stuck to command line handling that is handled by both the llvm*/clang* = toochain(s) and the gcc* toolchain(s): It expects aarch64 targeting to accept -x assembler-with-cpp -m32 in order to assemble armv6 notation ( such as = for cloudabi_vdso_armv6_on_64bit.S ). gcc* does not support that way of = specifying such behavior. The context is head -r365932 . For reference: the prior notational differences that I worked around to see "what would be next" were (up to E-mail whitespace variations): # svnlite diff /usr/src/sys/arm64/arm64/locore.S Index: /usr/src/sys/arm64/arm64/locore.S =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- /usr/src/sys/arm64/arm64/locore.S (revision 365932) +++ /usr/src/sys/arm64/arm64/locore.S (working copy) @@ -296,10 +296,10 @@ cmp x2, #(ID_AA64PFR0_GIC_CPUIF_EN >> ID_AA64PFR0_GIC_SHIFT) b.ne 2f =20 - mrs x2, icc_sre_el2 + mrs x2, s3_4_c12_c9_5 /* icc_sre_el2 */ orr x2, x2, #ICC_SRE_EL2_EN /* Enable access from insecure = EL1 */ orr x2, x2, #ICC_SRE_EL2_SRE /* Enable system = registers */ - msr icc_sre_el2, x2 + msr s3_4_c12_c9_5, x2 /* icc_sre_el2 */ 2: =20 /* Set the address to return to our return address */ (I've not tested if llvm/clang allows the alternate notation used above yet.) machine/db_machdep.h has a && that appears to me to be wrong: should instead be || . Look for the "/* <<<=3D=3D=3D was: && */" comment in the below. #define is_load_instr(ins) ((((ins) & 0x3b000000u) =3D=3D = 0x18000000u) || /* literal */ \ (((ins) & 0x3f400000u) =3D=3D = 0x08400000u) || /* exclusive */ \ (((ins) & 0x3bc00000u) =3D=3D = 0x28400000u) || /* no-allocate pair */ \ ((((ins) & 0x3b200c00u) =3D=3D = 0x38000400u) && \ (((ins) & 0x3be00c00u) !=3D = 0x38000400u) && \ (((ins) & 0xffe00c00u) !=3D = 0x3c800400u)) || /* immediate post-indexed */ \ ((((ins) & 0x3b200c00u) =3D=3D = 0x38000c00u) && \ (((ins) & 0x3be00c00u) !=3D = 0x38000c00u) && \ (((ins) & 0xffe00c00u) !=3D = 0x3c800c00u)) || /* immediate pre-indexed */ \ ((((ins) & 0x3b200c00u) =3D=3D = 0x38200800u) && \ (((ins) & 0x3be00c00u) !=3D = 0x38200800u) && \ (((ins) & 0xffe00c00u) !=3D = 0x3ca00c80u)) || /* register offset */ \ ((((ins) & 0x3b200c00u) =3D=3D = 0x38000800u) && \ (((ins) & 0x3be00c00u) !=3D = 0x38000800u)) || /* unprivileged */ \ ((((ins) & 0x3b200c00u) =3D=3D = 0x38000000u) && \ (((ins) & 0x3be00c00u) !=3D = 0x38000000u) && \ (((ins) & 0xffe00c00u) !=3D = 0x3c800000u)) || /* unscaled immediate */ \ ((((ins) & 0x3b000000u) =3D=3D = 0x39000000u) && \ (((ins) & 0x3bc00000u) !=3D = 0x39000000u) && \ (((ins) & 0xffc00000u) !=3D = 0x3d800000u)) || /* <<<=3D=3D=3D was: && */ /* unsigned immediate */ \ (((ins) & 0x3bc00000u) =3D=3D = 0x28400000u) || /* pair (offset) */ \ (((ins) & 0x3bc00000u) =3D=3D = 0x28c00000u) || /* pair (post-indexed) */ \ I will note that the above lists (((ins) & 0x3bc00000u) =3D=3D = 0x28400000u) twice, once for "no-allocate pair" and once for "pair (offset)", such = that the 2nd use can not change the overall expression true vs. false status. But the following has that same issue in a simpler context: #define is_store_instr(ins) ((((ins) & 0x3f400000u) =3D=3D = 0x08000000u) || /* exclusive */ \ (((ins) & 0x3bc00000u) =3D=3D = 0x28000000u) || /* no-allocate pair */ \ ((((ins) & 0x3be00c00u) =3D=3D = 0x38000400u) || \ (((ins) & 0xffe00c00u) =3D=3D = 0x3c800400u)) || /* immediate post-indexed */ \ ((((ins) & 0x3be00c00u) =3D=3D = 0x38000c00u) || \ (((ins) & 0xffe00c00u) =3D=3D = 0x3c800c00u)) || /* immediate pre-indexed */ \ ((((ins) & 0x3be00c00u) =3D=3D = 0x38200800u) || \ (((ins) & 0xffe00c00u) =3D=3D = 0x3ca00800u)) || /* register offset */ \ (((ins) & 0x3be00c00u) =3D=3D = 0x38000800u) || /* unprivileged */ \ ((((ins) & 0x3be00c00u) =3D=3D = 0x38000000u) || \ (((ins) & 0xffe00c00u) =3D=3D = 0x3c800000u)) || /* unscaled immediate */ \ ((((ins) & 0x3bc00000u) =3D=3D = 0x39000000u) || \ (((ins) & 0xffc00000u) =3D=3D = 0x3d800000u)) || /* unsigned immediate */ \ (((ins) & 0x3bc00000u) =3D=3D = 0x28000000u) || /* pair (offset) */ \ (((ins) & 0x3bc00000u) =3D=3D = 0x28800000u) || /* pair (post-indexed) */ \ (((ins) & 0x3bc00000u) =3D=3D = 0x29800000u)) /* pair (pre-indexed) */ So the redundancy does not seem surprising but just happens to be textually just-after the && vs. || issue. The && vs. || was noticed via gcc complaining about the lack of explicit ()'s to be explicit about && vs. || binding power in interpreting the expression as it was (&& use). =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-arm@freebsd.org Fri Oct 23 13:42:57 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8E898447129 for ; Fri, 23 Oct 2020 13:42:57 +0000 (UTC) (envelope-from gbe@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CHlmK3GbMz47mb for ; Fri, 23 Oct 2020 13:42:57 +0000 (UTC) (envelope-from gbe@freebsd.org) Received: from localhost (p4fd3a1f5.dip0.t-ipconnect.de [79.211.161.245]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) (Authenticated sender: gbe) by smtp.freebsd.org (Postfix) with ESMTPSA id D60832330A for ; Fri, 23 Oct 2020 13:42:56 +0000 (UTC) (envelope-from gbe@freebsd.org) Date: Fri, 23 Oct 2020 15:42:56 +0200 From: Gordon Bergling To: freebsd-arm@freebsd.org Subject: Packages from arm64 Message-ID: <20201023134256.GA91582@lion.0xfce3.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Url: X-Operating-System: FreeBSD 12.2-STABLE amd64 X-Host-Uptime: 3:38PM up 1 day, 22:01, 4 users, load averages: 0.26, 0.49, 0.51 X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Oct 2020 13:42:57 -0000 Hi, does anyone know in which frequency packages for arm64 are generated? I have seen that for -CURRENT on amd64 the packages are getting updated very quickly, but on arm64 I haven't seen any updated packages since at least 4 weeks. --Gordon From owner-freebsd-arm@freebsd.org Fri Oct 23 16:32:29 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6AD6144A5CC; Fri, 23 Oct 2020 16:32:29 +0000 (UTC) (envelope-from melounmichal@gmail.com) Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com [IPv6:2a00:1450:4864:20::32b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CHqWw1K5Cz4LC2; Fri, 23 Oct 2020 16:32:28 +0000 (UTC) (envelope-from melounmichal@gmail.com) Received: by mail-wm1-x32b.google.com with SMTP id c16so2422422wmd.2; Fri, 23 Oct 2020 09:32:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:reply-to:subject:to:cc:references:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=LSOUjr7+2UhBbH5lMeRHhjAn/gqy5kvPoW84yNGTZAA=; b=JScP73GioJFIFR/nia9G09N5NvCWikDEuGB+eQQQ+A7uHXSO3ZS0Kue/MoJT2SOsqr bRUR9XjX8DqaxWt5ySF03/aMuqOowM+8gNUHVeTPbGZMZEe6y5gL7kAA/Kh7tn4bTjGt 9Aht1Qm0PoTCCW+2cAPBTbN6NIhnIWtE7K7C0MmrX6ckRExczFZtGSo4kewszJfuLu18 9lVK9BEoNYTCHc9KSY0ioGLKY1iNTxyoS/EEf1F5uzmswV3ZtN3VJtY8ghajAy/g6ScF 1tByGk7ovIpdOeLhwtSc5kquooyLbXGj8VNaw3ptXm6n8eZt6s+EAZGhQafRxvMHbjiz hu1A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:reply-to:subject:to:cc:references :message-id:date:user-agent:mime-version:in-reply-to :content-language:content-transfer-encoding; bh=LSOUjr7+2UhBbH5lMeRHhjAn/gqy5kvPoW84yNGTZAA=; b=oojZWDiTuSiA72HfTbyeQvfzdtKdWK/UGgFms7+c1cvRhBCSSnWdSRsTBqdg6h+inO J2Nm8hOxVnZN59Us7ys/O+BtEyMpRhpDEoLZBcRIxs6kDq1wvPFuuoAL3EO+mSPqVXdN 51VBt3n1J7Jxrara9eY16qq8knQ3yFz8RJGaIkTGVeQVKJcnIfMXJWpyGdLmqkYkS5K7 PCmhFIaDak1hQYI3wSJ8l8WgvKNAkY3uYj3eCuLGtBDcpj5nIW/AjncOfOju9ZCEh+tv GZHuTCT2mtVN9/X4+B7iH7RHvmckYVVNbPgO3ZfTwJSL6owkNKdH2/9JhGWR5AYJR6Ku D3Uw== X-Gm-Message-State: AOAM530t2ok5VYMNuo+9NUze7iEKo6dM0kK3iHyqFZw52FGJMeSbKjUa zUxvMVOGVxW7excNyb4UhDR8mpsRILY= X-Google-Smtp-Source: ABdhPJz2oGzdzc88aTvyy51uChxw46YxZAdpTt8eOt+SYv9OB/9wUlxfhZuHVZFleFXhFAcCdULuug== X-Received: by 2002:a7b:c114:: with SMTP id w20mr57783wmi.105.1603470746483; Fri, 23 Oct 2020 09:32:26 -0700 (PDT) Received: from [88.208.79.100] (halouny.humusoft.cz. [88.208.79.100]) by smtp.gmail.com with ESMTPSA id x1sm3961916wrl.41.2020.10.23.09.32.25 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 23 Oct 2020 09:32:25 -0700 (PDT) Sender: Michal Meloun From: Michal Meloun X-Google-Original-From: Michal Meloun Reply-To: mmel@freebsd.org Subject: Re: panic: non-current pmap 0xffffa00020eab8f0 on Rpi3 To: Mark Johnston Cc: bob prohaska , freebsd-current@freebsd.org, freebsd-arm@freebsd.org References: <20201006021029.GA13260@www.zefox.net> <20201006133743.GA96285@raichu> <20201019203954.GC46122@raichu> Message-ID: <454e1e9f-e839-8961-2ae1-9ddd86f1cefd@freebsd.org> Date: Fri, 23 Oct 2020 18:32:25 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.4.0 MIME-Version: 1.0 In-Reply-To: <20201019203954.GC46122@raichu> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4CHqWw1K5Cz4LC2 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=JScP73Gi; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of melounmichal@gmail.com designates 2a00:1450:4864:20::32b as permitted sender) smtp.mailfrom=melounmichal@gmail.com X-Spamd-Result: default: False [-2.43 / 15.00]; HAS_REPLYTO(0.00)[mmel@freebsd.org]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-0.46)[-0.459]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; TAGGED_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.994]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; NEURAL_HAM_LONG(-0.98)[-0.976]; MIME_GOOD(-0.10)[text/plain]; REPLYTO_DOM_NEQ_FROM_DOM(0.00)[]; MID_RHS_MATCH_TO(1.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::32b:from]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-arm,freebsd-current] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Oct 2020 16:32:29 -0000 On 19.10.2020 22:39, Mark Johnston wrote: > On Fri, Oct 16, 2020 at 11:53:56AM +0200, Michal Meloun wrote: >> >> >> On 06.10.2020 15:37, Mark Johnston wrote: >>> On Mon, Oct 05, 2020 at 07:10:29PM -0700, bob prohaska wrote: >>>> Still seeing non-current pmap panics on the Pi3, this time a B+ running >>>> 13.0-CURRENT (GENERIC-MMCCAM) #0 71e02448ffb-c271826(master) >>>> during a -j4 buildworld. The backtrace reports >>>> >>>> panic: non-current pmap 0xffffa00020eab8f0 >>> >>> Could you show the output of "show procvm" from the debugger? >> >> I see same panic too, in my case its very rare - typical scenario is >> rebuild of kf5 ports (~250, 2 days of full load). Any idea how to debug >> this? >> Michal > > I suspect that there is some race involving the pmap switching in > vmspace_exit(), but I can't see it. In the example below, presumably > process 22604 on CPU 0 is also exiting? Could you show the backtrace?> > It would also be useful to see the value of PCPU_GET(curpmap) at the > time of the panic. I'm not sure if there's a way to get that from DDB, > but I suspect it should be equal to &vmspace0->vm_pmap. Mark, I think that I found problem. The PCPU_GET() is not (and is not supposed to be) an atomic operation, it expects that thread is at least pinned. This is not true for pmap_remove_pages() - so I think that the KASSERT is racy and shoud be removed (or at least covered by sched_pin()/sched_unpin() pair). What do you think? > > I think vmspace_exit() should issue a release fence with the cmpset and > an acquire fence when handling the refcnt == 1 case, Yep, true, fully agree. Michal but I don't see why > that would make a difference here. So, if you can test a debug patch, > this one will yield a bit more debug info. If you can provide access to > a vmcore and kernel debug symbols, that'd be even better. > > diff --git a/sys/arm64/arm64/pmap.c b/sys/arm64/arm64/pmap.c > index 284f00b3cc0d..3c53ae3b4c1e 100644 > --- a/sys/arm64/arm64/pmap.c > +++ b/sys/arm64/arm64/pmap.c > @@ -4838,7 +4838,8 @@ pmap_remove_pages(pmap_t pmap) > int allfree, field, freed, idx, lvl; > vm_paddr_t pa; > > - KASSERT(pmap == PCPU_GET(curpmap), ("non-current pmap %p", pmap)); > + KASSERT(pmap == PCPU_GET(curpmap), > + ("non-current pmap %p %p", pmap, PCPU_GET(curpmap))); > > lock = NULL; > > diff --git a/sys/vm/vm_map.c b/sys/vm/vm_map.c > index c20005ae64cf..0ad415e3b88c 100644 > --- a/sys/vm/vm_map.c > +++ b/sys/vm/vm_map.c > @@ -358,7 +358,10 @@ vmspace_exit(struct thread *td) > p = td->td_proc; > vm = p->p_vmspace; > atomic_add_int(&vmspace0.vm_refcnt, 1); > - refcnt = vm->vm_refcnt; > + refcnt = atomic_load_int(&vm->vm_refcnt); > + > + KASSERT(vmspace_pmap(vm) == PCPU_GET(curpmap), > + ("non-current pmap %p %p", pmap, PCPU_GET(curpmap))); > do { > if (refcnt > 1 && p->p_vmspace != &vmspace0) { > /* Switch now since other proc might free vmspace */ > From owner-freebsd-arm@freebsd.org Fri Oct 23 18:08:10 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0B48E44C44A for ; Fri, 23 Oct 2020 18:08:10 +0000 (UTC) (envelope-from ronald-lists@klop.ws) Received: from smarthost1.greenhost.nl (smarthost1.greenhost.nl [195.190.28.88]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4CHsfJ5V2Lz4RK1; Fri, 23 Oct 2020 18:08:08 +0000 (UTC) (envelope-from ronald-lists@klop.ws) Received: from smtp.greenhost.nl ([213.108.110.112]) by smarthost1.greenhost.nl with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1kW1Tv-0005sY-Nk; Fri, 23 Oct 2020 20:08:06 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=klop.ws; s=mail; h=In-Reply-To:Message-ID:From:Content-Transfer-Encoding:MIME-Version: Date:References:Subject:To:Content-Type:Sender:Reply-To:Cc:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=imtOl3JeahXMCdAsdR6XV3eekp1SNt4Ul/bPMu3/sLs=; b=NbZJMDy9meOuwI0hXRp75CxyHI 79Xp7MfM44s/bQztrYXjByqeO5jIyX5whJPAM1JQRiB85AjaqFGDUw7uMEbQvfpVTAW0Cbs8KGgia v3NgavRiUQ1IKDKVNlYRLUPnTBqfv/zClykwBV/+0LbSLGzuuoQ/a58lcdKobQOaqoko=; Content-Type: text/plain; charset=utf-8; format=flowed; delsp=yes To: freebsd-arm@freebsd.org, "Gordon Bergling" Subject: Re: Packages from arm64 References: <20201023134256.GA91582@lion.0xfce3.net> Date: Fri, 23 Oct 2020 20:08:01 +0200 MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: "Ronald Klop" Message-ID: In-Reply-To: <20201023134256.GA91582@lion.0xfce3.net> User-Agent: Opera Mail/12.16 (FreeBSD) X-Authenticated-As-Hash: 398f5522cb258ce43cb679602f8cfe8b62a256d1 X-Virus-Scanned: by clamav at smarthost1.samage.net X-Spam-Level: / X-Spam-Score: -0.4 X-Spam-Status: No, score=-0.4 required=5.0 tests=ALL_TRUSTED, BAYES_50, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF autolearn=disabled version=3.4.2 X-Scan-Signature: c74461a82029b6293650421ecb57b64a X-Rspamd-Queue-Id: 4CHsfJ5V2Lz4RK1 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=klop.ws header.s=mail header.b=NbZJMDy9; dmarc=none; spf=pass (mx1.freebsd.org: domain of ronald-lists@klop.ws designates 195.190.28.88 as permitted sender) smtp.mailfrom=ronald-lists@klop.ws X-Spamd-Result: default: False [-2.10 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[klop.ws:s=mail]; NEURAL_HAM_MEDIUM(-0.96)[-0.957]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:195.190.28.64/27]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[klop.ws]; NEURAL_HAM_LONG(-0.96)[-0.963]; RWL_MAILSPIKE_GOOD(0.00)[195.190.28.88:from]; DKIM_TRACE(0.00)[klop.ws:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[195.190.28.88:from]; NEURAL_HAM_SHORT(-0.18)[-0.181]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; MID_RHS_NOT_FQDN(0.50)[]; ASN(0.00)[asn:47172, ipnet:195.190.28.0/24, country:NL]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-arm] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Oct 2020 18:08:10 -0000 On Fri, 23 Oct 2020 15:42:56 +0200, Gordon Bergling wrote: > Hi, > > does anyone know in which frequency packages for arm64 are generated? > > I have seen that for -CURRENT on amd64 the packages are getting updated > very > quickly, but on arm64 I haven't seen any updated packages since at least > 4 weeks. The server doing the arm64 package building is broken. I am not involved in it so I don't have more information. Regards, Ronald. > --Gordon > _______________________________________________ > freebsd-arm@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-arm > To unsubscribe, send any mail to "freebsd-arm-unsubscribe@freebsd.org" From owner-freebsd-arm@freebsd.org Sat Oct 24 19:37:42 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id F3744452917; Sat, 24 Oct 2020 19:37:42 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-qk1-x731.google.com (mail-qk1-x731.google.com [IPv6:2607:f8b0:4864:20::731]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CJWbB1dPfz417P; Sat, 24 Oct 2020 19:37:41 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: by mail-qk1-x731.google.com with SMTP id b69so4835244qkg.8; Sat, 24 Oct 2020 12:37:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=e85WFxtcM8vw1yb7YkMm1o5VYF29zO92gdczz6GqUVo=; b=EFeZcjIGkFkLtKupT7s2YZrUXD0SjX8eL/0P8bKf8LV3K2lPz7easGl5IVb7zRxe7E onPOX3RC60iWQ8LHGq81bw0nwpRoVEBxV21mKVI97GFALcjQGoPhQnHS5wygZJf+5aL7 O7VhINyBaoYguGSt6v1XfG/A3Tgq7U8f4sJoCPoi4rQMaZ4tRxaOFnOMfINV4noMg/Jc Lt9U2x3i3uVuiFnvXBNwv4NSBTiywclqn4oZ2bzagXD0fWsZhUB02868EFrmY6r2quQy 8or7J02Q2gRIBDDC0hXu30U1yNQzdMn0BFAb7l3NgvDwTiJlYixR25mxw16sIf00evY8 q3rQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :references:mime-version:content-disposition:in-reply-to; bh=e85WFxtcM8vw1yb7YkMm1o5VYF29zO92gdczz6GqUVo=; b=TMEieT6QqIoF4ZeSrCIsSuxmShA6YTTpSZqr7hfv/QPipYcnQ5+KmdU388SaCja7WL qX/NdrLxRHWIN2kcMnS4b5YI6gferBFvQngRThLRXwP8TrQL9zngB4wINzlYfr9QYasO zDFSnkgOUwtuZIhEdafQCF8rzXnEldDbHnOFS5te+eSpe2PVB3+wtmo3gAi1C1dTkzTw x0snomkTZU9ZsGzHn1TANxTA+9+74yM7lM1fu+f7iiIRqD8kGF2Zd3tnAYGfNyTmZ+G5 A8x4v1h13N7S30sUrDFGsZn7f8VVyUWIRGkeg93Pp6nKUpV90u7sfbkS8fWPN01r0fdI rAYA== X-Gm-Message-State: AOAM533VpC6exYKdWFGvcRNpVGhdy7WGRD0g5xambk1vAq3GP/YppB/9 PkBx+B6RgSCSLIzDZnc6qBlQMl7uZs8= X-Google-Smtp-Source: ABdhPJw8yryx1PcxuEZ0tBIni/FP8pKTPzv24Z5nk9VgnRnMtizA6lETmKy8ca9JwwYHiOz/o0ScHA== X-Received: by 2002:a05:620a:142d:: with SMTP id k13mr8389579qkj.315.1603568260845; Sat, 24 Oct 2020 12:37:40 -0700 (PDT) Received: from raichu (toroon0560w-lp130-01-174-88-77-103.dsl.bell.ca. [174.88.77.103]) by smtp.gmail.com with ESMTPSA id f3sm3349914qkl.134.2020.10.24.12.37.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 24 Oct 2020 12:37:39 -0700 (PDT) Sender: Mark Johnston Date: Sat, 24 Oct 2020 15:37:35 -0400 From: Mark Johnston To: mmel@freebsd.org Cc: bob prohaska , freebsd-current@freebsd.org, freebsd-arm@freebsd.org Subject: Re: panic: non-current pmap 0xffffa00020eab8f0 on Rpi3 Message-ID: <20201024193735.GA7755@raichu> References: <20201006021029.GA13260@www.zefox.net> <20201006133743.GA96285@raichu> <20201019203954.GC46122@raichu> <454e1e9f-e839-8961-2ae1-9ddd86f1cefd@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <454e1e9f-e839-8961-2ae1-9ddd86f1cefd@freebsd.org> X-Rspamd-Queue-Id: 4CJWbB1dPfz417P X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=EFeZcjIG; dmarc=none; spf=pass (mx1.freebsd.org: domain of markjdb@gmail.com designates 2607:f8b0:4864:20::731 as permitted sender) smtp.mailfrom=markjdb@gmail.com X-Spamd-Result: default: False [-2.01 / 15.00]; RCVD_TLS_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; MIME_GOOD(-0.10)[text/plain]; MID_RHS_NOT_FQDN(0.50)[]; DMARC_NA(0.00)[freebsd.org]; NEURAL_HAM_LONG(-0.97)[-0.974]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; NEURAL_HAM_SHORT(-0.28)[-0.284]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::731:from]; NEURAL_HAM_MEDIUM(-1.05)[-1.053]; FORGED_SENDER(0.30)[markj@freebsd.org,markjdb@gmail.com]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[markj@freebsd.org,markjdb@gmail.com]; MAILMAN_DEST(0.00)[freebsd-current,freebsd-arm] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Oct 2020 19:37:43 -0000 On Fri, Oct 23, 2020 at 06:32:25PM +0200, Michal Meloun wrote: > > > On 19.10.2020 22:39, Mark Johnston wrote: > > On Fri, Oct 16, 2020 at 11:53:56AM +0200, Michal Meloun wrote: > >> > >> > >> On 06.10.2020 15:37, Mark Johnston wrote: > >>> On Mon, Oct 05, 2020 at 07:10:29PM -0700, bob prohaska wrote: > >>>> Still seeing non-current pmap panics on the Pi3, this time a B+ running > >>>> 13.0-CURRENT (GENERIC-MMCCAM) #0 71e02448ffb-c271826(master) > >>>> during a -j4 buildworld. The backtrace reports > >>>> > >>>> panic: non-current pmap 0xffffa00020eab8f0 > >>> > >>> Could you show the output of "show procvm" from the debugger? > >> > >> I see same panic too, in my case its very rare - typical scenario is > >> rebuild of kf5 ports (~250, 2 days of full load). Any idea how to debug > >> this? > >> Michal > > > > I suspect that there is some race involving the pmap switching in > > vmspace_exit(), but I can't see it. In the example below, presumably > > process 22604 on CPU 0 is also exiting? Could you show the backtrace?> > > It would also be useful to see the value of PCPU_GET(curpmap) at the > > time of the panic. I'm not sure if there's a way to get that from DDB, > > but I suspect it should be equal to &vmspace0->vm_pmap. > Mark, > I think that I found problem. > The PCPU_GET() is not (and is not supposed to be) an atomic operation, > it expects that thread is at least pinned. > This is not true for pmap_remove_pages() - so I think that the KASSERT > is racy and shoud be removed (or at least covered by > sched_pin()/sched_unpin() pair). > What do you think? I think you're right. On amd64 curpmap is loaded using a single instruction so the assertion happens to work properly. On arm64 we have: 0xffff0000007ff138 <+32>: mov x8, x18 0xffff0000007ff13c <+36>: ldr x8, [x8, #216] 0xffff0000007ff140 <+40>: mov x26, x0 0xffff0000007ff144 <+44>: cmp x8, x0 Though, it looks like arm64's PCPU_GET could be modified to combine the first two instructions. To fix it, we could perhaps change the KASSERT to verify that pmap == vmspace_pmap(curthread->td_proc->p_vmspace). The various implementations of pmap_remove_pages() have different flavours of the same check and it would be nice to unify them. Using sched_pin() would also be fine I think. > > I think vmspace_exit() should issue a release fence with the cmpset and > > an acquire fence when handling the refcnt == 1 case, > Yep, true, fully agree. Alan pointed out in the review that pmap_remove_pages() acquires the pmap lock, which I missed, so I don't think the extra barriers are necessary after all.