From owner-freebsd-arm@freebsd.org Wed Jul 8 15:21:41 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 21D16365601 for ; Wed, 8 Jul 2020 15:21:41 +0000 (UTC) (envelope-from dan.kotowski@a9development.com) Received: from mail-40131.protonmail.ch (mail-40131.protonmail.ch [185.70.40.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "protonmail.com", Issuer "SwissSign Server Gold CA 2014 - G22" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4B231c21Zwz4BnM for ; Wed, 8 Jul 2020 15:21:40 +0000 (UTC) (envelope-from dan.kotowski@a9development.com) Date: Wed, 08 Jul 2020 15:21:30 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=a9development.com; s=protonmail; t=1594221696; bh=69LvxhHU2WnQd1K0V2CC/pEGmNGucRNs5/GAcuw2idQ=; h=Date:To:From:Cc:Reply-To:Subject:In-Reply-To:References:From; b=Z7Esfz3kqjx0W+Estt+yjHO2POl+R0yKlyoBx4MuxKZadnBQpJs4OTlgxoNQV9NRK Xd9HxJYPDNOr5xTzPMDNeQWSIs0GmmA5fRBvu0fB6JfuKF1Hqck4rsNLq3JPeVyNWH DnFzmoCyFZBHz1ED8Fcu5BnU0dAL6nw42zpGXt+Q= To: Dan Kotowski From: Dan Kotowski Cc: "greg@unrelenting.technology" , freebsd-arm Reply-To: Dan Kotowski Subject: Re: FreeBSD on Layerscape/QorIQ LX2160X Message-ID: In-Reply-To: References: =?us-ascii?Q?<82UUBttUqS5j5wotn5ibAhp3w3JveSof3dsBLqW68NkaOu1xc6txd62UJeyJgV6hk9mLNYqhAJDyEej=5FPPtiv=5Fps9vROdUI529pKfxp4lbs=3D@a9development.com>__=5F=5F=5F=5F=5F<71481BF5-3972-45A3-8287-FCEB1FCCDC41@unrelenting.technology>____<8a3f78ddd5136ef22c59e9f7b1b23ca6@unrelenting.technology>__?= MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Status: No, score=-1.2 required=7.0 tests=ALL_TRUSTED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF shortcircuit=no autolearn=disabled version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on mail.protonmail.ch X-Rspamd-Queue-Id: 4B231c21Zwz4BnM X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=a9development.com header.s=protonmail header.b=Z7Esfz3k; dmarc=pass (policy=none) header.from=a9development.com; spf=pass (mx1.freebsd.org: domain of dan.kotowski@a9development.com designates 185.70.40.131 as permitted sender) smtp.mailfrom=dan.kotowski@a9development.com X-Spamd-Result: default: False [-3.86 / 15.00]; HAS_REPLYTO(0.00)[dan.kotowski@a9development.com]; TO_DN_EQ_ADDR_SOME(0.00)[]; R_DKIM_ALLOW(-0.20)[a9development.com:s=protonmail]; REPLYTO_EQ_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:185.70.40.0/24]; MIME_GOOD(-0.10)[text/plain]; ARC_NA(0.00)[]; RWL_MAILSPIKE_GOOD(0.00)[185.70.40.131:from]; NEURAL_HAM_LONG(-0.98)[-0.984]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[a9development.com:+]; DMARC_POLICY_ALLOW(-0.50)[a9development.com,none]; NEURAL_HAM_SHORT(-0.76)[-0.757]; NEURAL_HAM_MEDIUM(-1.02)[-1.021]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:62371, ipnet:185.70.40.0/24, country:CH]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[185.70.40.131:from] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Jul 2020 15:21:41 -0000 > > > > In the old firmware, under the Root Complex Nodes, `Output referenc= e: 0x30` points to the ITS Node > > > > with node offset 0x30. > > > > Then in the new firmware, under the Root Copmlex Nodes, `Output ref= erence: 0x48` points to the SMMU > > > > with node offset 0x48. > > > > Is this what you meant when you said everything is "behind the SMMU= "? > > > > Yes. > > > > What patches are we applying right now? I think some new builds wou= ld be good, including one with > > > > all the stuff we've fixed - like AHCI - but with the NXP PCIe code = so I can test on the old > > > > firmware. If I'm keeping track correctly, this includes: > > > > > > > > - D20835: enable tagged pointers > > > > - D20974: Port sbsawdt drive from NetBSD > > > > - D21017: armv8crypto: add AES-XTS support > > > > - D24423: arm/pmu: add ACPI attachment, more FDT names > > > > > > > > These are not directly related to running on NXP, just random impro= vements :) > > > > > > > > - D25145: acpi_resource: support multiple IRQs > > > > - D25157: ahci_generic: add quirk for NXP0004 > > > > - D25179: acpi_iort: fix mapping end calculation > > > > > > > > Yes, these three. > > > > Here's the pci_layerscape patch: > > > > https://github.com/DankBSD/base/commit/c1ea44aa33b29f74daed89eee82b= 3dfeb105d376.patch > > > > If we haven't tried it + acpi_iort fix before, which is quite likel= y, maybe that's a combination > > > > that would work. > > > > (I remember when we tried it only, there was only the same interrup= t problem as usual..) > > > > It's honestly kinda weird that "old" FW requires the custom control= ler access while "new" FW > > > > requires not doing it >_< > > > > > > Any tips on where to add verbosity to dmesg during boot? You had a ha= ndful of extras in there to > > > print things things from the IORT. > > > > That thing is here: > > https://github.com/freebsd/freebsd/blob/6cee1596c05e8a9ab64812444627b61= c584ca6bc/sys/arm64/acpica/acpi_iort.c#L169 > > > > > Also seeing that NVMe hangs in nvme_ctrlr_start_config_hook, > > > maybe we want to add some verbosity to sys/dev/nvme/nvme_ctrlr.c to s= ee what it's waiting for? > > > > We know it's an interrupt.. everything is waiting for an interrupt. > > Well, at least we're stable enough to buildworld and buildkernel on-syste= m! :partyparrot: > > Waiting on a new SSD, but building on a USB uSD: > > -------------------------------------------------------------------------= --------------------------------------------------------------------- > > > > > World build completed on Wed Jul 8 02:47:59 UTC 2020 > > > > World built in 6897 seconds, ncpu: 16, make -j16 > > Forgot to copy off the buildkernel time, but it was just under 900 second= s. > > Interestingly, we have real temp readouts from thermal zones, which is ne= w. Did I add something back in that we had previously removed? Here's something: https://gist.github.com/agrajag9/11efe00514513100232999e0= a9ec612c NVMe is working. It mapped an interrupt. I have no idea what changed to mak= e this work other than compiling my own kernel. I applied the 7 patches abo= ve and added a bunch of my own debug printfs (dmesg.boot lines beginning wi= th A9DEBUG), but it's the same POC ECAM firmware we've been using for a whi= le. I'll test with the HBA later this week - it's currently in use elsehwer= e in my homelab doing zfs send|receive things.