From owner-freebsd-arm@freebsd.org Fri Jun 5 11:46:06 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A8076328446 for ; Fri, 5 Jun 2020 11:46:06 +0000 (UTC) (envelope-from dan.kotowski@a9development.com) Received: from mail-40134.protonmail.ch (mail-40134.protonmail.ch [185.70.40.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "protonmail.com", Issuer "SwissSign Server Gold CA 2014 - G22" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49dgp54F9Dz4pjx for ; Fri, 5 Jun 2020 11:46:05 +0000 (UTC) (envelope-from dan.kotowski@a9development.com) Date: Fri, 05 Jun 2020 11:45:57 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=a9development.com; s=protonmail; t=1591357561; bh=kjfbwzQfr9awUkkv7qWPixl/CSmYEZZmuM7hNVHWu/U=; h=Date:To:From:Cc:Reply-To:Subject:In-Reply-To:References:From; b=cnlXV3Ch65ZCAc7IJMgGi6SQMFhqFNiT8kWgjOEZil4OeuKb77kGm1DxuQMzC/hIY 8JeCFZBjKonaZ3DqBFIgfvbZrdCMnRCwmvqIH3B+rgvMZCrue5RQOqmVWzVNRtqKTT EoohdRAihLcehCp7JHszvMiiPiKabFDFP7PKxbgo= To: "greg@unrelenting.technology" From: Dan Kotowski Cc: freebsd-arm Reply-To: Dan Kotowski Subject: Re: FreeBSD on Layerscape/QorIQ LX2160X Message-ID: In-Reply-To: References: <8951311F-77F7-40B8-AEA0-F8CBCB1A05DE@yahoo.com> <4ad62e6669044f82e71a9d86fd493356@unrelenting.technology> <31D3FA64-8296-4CA5-92A2-F7FE7C4AE981@unrelenting.technology> <32d1c173d986884efb9b28932c0ead52@unrelenting.technology> <5e1b4bfe845e62bbcd8b827fa37f2b98@unrelenting.technology> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Spam-Status: No, score=-1.2 required=7.0 tests=ALL_TRUSTED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF shortcircuit=no autolearn=disabled version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on mail.protonmail.ch X-Rspamd-Queue-Id: 49dgp54F9Dz4pjx X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=a9development.com header.s=protonmail header.b=cnlXV3Ch; dmarc=pass (policy=none) header.from=a9development.com; spf=pass (mx1.freebsd.org: domain of dan.kotowski@a9development.com designates 185.70.40.134 as permitted sender) smtp.mailfrom=dan.kotowski@a9development.com X-Spamd-Result: default: False [-3.60 / 15.00]; HAS_REPLYTO(0.00)[dan.kotowski@a9development.com]; TO_DN_EQ_ADDR_SOME(0.00)[]; R_DKIM_ALLOW(-0.20)[a9development.com:s=protonmail]; REPLYTO_EQ_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:185.70.40.0/24]; NEURAL_HAM_LONG(-0.98)[-0.980]; MIME_GOOD(-0.10)[text/plain]; ARC_NA(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[185.70.40.134:from]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[a9development.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[a9development.com,none]; NEURAL_HAM_SHORT(-0.51)[-0.514]; NEURAL_HAM_MEDIUM(-1.01)[-1.009]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:62371, ipnet:185.70.40.0/24, country:CH]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[185.70.40.134:from] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jun 2020 11:46:06 -0000 > > > Oops, I've been adding some debug prints to gicv2 not gicv3. > > > One thing I noticed is the interesting irq number the nvme admin queu= e gets.. > > > it's the same as gic_nirqs. > > > gic0: SPIs: 288, IDs: 65535 > > > nvme0: attempting to allocate 17 MSI-X vectors (33 supported) > > > nvme0: using IRQs 21-37 for MSI-X > > > acpi0: allocating via sysres: res 0xfffffd0000726280, start 21 + coun= t 1 - 1 =3D? end 21 > > > intr_setup_irq(): irq 288 add handler error 0 on nvme0 > > > maybe that's just like.. the first ITS handled interrupt? > > > (funnily enough, NetBSD lists MSIs as IRQs starting from 8192) > > > More SATA debugging: https://send.firefox.com/download/9de5357a2e58ed= d9/#s9ZaU_k2NHlO-tLdyGk5iA > > > > https://gist.github.com/agrajag9/1aaf18f48ee883b917907dd417803711 > > We have SATA! > > Wow that was silly. > > FreeBSD did not support multiple interrupts in an ACPI device, at all. > Since forever, nobody else has needed this, apparently. > It's really stupid that the error was returned this quietly >_< > > In the build below, it would try to attach all interrupts, not just the f= irst one. > If that succeeds, I'll post that as a patch. > > > NVMe and mps still cause hangs, but progress is progress! > > Well I didn't say I've tried to actually fix PCIe interrupts yet, > but I did add some debug logging that seems to be useful. > Namely, IORT stuff. IORT is the ACPI table describing how interrupts flow= . > > Aaaand this is concerning: > > iort: Found PCI RC node with segment and device ID. its? 1 smmu3? 0 smmu?= 1 | nxtid 6400 rid 256 > iort: entry lookup fail > > Wait a second, SMMU, that's not what IORT says the PCI nodes output to! > > https://github.com/SolidRun/edk2-platforms/blob/eec706c2d693be0b3793d9180= e7d1a4813a526cf/Silicon/NXP > LX2160A/AcpiTables/Iort.aslc > > .PciRcNode[0] =3D { .PciRcIdMapping =3D { ... > .OutputReference =3D OFFSET_OF (NXP_EFI_ACPI_6_0_IO_REMAPPING_TABLE, ItsN= ode), }, > > Looks like our IORT parsing might be busted! > > The new build should output more info and force usage of the ITS node. > So if that was the issue, and I did the forcing correctly, you should hav= e PCIe interrupts working now :) > > (btw: NetBSD for now doesn't care about OutputReference at all, lol) > > https://send.firefox.com/download/d655b27c2bd73021/#O9tR6di9HbS8FHp0isgFk= A https://gist.github.com/agrajag9/749f504dcac741e902f87c2ea03ae9ac >From jnettlet: BEGIN QUOTE the V1 silicon has a SATA errata. I have partially worked around the errat= a moving some configuration into the firmware, but I still need to sort out= the remaining bits. Sometimes just unplugging and replugging your SATA ca= ble will help to reseat the connector and help. Also it is very sensitive= to marginal SATA cables. END QUOTE I played around a bit and could create and destroy GPTs, make filesystems, = read and write to said filesystems, and hot-plug even seems to work. But I'= m not sure if jon's comment really explains the AHCI error we're still seei= ng: ``` (aprobe3:ahcich3:0:15:0): NOP FLUSHQUEUE. ACB: 00 00 00 00 00 00 00 00 00 0= 0 00 00 (aprobe3:ahcich3:0:15:0): CAM status: Command timeout (aprobe3:ahcich3:0:15:0): Error 5, Retries exhausted ```