From nobody Thu Jun 8 22:54:29 2023 X-Original-To: wireless@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Qcfg54tZXz4bpnc for ; Thu, 8 Jun 2023 22:54:57 +0000 (UTC) (envelope-from lists@jnielsen.net) Received: from webmail5.jnielsen.net (webmail5.jnielsen.net [IPv6:2607:f170:34:11::b0]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "mail.freebsdsolutions.net", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Qcfg35hLvz3Mhr; Thu, 8 Jun 2023 22:54:55 +0000 (UTC) (envelope-from lists@jnielsen.net) Authentication-Results: mx1.freebsd.org; dkim=none; spf=pass (mx1.freebsd.org: domain of lists@jnielsen.net designates 2607:f170:34:11::b0 as permitted sender) smtp.mailfrom=lists@jnielsen.net; dmarc=none Received: from smtpclient.apple ([50.207.241.62]) (authenticated bits=0) by webmail5.jnielsen.net (8.17.1/8.17.1) with ESMTPSA id 358Msi6u068226 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 8 Jun 2023 16:54:47 -0600 (MDT) (envelope-from lists@jnielsen.net) X-Authentication-Warning: webmail5.jnielsen.net: Host [50.207.241.62] claimed to be smtpclient.apple Content-Type: text/plain; charset=utf-8 List-Id: Discussions List-Archive: https://lists.freebsd.org/archives/freebsd-wireless List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-wireless@freebsd.org X-BeenThere: freebsd-wireless@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.600.7\)) Subject: Re: Help me grok the ath(4) device attach code From: John Nielsen In-Reply-To: <8CEDC118-5FCA-434D-B7F7-B0C4A7AC3ACC@jnielsen.net> Date: Thu, 8 Jun 2023 16:54:29 -0600 Cc: "Bjoern A. Zeeb" , "wireless@freebsd.org" Content-Transfer-Encoding: quoted-printable Message-Id: <3AD06513-B42D-4C65-83AC-F9AC588C0E59@jnielsen.net> References: <49AEA1CB-FA85-432F-89D7-8C49B5F3A344@jnielsen.net> <85BF3AB2-2EBF-4398-A507-ABA35505A56C@jnielsen.net> <5264D290-EB82-4D24-A812-85E3E6B5C88E@jnielsen.net> <8CEDC118-5FCA-434D-B7F7-B0C4A7AC3ACC@jnielsen.net> To: Adrian Chadd X-Mailer: Apple Mail (2.3731.600.7) X-Spamd-Result: default: False [-2.79 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.99)[-0.991]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+mx]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[jnielsen.net]; MLMMJ_DEST(0.00)[wireless@freebsd.org]; R_DKIM_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:6364, ipnet:2607:f170:30::/44, country:US]; MIME_TRACE(0.00)[0:+]; BLOCKLISTDE_FAIL(0.00)[50.207.241.62:server fail,2607:f170:34:11::b0:server fail]; TO_DN_EQ_ADDR_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; RCPT_COUNT_THREE(0.00)[3]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_SOME(0.00)[]; HAS_XAW(0.00)[]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; RCVD_TLS_ALL(0.00)[] X-Rspamd-Queue-Id: 4Qcfg35hLvz3Mhr X-Spamd-Bar: -- X-ThisMailContainsUnwantedMimeParts: N > On Jun 1, 2023, at 11:28 PM, John Nielsen wrote: >=20 >> On May 31, 2023, at 1:52 PM, John Nielsen wrote: >>=20 >>> On May 30, 2023, at 11:17 PM, Adrian Chadd = wrote: >>>=20 >>> On Tue, 30 May 2023 at 22:12, John Nielsen = wrote: >>>> On May 30, 2023, at 10:56 PM, Adrian Chadd = wrote: >>>>=20 >>>> On Tue, 30 May 2023 at 20:56, John Nielsen = wrote: >>>> > On May 30, 2023, at 8:02 PM, Adrian Chadd = wrote: >>>> >=20 >>>> > Err, if it's coming up w/ that MAC then it's not finding and = attaching right to the OTP/EEPROM calibration information. That's the = big red flag that it in general won't work correctly. >>>> >=20 >>>> > Can you provide the rest of the ath_hal messages? I'd like to see = what it's saying during boot around it checking the EEPROM/OTP contents. = It's possible there's some work around required for this NIC. >>>>=20 >>>> He speaks! Thanks for taking the time. I just realized that = ath_hal_printf doesn=E2=80=99t prepend =E2=80=9Cath%d=E2=80=9D so I=E2=80=99= ve been missing those messages when grep-ing. Here=E2=80=99s the whole = snippet: >>>>=20 >>>> ath0: mem 0xf7a00000-0xf7a7ffff at device = 0.0 on pci4 >>>> ar9300_flash_map: unimplemented for now >>>> Restoring Cal data from DRAM >>>> Restoring Cal data from EEPROM >>>> Restoring Cal data from Flash >>>> Restoring Cal data from Flash >>>> Restoring Cal data from OTP >>>> ar9300_eeprom_restore_internal[4338] No vaid CAL, calling default = template >>>> ar9300_hw_attach: ar9300_eeprom_attach returned 0 >>>>=20 >>>> Yeah, this bit right here is the problem. It's not finding a valid = calibration. >>>=20 >>>=20 >>>> oh err, is there a wifi enable/disable switch or something? maybe = it's asserted and somehow it's mucking up the NIC? >>>=20 >>> There is a physical switch and it=E2=80=99s in the =E2=80=9Cenable=E2=80= =9D position. >>>=20 >>>> I wonder what ath9k is doing here? Is there some weird pci based = workaround/flag for the given NIC PCI id? >>>=20 >>> That was the first breadcrumb BZ threw me but I can=E2=80=99t find = anything. There are some .driver_data hints for adjacent subdevice IDs = but none for this one (Dell 0x020d) in either FreeBSD or Linux that I = could find. >>>=20 >>> [snip] >>> There is this commit in ath9k which mentions an alternative EEPROM = address, but I=E2=80=99m not sure if that=E2=80=99s relevant. =46rom = what I can tell the probe should succeed at the normal base_address = 0x3ff instead of needing to try the =E2=80=9C4k=E2=80=9D one 0xfff. >>> = https://git.kernel.org/pub/scm/linux/kernel/git/kvalo/ath.git/commit/drive= rs/net/wireless/ath/ath9k?id=3D528782ecf59f7bab2f1368628a479f49be59b512 >>>=20 >>> Yeah i'd try that. It'd be nice if I knew that the NIC used OTP or = EEPROM though. >>>=20 >>> There's known issues with all the Atheros chips (sigh) with how the = EEPROM and PCIe bus reset .. interact. >>> (If the bus reset is too short then the EEPROM state machine gets = stuck and nothing gets read.) It makes debugging this hard because the = NIC itself will work in another device fine, because it's the BIOS/ACPI = code. :( >>=20 >>=20 >> The 4k EEPROM read didn=E2=80=99t work. But I did notice that where = it says it=E2=80=99s restoring Cal data from OTP it actually just does = an EEPROM read again. Shouldn=E2=80=99t this line be a call to = ar9300_otp_read() instead of ar9300_eeprom_restore_internal_address()? >>=20 >> = https://cgit.freebsd.org/src/tree/sys/contrib/dev/ath/ath_hal/ar9300/ar930= 0_eeprom.c#n4292 >>=20 >> Otherwise it doesn=E2=80=99t use the 0x14000 (0x30000 for some cards) = OTP offset as a starting point. >=20 >=20 > I=E2=80=99d love to know if I=E2=80=99m up in the night about my = theory above about the OTP call not actually happening. In the meantime = I=E2=80=99ll carry on either trying to find/build a Linux kernel with = CONFIG_ATH_DEBUG enabled that I can boot on the laptop (to see which = EEPROM retrieval/check actually succeeds) and/or try to reimplement bits = from the ath9k ar9300_eeprom_restore_internal() to match its behavior on = FreeBSD. > ( = https://git.kernel.org/pub/scm/linux/kernel/git/kvalo/ath.git/tree/drivers= /net/wireless/ath/ath9k/ar9003_eeprom.c#n3266 ) After learning more than I planned to about archiso and building kernel = packages for Arch I managed to boot the laptop on a Linux kernel with = CONFIG_ATH_DEBUG enabled and set as ath9k.debug=3D0x4 to get the EEPROM = debug output. And.. it looks a lot more normal than I would have = guessed. The EEPROM is located, identified and loaded from the default = 0x3ff address (first try!) and everything just goes smoothly. Full = output below. That at least tells me where to focus in the FreeBSD code but as always = I=E2=80=99d love some input or advice from Adrian or anyone who=E2=80=99s = looked at this more than I have. Thanks! -JN =3D=3D=3D dmesg|grep ath output =3D=3D=3D [ 0.039398] Kernel command line: = BOOT_IMAGE=3D/arch/boot/x86_64/vmlinuz-linux-jn archisobasedir=3Darch = archisodevice=3DUUID=3D2023-06-08-21-40-20-00 ath9k.debug=3D0x4 [ 17.692752] ath9k 0000:04:00.0: enabling device (0000 -> 0002) [ 17.696711] ath: phy0: Trying EEPROM access at Address 0x03ff [ 17.697238] ath: phy0: Found valid EEPROM data [ 17.697763] ath: phy0: Found block at 3ff: code=3D3 ref=3D4 = length=3D794 major=3D4 minor=3D6 [ 17.803208] ath: phy0: checksum a5cc a5cc [ 17.803214] ath: phy0: restore eeprom 0: block, reference 4, length = 794 [ 17.803217] ath: phy0: Restore at 0: spot=3D2 offset=3D2 length=3D22 [ 17.803219] ath: phy0: Restore at 24: spot=3D28 offset=3D4 length=3D1 [ 17.803222] ath: phy0: Restore at 27: spot=3D35 offset=3D6 length=3D1 [ 17.803224] ath: phy0: Restore at 30: spot=3D43 offset=3D7 length=3D1 [ 17.803226] ath: phy0: Restore at 33: spot=3D48 offset=3D4 length=3D31 [ 17.803228] ath: phy0: Restore at 66: spot=3D106 offset=3D27 = length=3D10 [ 17.803230] ath: phy0: Restore at 78: spot=3D128 offset=3D12 length=3D8= [ 17.803232] ath: phy0: Restore at 88: spot=3D141 offset=3D5 length=3D3 [ 17.803234] ath: phy0: Restore at 93: spot=3D147 offset=3D3 length=3D3 [ 17.803236] ath: phy0: Restore at 98: spot=3D153 offset=3D3 length=3D3 [ 17.803238] ath: phy0: Restore at 103: spot=3D159 offset=3D3 length=3D3= [ 17.803240] ath: phy0: Restore at 108: spot=3D165 offset=3D3 length=3D3= [ 17.803241] ath: phy0: Restore at 113: spot=3D171 offset=3D3 length=3D3= [ 17.803243] ath: phy0: Restore at 118: spot=3D205 offset=3D31 = length=3D31 [ 17.803245] ath: phy0: Restore at 151: spot=3D240 offset=3D4 = length=3D10 [ 17.803247] ath: phy0: Restore at 163: spot=3D254 offset=3D4 = length=3D10 [ 17.803249] ath: phy0: Restore at 175: spot=3D268 offset=3D4 = length=3D10 [ 17.803251] ath: phy0: Restore at 187: spot=3D282 offset=3D4 = length=3D10 [ 17.803253] ath: phy0: Restore at 199: spot=3D296 offset=3D4 = length=3D10 [ 17.803255] ath: phy0: Restore at 211: spot=3D328 offset=3D22 = length=3D9 [ 17.803257] ath: phy0: Restore at 222: spot=3D344 offset=3D7 length=3D9= [ 17.803259] ath: phy0: Restore at 233: spot=3D356 offset=3D3 = length=3D79 [ 17.803261] ath: phy0: Restore at 314: spot=3D438 offset=3D3 length=3D5= [ 17.803263] ath: phy0: Restore at 321: spot=3D479 offset=3D36 = length=3D2 [ 17.803265] ath: phy0: Restore at 325: spot=3D489 offset=3D8 = length=3D25 [ 17.803267] ath: phy0: Restore at 352: spot=3D517 offset=3D3 length=3D3= [ 17.803269] ath: phy0: Restore at 357: spot=3D523 offset=3D3 length=3D3= [ 17.803271] ath: phy0: Restore at 362: spot=3D529 offset=3D3 length=3D3= [ 17.803273] ath: phy0: Restore at 367: spot=3D535 offset=3D3 length=3D3= [ 17.803275] ath: phy0: Restore at 372: spot=3D541 offset=3D3 length=3D3= [ 17.803277] ath: phy0: Restore at 377: spot=3D547 offset=3D3 length=3D3= [ 17.803279] ath: phy0: Restore at 382: spot=3D553 offset=3D3 length=3D3= [ 17.803280] ath: phy0: Restore at 387: spot=3D559 offset=3D3 length=3D3= [ 17.803282] ath: phy0: Restore at 392: spot=3D565 offset=3D3 length=3D3= [ 17.803284] ath: phy0: Restore at 397: spot=3D571 offset=3D3 length=3D3= [ 17.803286] ath: phy0: Restore at 402: spot=3D577 offset=3D3 length=3D3= [ 17.803288] ath: phy0: Restore at 407: spot=3D583 offset=3D3 length=3D3= [ 17.803290] ath: phy0: Restore at 412: spot=3D589 offset=3D3 length=3D3= [ 17.803292] ath: phy0: Restore at 417: spot=3D595 offset=3D3 length=3D3= [ 17.803294] ath: phy0: Restore at 422: spot=3D601 offset=3D3 length=3D3= [ 17.803296] ath: phy0: Restore at 427: spot=3D678 offset=3D74 = length=3D43 [ 17.803298] ath: phy0: Restore at 472: spot=3D725 offset=3D4 = length=3D10 [ 17.803300] ath: phy0: Restore at 484: spot=3D739 offset=3D4 = length=3D10 [ 17.803302] ath: phy0: Restore at 496: spot=3D753 offset=3D4 = length=3D10 [ 17.803304] ath: phy0: Restore at 508: spot=3D767 offset=3D4 = length=3D10 [ 17.803306] ath: phy0: Restore at 520: spot=3D781 offset=3D4 = length=3D10 [ 17.803308] ath: phy0: Restore at 532: spot=3D795 offset=3D4 = length=3D10 [ 17.803309] ath: phy0: Restore at 544: spot=3D809 offset=3D4 = length=3D10 [ 17.803311] ath: phy0: Restore at 556: spot=3D823 offset=3D4 = length=3D10 [ 17.803313] ath: phy0: Restore at 568: spot=3D837 offset=3D4 = length=3D10 [ 17.803315] ath: phy0: Restore at 580: spot=3D851 offset=3D4 = length=3D10 [ 17.803317] ath: phy0: Restore at 592: spot=3D865 offset=3D4 = length=3D10 [ 17.803319] ath: phy0: Restore at 604: spot=3D879 offset=3D4 = length=3D10 [ 17.803321] ath: phy0: Restore at 616: spot=3D893 offset=3D4 = length=3D10 [ 17.803323] ath: phy0: Restore at 628: spot=3D907 offset=3D4 = length=3D10 [ 17.803325] ath: phy0: Restore at 640: spot=3D921 offset=3D4 = length=3D10 [ 17.803327] ath: phy0: Restore at 652: spot=3D945 offset=3D14 = length=3D15 [ 17.803329] ath: phy0: Restore at 669: spot=3D963 offset=3D3 length=3D5= [ 17.803331] ath: phy0: Restore at 676: spot=3D971 offset=3D3 = length=3D37 [ 17.803333] ath: phy0: Restore at 715: spot=3D1011 offset=3D3 = length=3D77 [ 17.805147] ath: EEPROM regdomain: 0x6c [ 17.805150] ath: EEPROM indicates we should expect a direct regpair = map [ 17.805152] ath: Country alpha2 being used: 00 [ 17.805154] ath: Regpair used: 0x6c [several more pages of calibration correction data omitted]