From nobody Mon Feb 12 00:59:15 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TY5hN59BKz595G8; Mon, 12 Feb 2024 00:59:32 +0000 (UTC) (envelope-from lars.sonchocky-helldorf@hamburg.de) Received: from longisland.snafu.de (longisland.snafu.de [IPv6:2001:1560:3:255::153]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4TY5hN2gvWz4mZ2; Mon, 12 Feb 2024 00:59:32 +0000 (UTC) (envelope-from lars.sonchocky-helldorf@hamburg.de) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=hamburg.de; s=snha; h=References:To:Cc:In-Reply-To:Date:Subject:Mime-Version:Content-Type :Message-Id:From:Sender:Reply-To:Content-Transfer-Encoding:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=5bQOZNuB9fHu028eTzhSmR043dl0t8Fc41ehyRbxt1U=; b=ook21qLKUms2cywhRj8yUAbQO3 gWeJVX3XG7NAuIaI4i2fsAexACLo4q2Qoqk9lg9xh8+zrjqibMuY2/jnTgIpMRiO3TDg9rBlvtOd4 8D2lEzt5OVqGsfTbAHLO+lfTMOIb+QMt4x96UOvcn1OtEI6R+DYiFGjTBJXJX0SatMqEppyWQ7i/b i3xZjJbLdCCWYRq3Le5g5ghbLPvpA76AQ0DwgJHRhlnj4S3EwydC3lydq7QZ4sCAbXamh3yyerH0L McbtI4KmHIk8SsGQlzCS8a0cHqDNi5CoCOGdIhPpQdrq16NqHSlnmNqAtkcpuWRXXbxeqBTABhc56 6zuCQH1A==; X-Trace: 507c6c6172732e736f6e63686f636b792d68656c6c646f72664068616d62757267 2e64657c39332e3233302e3137352e34337c31725a4b664f2d3030303030303030 4d485a2d314a39397c31373037363939353636 Received: from longisland.snafu.de ([10.153.10.15] helo=localhost) by longisland.snafu.de with esmtpsa (Exim 4.97.1) id 1rZKfO-00000000MHZ-1J99; Mon, 12 Feb 2024 01:59:28 +0100 From: "lars.sonchocky-helldorf@hamburg.de" Message-Id: <011B3051-E189-41AF-AAE7-9867010017C1@hamburg.de> Content-Type: multipart/alternative; boundary="Apple-Mail=_AD026309-0ADD-46C2-B490-8CF26003A37D" List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.400.31\)) Subject: Re: How to use the L4 Microkernel with a FreeBSD userland. Date: Mon, 12 Feb 2024 01:59:15 +0100 In-Reply-To: Cc: Mark Millard , freebsd-arm , freebsd-hackers , FreeBSD Current To: Mario Marietto References: <071E080E-C0E6-40F0-A0DF-4FCC22FC004D@yahoo.com> X-Mailer: Apple Mail (2.3774.400.31) X-VISP-ShouldScan: 1 X-VISP-Virus-Check: clean X-VISP-Spam-Score: 0.1 (/) X-VISP-Spam-Report: This message has been scanned on "longisland.snafu.de" to identify if it is considered spam or not. Contact the support hotline for details. Content analysis details: (0.1 points) pts rule name description ---- ---------------------- -------------------------------------------------- -1.0 ALL_TRUSTED Passed through trusted hosts only via SMTP 0.2 KAM_DMARC_NONE DKIM has Failed or SPF has failed on the message and the domain has no DMARC policy 0.0 KAM_DMARC_STATUS Test Rule for DKIM or SPF Failure with Strict Alignment 0.8 BAYES_50 BODY: Bayes spam probability is 40 to 60% [score: 0.5000] 0.1 TW_KL BODY: Odd Letter Triples with KL 0.0 HTML_MESSAGE BODY: HTML included in message -0.0 T_SCC_BODY_TEXT_LINE No description available. X-VISP-Spam-Max-Score: +++++ X-SA-Exim-Connect-IP: 93.230.175.43 X-SA-Exim-Mail-From: lars.sonchocky-helldorf@hamburg.de X-SA-Exim-Scanned: No (on longisland.snafu.de); SAEximRunCond expanded to false X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:34171, ipnet:2001:1560::/32, country:DE] X-Rspamd-Queue-Id: 4TY5hN2gvWz4mZ2 X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated --Apple-Mail=_AD026309-0ADD-46C2-B490-8CF26003A37D Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 hi Mario, I think the closest thing to what you=E2=80=99re aspiring to is the = =E2=80=9EDarbat=E2=80=9C-Kernel: a Darwin Kernel ported to L4: https://trustworthy.systems/publications/papers/Lee_Gray_06.abstract But then again the project is no longer active, Googling it brings a lot = of archived stuff. But still, it might work as an Inspiration (if you=E2=80=99re able to = dig up the sources, I didn=E2=80=99t look. Kind regards, Lars > Am 11.02.2024 um 22:20 schrieb Mario Marietto = : >=20 > I will do it as soon as I get all the necessary tools to turn on the = Raspberry Pi 4b. I was thinking that L4 worked like the old project = coLinux,where Linux ran as a list of processes under WIndows. In my sick = mind I'd thought that L4 allows FreeBSD to run as a list of processes = with the L4 microkernel itself on "top" of it. Do you know if something = like this exists ?=20 >=20 >=20 >=20 > On Sun, Feb 11, 2024 at 9:01=E2=80=AFPM Mark Millard = > wrote: >> [Only replying to what I've subscribed to --and I dropped >> Warner as well.] >>=20 >> On Feb 11, 2024, at 11:43, Mario Marietto > wrote: >>=20 >> > ok. But what does this mean ? That I can use whatever Linux distro = I want ? Or even the FreeBSD world ? >>=20 >> Only to build L4Re. >>=20 >> The LR4e built will not contain any Linux userland materials, >> nor any FreeBSD userland materials. LR4e has its own userland >> materials that will be present instead. >>=20 >> = http://os.inf.tu-dresden.de/download/snapshots/pre-built-images/arm64/ >>=20 >> already contains pre-built .elf and .uimage files Why not use one >> of those on the RPi4B? >>=20 >> By size (larger), the most complete ones for the RPi4B seem to be >> (both formats): >>=20 >> = http://os.inf.tu-dresden.de/download/snapshots/pre-built-images/arm64/boot= strap_vm-multi_rpi4.elf >> = http://os.inf.tu-dresden.de/download/snapshots/pre-built-images/arm64/boot= strap_vm-multi_rpi4.uimage >>=20 >> = http://os.inf.tu-dresden.de/download/snapshots/pre-built-images/arm64/boot= strap_vm-basic_rpi4.elf >> = http://os.inf.tu-dresden.de/download/snapshots/pre-built-images/arm64/boot= strap_vm-basic_rpi4.uimage >>=20 >>=20 >> > On Sun, Feb 11, 2024 at 7:59=E2=80=AFPM Mark Millard = > wrote: >> >=20 >> >=20 >> > On Feb 11, 2024, at 05:44, Mario Marietto > wrote: >> >=20 >> > > I'm trying to understand how to use the L4 Microkernel with a = FreeBSD userland. I've asked the same to a L4 developer,but he told me = that he does not know FreeBSD,so I'm here to ask the same question. = First of all I'm sure that it can be done,because it is written clearly = on their website : >> > >=20 >> > >=20 >> > > http://os.inf.tu-dresden.de/L4Re/download/snapshots/ >> > >=20 >> > >=20 >> > > on the section : >> > > Host system requirements >> > > The host system shall be a 64bit-based system with a recent Linux = distribution installed and at least 2GB of free disk space. >> > > All necessary tools required by the build are available from the = provided packages of the Linux distributions, including cross compilers. = But there are also other cross compiler packages available (see below). = You might want to run make check_build_tools in the src/l4 directory to = verify the common tools are installed. >> > > You are free to use any Linux distribution you like, or even BSDs = or any of its derivatives. But then you should know the game. Especially = tool versions should be recent, as installed on the listed distributions = below. >> > > We are confident that the snapshot works on the following = distributions: >> > > =E2=80=A2 Debian 11 or later >> > > =E2=80=A2 Ubuntu 22.04 or later >> > >=20 >> > > Let's say I want to use the L4 microkernel + FreeBSD 14 on my = Raspberry Pi 4,the first step I did was to build L4Re for the = Rpi,according with this instructions : >> > >=20 >> > >=20 >> > > http://os.inf.tu-dresden.de/L4Re/rpi.html=20 >> > >=20 >> > > This is the log file of the compilation,that hasn't given any = error : >> > >=20 >> > >=20 >> > > https://pastebin.ubuntu.com/p/6SwN2mpJBM/ >> > >=20 >> > >=20 >> > > Or I could have taken a pre built image of the L4 microkernel = here :=20 >> > >=20 >> > >=20 >> > > = http://os.inf.tu-dresden.de/download/snapshots/pre-built-images/arm64/ >> > >=20 >> > >=20 >> > >=20 >> > > At this point the tutorial says that I should use a Linux distro. = They suggest the official distro for the Raspberry Pi 4,that's RaspBian. = But I don't want to use Linux as a userland,I want to use FreeBSD. The = question now is : what should I do to achieve that goal ? How can I link = the L4 microkernel with the ubldr bootloader of FreeBSD ? Or should I = link it to the kernel of FreeBSD ? Can someone explain to me the missing = step ? thanks. >> >=20 >> > QUOTING the "Configuring yourself" section: >> > The make setup step configures predefined setups for both the L4Re = microkernel (Fiasco) and the L4Re user-level software, and connects both = together so the images for the target system can be built. >> > END QUOTE >> >=20 >> > So L4Re has its own user-level software, not just a kernel. There = is no use of a Linux or FreeBSD user-level software >> > when L4Re is booted. (They are just used for building.) >> >=20 >> > "The host system" is just a host for building the L4Re parts and = assembling the image from the parts. The "Pulling it together" section = is about combining the parts (including the microkernel and the = user-level software) to make the overall image that does not include = Linux or FreeBSD code. >>=20 >>=20 >> =3D=3D=3D >> Mark Millard >> marklmi at yahoo.com >>=20 >=20 >=20 > -- > Mario. --Apple-Mail=_AD026309-0ADD-46C2-B490-8CF26003A37D Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8
hi = Mario,

I think the closest thing to what you=E2=80=99r= e aspiring to is the =E2=80=9EDarbat=E2=80=9C-Kernel: a Darwin Kernel = ported to L4:






Lars

Am 11.02.2024 um 22:20 schrieb Mario Marietto = <marietto2008@gmail.com>:

I will do = it as soon as I get all the necessary tools to turn on the Raspberry Pi = 4b. I was thinking that L4 worked like the old project coLinux,where = Linux ran as a list of processes under WIndows. In my sick mind I'd = thought that L4 allows FreeBSD to run as a list of processes with the L4 = microkernel itself on "top" of it. Do you know if something like this = exists ? 



On Sun, Feb = 11, 2024 at 9:01=E2=80=AFPM Mark Millard <marklmi@yahoo.com> = wrote:
[Only replying to what I've = subscribed to --and I dropped
Warner as well.]

On Feb 11, 2024, at 11:43, Mario Marietto <marietto2008@gmail.com> wrote:

> ok. But what does this mean ? That I can use whatever Linux distro = I want ? Or even the FreeBSD world ?

Only to build L4Re.

The LR4e built will not contain any Linux userland materials,
nor any FreeBSD userland materials. LR4e has its own userland
materials that will be present instead.

http://os.inf.tu-dresden.de/download/snapshots/pre-built= -images/arm64/

already contains pre-built .elf and .uimage files Why not use one
of those on the RPi4B?

By size (larger), the most complete ones for the RPi4B seem to be
(both formats):

http://os.inf.tu-dresden.de/download/snapshots/pre-built= -images/arm64/bootstrap_vm-multi_rpi4.elf
http://os.inf.tu-dresden.de/download/snapshots/pre-built= -images/arm64/bootstrap_vm-multi_rpi4.uimage

http://os.inf.tu-dresden.de/download/snapshots/pre-built= -images/arm64/bootstrap_vm-basic_rpi4.elf
http://os.inf.tu-dresden.de/download/snapshots/pre-built= -images/arm64/bootstrap_vm-basic_rpi4.uimage


> On Sun, Feb 11, 2024 at 7:59=E2=80=AFPM Mark Millard <marklmi@yahoo.com> wrote:
>
>
> On Feb 11, 2024, at 05:44, Mario Marietto <marietto2008@gmail.com> wrote:
>
> > I'm trying to understand how to use the L4 Microkernel with a = FreeBSD userland. I've asked the same to a L4 developer,but he told me = that he does not know FreeBSD,so I'm here to ask the same question. = First of all I'm sure that it can be done,because it is written clearly = on their website :
> >
> >
> > http://os.inf.tu-dresden.de/L4Re/download/snapshots/=
> >
> >
> > on the section :
> > Host system requirements
> > The host system shall be a 64bit-based system with a recent = Linux distribution installed and at least 2GB of free disk space.
> > All necessary tools required by the build are available from = the provided packages of the Linux distributions, including cross = compilers. But there are also other cross compiler packages available = (see below). You might want to run make check_build_tools in the src/l4 = directory to verify the common tools are installed.
> > You are free to use any Linux distribution you like, or even = BSDs or any of its derivatives. But then you should know the game. = Especially tool versions should be recent, as installed on the listed = distributions below.
> > We are confident that the snapshot works on the following = distributions:
> >     =E2=80=A2 Debian 11 or later
> >     =E2=80=A2 Ubuntu 22.04 or later
> >
> > Let's say I want to use the L4 microkernel + FreeBSD 14 on my = Raspberry Pi 4,the first step I did was to build L4Re for the = Rpi,according with this instructions :
> >
> >
> > http://os.inf.tu-dresden.de/L4Re/rpi.html
> >
> > This is the log file of the compilation,that hasn't given = any  error :
> >
> >
> > https://pastebin.ubuntu.com/p/6SwN2mpJBM/
> >
> >
> > Or I could have taken a pre built image of the L4 microkernel = here :
> >
> >
> > http://os.inf.tu-dresden.de/download/snapshots/pre-built= -images/arm64/
> >
> >
> >
> > At this point the tutorial says that I should use a Linux = distro. They suggest the official distro for the Raspberry Pi 4,that's = RaspBian. But I don't want to use Linux as a userland,I want to use = FreeBSD. The question now is : what should I do to achieve that goal ? = How can I link the L4 microkernel with the ubldr bootloader of FreeBSD ? = Or should I link it to the kernel of FreeBSD ? Can someone explain to me = the missing step ? thanks.
>
> QUOTING the "Configuring yourself" section:
> The make setup step configures predefined setups for both the L4Re = microkernel (Fiasco) and the L4Re user-level software, and connects both = together so the images for the target system can be built.
> END QUOTE
>
> So L4Re has its own user-level software, not just a kernel. There = is no use of a Linux or FreeBSD user-level software
> when L4Re is booted. (They are just used for building.)
>
> "The host system" is just a host for building the L4Re parts and = assembling the image from the parts. The "Pulling it together" section = is about combining the parts (including the microkernel and the = user-level software) to make the overall image that does not include = Linux or FreeBSD code.


=3D=3D=3D
Mark Millard
marklmi at yahoo.com



--
Mario.

= --Apple-Mail=_AD026309-0ADD-46C2-B490-8CF26003A37D-- From nobody Mon Feb 12 17:32:34 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TYWkF2YQ6z5BCXl; Mon, 12 Feb 2024 17:32:37 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TYWkF0gVcz4LrM; Mon, 12 Feb 2024 17:32:37 +0000 (UTC) (envelope-from jhb@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1707759157; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LsyJLlg3EirIgXm1spHayzb3Tu9WXObkdrJBXLjF7dk=; b=lvIC3N0tuOvBoWEz2v+riXovBl+OKlRVs3GRQG5jGMsx0JE1Bj2IUIJ2J7HC+I3bS6JH5s 7xXdU9Q4pEXmsUZpGrNhmkiXbV2HlASjT0FkLciwCMFNEi2+SYHfiRG5ehrmzU+K/BEo06 +ex6WhDubZdXGV8dgqyjm9/NMHAr+4z2x8mCGtpa1YdJAFm9jqxTVpFmE94NAYRf+MGTR7 gt5bMuWMQwme9dGtofa15XV15e+4t+wPmgZ1HLo/l7tpXB6EVbtHnXs86lV2RnQRQM8je4 alCPWDO/DHbu15fK92eFCDfpp4fvuhqxnbBN6DzbsZA/WhFpN3z6bx2JBuOYgw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1707759157; a=rsa-sha256; cv=none; b=IMEaPIfAHELyBe6wiyKiEM3t0uyAqly3U8HuKSv8Hl05qRIqDVMoKoWkIx+3npVEXTDVtt p48+w+nivJHlOZ8BZ/TS5RYAoF1MpQnm/7ZbQM2VXBFnU9+b8H8HtvESmZeLmDNckASu0N /xmdJSp2+bs9AtTYee9cEH9bqBi/NEs0enRvxJbe6UrPG1lq5ekkRAD6mHhbZcCM0N7kM/ WdHpcIuCuNnAB+tnsiMHakRwLm+sSFmq3TDo1U+XVZQVO3dgk0YWzen88pLsmtW6TtBrud 5mjJ1Wpwkre3J1933G4kDTqiJkjeBLGlh0mw+08C6k3xL/cjIWIwT8Xqe6ul1A== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1707759157; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LsyJLlg3EirIgXm1spHayzb3Tu9WXObkdrJBXLjF7dk=; b=fz8Gxhqmjf1WxMCSwiu1XDcxZKmxN2CLXbQH4brHe1JAgfLrmxLKoY1pLC/PssnT1ObM+l U/CAnceU20/V3lW2xBZEQARITEgtd+nbJSmPdGTG4aK7tqLwVBKkaj3LnJNZMd7YDAu46L u0yZgAto3FD15IekoSb4ZruiUDAO2nRrBLX3pWhg1PRBTSGYf5HmR7cLEl2QtMLx6ngOMk 3xUyMexxgzSJHyTf5mIOJBlPlFOgOK/VC0luPEef681nVJORszLdyNzDAY6mmsw041iXM1 FH1hPi3wSz+1C75I1r47VZA2PfxZbBSc1voebXr4BPQhjnzY/lWKXk4B6ji6KQ== Received: from [IPV6:2601:644:937c:5920:4c63:23c7:5c22:d7ba] (unknown [IPv6:2601:644:937c:5920:4c63:23c7:5c22:d7ba]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: jhb) by smtp.freebsd.org (Postfix) with ESMTPSA id 4TYWkD3XvBzTfh; Mon, 12 Feb 2024 17:32:36 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Message-ID: Date: Mon, 12 Feb 2024 09:32:34 -0800 List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: Recent commits reject RPi4B booting: pcib0 vs. pcib1 "rman_manage_region: request" leads to panic Content-Language: en-US To: Mark Millard , FreeBSD ARM List , Current FreeBSD Cc: Warner Losh References: <76AB969F-5BC5-4116-8AF4-3ED2CABEBBA5.ref@yahoo.com> <76AB969F-5BC5-4116-8AF4-3ED2CABEBBA5@yahoo.com> From: John Baldwin In-Reply-To: <76AB969F-5BC5-4116-8AF4-3ED2CABEBBA5@yahoo.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit On 2/9/24 8:13 PM, Mark Millard wrote: > Summary: > > pcib0: mem 0x7d500000-0x7d50930f irq 80,81 on simplebus2 > pcib0: parsing FDT for ECAM0: > pcib0: PCI addr: 0xc0000000, CPU addr: 0x600000000, Size: 0x40000000 > . . . > rman_manage_region: request: start 0x600000000, end 0x6000fffff > panic: Failed to add resource to rman Hmmm, I suspect this is due to the way that bus_translate_resource works which is fundamentally broken. It rewrites the start address of a resource in-situ instead of keeping downstream resources separate from the upstream resources. For example, I don't see how you could ever release a resource in this design without completely screwing up your rman. That is, I expect trying to detach a PCI device behind a translating bridge that uses the current approach should corrupt the allocated resource ranges in an rman long before my changes. That said, that doesn't really explain the panic. Hmm, the panic might be because for PCI bridge windows the driver now passes RF_ACTIVE and the bus_translate_resource hack only kicks in the activate_resource method of pci_host_generic.c. > Detail: > > > . . . > pcib0: mem 0x7d500000-0x7d50930f irq 80,81 on simplebus2 > pcib0: parsing FDT for ECAM0: > pcib0: PCI addr: 0xc0000000, CPU addr: 0x600000000, Size: 0x40000000 This indicates this is a translating bus. > pcib1: irq 91 at device 0.0 on pci0 > rman_manage_region: request: start 0x1, end 0x1 > pcib0: rman_reserve_resource: start=0xc0000000, end=0xc00fffff, count=0x100000 > rman_reserve_resource_bound: request: [0xc0000000, 0xc00fffff], length 0x100000, flags 102, device pcib1 > rman_reserve_resource_bound: trying 0xffffffff <0xc0000000,0xfffff> > considering [0xc0000000, 0xffffffff] > truncated region: [0xc0000000, 0xc00fffff]; size 0x100000 (requested 0x100000) > candidate region: [0xc0000000, 0xc00fffff], size 0x100000 > allocating from the beginning > rman_manage_region: request: start 0x600000000, end 0x6000fffff The fact that we are trying to reserve the CPU addresses in the rman is because bus_translate_resource rewrote the start address in the resource after it was allocated. That said, I can't see why rman_manage_region would actually fail. At this point the rman is empty (this is the first call to rman_manage_region for "pcib1 memory window"), so only the check that should be failing are the checks against rm_start and rm_end. For the memory window, rm_start is always 0, and rm_end is always 0xffffffff, so both the old (0xc00000000 - 0xc00fffff) and new (0x60000000 - 0x600fffffff) ranges are within those bounds. I would instead expect to see some other issue later on where we fail to allocate a resource for a child BAR, but I wouldn't expect rman_manage_region to fail. Logging the return value from rman_manage_region would be the first step I think to see which error value it is returning. Probably I should fix pci_host_generic.c to handle translation properly however. I can work on a patch for that. -- John Baldwin From nobody Mon Feb 12 17:36:46 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TYWq50h03z5BCjm; Mon, 12 Feb 2024 17:36:49 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TYWq509qdz4Pl4; Mon, 12 Feb 2024 17:36:49 +0000 (UTC) (envelope-from jhb@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1707759409; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9pxY1kA+m6XoNdh8a1flJdAb3IyijMRAYas9FhCoImA=; b=wtWHCesOkUtCbRIf3bB568DyLXoY9Fq6WNiQEZu/sld8NUFXwEnz4/vusvd+jdYFrdWZwp 0CWfYXqBTGIA4h6n8M7Gl9q+1CPUFncl8zlJdvsBWeYy2kFuOpBmAR4ybn9FFFLmxb6rYE p4QzuEMtt0pkAhZ9L/nQNQNdHVB41a9zCkneDntR7E+TbE5Fs1McxB6pP49YMKgUQ2wiIG cinwooe3oe2Z4GaSVurUFCw4A2wtZruzDWLIbfOIXXymWIQKvCFkK1CbK3HIPWMbCl/Odw /AOaR65fEGbA9b+h6RHPec6KqRKsMkev8qteVOQuvXMxXVz5bp0yArUyX6HLQA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1707759409; a=rsa-sha256; cv=none; b=u+afM9/+JzN2QcPvF43sXq3hIMOatCQ5u/qpvcEP6e1wBG6OookvawofhwPMl6JxpLjOpM 6kO45SLWgAHeKEt46raBWMYlb3b50qWeDW+MLW09nOGVbGARJjM/DXdzVVlPOXO+jZlGea s7+e6Fmw0Hm76fS0XGiSAZKrUTXKUYAbs/K/r8reNj+D+V26o84lnp1a0ZFKjCBi9yb7Jy UH1vUwtX06F5Tl06Y/5LlBRWm3ASKuKwgyMl9ah8K7SM1gT7qGQ4M112SX0VVKQeZbGfOr jqYAsPoVbMCxrSauSSZC+Ed2uMn2ENlb2pbFN9IijBZFMWW576rEB7WqNZ9nUQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1707759409; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9pxY1kA+m6XoNdh8a1flJdAb3IyijMRAYas9FhCoImA=; b=JYgK58FAapPgBQnRwBPI2Yg4MVs9RgWf6wXMz6g7Jj3WEllEn7i9YpC5HvoXkNltxaKa0u WoOV8RvCGKxcCjxv/HQ7TQ5lb0dEPN94xdvC2DSwCn27ZZLN3750CWMkntvKrNYoc4BQqB kHm4HSMfcnmF+tkzwL9rpNp16exfdLMqWP6zPxhbLW/ZAgyArRAopu2PWB5eyjToWokQ0J rL4Ac6gr5chBVQSRD8eNgUxc2DNH/afR/SQWRbI3mw6joJhzzfb4iYozePYVi7stt/Cl2w VAM3IOXRY6F6xGg85ihnl3iKmypTTIAO4qrlzcSLzvHS/skfkjvSY70M+QVFJw== Received: from [IPV6:2601:644:937c:5920:9426:11ce:b04a:f99b] (unknown [IPv6:2601:644:937c:5920:9426:11ce:b04a:f99b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: jhb) by smtp.freebsd.org (Postfix) with ESMTPSA id 4TYWq422t8zTVl; Mon, 12 Feb 2024 17:36:48 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Message-ID: Date: Mon, 12 Feb 2024 09:36:46 -0800 List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: Recent commits reject RPi4B booting: pcib0 vs. pcib1 "rman_manage_region: request" leads to panic Content-Language: en-US To: Michael Butler , Mark Millard , FreeBSD ARM List , Current FreeBSD Cc: Warner Losh , Bakul Shah References: <76AB969F-5BC5-4116-8AF4-3ED2CABEBBA5.ref@yahoo.com> <76AB969F-5BC5-4116-8AF4-3ED2CABEBBA5@yahoo.com> <2FE8FA48-180B-4F0D-BCD8-F7F33053B0F7@iitbombay.org> <986D2CD6-6241-4EBE-8BD2-9821AB693BA7@yahoo.com> From: John Baldwin In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit On 2/10/24 2:09 PM, Michael Butler wrote: > I have stability problems with anything at or after this commit > (b377ff8) on an amd64 laptop. While I see the following panic logged, no > crash dump is preserved :-( It happens after ~5-6 minutes running in KDE > (X). > > Reverting to 36efc64 seems to work reliably (after ACPI changes but > before the problematic PCI one) > > kernel: Fatal trap 12: page fault while in kernel mode > kernel: cpuid = 2; apic id = 02 > kernel: fault virtual address = 0x48 > kernel: fault code = supervisor read data, page not present > kernel: instruction pointer = 0x20:0xffffffff80acb962 > kernel: stack pointer = 0x28:0xfffffe00c4318d80 > kernel: frame pointer = 0x28:0xfffffe00c4318d80 > kernel: code segment = base 0x0, limit 0xfffff, type 0x1b > kernel: = DPL 0, pres 1, long 1, def32 0, gran 1 > kernel: processor eflags = interrupt enabled, resume, IOPL = 0 > kernel: current process = 2 (clock (0)) > kernel: rdi: fffff802e460c000 rsi: 0000000000000000 rdx: 0000000000000002 > kernel: rcx: 0000000000000000 r8: 000000000000001e r9: fffffe00c4319000 > kernel: rax: 0000000000000002 rbx: fffff802e460c000 rbp: fffffe00c4318d80 > kernel: r10: 0000000000001388 r11: 000000007ffc765d r12: 000f000000000000 > kernel: r13: 0002000000000000 r14: fffff8000193e740 r15: 0000000000000000 > kernel: trap number = 12 > kernel: panic: page fault > kernel: cpuid = 2 > kernel: time = 1707573802 > kernel: Uptime: 6m19s > kernel: Dumping 942 out of 16242 > MB:..2%..11%..21%..31%..41%..51%..62%..72%..82%..92% > kernel: Dump complete > kernel: Automatic reboot in 15 seconds - press a key on the console to abort Without a stack trace it is pretty much impossible to debug a panic like this. Do you have KDB_TRACE enabled in your kernel config? I'm also not sure how the PCI changes can result in a panic post-boot. If you were going to have problems they would be during device attach, not after you are booted and running X. Short of a stack trace, you can at least use lldb or gdb to lookup the source line associated with the faulting instruction pointer (as long as it isn't in a kernel module), e.g. for gdb you would use 'gdb /boot/kernel/kernel' and then 'l *', e.g. from above: 'l *0xffffffff80acb962' -- John Baldwin From nobody Mon Feb 12 18:02:41 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TYXTs5ndHz5BGhB; Mon, 12 Feb 2024 18:06:57 +0000 (UTC) (envelope-from warlock@phouka1.phouka.net) Received: from phouka1.phouka.net (phouka1.phouka.net [107.170.196.116]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "phouka.net", Issuer "Go Daddy Secure Certificate Authority - G2" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TYXTs19T5z4bTQ; Mon, 12 Feb 2024 18:06:57 +0000 (UTC) (envelope-from warlock@phouka1.phouka.net) Authentication-Results: mx1.freebsd.org; none Received: from phouka1.phouka.net (localhost [127.0.0.1]) by phouka1.phouka.net (8.17.1/8.17.1) with ESMTPS id 41CI2fWC028945 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Mon, 12 Feb 2024 10:02:41 -0800 (PST) (envelope-from warlock@phouka1.phouka.net) Received: (from warlock@localhost) by phouka1.phouka.net (8.17.1/8.17.1/Submit) id 41CI2fTM028944; Mon, 12 Feb 2024 10:02:41 -0800 (PST) (envelope-from warlock) Date: Mon, 12 Feb 2024 10:02:41 -0800 From: John Kennedy To: John Baldwin Cc: Michael Butler , Mark Millard , FreeBSD ARM List , Current FreeBSD , Warner Losh , Bakul Shah Subject: Re: Recent commits reject RPi4B booting: pcib0 vs. pcib1 "rman_manage_region: request" leads to panic Message-ID: References: <76AB969F-5BC5-4116-8AF4-3ED2CABEBBA5.ref@yahoo.com> <76AB969F-5BC5-4116-8AF4-3ED2CABEBBA5@yahoo.com> <2FE8FA48-180B-4F0D-BCD8-F7F33053B0F7@iitbombay.org> <986D2CD6-6241-4EBE-8BD2-9821AB693BA7@yahoo.com> List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:14061, ipnet:107.170.192.0/18, country:US] X-Rspamd-Queue-Id: 4TYXTs19T5z4bTQ X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated On Mon, Feb 12, 2024 at 09:36:46AM -0800, John Baldwin wrote: > Without a stack trace it is pretty much impossible to debug a panic like this. > Do you have KDB_TRACE enabled in your kernel config? I'm also not sure how the > PCI changes can result in a panic post-boot. If you were going to have problems > they would be during device attach, not after you are booted and running X. > > Short of a stack trace, you can at least use lldb or gdb to lookup the source > line associated with the faulting instruction pointer (as long as it isn't in > a kernel module), e.g. for gdb you would use 'gdb /boot/kernel/kernel' and then > 'l *', e.g. from above: 'l *0xffffffff80acb962' I know on my RPI4 where I saw that, my USB keyboard was dead at that point and I couldn't get a trace or continue along to get the crashdump. From nobody Mon Feb 12 18:41:55 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TYYGZ0DJbz5BKkZ for ; Mon, 12 Feb 2024 18:42:14 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic311-24.consmr.mail.gq1.yahoo.com (sonic311-24.consmr.mail.gq1.yahoo.com [98.137.65.205]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4TYYGY1WWSz4hd5 for ; Mon, 12 Feb 2024 18:42:12 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1707763330; bh=Yd2DIrIA/QSXA+wsPbV5dwmQIQlse/+QachCcoyatx0=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=ouwnnp41isMVep5LIW9Zs+OCheOFdya+RRdq/6Ey7ByFhyhy6T3ycVf432kq2qUQ7AOh0gvebrEHCgEcZDjUOrsRj9ANxd/lM7m7Wp0MejTLNYHKc0smtH/1nMC2pAHAzV65e/zw/qxVGNcVDAUCn30F159MasIWs/DfFBSgVr4kFckGueg/bw4lzBgHTkxbznXovVPtAMgVCfN8m2Hi9yOXUQrQ5XANJyEmlF1DTWuCOymcW0XKVlnkPyKXgWy99EmXkDtc+n4OclvWEYIjwutZ2eHPupUPvLf7HfGqlB1XdUdcujXubzXG9WbiXWOfn7EcVYg4BNDymOkA4YvgfQ== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1707763330; bh=kcG0vqm6PtEPB3dGxBQwRKT0umVjSv1YuBoAdKAX5p0=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=TnTytltks55wRUSUMhYOBwTMZz/Xn0ZZi4LvcsftYEfOWjxf43GHBvZLP++Tz1l/rFgdFjbPRlbzsjPXsv8yP1iVqEfr2T8opJWyciG/T2wg+xNut/qh46qUubajM0IQQ1V8Zt5aGPgfdwvv/3zyErs2KUtVcvH0TLbsTGj4j8an4q+Eba+DT8Zew3uCbCJqerSlc/BHLf067hz9QgzPLzFLDIyBKSzEpLTLRfCdh7ulF0KFcabnKX6nYrCuUlGi2uzg1NJjbI0oIZnaHGNuzeMsF9lyJ86lEUSaJD16qA7b8Qb8wT8QWV0w+W5lFphxrEGDao0KkmLR/mFlatbSeA== X-YMail-OSG: 50VU.9AVM1mhymbfJuxht2FXtzGb7OcxH53SHdpUffCyG9hh_5vLO.LYTHofVy. MrecZNVJo9Cr.im8JE0BD3SrEMc271mJvp9EZ_Hx8RY7WpDkp_Ab.Je9.Bk2wsJ.Uidfy4KuonIT jsLVW998urMrV8MSVFD.sxk34wvU1RYMHCfT8C7BvnPXwUp61UoOxv3ld5T9K6E7eJn8CKFhX19. zV8MTZk29QazdhH5Hh17IfYs1WuVnTwERfUD1bYFnI581we0aIX632IR1ZEMga9DrHxLysK1CVT_ MYZIn5AnolY1TNJ8GrM11fHlv2c2RtgVj.q_vZ0Nm2VMK416Rdq8VpfgyKQD6eFj_vxdlx0RRmr1 OxphENkPCphELkQvTTvBMo34nRZieWtVyBjlGSbRepBYZ5RfipfockEo_wVWa3NjWaHghkhdkT1x I4d0UWOFd7nxLuh0BXhelKnI39hrdTCIMBuvexlfJRZnJUzA1anmHOXc2j3rwfrLYfVjUYr9bBCc QeNYop9XJAoFZ.E1gW5x6TPaTNg9PNMdYsEkhXyUlIgAJ61ckq4hFTngjUaPTBbyT9v9T0XVKcOG HUgokDaNhy8pXofU5am2ycBGN1WPpnVGVDfLTWbahj2_s3faDPAoTjgTHl7mSA2XJBnksfAsmBW9 aHj3LMID0KCCYgDsdKEEF_3rFaeugHL1D.AWP1sNcfI6mUXpYKSa1ByVZEoYoeMAwrot1tOLrNsN ry0p.xU6fJnFpRN16RL4nTA_ElQsL8d9OL2UpG.LQ6mstz8thjwtW6q.Ls4XufxFXhlteKQlFqWV CrNv0WOj5gQdKbuVhdX4laCmf1UKiUS79YHE_HW9YCJfHpWOLSur2Pg6M0Grhz5vqPowpZLLHc0Z 8Xl8cHTUggN_u22mrcxf9BpulZv.kxCY89njWEieFRDOwt_79Jy1Kvz1M7Yzg1vVkaeQFUazVUIQ EsGom6AhSCVNemgLozTgWwtRc07A0094TPTu_FZxY7Ffavd5pMR213o5bIRiaWgvdhzhPOn7P3s. 89qyqF0bCCe_WPpaxhgva5gDrM_2zeWbF6AzMRUNTOVLX3tB.FKrvJw9FNAmdtp9cr4cdVsBDz4P zTWyTPvtLjR0b_WS9qnGV4LFchVYQa7qS.EikI7RgM474XeO6kMNvS2SLoa0uLdfb0Ytb0CIeSXc cxqCBQLKBLNxdsfsmWEe.C.cTzjdaFpsRgOEqQmBqLUAO_GlllnxIHxsxDmmerb8CnZLAUD_Dewp SeSp9oAuzkc7abeNCeHW2htdl95Wv6wlxIUfD35wKFjaX56GHjKpvnsl.8S8UQz8o.1G9ddDVc8P dZrAgQcQgzxY.sQ729tbOHlQ0FgzpRj3I98ASqSFiqpMvx0rigA4NIsAlMACaIwdhc2iAoHuZ2Hk z5LDh9KRrK1NEayOgV.c1QcqOeYFT9rz_Nwl8TFYQ56Q84PUiz9I9NauUVTSFl31kamtO0rkhpBy 8c3StunSeibwMIkqVikD5Zqi1ZpOYf_uQuleKYCh.zsixNnuZIXZipmGORs8LEd9Jn89ngBcnvZN y82njc6OEatuVFx5tlDrlM0kvvK0p9JCtKWXEUci.Gaw.V1RDMuoUV1LL_ZVL3I2aHDtZBJrjbUy _K6hPykkXUl4Uglpek1HXwh_SdJ0k.Tb.XO12gu0dc_HtBmBP1YYatT2Meyka2HdvTBKj6gjbVfG Qao8jWjHHI.tGsl9fXR7fv4eRzTpfCyUHcGtsnObiMgCHeRw8W_O1fRB2_hVD2._udyTwEvFWRWF WEB_7nPAMMBn5ZDcE1E76F0Uq8.684aYAy7f4GGvmDQBY46Eh4OH4zLfyIobY1en96cnARqRuKdE H0FaTjtjnTV46NiGk5DjlMNsnrRMYe3YqJyDdAKP5FwZaR8XRJaia3ViXxczwesQZWlnWtSmIA1O AcNTitZressfYLH54CRyxXd35rcNLBSs9tLfLM54yiLXfBrqnvuhsqqT8rZmAvB.iCgrFXTEQ2Vu vvOyVkZ8nYM5CBwFipje28FrDP797YdOPUIBpklkXRKRlRwpYYcrWGSHcCbkU3iHyBBE0WD7t63j vCS_nYPDhc2NtOeiden.05hvCQoiaFNTTWRDZ2.w9wS7Sgul1Pno2a_QjA0h8snxPM2c1zoS.yxw StUxXeIX88Bv3XT5nmgtIqgt0651QYmXeN5Em6yOmvA_gv9vYpyBDB.MsU3MvRlYrylFEQq3E0iw qh3cceHENqnV6u6nWRPj8XJnrQegdYrRxd.Iw8Z_.EeS.3THCYY8VskanlUboJYo56DUb8edH_Sk - X-Sonic-MF: X-Sonic-ID: 6b369b64-70c3-40aa-b0af-62fa45efb13c Received: from sonic.gate.mail.ne1.yahoo.com by sonic311.consmr.mail.gq1.yahoo.com with HTTP; Mon, 12 Feb 2024 18:42:10 +0000 Received: by hermes--production-gq1-5c57879fdf-27p5r (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 040bb44139625531b00f5b5a08bc626a; Mon, 12 Feb 2024 18:42:06 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.400.31\)) Subject: Re: Recent commits reject RPi4B booting: pcib0 vs. pcib1 "rman_manage_region: request" leads to panic From: Mark Millard In-Reply-To: Date: Mon, 12 Feb 2024 10:41:55 -0800 Cc: FreeBSD ARM List , Current FreeBSD , Warner Losh Content-Transfer-Encoding: quoted-printable Message-Id: <1F704317-FDB8-4BDA-8A67-61CF48794DFE@yahoo.com> References: <76AB969F-5BC5-4116-8AF4-3ED2CABEBBA5.ref@yahoo.com> <76AB969F-5BC5-4116-8AF4-3ED2CABEBBA5@yahoo.com> To: John Baldwin X-Mailer: Apple Mail (2.3774.400.31) X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US] X-Rspamd-Queue-Id: 4TYYGY1WWSz4hd5 X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated On Feb 12, 2024, at 09:32, John Baldwin wrote: > On 2/9/24 8:13 PM, Mark Millard wrote: >> Summary: >> pcib0: mem = 0x7d500000-0x7d50930f irq 80,81 on simplebus2 >> pcib0: parsing FDT for ECAM0: >> pcib0: PCI addr: 0xc0000000, CPU addr: 0x600000000, Size: 0x40000000 >> . . . >> rman_manage_region: request: start 0x600000000, = end 0x6000fffff >> panic: Failed to add resource to rman >=20 > Hmmm, I suspect this is due to the way that bus_translate_resource = works which is > fundamentally broken. It rewrites the start address of a resource = in-situ instead > of keeping downstream resources separate from the upstream resources. = For example, > I don't see how you could ever release a resource in this design = without completely > screwing up your rman. That is, I expect trying to detach a PCI = device behind a > translating bridge that uses the current approach should corrupt the = allocated > resource ranges in an rman long before my changes. >=20 > That said, that doesn't really explain the panic. Hmm, the panic = might be because > for PCI bridge windows the driver now passes RF_ACTIVE and the = bus_translate_resource > hack only kicks in the activate_resource method of pci_host_generic.c. >=20 >> Detail: >> . . . >> pcib0: mem = 0x7d500000-0x7d50930f irq 80,81 on simplebus2 >> pcib0: parsing FDT for ECAM0: >> pcib0: PCI addr: 0xc0000000, CPU addr: 0x600000000, Size: 0x40000000 >=20 > This indicates this is a translating bus. >=20 >> pcib1: irq 91 at device 0.0 on pci0 >> rman_manage_region: request: start 0x1, end 0x1 >> pcib0: rman_reserve_resource: start=3D0xc0000000, end=3D0xc00fffff, = count=3D0x100000 >> rman_reserve_resource_bound: request: [0xc0000000, = 0xc00fffff], length 0x100000, flags 102, device pcib1 >> rman_reserve_resource_bound: trying 0xffffffff <0xc0000000,0xfffff> >> considering [0xc0000000, 0xffffffff] >> truncated region: [0xc0000000, 0xc00fffff]; size 0x100000 (requested = 0x100000) >> candidate region: [0xc0000000, 0xc00fffff], size 0x100000 >> allocating from the beginning >> rman_manage_region: request: start 0x600000000, = end 0x6000fffff >=20 > The fact that we are trying to reserve the CPU addresses in the rman = is because > bus_translate_resource rewrote the start address in the resource after = it was allocated. >=20 > That said, I can't see why rman_manage_region would actually fail. At = this point the > rman is empty (this is the first call to rman_manage_region for "pcib1 = memory window"), > so only the check that should be failing are the checks against = rm_start and > rm_end. For the memory window, rm_start is always 0, and rm_end is = always > 0xffffffff, so both the old (0xc00000000 - 0xc00fffff) and new = (0x60000000 - 0x600fffffff) > ranges are within those bounds. I would instead expect to see some = other issue later > on where we fail to allocate a resource for a child BAR, but I = wouldn't expect > rman_manage_region to fail. >=20 > Logging the return value from rman_manage_region would be the first = step I think > to see which error value it is returning. Looking at the code in sys/kern/subr_rman.c for rman_manage_region I see the (mostly) return rv related logic only has ENONMEM (explicit return) = and EBUSY as non-0 possibilities (no other returns). All the rv references = and all the returns are shown below: int rv =3D 0; . . . r =3D int_alloc_resource(M_NOWAIT); if (r =3D=3D NULL) return ENOMEM; . . . /* Check for any overlap with the current region. */ if (r->r_start <=3D s->r_end && r->r_end >=3D = s->r_start) { rv =3D EBUSY; goto out; } /* Check for any overlap with the next region. */ t =3D TAILQ_NEXT(s, r_link); if (t && r->r_start <=3D t->r_end && r->r_end >=3D = t->r_start) { rv =3D EBUSY; goto out; } . . . out: mtx_unlock(rm->rm_mtx); return rv; int_alloc_resource failure would be failure (NULL) from the: struct resource_i *r; =20 r =3D malloc(sizeof *r, M_RMAN, malloc_flag | M_ZERO); (associated with the M_NOWAIT argument). The malloc failure would likely go in a very different direction. Side note: looks like the EBUSY cases leak what r references. > Probably I should fix pci_host_generic.c to handle translation = properly however. > I can work on a patch for that. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Mon Feb 12 20:00:16 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TYb0x1lS6z5BTMX for ; Mon, 12 Feb 2024 20:00:33 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic310-21.consmr.mail.gq1.yahoo.com (sonic310-21.consmr.mail.gq1.yahoo.com [98.137.69.147]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4TYb0w38rcz43F5 for ; Mon, 12 Feb 2024 20:00:32 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=kbzG8HaK; dmarc=pass (policy=reject) header.from=yahoo.com; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.69.147 as permitted sender) smtp.mailfrom=marklmi@yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1707768029; bh=G8j0tnRXRCO2xAZjQvWZnJ4hPvXSgdIDKRGQKVs/SVI=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=kbzG8HaKxtvkCoILEfYw9jWzvCRGccLjQU+f9hHnZmF9ZahGWxDhcLZtQdK/UzcuJ/TH3EduXee8Gs08GtuGq3j9EcIQGO3xcxCCoBhc08XXJfjCrkThoEyj86HESvN85UK3wfSZDutVMdxAG4TLdcCE9Ffgst8BW/NjsJdpdd4cGe2dO4DHOsiqjEgnw77dLFA+iou80vG91I6avtqsqEudXnZihsVf1L+DHlwmVQuwGvTWRrmM/mcD/uQVSEBu3/GLfwC3JnGSRnHKDqrOO0b69XtYCgm/Qo9icfhMTo764yKCBMP/91TacUWBvTWabCNueXSt5LufGSKIM0M4XA== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1707768029; bh=ts6CMOyIS7djgJxcf2y2dNlZjt3Y6hiSnSdAeIfgnCH=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=GC6rMmiU6a+ryTOd+9/3qR9RLM05pdwb6FxtcixkrJA05W0OVv2eZMEti8oowlgnBHyD8h9/tFjh/s8MB8JolaRyxv1UGCA2KNS6wH+IRagm0yIGcckH2OWq1QYPr4np4EceeSmckg6bBaY3YS7JEbPepBwUag/B5UStFhC0tZUywqkp9Ak3LoKAZ2sAYvAInuToQ0bQiJCYSnw5FqkIJsDzAe6zUmyhjAxr5+nz1JmTOLGVTRU+yQi7cf59Dmy/OYbmTtKn+GSNs+cXUq7w7B5RHflr5k4ft/Vr3G2yznKHRYLGdzdNQJ4Tho7XK/6BVv6U3t8hAyHaCYJTg4CB6Q== X-YMail-OSG: gxcR.tsVM1mhndE7Dc4gWvyNQvxEm2tIM.TSMNtir8..IfpVAHMn1Zo4ipTsywm DQy194QeI_1UUHFerRzVyulbSpkYdedfJ411b0vmGJkCSVv4dZJTy2BJb1qD_wwYc9r4eEwypZPD G.VuL.5CFr.y8c567ou_Gbx5dYJfDPWiO677CqEWvRLMu42tt0fsL5QG3rRpVnQ_jGShEpZNrt2p wGSL1SBzoKVtYfAmSFWAeb_Bo.PhN0APCYHdC..3Jsi7vFnlKDgTySpK4_ERZMLIJzeqDZyU2bLc b0ALzKYLk6CFqIHTRToOgL_jKYzYznSQM3ckMehbIIUipSCi7KU6iUMFdxZtQtpQUggZaHeQP1P8 Bj2bqvG1sWmPu1dGLXPPAlVx0vK0lbWPeh8h9iqtVnkS7lVDi25K3GyZgYLWrr3za1rIVkOGBqhC lXWwazIEuISC05OHBoJoTF3ulrrw722EWCtEnB.7PDQX6BZlQFtgsSRbJFYt1zwD5hpan5JmCb1F _xBA1_UBBCU9hJ8bmz_2AwRXcQxniA_BvQ.ts.KMWam4tNXxjEa0Li2zhc1ngDxWOAQxOWOQJqI0 xy3BX.BCPJkL7b_R3KJ0aEm_dKkPoLjTN8S5NxMMPlm.W07ei9LQq9E1KdH9PS_b0Zu8ju2zYc9i 0wlqbeM8.RHBicdVExEJUbu.uLUwHAEepsLZwYuJeP3MvR2KF8Aq4pggT5TAhcN0GEWNu10jbE4W JZW77q.RNJyJghSS59eTFQE7Jh5GZLi.7kCqz68PxKr4k4LTTl2GiYDmsyXrNJK8bZeGv1d4gqQN ofW92t_pFZcfQQBLKeiMu5CC8_C4CszQMzUswEnaMrDfWCwyzzQgKmfOn0sSYD.Ah9AMf5PT.AXs IgfvV0GSwrT8IP__1.yS9vTeZoYAx_R6CWWh2GldRgLaPWDiPBymX.rw1.G2RLlMAz3NJqS47XLS hY7knZuYZxdvwQQV4GciTdms2DNz7gsXubOuOSBXPh3DML.aC7lsKpcDi7nQ4Orwsky3zduCfG2v EC5BXKeAATRVGrImJyG1EhAkM1c2LQa4u7lnUk9Wb87cJho1tCUjVGi6qTlzJdMAw9k1vKGZaLWg .uWsDaHa5Hcu5NirCAligClxMzGH3l5zhbGRcmXQUlxwS1TrMJXCcbYhklxX5NjmvhkrfYF5jC9N 6bxfJBVEr1jdoGhI57MJ1s_6MeTLDETXU9PKf.lZIur2fkFO57VU4S5UgkJOdVpUspOb6Zh0bs1I CVvHlTtWfwYYhBe.1aAMY2oNtU2dZNFDcCaU5xQkiW1bhoQ9lH3rTSRMNWT46nXnSIgXxYgHXJ8l wbTqIR6lnBDyLeIWiuiX5DjYaphXOcXziuy9sWJljgyMLxOtDG98__tSqYyZmKl.htm3dTtw47Pt Egt3qB0hDQdJIzoS1CnkSP8CE2kBZrMpr.F8BnY.ko3chIJilDJBcuiorHQZkLNi0p9pL0FYqMHF _EzACguOu5TuG1Zy0iULI_tB._sPA2VkmpfgPDDqHej98aS.nat1Y1UYX75hQ26fEuYS.hbrEAKL KR3U_dsdH5Msbh0WDvrXuwy5kihQSfEp0xKVQ4DJ6Rb51QIujdgBB65Lw3jlWhhaCLSs1NzQVEvT McM29l34CTnZ.a17wAYHI.QOZTRZQXqwZhTTwF7Z1__rV1r8Q3fEBWXvis4ZoVA0N4K9doz4y_rh _gY8u7f7mFAPl0F6kuQX_8gqYTb7rHWiN2.WYU7EmBnBFNV3UumhtlVH5g0qANIcTFhkJmVXkktD zyYv_HXTBJUvbjkzFPFTpOxFzo8QNbwDfa8t5jaXBazsbS63tLXGKizpEIFvh2rKGDhMOL2jmaQJ KMGGfPNcZtvKS4Otpm6v4.KRDtXsfRv4Pn3rt5f81wl_Jbc_48FgmxViWr7APOoC5Lf1iC2Z4ekd zcmoeLPC_PyeaYUawqsGwLPRPx3L7Oj4RRmU20zFcs1ZQG1FDbbdug0ecukEwN.08Wkj8zDbR..S xZz5D5em96sf7Da35WCdOiP.ocRWIyLFPjWaNIvNi1oINqFbhNSplTU3KI4YJL3hWmYzJXKfCH1q S8Og3fxJ5NpGYU01unSGWy48TDOxIvOBDeGlhkYKDev.jOSz7JsnkgCs19B0yFAxBfi9_yLW33i5 hnsxUMScWxhOeyl9mz_3VNlwtpl_ROm_x4S2iLMgc2pm4JVT3oJZ8akEX74sOu2uJoJrYtqDW4LR 78h7tC6tBmu2rMWQeReR7F3Wc4hCRFvjvFFfzNy4yNtZ.v2j0HawnHQoZ8E30wOZzs5f3XDY46ts - X-Sonic-MF: X-Sonic-ID: 6c6cc0f3-3a3b-4919-ad8f-cddc5ce9064e Received: from sonic.gate.mail.ne1.yahoo.com by sonic310.consmr.mail.gq1.yahoo.com with HTTP; Mon, 12 Feb 2024 20:00:29 +0000 Received: by hermes--production-gq1-5c57879fdf-qprqq (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 0ec8a9cfc035de1a91425f5ef22695fc; Mon, 12 Feb 2024 20:00:26 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.400.31\)) Subject: Re: Recent commits reject RPi4B booting: pcib0 vs. pcib1 "rman_manage_region: request" leads to panic From: Mark Millard In-Reply-To: <1F704317-FDB8-4BDA-8A67-61CF48794DFE@yahoo.com> Date: Mon, 12 Feb 2024 12:00:16 -0800 Cc: FreeBSD ARM List , Current FreeBSD , Warner Losh Content-Transfer-Encoding: quoted-printable Message-Id: <9AFDF067-96E4-4E67-90D2-F40DAF3F138F@yahoo.com> References: <76AB969F-5BC5-4116-8AF4-3ED2CABEBBA5.ref@yahoo.com> <76AB969F-5BC5-4116-8AF4-3ED2CABEBBA5@yahoo.com> <1F704317-FDB8-4BDA-8A67-61CF48794DFE@yahoo.com> To: John Baldwin X-Mailer: Apple Mail (2.3774.400.31) X-Spamd-Bar: --- X-Spamd-Result: default: False [-4.00 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; DKIM_TRACE(0.00)[yahoo.com:+]; TO_DN_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; MIME_TRACE(0.00)[0:+]; RCPT_COUNT_THREE(0.00)[4]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.69.147:from]; MLMMJ_DEST(0.00)[freebsd-arm@freebsd.org]; APPLE_MAILER_COMMON(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.69.147:from]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim] X-Rspamd-Queue-Id: 4TYb0w38rcz43F5 [Gack: I was looking at the wrong vintage of source code, predating your changes: wrong system used.] On Feb 12, 2024, at 10:41, Mark Millard wrote: > On Feb 12, 2024, at 09:32, John Baldwin wrote: >=20 >> On 2/9/24 8:13 PM, Mark Millard wrote: >>> Summary: >>> pcib0: mem = 0x7d500000-0x7d50930f irq 80,81 on simplebus2 >>> pcib0: parsing FDT for ECAM0: >>> pcib0: PCI addr: 0xc0000000, CPU addr: 0x600000000, Size: = 0x40000000 >>> . . . >>> rman_manage_region: request: start = 0x600000000, end 0x6000fffff >>> panic: Failed to add resource to rman >>=20 >> Hmmm, I suspect this is due to the way that bus_translate_resource = works which is >> fundamentally broken. It rewrites the start address of a resource = in-situ instead >> of keeping downstream resources separate from the upstream resources. = For example, >> I don't see how you could ever release a resource in this design = without completely >> screwing up your rman. That is, I expect trying to detach a PCI = device behind a >> translating bridge that uses the current approach should corrupt the = allocated >> resource ranges in an rman long before my changes. >>=20 >> That said, that doesn't really explain the panic. Hmm, the panic = might be because >> for PCI bridge windows the driver now passes RF_ACTIVE and the = bus_translate_resource >> hack only kicks in the activate_resource method of = pci_host_generic.c. >>=20 >>> Detail: >>> . . . >>> pcib0: mem = 0x7d500000-0x7d50930f irq 80,81 on simplebus2 >>> pcib0: parsing FDT for ECAM0: >>> pcib0: PCI addr: 0xc0000000, CPU addr: 0x600000000, Size: 0x40000000 >>=20 >> This indicates this is a translating bus. >>=20 >>> pcib1: irq 91 at device 0.0 on pci0 >>> rman_manage_region: request: start 0x1, end 0x1 >>> pcib0: rman_reserve_resource: start=3D0xc0000000, end=3D0xc00fffff, = count=3D0x100000 >>> rman_reserve_resource_bound: request: [0xc0000000, = 0xc00fffff], length 0x100000, flags 102, device pcib1 >>> rman_reserve_resource_bound: trying 0xffffffff <0xc0000000,0xfffff> >>> considering [0xc0000000, 0xffffffff] >>> truncated region: [0xc0000000, 0xc00fffff]; size 0x100000 (requested = 0x100000) >>> candidate region: [0xc0000000, 0xc00fffff], size 0x100000 >>> allocating from the beginning >>> rman_manage_region: request: start = 0x600000000, end 0x6000fffff >>=20 >> The fact that we are trying to reserve the CPU addresses in the rman = is because >> bus_translate_resource rewrote the start address in the resource = after it was allocated. >>=20 >> That said, I can't see why rman_manage_region would actually fail. = At this point the >> rman is empty (this is the first call to rman_manage_region for = "pcib1 memory window"), >> so only the check that should be failing are the checks against = rm_start and >> rm_end. For the memory window, rm_start is always 0, and rm_end is = always >> 0xffffffff, so both the old (0xc00000000 - 0xc00fffff) and new = (0x60000000 - 0x600fffffff) >> ranges are within those bounds. I would instead expect to see some = other issue later >> on where we fail to allocate a resource for a child BAR, but I = wouldn't expect >> rman_manage_region to fail. >>=20 >> Logging the return value from rman_manage_region would be the first = step I think >> to see which error value it is returning. >=20 > Looking at the code in sys/kern/subr_rman.c for rman_manage_region I = see > the (mostly) return rv related logic only has ENONMEM (explicit = return) and > EBUSY as non-0 possibilities (no other returns). The modern code also has EINVAL via an explicit return. > All the rv references and > all the returns are shown below: >=20 > int rv =3D 0; > . . . The modern code also has here: DPRINTF(("rman_manage_region: <%s> request: start %#jx, end = %#jx\n", rm->rm_descr, start, end)); if (start < rm->rm_start || end > rm->rm_end) return EINVAL; Adding rm->rm_start and rm->rm_end to the message might be appropriate --and would also be enough additional information to know if EINVAL would be returned. > r =3D int_alloc_resource(M_NOWAIT); > if (r =3D=3D NULL) > return ENOMEM; > . . . > /* Check for any overlap with the current region. */ > if (r->r_start <=3D s->r_end && r->r_end >=3D = s->r_start) { > rv =3D EBUSY; > goto out; > } >=20 > /* Check for any overlap with the next region. */ > t =3D TAILQ_NEXT(s, r_link); > if (t && r->r_start <=3D t->r_end && r->r_end >=3D = t->r_start) { > rv =3D EBUSY; > goto out; > } > . . . > out: > mtx_unlock(rm->rm_mtx); > return rv; >=20 > int_alloc_resource failure would be failure (NULL) from the: >=20 > struct resource_i *r; >=20 > r =3D malloc(sizeof *r, M_RMAN, malloc_flag | M_ZERO); >=20 > (associated with the M_NOWAIT argument). >=20 > The malloc failure would likely go in a very different direction. >=20 >=20 >=20 > Side note: looks like the EBUSY cases leak what r references. Still true in the newer code. >> Probably I should fix pci_host_generic.c to handle translation = properly however. >> I can work on a patch for that. >=20 =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Mon Feb 12 20:21:27 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TYbTN682wz5BWjD for ; Mon, 12 Feb 2024 20:21:44 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic309-21.consmr.mail.gq1.yahoo.com (sonic309-21.consmr.mail.gq1.yahoo.com [98.137.65.147]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4TYbTN3QDgz46Yn for ; Mon, 12 Feb 2024 20:21:44 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1707769302; bh=iRklNIyJW3dgjnbFU/rJc84i78OnTiX8kR3DnpKBESE=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=YLTPUr+MvbuYQjQ1uZeqggjl05NbPbpFE4gWMPOBIB/fSRngo4QmfZcyEZNCb586ckFG3aUcsAIV11iOtUfGF+3y0VX51bWiHvL38ckfePtFV2XHSzM7qjQ5HH20jPDBaM/7rDNdhLMIzTOcOqBnXT0wyu4wsMdHenNH59X4rGWO5n5MBXxaJcGbhvj+oBHPaxxNLj10hUC/2Rum8029Kc2isKTPG42eZtV+4wHejwsuHM0P1oaBJ6ChN8+xrUEskYY/3LDc62lbZbkUbHYnRarXy7o/rv6kH2ZlYrgXzzw5nyRFCyvK3RznpUd/AGbZaMAH5+eiZ107uDSsqyPA1Q== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1707769302; bh=trd+Tz3QDcrWyLEE7TR6ftyzwFObMNl5xCVIpeXvTIb=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=c8Mwk9i85cyzG3A6qk9bOcMj3N/iicUs2S8360BrsANKfKllmqH7YDDDIZaupz1Z2aFjCpnX3V5Z2K3yelMCFZvCmCrc7ezCV4c+4dVOOwEKc8wsrktv80weXSH8aPWs3jJJ1JKCdPtuwjrhshR3HwZ/d4vVQzl1t1mLQuvjp5O/Sixpn4IZ8mAUWb6h2uI/qwfJNPlnYxAoy0okHSvoC6jm0eCNv/q8tKMtarD84lZG9bk3Wq/2KwE/a6HeTeMC7nTfDVj901e/8sI4/FZxWD8ojdaT8oCuQ8WTmzmSbW8xNiIC4R9dMSnGc/LADNTEPbk4g62iviuNk5r//qCxtw== X-YMail-OSG: VhJH12gVM1km9EXr5l6yxnfUiOPWU_OmgHjErSI.Si072OoweiHfbLSZ79nK2x0 GnCanvZbGhrJJBzIG.ZUkKFMHemEqXKNPr9ownT0BIyuDAWTcpLOpp5t8e7tPtH24xkJkd2Iv1hg SS7nSPehekTF7sFuQHBgJrk9XtM5xpgYkO0Qf3llBfBghKFhVNldyxGrXlHcZugeCZMAJfzDdY2h NOH3DKm_5V92qOJZdVBD1t230ZhpeJpXgdgihMI05ucQRa8riejG_y18u7qIFPAbfID3GR4KMiDg wHfdd0YCZq3aCCil5FfmgqQi_GBjCl8OpToewt0UCKbm.hsQqSqG0nMNrnGiZl4bdUCYLMo7QP78 WzpVNTO43DM70_FH_kUVdfi8w.vS.uVjq3KQ9VggFQl0ZF0XQK0du7nIvgo1UMH08Y.iHpSNC8W3 gj8MMiO0XzLWmjzmtlIlya6ckYUuY0hLQitVUyFyZ8QJQ1qCJMfDCr6PoB6uxpxBq7omRAy5cYcI IDafm6B.lbHTsG6bMrDk.9a4DgMBTXE2WXs2Hi6Cx343qRUC4loYLceV7_Xh9eJ3GhqByj9hX.F6 zC5UOA02YJE5rNZpnsFV2z1HSeEspZE_aZjqfGd5T0ftQsXCVMpVlmpTga0vEyjS2H2tWnee0rBy s_97IWM6nMz7mZF1zpKwHvP61UPIOmOi7ZTE_J5xIvrO4_Xo4FzFH3XNCKFyJqaptFrUClp.A7tU Roio.CqYOC0ym5swrypDP5OqtBo.24c1SHywvcwf.YO2OOs41Fnpt.F5NNkfAovlvS6MkZ_eMo6F RftB2mdC2CToJA11_tnCMb2h9JTCb36vXS5qfOzgjm6t9hlvrPqPrIWaRNbOdnm70CuYV4Dxl3Ee D7yHrHTzyMuDpYhuL1cxrYBlNSYUY_1xsTrdLmPizF7NvIdMEsk3EVF6Kkr3241Rpf4UujjvGRnq 55iL42sQxWeKCYUUC8YUwsHCko7zx0gCdN9X5bJzsuYn6QeACwgCioMsw82Chnq7QWDk445Ecn56 0zZrefq8GA_SDSc1t6Pe7_0lT8A93iHGEbAIk4t_2c_px7NLj7UBY7OiowGzK3aVV8LHGewJP2Yh k1.dwA9IrBtDJz_9rVxYyPobex.q0378HKpdUUfbLg7lNMKp6gDPefkLbyKVpwJib47BU8VY3qiE .9zTOMO76y9MUafQJt6qQJtEJBqGqUOJLZCqeeqmyjKPe90n5n9IdMJqocgbUr_FmRwOgEly7qaT Dzn63vtJL..euK4kjafwpN_a6YMSWuHPdBWMSyLGl6dPsPECCTotbMqoY5DOYjBl7zrW6KS6V7e5 L3ozwEKp.aTnJu2IuV6p8OgSInaIVrgBM46eRY1d5WnfRjtQFc1r.Z.PQIVHEkzSSL4m1fpw7D2F dTg3uxQ_0.Q8aADcwt4FDG75Pps7GuJ1V2oDy6WLgP359xmmzcuBf.PDhCu9QqLHGNgvWf4VL8oX 6FUd15K8Bops3O0BCs6uosgvUoqI63_0a9ob21Xw3eHEFzX4KriP3EcrQtoUcUAvhErCoJUuQX1q WelKD1a482.aLHxTqPNBqTC2D6Fh0qcj2Ut3.ii3nzzVvdVSxgaz7Nq34gvFPAr7R0y8iJXnueJl DezqFqLOCXq21zTbpUwOKeDJBGrwoqzCndM3SKN.fUdG4AP7i2hmk4CeotU6d1YYNbMXuAuSJNEs j.u_deY5V1q3E05nmWlIyfzspS6CfHxANlabN6GY8JRjozE0Z0oo0Wd2vAM4w0fc0yUlKaRFZNm4 IytEXft90r0HBrSXzVAdNjgLvtbqH7cE1x.2pvg6p4PIvotn5ntr1l.j2ZdtUSBhppOHU3wurh0r cEJjt3tK6DhcEZdi07XLj64Bm4pP_lAqmlZWdk__K5CvkFg8L79rVDUPxEmj0kOPT7e0iO3mobGV hWoAMJxeiDrdqguPVeE8golnE6g.Kq477aD18WYVfLDZ22dArCmm7_eaYhp6ysPVHcgin3wYcSCG ihqGmbBTGqV.zsjOComGdddKZPHc9E2k56MS0CrCOWmrRmB3wExCcg7WqeqV6dJT0nqT5qz92VMv 5a0ZQ5bkqye_XLTBQUyxvPOTyDDkyeUZZsDcUB7pJSycd8TfXaNcXBC9Ejp6MEqAyb_Oq2IPqG7N 42pyk6LWhnQ9JkfnP17qbKsvMRweeJ9a_4b8tkFiQWg3R.tu_BHPRIAK8fY7A7MVvCEJuT31jS0M yUpCoJhGAm_XJQw1z2ztVHUPVEPzMzZTal5AEZBmVzcc8Xvc_0YVKL.oqIr3fJC3q4IHZwuRof1e yXQ-- X-Sonic-MF: X-Sonic-ID: c5e7cf0a-ae65-4221-b7ec-903391a33421 Received: from sonic.gate.mail.ne1.yahoo.com by sonic309.consmr.mail.gq1.yahoo.com with HTTP; Mon, 12 Feb 2024 20:21:42 +0000 Received: by hermes--production-gq1-5c57879fdf-vxz7c (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 0aab6ab516ff3bc8a096222fb49422f8; Mon, 12 Feb 2024 20:21:37 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.400.31\)) Subject: Re: Recent commits reject RPi4B booting: pcib0 vs. pcib1 "rman_manage_region: request" leads to panic From: Mark Millard In-Reply-To: Date: Mon, 12 Feb 2024 12:21:27 -0800 Cc: John Baldwin , Michael Butler , FreeBSD ARM List , Current FreeBSD , Warner Losh , Bakul Shah Content-Transfer-Encoding: quoted-printable Message-Id: <5AEF0FB0-F8CB-46C9-B14F-0723136AA116@yahoo.com> References: <76AB969F-5BC5-4116-8AF4-3ED2CABEBBA5.ref@yahoo.com> <76AB969F-5BC5-4116-8AF4-3ED2CABEBBA5@yahoo.com> <2FE8FA48-180B-4F0D-BCD8-F7F33053B0F7@iitbombay.org> <986D2CD6-6241-4EBE-8BD2-9821AB693BA7@yahoo.com> To: John Kennedy X-Mailer: Apple Mail (2.3774.400.31) X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US] X-Rspamd-Queue-Id: 4TYbTN3QDgz46Yn X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated On Feb 12, 2024, at 10:02, John Kennedy wrote: > On Mon, Feb 12, 2024 at 09:36:46AM -0800, John Baldwin wrote: >> Without a stack trace it is pretty much impossible to debug a panic = like this. >> Do you have KDB_TRACE enabled in your kernel config? I'm also not = sure how the >> PCI changes can result in a panic post-boot. If you were going to = have problems >> they would be during device attach, not after you are booted and = running X. >>=20 >> Short of a stack trace, you can at least use lldb or gdb to lookup = the source >> line associated with the faulting instruction pointer (as long as it = isn't in >> a kernel module), e.g. for gdb you would use 'gdb = /boot/kernel/kernel' and then >> 'l *', e.g. from above: 'l = *0xffffffff80acb962' >=20 > I know on my RPI4 where I saw that, my USB keyboard was dead at that = point > and I couldn't get a trace or continue along to get the crashdump. My serial console context still works at the panic. But: . . . KDB: enter: panic [ thread pid 0 tid 100000 ] Stopped at kdb_enter+0x4c: str xzr, [x19, #1280] db> dump Cannot dump: no dump device specified. db>=20 This is despite: # grep dumpdev /boot/loader.conf dumpdev=3D"/dev/da0p3" and: # gpart show -p =3D> 40 468862048 da0 GPT (224G) 40 32728 - free - (16M) 32768 102400 da0p1 efi (50M) 135168 451809280 da0p2 freebsd-ufs (215G) 451944448 16916480 da0p3 freebsd-swap (8.1G) 468860928 1160 - free - (580K) So, the panic may be before dumping is set up, at least for USB3 media. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Tue Feb 13 00:10:38 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TYhYr3wgnz59Gls for ; Tue, 13 Feb 2024 00:10:56 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic308-8.consmr.mail.gq1.yahoo.com (sonic308-8.consmr.mail.gq1.yahoo.com [98.137.68.32]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4TYhYq42Ktz4bHt for ; Tue, 13 Feb 2024 00:10:55 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=JFvBMZJk; dmarc=pass (policy=reject) header.from=yahoo.com; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.68.32 as permitted sender) smtp.mailfrom=marklmi@yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1707783053; bh=fmC1BrYywDybpqqpl/CAHYMD81hpa1dY1DeQd9rDaKo=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=JFvBMZJkgLZAR2oAqbklyugsMe9H5B5ZOp9faKAG5xGl6RpACCjq1kFlbW8Y8PFGWIharBLm2v6F+JpwRhvS2pFS8GOoB6jNpAHa5W/jQp3jyOKFZxc1reeWYnxX9us+pNGUB+ys/ATkZav2P865JAGjlF+ou3F/NcZbnNzrXQJ1X4AH6m9as1p+Q5UWUOyvf0qr0xr/Yzbqb29WL6hAhcBmTjWMIVy5kX5pS6hzghrY0QFBhzLx4EXyfUmUclr7D7u6sP5vaKJcu09qH1OE4HCKVecvNivIKEjnIWQAXQwFeODCkRm4j6YlBdBlod7NsWjSPQWjD4DsJD+QgqikjQ== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1707783053; bh=0phXzRBikRx/pKt+HA5L9XzzXDkskjqKXaT0cXfcWa1=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=bmIjFr+ktnSCNK7E4Ef188gzGBJrYG/i8dapzaLfd3IcHIi36+n5j5Yy2gpx3dV/GPGRDX7+Di/5uj7BEaGovZQ5ZcyQEgbyGeWT2JKUEFD08GcCiJ3TgKpyiFJxXREk/fWQpNl634gVJrbYzbKz0z273Le5U3Bn0tCz7LnuEzfnw8aSrfG9GCcVjqeSBCO9BGJ5sGQjaWqZ4iRQNxOk4kS00T5bVjSFJWtChFei0R1MPDR0IICNj8xAlxs0uHxjMEm7IMxpuknBMJ5YgxDhUdOQf9+WDnDgTRJzeqngFoOcXU4wY3x+CIyFic1gXJoDGMFeZLhAWRMal84IRyZskQ== X-YMail-OSG: u9sXt8MVM1mJP6XZXF03tSwo1sIPTKZuLfDx8H5YWvALYpfQ2z8yaFKZMT092Nl 1aY8f9sVSVOY0D1YM68R4xK6XbNB0diBZXBs1I0nKechScm1GGGGu12k4gBA46A3qR7R9gUbEgCE iJAq_ovzH.suy3f1XFDvgsJCkM08KXvd9X1siS.pYknWNpJOpys1sIuon80UedSmoPpLBx4Ooln_ gCsRtnPDhmgfkI.nlKPGSmlAPvJk9g7iGfK1OvtHGOv55_RzMBzVyg_AupqRzw9ah8S0cqXwc_.m .ljsQ4.tKqvVTIQFCbeCH42q.grH5RNWFnk1eqcigQRe1PW5MMg5E9JfnqDkTeb9NPeuThP.AjNz rd.EWhgzYgz2W.Kw8XTnLN4LlR5i_WqowUZlPKeDbEC.I1d5dCvEoZ6rUS8yC.53mCMGf9nmkEZ_ jCScj5Lum69Z_ItTDtupEmMMlZ_.OMZsQRnIOc.mPlbP.mOO4lGlSbhUw2Cx0za3E9WyFo.msHqN X2aJBypeR2BN3jo6hxWZWXwWUj7IVQMS.LOgQ7lLzvyo0339UU__Caucz.8WrMrPN.ZdK8aFdcM. GdkNu9my5KX_5BJZZ1gsGblgQg_I.vsBZPAmy8M6LIHVFsVsALONMMu8gXpiJm.s3VQouI9gIJKA ZTNs5EpamQouaaSOHl8AFH2WMwpzncOuAZKe9eUcuyHMAu4is3RCQ8vVW_q.e7BdjKqpDp4j8beg g2gKOYSEn0Cu3kaweeBokLdVbReWibK1WQg8svQ.NfRvoek7zZ_T4xjcMj_K.5XDXxIFuR2sry6R xJou81bgwlx5uEFY5DP.AT1e7.OCjMlXFQ5Wf4AS_4oUxc4bIZvpyg3HjspEJmh6WRGXyCQsRLvB mmEnMe1y3VBROKYaxG.PzPlWHjgwGyy..7Ygs3nzRKubTjyxymnmmTW3P3Bz.WB.u4vxoMpfMYzC E5aECfiFRa7hEkh16XYOSF72xfVJxyHsDLo1dAp320RKKjlEdl3iiYCOohJ_R9Oj_Wmqx7CRQfmq VWLXZKjfddMcK9e6B7Hw5.ZL3jpKJ.tFtBcKFfYfIrVUB__HduVVt4cDpjPIu1xMcJzdM9b9oUhq 6gnHbpY0J91HkIeBW_fi7P5Hx8lN_J4tDEGXu0aO8.Tm8QOJuCmnglbYHpn1kEf6Vo1JFRMz6rE3 QhosLUigwMG5d5BAIzIuAdlmeZl4PF3Zk32igHYRgEjwb4XRNNnG0x.2rjZOKEkM1fwFLfENCoXq Q6c2gowX132XEUO9PfKKGzWWIORoMi5YaFD1IVLujExj4rHtD_nl3ud5g_qzAvNypL4qH_1uLYD_ sO4RfBguHM9E2ITvebMUowCb6fd5XXMJUHLDhZ1G6oRfaxlfD_F9eFA6OAKH7WLTxcwMjBJa5sZf rHP4zGlVMLiZrJE7_aNN00LP92WYW3Iy_z9AVVuhAmVspJo4pYr08SglFuf8i9IJMAr0CvLA_qZ0 ECqimfr0otUaXVP0mDEAxyTzYwUbUrgsg2QG2oAZfenzUJO2XiI9SI6IYWz4wnef_5vlp4fOrz03 KjZNOJ6c0Z9KugoTn8MxmZkrczwQOpN8f3v7yeYh6eg9vIIaCJnTf5s361ENk1Nl5yl2By.jP2cO ZniqYhoHjFbaKo.WJN2AsBlzqXOoj_FRQQucxmqleRKYKrQS9CFVbS_8XG.yF_DlF07IT5fwsFhs pDDCgk1WZz1eygL1pDNIY4M0xR7IXacm78_lTZ4Xg3NyyNGhVV8we35157d2lnRIhXJZ78TTm40e iWIOOL736ZQJvLWb6Pp6Mu5P584tPMESlZJnYmJ.JfGWrlKw3yaEGJWWQLlSRzW3zsye0y_7S1lQ Z09BvIddINavMvXmUbMISavkOdJaNbvYqIn8HZyxBeqLsZR24bECLk9uryjG0X83XcL7zFUZkVBO OmonfQWVsF_3ZGFtxdHGeJtj5rYivad1Inzibx_K_E0vP3xFS5j0ygkfj8tvSUzonJ4xJ2brh7VQ M53hvXJPZwKsRaTPWjHhawxRHNhYsI0Ul_QI8pQ7E3bjKuoanMpdb5Umzij83zh9LrRBT6U9ZlC1 43ZqO8Wodjl8EpaIvrsPtSmIvBKSjaBdEz_a1cLcr1c_ts3kL0ktom.tD92j.tIyuhj3QE9O9vXv L0xIMQBxgGVqiLIWnMoTw4m2Sin9Pw002MeCj65pN9hwD6_b0VQ3D1JBeZOjfxqf_1jaRY.tm2T. iLfhwqEOxl_U10V6.B.85lzEebD2LIqmXZXsO5721QDmis0XFS3CI5dPoFVhpIrd7Xgcb.VwM2Sc 0zR54GQ-- X-Sonic-MF: X-Sonic-ID: c8d9593c-dcac-4a56-b259-740cbef7bed3 Received: from sonic.gate.mail.ne1.yahoo.com by sonic308.consmr.mail.gq1.yahoo.com with HTTP; Tue, 13 Feb 2024 00:10:53 +0000 Received: by hermes--production-gq1-5c57879fdf-4h5cs (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID cee797ca66f1f6d12376825ab98ef016; Tue, 13 Feb 2024 00:10:49 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.400.31\)) Subject: Re: Recent commits reject RPi4B booting: pcib0 vs. pcib1 "rman_manage_region: request" leads to panic From: Mark Millard In-Reply-To: <9AFDF067-96E4-4E67-90D2-F40DAF3F138F@yahoo.com> Date: Mon, 12 Feb 2024 16:10:38 -0800 Cc: FreeBSD ARM List , Current FreeBSD , Warner Losh Content-Transfer-Encoding: quoted-printable Message-Id: <4C279710-5F88-4295-B1A4-7C395F3587E5@yahoo.com> References: <76AB969F-5BC5-4116-8AF4-3ED2CABEBBA5.ref@yahoo.com> <76AB969F-5BC5-4116-8AF4-3ED2CABEBBA5@yahoo.com> <1F704317-FDB8-4BDA-8A67-61CF48794DFE@yahoo.com> <9AFDF067-96E4-4E67-90D2-F40DAF3F138F@yahoo.com> To: John Baldwin X-Mailer: Apple Mail (2.3774.400.31) X-Spamd-Bar: --- X-Spamd-Result: default: False [-4.00 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; DKIM_TRACE(0.00)[yahoo.com:+]; TO_DN_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; MIME_TRACE(0.00)[0:+]; RCPT_COUNT_THREE(0.00)[4]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; MLMMJ_DEST(0.00)[freebsd-arm@freebsd.org]; APPLE_MAILER_COMMON(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.68.32:from]; RCVD_IN_DNSWL_NONE(0.00)[98.137.68.32:from] X-Rspamd-Queue-Id: 4TYhYq42Ktz4bHt On Feb 12, 2024, at 12:00, Mark Millard wrote: > [Gack: I was looking at the wrong vintage of source code, predating > your changes: wrong system used.] >=20 >=20 > On Feb 12, 2024, at 10:41, Mark Millard wrote: >=20 >> On Feb 12, 2024, at 09:32, John Baldwin wrote: >>=20 >>> On 2/9/24 8:13 PM, Mark Millard wrote: >>>> Summary: >>>> pcib0: mem = 0x7d500000-0x7d50930f irq 80,81 on simplebus2 >>>> pcib0: parsing FDT for ECAM0: >>>> pcib0: PCI addr: 0xc0000000, CPU addr: 0x600000000, Size: = 0x40000000 >>>> . . . >>>> rman_manage_region: request: start = 0x600000000, end 0x6000fffff >>>> panic: Failed to add resource to rman >>>=20 >>> Hmmm, I suspect this is due to the way that bus_translate_resource = works which is >>> fundamentally broken. It rewrites the start address of a resource = in-situ instead >>> of keeping downstream resources separate from the upstream = resources. For example, >>> I don't see how you could ever release a resource in this design = without completely >>> screwing up your rman. That is, I expect trying to detach a PCI = device behind a >>> translating bridge that uses the current approach should corrupt the = allocated >>> resource ranges in an rman long before my changes. >>>=20 >>> That said, that doesn't really explain the panic. Hmm, the panic = might be because >>> for PCI bridge windows the driver now passes RF_ACTIVE and the = bus_translate_resource >>> hack only kicks in the activate_resource method of = pci_host_generic.c. >>>=20 >>>> Detail: >>>> . . . >>>> pcib0: mem = 0x7d500000-0x7d50930f irq 80,81 on simplebus2 >>>> pcib0: parsing FDT for ECAM0: >>>> pcib0: PCI addr: 0xc0000000, CPU addr: 0x600000000, Size: = 0x40000000 >>>=20 >>> This indicates this is a translating bus. >>>=20 >>>> pcib1: irq 91 at device 0.0 on pci0 >>>> rman_manage_region: request: start 0x1, end 0x1 >>>> pcib0: rman_reserve_resource: start=3D0xc0000000, end=3D0xc00fffff, = count=3D0x100000 >>>> rman_reserve_resource_bound: request: [0xc0000000, = 0xc00fffff], length 0x100000, flags 102, device pcib1 >>>> rman_reserve_resource_bound: trying 0xffffffff <0xc0000000,0xfffff> >>>> considering [0xc0000000, 0xffffffff] >>>> truncated region: [0xc0000000, 0xc00fffff]; size 0x100000 = (requested 0x100000) >>>> candidate region: [0xc0000000, 0xc00fffff], size 0x100000 >>>> allocating from the beginning >>>> rman_manage_region: request: start = 0x600000000, end 0x6000fffff What you later typed does not match: 0x600000000 0x6000fffff You later typed: 0x60000000 0x600fffffff This seems to have lead to some confusion from using the wrong figure(s). >>> The fact that we are trying to reserve the CPU addresses in the rman = is because >>> bus_translate_resource rewrote the start address in the resource = after it was allocated. >>>=20 >>> That said, I can't see why rman_manage_region would actually fail. = At this point the >>> rman is empty (this is the first call to rman_manage_region for = "pcib1 memory window"), >>> so only the check that should be failing are the checks against = rm_start and >>> rm_end. For the memory window, rm_start is always 0, and rm_end is = always >>> 0xffffffff, so both the old (0xc00000000 - 0xc00fffff) and new = (0x60000000 - 0x600fffffff) >>> ranges are within those bounds. No: 0xffffffff .vs (actual): 0x600000000 0x6000fffff Both the actuals are larger then the 0xffffffff figure you list above. I've no clue if the 0xffffffff might have its own typos. >>> I would instead expect to see some other issue later >>> on where we fail to allocate a resource for a child BAR, but I = wouldn't expect >>> rman_manage_region to fail. >>>=20 >>> Logging the return value from rman_manage_region would be the first = step I think >>> to see which error value it is returning. >>=20 >> Looking at the code in sys/kern/subr_rman.c for rman_manage_region I = see >> the (mostly) return rv related logic only has ENONMEM (explicit = return) and >> EBUSY as non-0 possibilities (no other returns). >=20 > The modern code also has EINVAL via an explicit return. >=20 >> All the rv references and >> all the returns are shown below: >>=20 >> int rv =3D 0; >> . . . >=20 > The modern code also has here: >=20 > DPRINTF(("rman_manage_region: <%s> request: start %#jx, end = %#jx\n", > rm->rm_descr, start, end)); > if (start < rm->rm_start || end > rm->rm_end) > return EINVAL; >=20 > Adding rm->rm_start and rm->rm_end to the message might be > appropriate --and would also be enough additional information > to know if EINVAL would be returned. I added such code and built, installed, and tried booting a separate kernel. The result was: rman_manage_region: range 0..0xffffffff : request: = start 0x600000000, end 0x6000fffff panic: Failed to add resource to rman So: 0 vs. start: 0x600000000 and: 0xffffffff vs. end: 0x6000fffff The 0x600000000..0x6000fffff range is not a range of RAM addresses [too large for that for the 8 GiByte RPi4B] and, so, is well above the 32 bit boundary too. That is the only line referencing "memory window". A 32 bit initial = range for some reason. For reference, the source change in question was: - DPRINTF(("rman_manage_region: <%s> request: start %#jx, end = %#jx\n", - rm->rm_descr, start, end)); + DPRINTF(("rman_manage_region: <%s> range %#jx..%#jx : request: = start %#jx, end %#jx\n", + rm->rm_descr, rm->rm_start, rm->rm_end, start, end)); if (start < rm->rm_start || end > rm->rm_end) return EINVAL; I'll show a larger boot output context at the bottom of the message for reference. >> r =3D int_alloc_resource(M_NOWAIT); >> if (r =3D=3D NULL) >> return ENOMEM; >> . . . >> /* Check for any overlap with the current region. */ >> if (r->r_start <=3D s->r_end && r->r_end >=3D = s->r_start) { >> rv =3D EBUSY; >> goto out; >> } >>=20 >> /* Check for any overlap with the next region. */ >> t =3D TAILQ_NEXT(s, r_link); >> if (t && r->r_start <=3D t->r_end && r->r_end >=3D = t->r_start) { >> rv =3D EBUSY; >> goto out; >> } >> . . . >> out: >> mtx_unlock(rm->rm_mtx); >> return rv; >>=20 >> int_alloc_resource failure would be failure (NULL) from the: >>=20 >> struct resource_i *r; >>=20 >> r =3D malloc(sizeof *r, M_RMAN, malloc_flag | M_ZERO); >>=20 >> (associated with the M_NOWAIT argument). >>=20 >> The malloc failure would likely go in a very different direction. >>=20 >>=20 >>=20 >> Side note: looks like the EBUSY cases leak what r references. >=20 > Still true in the newer code. >=20 >>> Probably I should fix pci_host_generic.c to handle translation = properly however. >>> I can work on a patch for that. >>=20 For reference: . . . pcib0: mem = 0x7d500000-0x7d50930f irq 80,81 on simplebus2 pcib0: parsing FDT for ECAM0: pcib0: PCI addr: 0xc0000000, CPU addr: 0x600000000, Size: 0x40000000 pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 pcib0: Bus is not cache-coherent rman_reserve_resource_bound: request: = [0xfd500000, 0xfd50930f], length 0x9310, flags 100, device pcib0 rman_reserve_resource_bound: trying 0x1fff <0xfd500000,0x930f> rman_reserve_resource_bound: tried 0x1fff <0xfd500000,0x930f> rman_reserve_resource_bound: tried 0x31bfffff <0xfd500000,0x930f> rman_reserve_resource_bound: tried 0x33298fff <0xfd500000,0x930f> rman_reserve_resource_bound: tried 0x39bf1fff <0xfd500000,0x930f> rman_reserve_resource_bound: tried 0x39c02fff <0xfd500000,0x930f> rman_reserve_resource_bound: tried 0x39c06fff <0xfd500000,0x930f> rman_reserve_resource_bound: tried 0x39c07fff <0xfd500000,0x930f> rman_reserve_resource_bound: tried 0x39c08fff <0xfd500000,0x930f> rman_reserve_resource_bound: tried 0x39c2afff <0xfd500000,0x930f> rman_reserve_resource_bound: tried 0x39c36fff <0xfd500000,0x930f> rman_reserve_resource_bound: tried 0x39c37fff <0xfd500000,0x930f> rman_reserve_resource_bound: tried 0x3b03ffff <0xfd500000,0x930f> rman_reserve_resource_bound: tried 0x3b04ffff <0xfd500000,0x930f> rman_reserve_resource_bound: tried 0x3b2fffff <0xfd500000,0x930f> rman_reserve_resource_bound: tried 0x3ee61fff <0xfd500000,0x930f> rman_reserve_resource_bound: tried 0x3ee63fff <0xfd500000,0x930f> rman_reserve_resource_bound: tried 0x3fffffff <0xfd500000,0x930f> rman_reserve_resource_bound: tried 0xfbffffff <0xfd500000,0x930f> considering [0xfc000000, 0xfd5d1fff] truncated region: [0xfd500000, 0xfd50930f]; size 0x9310 (requested = 0x9310) candidate region: [0xfd500000, 0xfd50930f], size 0x9310 splitting region in three parts: [0xfc000000, 0xfd4fffff]; [0xfd500000, = 0xfd50930f]; [0xfd509310, 0xfd5d1fff] rman_manage_region: range 0..0xffffffffffffffff : request: = start 0xc0000000, end 0xffffffff pcib0: hardware identifies as revision 0x304. pcib0: note: reported link speed is 5.0 GT/s. rman_reserve_resource_bound: request: [0x51, 0x51], length = 0x1, flags 0, device pcib0 rman_reserve_resource_bound: trying 0 <0x51,0> rman_reserve_resource_bound: tried 0 <0x51,0> rman_reserve_resource_bound: tried 0x1 <0x51,0> rman_reserve_resource_bound: tried 0x2 <0x51,0> rman_reserve_resource_bound: tried 0x3 <0x51,0> rman_reserve_resource_bound: tried 0x4 <0x51,0> rman_reserve_resource_bound: tried 0x5 <0x51,0> rman_reserve_resource_bound: tried 0x6 <0x51,0> rman_reserve_resource_bound: tried 0x7 <0x51,0> rman_reserve_resource_bound: tried 0xc <0x51,0> rman_reserve_resource_bound: tried 0xd <0x51,0> rman_reserve_resource_bound: tried 0xe <0x51,0> rman_reserve_resource_bound: tried 0xf <0x51,0> rman_reserve_resource_bound: tried 0x10 <0x51,0> rman_reserve_resource_bound: tried 0x11 <0x51,0> rman_reserve_resource_bound: tried 0x12 <0x51,0> rman_reserve_resource_bound: tried 0x17 <0x51,0> rman_reserve_resource_bound: tried 0x18 <0x51,0> rman_reserve_resource_bound: tried 0x1a <0x51,0> rman_reserve_resource_bound: tried 0x1b <0x51,0> rman_reserve_resource_bound: tried 0x22 <0x51,0> rman_reserve_resource_bound: tried 0x23 <0x51,0> rman_reserve_resource_bound: tried 0x24 <0x51,0> rman_reserve_resource_bound: tried 0x25 <0x51,0> rman_reserve_resource_bound: tried 0x26 <0x51,0> rman_reserve_resource_bound: tried 0x27 <0x51,0> rman_reserve_resource_bound: tried 0x28 <0x51,0> rman_reserve_resource_bound: tried 0x29 <0x51,0> rman_reserve_resource_bound: tried 0x4e <0x51,0> rman_reserve_resource_bound: tried 0x4f <0x51,0> considering [0x50, 0xffffffffffffffff] truncated region: [0x51, 0x51]; size 0x1 (requested 0x1) candidate region: [0x51, 0x51], size 0x1 splitting region in three parts: [0x50, 0x50]; [0x51, 0x51]; [0x52, = 0xffffffffffffffff] pci0: on pcib0 rman_manage_region: range 0..0xff : request: = start 0, end 0xff rman_reserve_resource_bound: request: [0, 0], = length 0x1, flags 0, device pci0 rman_reserve_resource_bound: trying 0xff <0,0> considering [0, 0xff] truncated region: [0, 0]; size 0x1 (requested 0x1) candidate region: [0, 0], size 0x1 allocating from the beginning pci0: domain=3D0, physical bus=3D0 found-> vendor=3D0x14e4, dev=3D0x2711, revid=3D0x00 domain=3D0, bus=3D0, slot=3D0, func=3D0 class=3D06-04-00, hdrtype=3D0x01, mfdev=3D0 cmdreg=3D0x0000, statreg=3D0x0010, cachelnsz=3D0 (dwords) lattimer=3D0x00 (0 ns), mingnt=3D0x00 (0 ns), maxlat=3D0x00 (0 = ns) intpin=3Da, irq=3D0 powerspec 3 supports D0 D3 current D0 secbus=3D1, subbus=3D1 rman_reserve_resource_bound: request: [0x1, = 0x1], length 0x1, flags 0, device (null) rman_reserve_resource_bound: trying 0 <0x1,0> rman_reserve_resource_bound: tried 0 <0x1,0> considering [0x1, 0xff] truncated region: [0x1, 0x1]; size 0x1 (requested 0x1) candidate region: [0x1, 0x1], size 0x1 allocating from the beginning pcib1: irq 91 at device 0.0 on pci0 rman_manage_region: range 0..0xff : request: start = 0x1, end 0x1 pcib0: rman_reserve_resource: start=3D0xc0000000, end=3D0xc00fffff, = count=3D0x100000 rman_reserve_resource_bound: request: [0xc0000000, = 0xc00fffff], length 0x100000, flags 102, device pcib1 rman_reserve_resource_bound: trying 0xffffffff <0xc0000000,0xfffff> considering [0xc0000000, 0xffffffff] truncated region: [0xc0000000, 0xc00fffff]; size 0x100000 (requested = 0x100000) candidate region: [0xc0000000, 0xc00fffff], size 0x100000 allocating from the beginning rman_manage_region: range 0..0xffffffff : request: = start 0x600000000, end 0x6000fffff panic: Failed to add resource to rman cpuid =3D 0 time =3D 1 KDB: stack backtrace: db_trace_self() at db_trace_self db_trace_self_wrapper() at db_trace_self_wrapper+0x38 vpanic() at vpanic+0x1a4 panic() at panic+0x48 pcib_add_window_resources() at pcib_add_window_resources+0xf4 pcib_alloc_window() at pcib_alloc_window+0xfc pcib_attach_common() at pcib_attach_common+0xa18 pcib_attach() at pcib_attach+0x14 device_attach() at device_attach+0x3fc device_probe_and_attach() at device_probe_and_attach+0x80 bus_generic_attach() at bus_generic_attach+0x1c pci_attach() at pci_attach+0xec device_attach() at device_attach+0x3fc device_probe_and_attach() at device_probe_and_attach+0x80 bus_generic_attach() at bus_generic_attach+0x1c device_attach() at device_attach+0x3fc device_probe_and_attach() at device_probe_and_attach+0x80 bus_generic_new_pass() at bus_generic_new_pass+0x100 bus_generic_new_pass() at bus_generic_new_pass+0xb0 bus_generic_new_pass() at bus_generic_new_pass+0xb0 bus_generic_new_pass() at bus_generic_new_pass+0xb0 bus_set_pass() at bus_set_pass+0x50 mi_startup() at mi_startup+0x1e0 virtdone() at virtdone+0x68 KDB: enter: panic [ thread pid 0 tid 100000 ] Stopped at kdb_enter+0x4c: str xzr, [x19, #1280] db>=20 =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Tue Feb 13 00:36:28 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TYj7h4W8kz59Kd8 for ; Tue, 13 Feb 2024 00:36:48 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic306-21.consmr.mail.gq1.yahoo.com (sonic306-21.consmr.mail.gq1.yahoo.com [98.137.68.84]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4TYj7g3GC6z4fww for ; Tue, 13 Feb 2024 00:36:47 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=OOQMW93e; dmarc=pass (policy=reject) header.from=yahoo.com; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.68.84 as permitted sender) smtp.mailfrom=marklmi@yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1707784603; bh=FkLc1IgGFEAIaFWdUc50wyrkuti9yEW8yGO+SkPReDY=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=OOQMW93eqt1KXB+bdguy+m3dUdbqvE8VeA5UWs5ed9GGHhtV4Yy94re+qZzzW7LXcTNzSgjwp5SriOdCUDBpp0mIc3jX/r0LTl8Ox7+ZHOdpR4QS3Ts6Y7So/k4mS/c/aSPRYKUXsQuqKBIQsE8TeOfb/abmI0uQ1q9yk/Yoa3CIxmaIVLTnokknWB1ymu/XkE2GeP1Ee09bnCLQQ67QBhQQp/0nK+LWWUxEes5MH/fr+kPq6iXIRMujPOmWtaiHabDFklJh+G3pe2k60LsA02nz3gD4PeS/k9MD8bwMkIXDSBWCghVX59o1VrLomG4SziaLYclyuGjM9mFn78p5TQ== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1707784603; bh=1GFWolNelIPabwY2vVZ5Apv+EHc1AaI8dBCFetbIP2y=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=l1Q0AO2O+0Bie6j1XEiyIzVBdttY4XhoyZCJNCWX6RfKuoh25o3vFJBDWahrQigjOaakxsliIu0jtX3wOtXjldbv26FEmfw2r1BEfZ1DXfZfb+k2rCgYsDDXppB0W5t5N1F0TjmcRLxU3HBYRoMPq8i57FMdsdLDsY2z47d4rmkSkVpaJpfuCNiN9eIiGmUgJbh52lMIaOPXcKGDjhbOiZ/jc7EILprfK8wIf4MeEWlc+p+EKc7ZgbAMg2vpLULJ+9YNr4rosxtKZtmMPHrZrXfDRwfWKwJe/DTU/M4zxOYm+de3OqgGo+VYDuv4r5zDbe70UtrbhXhSK7gicK/nBg== X-YMail-OSG: ZiGa9cIVM1l956LKTBTsZv6B8Hh.R0Iw5ABN7k2iW4YW.KohnXHoS4XKMfuIHeD uDHL7u_yZo_SnWp9vAq172RruIMa3jexBCOxAnvIOubf587MPN6AiSaLkhcFxQ6q6bGhqy4vLGei ZhHWWJQ5fEJTqkUAPG_a3yISL6hmg8i9daZuvf18PuUHFKMa_67BfbcRSQSHxh7b1IQWe0oUBhiN aQhHBd.YxbK_c7Fuzd9lD_pL.4TnwT8llehA_UqgRNlhlxx2nAH0bMMGVy7e7bsCcsD7ix3rRQ9Z Gozzv4veqXgAZCOXIDUCQ1bUidUR1SUhxNRk7UX0lvrwi_xt8Ak_IxbXENvcqQT6usLbgK_BFXc1 OoaPEFX1ADmoRsskQHjZGH0Fuwcv5QHDNNms945TFyx69pUID5gLH_e7LQFizv1BL.p_uTQEm0uT I58hcnaEUOGdgO9vGFC4Z55FEK4P2pB.q04y86oBrxNpmCu_kNs9i0ODIHIOKBkZ4Bsix8MnTm4_ 9SuPIsL.WGxIjRTGLhu6i8s7OOKv8xEY0a6s35fWNTZQebdqs8DYvg9WYX4hRqCznq3OaW9lf04F FSsFOWFPL8y0j1Nv5hFFjEGyCpHdXLsPIEarAHHE0NdUDrbFNUjfT80Tr1rwEEOscWIJBbVi.6JY LqwR3iPpiZD4N.xICljXM173MHw5kQVRSqfQlB.CaQtwF9TzK_jKD3N2KsLwJlK7NLd5jcOybqDg q6Gc77gsO.MAXyGe3R68u7VlPAuiNmrkhhuqwvIxLps3dc5TYiObkpUSdj8Ic1zuztHnUJmZGUPG tbZ.1u89V1UQN_uUlYllbvy0lDv2sZhcai52p6TgecSr6pm_82rjwF44GP.k56nHd.8xDpFOEi6_ rZmXYdzRkrRiqnNefeaxiiBbTE2pn2NMGegYtFLIg1r9HBhmrWC7CUhHPzfOse1mgQatFqcQ9jlc F8FcZgprYR9xvlq7PBBNRo7jeX5uFUD1QVEf5Qk7bqVMWO9mziIyIAhRII8GqoX23hNX7mG8vLzT jDE25Qg.dPJj93h7xCXOtDO7sKarvIvnisIqoGMEC4RDmvI6DW60L3zZnP0SaVCz9uDd08Nnhids 4vlWQAA_u_MJ9WWThL907CXQ2Pvh802MvHtSf2s10FH9ssylejLOKYGNjZRYg3SIkeNObr6Equjl y47km6Y7AUMA1jpi3PvfU9MDLoGKm4EtxuP8NRbixrsBfydK3bh1Yq4zi3DXAt1h9_1drCwhJKMO HGcHPsChrTrmM6jIDskF6UMKGwPq1IcMaXBJMBrXNrdZCXeZ4jhucPq7fYc2D_tlBza7qO5UdIE2 DORAd96Dpi87B5Z_mYIVa1RM6J43bJn1g4gRw37N4lmhkl4eSnGpLPXi7eocyvTSnGzrRfbZqZP0 24OIYgHV6AMw6E7kZbqFzykZOO7vZtIQMCHOs3bKy6BcqAEgvW1fOzgGCIzb_3nOFXhNO2i64o66 zQ0i7mk03ktAcEJiDz_6LH94UJRxl7jpogrlwbpm.SgmReewrkFjbe5cP2NHKj3TU3rIsBJAMhCT krwHbzCdMHI2FO8t.eI3L5TuBq6sjGcA1uk7iq05Ezvv_df8R5dcVAg48VaS_whBM26H_OVjnEtC 78OOiYNXD_sDSCYkb7eNXzvT64.FCw7ZzUsSGDpYRPDb1wTyH0KNoVrkl9rnK9Y9YPL8KqgoLSIk UsIuxR0pw86tYN_3EZPPmYUlN71n4NEPIi_4pEOtq12aiByR.npH029S0egKnuHdGhQIdz9aQ5vJ A9.Im5byzafyt.A6ywCJnxO2JyJLPKOqatM5oZdAMFJNN3XsPPdYphi6r7evRJPtUMbdbh7jaLaa Q6DSri_vd5VaBYnKSbei29gD5vP1QkMnWG3cEnsGauDenVqSTklWGpKn_MwNg5vVYb7gEHLTmsmy JFxzphMUdTxN3MP6IYXTYCLWvwnoZOv.bao.6FuDnyOXLIkDTYG5WGoBVnYYHZUWmjl0nD8wJLVK kviJPu64nVCXQBsZGndUT9BDUGgujAfgHBVPhLy2XNe_WAW1uVTCnovEe7b_7TXYgoJT11rRgt00 etdcksN9NOX_XlIXAKbpIAzwqfzxfTVWs.mHp5sRPfJ9IPJkTdujdx7A6AwZgX84h0L1GxAEAVJ_ XG7Rdn8.nsNuKLKD_WLwGahgEC0NF8GKRhXQ16OSjH3ElPvxwfqeD58wnYT84LF4x4ktnF6FVUZd hr1onHqyZ0irqqKj2kTXyo_TX28.ooaSy5BWrlpYbgwktN513bbc_2ISiixYf05jlmj1axPT_nYu WwQ-- X-Sonic-MF: X-Sonic-ID: a4421fc4-f21c-43f3-9c72-84142ed6a712 Received: from sonic.gate.mail.ne1.yahoo.com by sonic306.consmr.mail.gq1.yahoo.com with HTTP; Tue, 13 Feb 2024 00:36:43 +0000 Received: by hermes--production-gq1-5c57879fdf-c7xks (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 2ad79929a038886e274ede3c44f9d34d; Tue, 13 Feb 2024 00:36:39 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.400.31\)) Subject: Re: Recent commits reject RPi4B booting: pcib0 vs. pcib1 "rman_manage_region: request" leads to panic From: Mark Millard In-Reply-To: <4C279710-5F88-4295-B1A4-7C395F3587E5@yahoo.com> Date: Mon, 12 Feb 2024 16:36:28 -0800 Cc: FreeBSD ARM List , Current FreeBSD , Warner Losh Content-Transfer-Encoding: quoted-printable Message-Id: <3A145420-399D-4EBD-9FF4-18924908AB1D@yahoo.com> References: <76AB969F-5BC5-4116-8AF4-3ED2CABEBBA5.ref@yahoo.com> <76AB969F-5BC5-4116-8AF4-3ED2CABEBBA5@yahoo.com> <1F704317-FDB8-4BDA-8A67-61CF48794DFE@yahoo.com> <9AFDF067-96E4-4E67-90D2-F40DAF3F138F@yahoo.com> <4C279710-5F88-4295-B1A4-7C395F3587E5@yahoo.com> To: John Baldwin X-Mailer: Apple Mail (2.3774.400.31) X-Spamd-Bar: --- X-Spamd-Result: default: False [-4.00 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; DKIM_TRACE(0.00)[yahoo.com:+]; TO_DN_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; MIME_TRACE(0.00)[0:+]; RCPT_COUNT_THREE(0.00)[4]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; MLMMJ_DEST(0.00)[freebsd-arm@freebsd.org]; APPLE_MAILER_COMMON(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.68.84:from]; RCVD_IN_DNSWL_NONE(0.00)[98.137.68.84:from] X-Rspamd-Queue-Id: 4TYj7g3GC6z4fww On Feb 12, 2024, at 16:10, Mark Millard wrote: > On Feb 12, 2024, at 12:00, Mark Millard wrote: >=20 >> [Gack: I was looking at the wrong vintage of source code, predating >> your changes: wrong system used.] >>=20 >>=20 >> On Feb 12, 2024, at 10:41, Mark Millard wrote: >>=20 >>> On Feb 12, 2024, at 09:32, John Baldwin wrote: >>>=20 >>>> On 2/9/24 8:13 PM, Mark Millard wrote: >>>>> Summary: >>>>> pcib0: mem = 0x7d500000-0x7d50930f irq 80,81 on simplebus2 >>>>> pcib0: parsing FDT for ECAM0: >>>>> pcib0: PCI addr: 0xc0000000, CPU addr: 0x600000000, Size: = 0x40000000 >>>>> . . . >>>>> rman_manage_region: request: start = 0x600000000, end 0x6000fffff >>>>> panic: Failed to add resource to rman >>>>=20 >>>> Hmmm, I suspect this is due to the way that bus_translate_resource = works which is >>>> fundamentally broken. It rewrites the start address of a resource = in-situ instead >>>> of keeping downstream resources separate from the upstream = resources. For example, >>>> I don't see how you could ever release a resource in this design = without completely >>>> screwing up your rman. That is, I expect trying to detach a PCI = device behind a >>>> translating bridge that uses the current approach should corrupt = the allocated >>>> resource ranges in an rman long before my changes. >>>>=20 >>>> That said, that doesn't really explain the panic. Hmm, the panic = might be because >>>> for PCI bridge windows the driver now passes RF_ACTIVE and the = bus_translate_resource >>>> hack only kicks in the activate_resource method of = pci_host_generic.c. >>>>=20 >>>>> Detail: >>>>> . . . >>>>> pcib0: mem = 0x7d500000-0x7d50930f irq 80,81 on simplebus2 >>>>> pcib0: parsing FDT for ECAM0: >>>>> pcib0: PCI addr: 0xc0000000, CPU addr: 0x600000000, Size: = 0x40000000 >>>>=20 >>>> This indicates this is a translating bus. >>>>=20 >>>>> pcib1: irq 91 at device 0.0 on pci0 >>>>> rman_manage_region: request: start 0x1, end = 0x1 >>>>> pcib0: rman_reserve_resource: start=3D0xc0000000, end=3D0xc00fffff, = count=3D0x100000 >>>>> rman_reserve_resource_bound: request: [0xc0000000, = 0xc00fffff], length 0x100000, flags 102, device pcib1 >>>>> rman_reserve_resource_bound: trying 0xffffffff = <0xc0000000,0xfffff> >>>>> considering [0xc0000000, 0xffffffff] >>>>> truncated region: [0xc0000000, 0xc00fffff]; size 0x100000 = (requested 0x100000) >>>>> candidate region: [0xc0000000, 0xc00fffff], size 0x100000 >>>>> allocating from the beginning >>>>> rman_manage_region: request: start = 0x600000000, end 0x6000fffff >=20 > What you later typed does not match: >=20 > 0x600000000 > 0x6000fffff >=20 > You later typed: >=20 > 0x60000000 > 0x600fffffff >=20 > This seems to have lead to some confusion from using the > wrong figure(s). >=20 >>>> The fact that we are trying to reserve the CPU addresses in the = rman is because >>>> bus_translate_resource rewrote the start address in the resource = after it was allocated. >>>>=20 >>>> That said, I can't see why rman_manage_region would actually fail. = At this point the >>>> rman is empty (this is the first call to rman_manage_region for = "pcib1 memory window"), >>>> so only the check that should be failing are the checks against = rm_start and >>>> rm_end. For the memory window, rm_start is always 0, and rm_end is = always >>>> 0xffffffff, so both the old (0xc00000000 - 0xc00fffff) and new = (0x60000000 - 0x600fffffff) >>>> ranges are within those bounds. >=20 > No: >=20 > 0xffffffff >=20 > .vs (actual): >=20 > 0x600000000 > 0x6000fffff >=20 > Both the actuals are larger then the 0xffffffff figure you list above. >=20 > I've no clue if the 0xffffffff might have its own typos. >=20 >>>> I would instead expect to see some other issue later >>>> on where we fail to allocate a resource for a child BAR, but I = wouldn't expect >>>> rman_manage_region to fail. >>>>=20 >>>> Logging the return value from rman_manage_region would be the first = step I think >>>> to see which error value it is returning. >>>=20 >>> Looking at the code in sys/kern/subr_rman.c for rman_manage_region I = see >>> the (mostly) return rv related logic only has ENONMEM (explicit = return) and >>> EBUSY as non-0 possibilities (no other returns). >>=20 >> The modern code also has EINVAL via an explicit return. >>=20 >>> All the rv references and >>> all the returns are shown below: >>>=20 >>> int rv =3D 0; >>> . . . >>=20 >> The modern code also has here: >>=20 >> DPRINTF(("rman_manage_region: <%s> request: start %#jx, end = %#jx\n", >> rm->rm_descr, start, end)); >> if (start < rm->rm_start || end > rm->rm_end) >> return EINVAL; >>=20 >> Adding rm->rm_start and rm->rm_end to the message might be >> appropriate --and would also be enough additional information >> to know if EINVAL would be returned. >=20 > I added such code and built, installed, and tried booting > a separate kernel. The result was: >=20 > rman_manage_region: range 0..0xffffffff : = request: start 0x600000000, end 0x6000fffff > panic: Failed to add resource to rman >=20 > So: >=20 > 0 > vs. start: > 0x600000000 >=20 > and: >=20 > 0xffffffff > vs. end: > 0x6000fffff >=20 > The 0x600000000..0x6000fffff range is not a range of RAM addresses > [too large for that for the 8 GiByte RPi4B] and, so, is well above > the 32 bit boundary too. >=20 > That is the only line referencing "memory window". A 32 bit initial = range > for some reason. For reference, the source change in question was: >=20 > - DPRINTF(("rman_manage_region: <%s> request: start %#jx, end = %#jx\n", > - rm->rm_descr, start, end)); > + DPRINTF(("rman_manage_region: <%s> range %#jx..%#jx : request: = start %#jx, end %#jx\n", > + rm->rm_descr, rm->rm_start, rm->rm_end, start, end)); > if (start < rm->rm_start || end > rm->rm_end) > return EINVAL; >=20 > I'll show a larger boot output context at the bottom of the message > for reference. >=20 >>> r =3D int_alloc_resource(M_NOWAIT); >>> if (r =3D=3D NULL) >>> return ENOMEM; >>> . . . >>> /* Check for any overlap with the current region. */ >>> if (r->r_start <=3D s->r_end && r->r_end >=3D = s->r_start) { >>> rv =3D EBUSY; >>> goto out; >>> } >>>=20 >>> /* Check for any overlap with the next region. */ >>> t =3D TAILQ_NEXT(s, r_link); >>> if (t && r->r_start <=3D t->r_end && r->r_end >=3D = t->r_start) { >>> rv =3D EBUSY; >>> goto out; >>> } >>> . . . >>> out: >>> mtx_unlock(rm->rm_mtx); >>> return rv; >>>=20 >>> int_alloc_resource failure would be failure (NULL) from the: >>>=20 >>> struct resource_i *r; >>>=20 >>> r =3D malloc(sizeof *r, M_RMAN, malloc_flag | M_ZERO); >>>=20 >>> (associated with the M_NOWAIT argument). >>>=20 >>> The malloc failure would likely go in a very different direction. >>>=20 >>>=20 >>>=20 >>> Side note: looks like the EBUSY cases leak what r references. >>=20 >> Still true in the newer code. >>=20 >>>> Probably I should fix pci_host_generic.c to handle translation = properly however. >>>> I can work on a patch for that. >>>=20 >=20 > For reference: >=20 > . . . > pcib0: mem = 0x7d500000-0x7d50930f irq 80,81 on simplebus2 > pcib0: parsing FDT for ECAM0: > pcib0: PCI addr: 0xc0000000, CPU addr: 0x600000000, Size: 0x40000000 > pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 > pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 > pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 > pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 > pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 > pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 > pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 > pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 > pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 > pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 > pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 > pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 > pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 > pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 > pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 > pcib0: Bus is not cache-coherent > rman_reserve_resource_bound: request: = [0xfd500000, 0xfd50930f], length 0x9310, flags 100, device pcib0 > rman_reserve_resource_bound: trying 0x1fff <0xfd500000,0x930f> > rman_reserve_resource_bound: tried 0x1fff <0xfd500000,0x930f> > rman_reserve_resource_bound: tried 0x31bfffff <0xfd500000,0x930f> > rman_reserve_resource_bound: tried 0x33298fff <0xfd500000,0x930f> > rman_reserve_resource_bound: tried 0x39bf1fff <0xfd500000,0x930f> > rman_reserve_resource_bound: tried 0x39c02fff <0xfd500000,0x930f> > rman_reserve_resource_bound: tried 0x39c06fff <0xfd500000,0x930f> > rman_reserve_resource_bound: tried 0x39c07fff <0xfd500000,0x930f> > rman_reserve_resource_bound: tried 0x39c08fff <0xfd500000,0x930f> > rman_reserve_resource_bound: tried 0x39c2afff <0xfd500000,0x930f> > rman_reserve_resource_bound: tried 0x39c36fff <0xfd500000,0x930f> > rman_reserve_resource_bound: tried 0x39c37fff <0xfd500000,0x930f> > rman_reserve_resource_bound: tried 0x3b03ffff <0xfd500000,0x930f> > rman_reserve_resource_bound: tried 0x3b04ffff <0xfd500000,0x930f> > rman_reserve_resource_bound: tried 0x3b2fffff <0xfd500000,0x930f> > rman_reserve_resource_bound: tried 0x3ee61fff <0xfd500000,0x930f> > rman_reserve_resource_bound: tried 0x3ee63fff <0xfd500000,0x930f> > rman_reserve_resource_bound: tried 0x3fffffff <0xfd500000,0x930f> > rman_reserve_resource_bound: tried 0xfbffffff <0xfd500000,0x930f> > considering [0xfc000000, 0xfd5d1fff] > truncated region: [0xfd500000, 0xfd50930f]; size 0x9310 (requested = 0x9310) > candidate region: [0xfd500000, 0xfd50930f], size 0x9310 > splitting region in three parts: [0xfc000000, 0xfd4fffff]; = [0xfd500000, 0xfd50930f]; [0xfd509310, 0xfd5d1fff] > rman_manage_region: range 0..0xffffffffffffffff : = request: start 0xc0000000, end 0xffffffff > pcib0: hardware identifies as revision 0x304. > pcib0: note: reported link speed is 5.0 GT/s. > rman_reserve_resource_bound: request: [0x51, 0x51], = length 0x1, flags 0, device pcib0 > rman_reserve_resource_bound: trying 0 <0x51,0> > rman_reserve_resource_bound: tried 0 <0x51,0> > rman_reserve_resource_bound: tried 0x1 <0x51,0> > rman_reserve_resource_bound: tried 0x2 <0x51,0> > rman_reserve_resource_bound: tried 0x3 <0x51,0> > rman_reserve_resource_bound: tried 0x4 <0x51,0> > rman_reserve_resource_bound: tried 0x5 <0x51,0> > rman_reserve_resource_bound: tried 0x6 <0x51,0> > rman_reserve_resource_bound: tried 0x7 <0x51,0> > rman_reserve_resource_bound: tried 0xc <0x51,0> > rman_reserve_resource_bound: tried 0xd <0x51,0> > rman_reserve_resource_bound: tried 0xe <0x51,0> > rman_reserve_resource_bound: tried 0xf <0x51,0> > rman_reserve_resource_bound: tried 0x10 <0x51,0> > rman_reserve_resource_bound: tried 0x11 <0x51,0> > rman_reserve_resource_bound: tried 0x12 <0x51,0> > rman_reserve_resource_bound: tried 0x17 <0x51,0> > rman_reserve_resource_bound: tried 0x18 <0x51,0> > rman_reserve_resource_bound: tried 0x1a <0x51,0> > rman_reserve_resource_bound: tried 0x1b <0x51,0> > rman_reserve_resource_bound: tried 0x22 <0x51,0> > rman_reserve_resource_bound: tried 0x23 <0x51,0> > rman_reserve_resource_bound: tried 0x24 <0x51,0> > rman_reserve_resource_bound: tried 0x25 <0x51,0> > rman_reserve_resource_bound: tried 0x26 <0x51,0> > rman_reserve_resource_bound: tried 0x27 <0x51,0> > rman_reserve_resource_bound: tried 0x28 <0x51,0> > rman_reserve_resource_bound: tried 0x29 <0x51,0> > rman_reserve_resource_bound: tried 0x4e <0x51,0> > rman_reserve_resource_bound: tried 0x4f <0x51,0> > considering [0x50, 0xffffffffffffffff] > truncated region: [0x51, 0x51]; size 0x1 (requested 0x1) > candidate region: [0x51, 0x51], size 0x1 > splitting region in three parts: [0x50, 0x50]; [0x51, 0x51]; [0x52, = 0xffffffffffffffff] > pci0: on pcib0 > rman_manage_region: range 0..0xff : = request: start 0, end 0xff > rman_reserve_resource_bound: request: [0, = 0], length 0x1, flags 0, device pci0 > rman_reserve_resource_bound: trying 0xff <0,0> > considering [0, 0xff] > truncated region: [0, 0]; size 0x1 (requested 0x1) > candidate region: [0, 0], size 0x1 > allocating from the beginning > pci0: domain=3D0, physical bus=3D0 > found-> vendor=3D0x14e4, dev=3D0x2711, revid=3D0x00 > domain=3D0, bus=3D0, slot=3D0, func=3D0 > class=3D06-04-00, hdrtype=3D0x01, mfdev=3D0 > cmdreg=3D0x0000, statreg=3D0x0010, cachelnsz=3D0 (dwords) > lattimer=3D0x00 (0 ns), mingnt=3D0x00 (0 ns), maxlat=3D0x00 (0 = ns) > intpin=3Da, irq=3D0 > powerspec 3 supports D0 D3 current D0 > secbus=3D1, subbus=3D1 > rman_reserve_resource_bound: request: [0x1, = 0x1], length 0x1, flags 0, device (null) > rman_reserve_resource_bound: trying 0 <0x1,0> > rman_reserve_resource_bound: tried 0 <0x1,0> > considering [0x1, 0xff] > truncated region: [0x1, 0x1]; size 0x1 (requested 0x1) > candidate region: [0x1, 0x1], size 0x1 > allocating from the beginning > pcib1: irq 91 at device 0.0 on pci0 > rman_manage_region: range 0..0xff : request: start = 0x1, end 0x1 > pcib0: rman_reserve_resource: start=3D0xc0000000, end=3D0xc00fffff, = count=3D0x100000 > rman_reserve_resource_bound: request: [0xc0000000, = 0xc00fffff], length 0x100000, flags 102, device pcib1 > rman_reserve_resource_bound: trying 0xffffffff <0xc0000000,0xfffff> > considering [0xc0000000, 0xffffffff] > truncated region: [0xc0000000, 0xc00fffff]; size 0x100000 (requested = 0x100000) > candidate region: [0xc0000000, 0xc00fffff], size 0x100000 > allocating from the beginning > rman_manage_region: range 0..0xffffffff : = request: start 0x600000000, end 0x6000fffff > panic: Failed to add resource to rman > cpuid =3D 0 > time =3D 1 > KDB: stack backtrace: > db_trace_self() at db_trace_self > db_trace_self_wrapper() at db_trace_self_wrapper+0x38 > vpanic() at vpanic+0x1a4 > panic() at panic+0x48 > pcib_add_window_resources() at pcib_add_window_resources+0xf4 > pcib_alloc_window() at pcib_alloc_window+0xfc > pcib_attach_common() at pcib_attach_common+0xa18 > pcib_attach() at pcib_attach+0x14 > device_attach() at device_attach+0x3fc > device_probe_and_attach() at device_probe_and_attach+0x80 > bus_generic_attach() at bus_generic_attach+0x1c > pci_attach() at pci_attach+0xec > device_attach() at device_attach+0x3fc > device_probe_and_attach() at device_probe_and_attach+0x80 > bus_generic_attach() at bus_generic_attach+0x1c > device_attach() at device_attach+0x3fc > device_probe_and_attach() at device_probe_and_attach+0x80 > bus_generic_new_pass() at bus_generic_new_pass+0x100 > bus_generic_new_pass() at bus_generic_new_pass+0xb0 > bus_generic_new_pass() at bus_generic_new_pass+0xb0 > bus_generic_new_pass() at bus_generic_new_pass+0xb0 > bus_set_pass() at bus_set_pass+0x50 > mi_startup() at mi_startup+0x1e0 > virtdone() at virtdone+0x68 > KDB: enter: panic > [ thread pid 0 tid 100000 ] > Stopped at kdb_enter+0x4c: str xzr, [x19, #1280] > db>=20 >=20 FYI: bcm2711-peripherals.pdf "Figure 1. BCM2711 Address Maps" documents the PCIe address space range as (their "_" notation): 0x6_0000_0000..0x7_FFFF_FFFF for both of the address maps: 'Full 35-bit Address Map' 'ARM view of the Address Map in "Low Peripheral" mode' The Low Peripheral mode is what is in use. So the 0x6_0000_0000..0x6_000f_ffff range looks to be a valid ARM system address space for the PCIe to map to. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Tue Feb 13 01:57:54 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TYkxd5YHtz59Tyv for ; Tue, 13 Feb 2024 01:58:13 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic305-21.consmr.mail.gq1.yahoo.com (sonic305-21.consmr.mail.gq1.yahoo.com [98.137.64.84]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4TYkxc37Kpz4sqx for ; Tue, 13 Feb 2024 01:58:12 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=UQpVBNSh; dmarc=pass (policy=reject) header.from=yahoo.com; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.64.84 as permitted sender) smtp.mailfrom=marklmi@yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1707789490; bh=yFdUWVGnKaC0hJq5OciUhv7RAE1dQWRhgaCkqTdoQmg=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=UQpVBNShUOYbXGYAon+rZcyxmqG1Vy0nYUQrWeXJ9MrljXEmBJG5QHc51ZKBJmdX/Qfw8cTL0hDU0ra35dIPJ/PVGJKtoOJMskaIDF8F5UzttNCkW+Mik2r3bk7FH/UqdD55e6H4m8pv6aTahZFnIE/jHAhtNUC/gzHpX1J4U4y2bYSc4PSfgpV7/81PhPviXFbAczHJFqUeX6DUBuwzExRUJiZVluFvtcMS72aoKI+39g4UGKFpsqm1WihBXxiSj7Bsx7iw94fj9RLpsn0P4bp3KfnSwOTlSAUk+mIwhr7rgec8S1XiKUcXB3mTbaYPzz6BM2/pYsX8Cj64dmyF2g== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1707789490; bh=UDt+I4D1OB/a7Jn4SUOBZVGxyvjYDHrK4/acf3Gqpqd=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=FOacDyF+Th9Fv+zMUOYQIGUMNuOD+Vjzn13XiigrLTUQw4HixqeL5q4vPUxMHsB+hlHCYS9laJ4w+2H8BqPFBW8Hu3dn6HJJx2S1iYa2Av0dXi38FlR4HwOBrf14eL+c9MfeYBRnZR1aqfPHtnt/sHiux30i7fqY++CAHs23MgnQA+UxL6RqY0sFc9hWfKJiGTX2WOiQ4sJETj5GASDOn983J2fHZL6lcdpEWpsHffxP1HKqnZku0a3+njig0i54StXjPngtVnDENtvVX8nlhBB1+8sqH5EVnexPo+XiPXhRWH9Fb7tbPbIk+z6O5zQHXIwzAolHFMwRFjbi7qFKwQ== X-YMail-OSG: WCeifLsVM1lRuZdxXxFjvlwTzuKew4d5K85ZvXCj5sr5uhrhzXs8C1dtuI4ulJa LH8GLPqBXT0u._3gi09JEL_445n_7WdGxbXdRB26iaR5AfQSv5cZQ0NBwpRUyREvEFOh3Bc_GIxi IchSR97jQf9z3B1wouusBqwOQLQUh8U3Lq.yluPn0PuMWGZ.be1o2CY28jg2Oae2Og4E1E5YP5Ys ifYxua6zleuOjaV3KGzvsxgA0lUvrON9ZQwfzloqkRHlFGyFkmZ3YMZcr82D5NJhh93kWvmAa_fY uF25qpuunxL8eRAoPrVj.1oUBH2XHHcXe7rB0HM8FElx_25x.zUuqSwrUlVEKTku60RED5QI5ibo s3Zgs66Yc2tShn5_Gb3HtZtHQZZMLorkNYCnaXnPnLRfgLt2KgDrkJ.c3KGo8Hx6fksgPLUfAP_5 oE.Hwf4nIGEpBwqFxPFvVsOocY5MfmaQanbM7vfpVgRaFlvxLk48XXcqil9JUmlVlUV7CIGZaf.1 PyVMznuCDUy1Aaq0rAZCackajthIKvlKrMAJNRSRpeMGQn2Dy7rZ.RU1YSGBe1P7Dmaf8qHzFM4E qWL6hr3ERwtgmsDoL5uLEbbAnmh9C97rqV2Xi56Wh_Gu9Ctj07YTOJsNq2ts8rxDfgqdHqhDOmh3 9dDeFv9AJf0zGFXCbww06WGkuFODmyPqs3JCnkl8OjSubKPVEnjtYLgG0CMJCG6ufdnCkcv1yc2K zPmZvLTzVLeUDhR2tmJ4y2TcesCLG.nu3ICGtzbJ.WPhE1hJRE6GBr3cNqLtfh30ySz._4zawfJX i3IcbVeXTgWAyh9sXf8vj11DrRuC1jT2oLxkSI3PjExrWg5D.mdMNqW0bo1QF4jg1oLFp8mF9hD1 B3CxVXPmAAZ2CnnK0BoRYWh2Qpb.C8WnOEVJs0qvnEpKe3Nl.goBLEKImqN2K3.sLxOtR6sVQcoN wyh9N94aMr1WrUe8hPfiNooJBhhwJyh86nXXVXSwCa50QPZabymW0UAKy97gEosWjBgUelAVYcZx NMUs1TsGYHlB7oQwRR.P6BK6p2ZJ0sCuHuHJFrOdFvCWHUo0ywSIM4JqQLpAIYSdGUZ3vGh5gU.B COUdfU2bz2v44h803u3Q.AoK77y1RlJD5zSIupKC6VpB1bXYdAM.DuaPz4zwLT3PhaKXMupvOiEy GoSgfENF1dC.iYcT3uyQJy1hDzPgSxuiamPhvhg_d6NAqJE2EVHR2MdFHZA6mfgf5HJR1eanV3nm rhFoXIxtkwbYb0O4thO.F5dhyc6zwCIjwYv8xB40nw.qb_fREBFGsLZanvX6UTXD9DIOruRWa1T. AlXzUZuq4CFvfQXZR7oGgeuR4H58HqQJB1DanbB6wWB.NxQ5WWFYfzPHSaEwQGmNZx_H786LzXJK cqOEsVmnGnwtEOxRH6xOVjagJiW5WWzglZinIvk6MFVBOeDM9XUJQnZ4ssiujoEvzlT1Xxtm.TQG xXKNj8XxxfWaomRJg_MUdfES0.4iiAWqgRBXCKCbX_LsqLQDxlN_jh1yHLS8JttBGwSqijlprYXo etvS5KJaagHP9JzIuGAPHdqiKCTqsr0aA.eCPb7NIJH8YzybidPl.v1m.6imUUGO5iNEYPS_wSTf 6UFKx79gfD4e9XNi8JWUbP.J8W.Xx.9uUjlU5gJVGfjw_vPBuz2RlA.MqakbTMIJlMSD1NXoKn2S iOWAfrXR1dtZmJVmYC61FXZ2vIPPABSAHDeRFyI_BRd8dhBt5V59lDvrhYR3hJWXiBLyXNR5Ybpn OJ_fP07Fs7HD3Huwpst5er1RJ33a1xxOdvKG6HRr4Oo_g19pPJ_Fsj8iIS4.0CCNBGKUvPvtY6a. 8VgYQ0CqelObqb8JP6j_c5UEiDYePlH2VC7R.Nyjz0hRibtGXKqGdJ6VgAOxOe_9J0wj60puJUY. vRhZX1pHTPdXGk0OB25SxShV.Qa3S4Tp5TApJ7HPNB2gNlBJBE7jkrJn.AcZxoUCKHII4hei5843 ifzYC_4tatcK9bpuNlcRPl_sOLtSI0EZPeSn5R1rvY_tembs7Zj_3rnMoix0qIhzpwgZOu1N85Nw rYciNCmQtMhk2diGU5POsuiBCmSWtQ593OlX40TNf22r1GIZrQfgdoJnJU.pvlPdWpGgz2jslI81 6qrAGmXJJkc_7Y8Z1rO2caQ4nQD5N9oh_Q32sEUukh6tOcBLvquUu1XzYI_HBX29FtlKKwx9abyu yJV.5TITw2s6MbyOpD0xRhIwTmgC7udr7L1AsXLD1XwDrbTK_Mm.fxNqTkT__zz_bvmlNSPkG3x8 88GE- X-Sonic-MF: X-Sonic-ID: db472841-ae12-47c3-a723-2fc5cedf1274 Received: from sonic.gate.mail.ne1.yahoo.com by sonic305.consmr.mail.gq1.yahoo.com with HTTP; Tue, 13 Feb 2024 01:58:10 +0000 Received: by hermes--production-gq1-5c57879fdf-27p5r (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID ae8256dfa2d8bb9483f05df3dc891bec; Tue, 13 Feb 2024 01:58:05 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.400.31\)) Subject: Re: Recent commits reject RPi4B booting: pcib0 vs. pcib1 "rman_manage_region: request" leads to panic From: Mark Millard In-Reply-To: <3A145420-399D-4EBD-9FF4-18924908AB1D@yahoo.com> Date: Mon, 12 Feb 2024 17:57:54 -0800 Cc: FreeBSD ARM List , Current FreeBSD , Warner Losh Content-Transfer-Encoding: quoted-printable Message-Id: <1298DF9C-0F82-4567-8E81-7332A608C7FC@yahoo.com> References: <76AB969F-5BC5-4116-8AF4-3ED2CABEBBA5.ref@yahoo.com> <76AB969F-5BC5-4116-8AF4-3ED2CABEBBA5@yahoo.com> <1F704317-FDB8-4BDA-8A67-61CF48794DFE@yahoo.com> <9AFDF067-96E4-4E67-90D2-F40DAF3F138F@yahoo.com> <4C279710-5F88-4295-B1A4-7C395F3587E5@yahoo.com> <3A145420-399D-4EBD-9FF4-18924908AB1D@yahoo.com> To: John Baldwin X-Mailer: Apple Mail (2.3774.400.31) X-Spamd-Bar: --- X-Spamd-Result: default: False [-4.00 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; DKIM_TRACE(0.00)[yahoo.com:+]; TO_DN_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; MIME_TRACE(0.00)[0:+]; RCPT_COUNT_THREE(0.00)[4]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; MLMMJ_DEST(0.00)[freebsd-arm@freebsd.org]; APPLE_MAILER_COMMON(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.64.84:from]; RCVD_IN_DNSWL_NONE(0.00)[98.137.64.84:from] X-Rspamd-Queue-Id: 4TYkxc37Kpz4sqx On Feb 12, 2024, at 16:36, Mark Millard wrote: > On Feb 12, 2024, at 16:10, Mark Millard wrote: >=20 >> On Feb 12, 2024, at 12:00, Mark Millard wrote: >>=20 >>> [Gack: I was looking at the wrong vintage of source code, predating >>> your changes: wrong system used.] >>>=20 >>>=20 >>> On Feb 12, 2024, at 10:41, Mark Millard wrote: >>>=20 >>>> On Feb 12, 2024, at 09:32, John Baldwin wrote: >>>>=20 >>>>> On 2/9/24 8:13 PM, Mark Millard wrote: >>>>>> Summary: >>>>>> pcib0: mem = 0x7d500000-0x7d50930f irq 80,81 on simplebus2 >>>>>> pcib0: parsing FDT for ECAM0: >>>>>> pcib0: PCI addr: 0xc0000000, CPU addr: 0x600000000, Size: = 0x40000000 >>>>>> . . . >>>>>> rman_manage_region: request: start = 0x600000000, end 0x6000fffff >>>>>> panic: Failed to add resource to rman >>>>>=20 >>>>> Hmmm, I suspect this is due to the way that bus_translate_resource = works which is >>>>> fundamentally broken. It rewrites the start address of a resource = in-situ instead >>>>> of keeping downstream resources separate from the upstream = resources. For example, >>>>> I don't see how you could ever release a resource in this design = without completely >>>>> screwing up your rman. That is, I expect trying to detach a PCI = device behind a >>>>> translating bridge that uses the current approach should corrupt = the allocated >>>>> resource ranges in an rman long before my changes. >>>>>=20 >>>>> That said, that doesn't really explain the panic. Hmm, the panic = might be because >>>>> for PCI bridge windows the driver now passes RF_ACTIVE and the = bus_translate_resource >>>>> hack only kicks in the activate_resource method of = pci_host_generic.c. >>>>>=20 >>>>>> Detail: >>>>>> . . . >>>>>> pcib0: mem = 0x7d500000-0x7d50930f irq 80,81 on simplebus2 >>>>>> pcib0: parsing FDT for ECAM0: >>>>>> pcib0: PCI addr: 0xc0000000, CPU addr: 0x600000000, Size: = 0x40000000 >>>>>=20 >>>>> This indicates this is a translating bus. >>>>>=20 >>>>>> pcib1: irq 91 at device 0.0 on pci0 >>>>>> rman_manage_region: request: start 0x1, end = 0x1 >>>>>> pcib0: rman_reserve_resource: start=3D0xc0000000, end=3D0xc00fffff,= count=3D0x100000 >>>>>> rman_reserve_resource_bound: request: [0xc0000000, = 0xc00fffff], length 0x100000, flags 102, device pcib1 >>>>>> rman_reserve_resource_bound: trying 0xffffffff = <0xc0000000,0xfffff> >>>>>> considering [0xc0000000, 0xffffffff] >>>>>> truncated region: [0xc0000000, 0xc00fffff]; size 0x100000 = (requested 0x100000) >>>>>> candidate region: [0xc0000000, 0xc00fffff], size 0x100000 >>>>>> allocating from the beginning >>>>>> rman_manage_region: request: start = 0x600000000, end 0x6000fffff >>=20 >> What you later typed does not match: >>=20 >> 0x600000000 >> 0x6000fffff >>=20 >> You later typed: >>=20 >> 0x60000000 >> 0x600fffffff >>=20 >> This seems to have lead to some confusion from using the >> wrong figure(s). >>=20 >>>>> The fact that we are trying to reserve the CPU addresses in the = rman is because >>>>> bus_translate_resource rewrote the start address in the resource = after it was allocated. >>>>>=20 >>>>> That said, I can't see why rman_manage_region would actually fail. = At this point the >>>>> rman is empty (this is the first call to rman_manage_region for = "pcib1 memory window"), >>>>> so only the check that should be failing are the checks against = rm_start and >>>>> rm_end. For the memory window, rm_start is always 0, and rm_end = is always >>>>> 0xffffffff, so both the old (0xc00000000 - 0xc00fffff) and new = (0x60000000 - 0x600fffffff) >>>>> ranges are within those bounds. >>=20 >> No: >>=20 >> 0xffffffff >>=20 >> .vs (actual): >>=20 >> 0x600000000 >> 0x6000fffff >>=20 >> Both the actuals are larger then the 0xffffffff figure you list = above. >>=20 >> I've no clue if the 0xffffffff might have its own typos. >>=20 >>>>> I would instead expect to see some other issue later >>>>> on where we fail to allocate a resource for a child BAR, but I = wouldn't expect >>>>> rman_manage_region to fail. >>>>>=20 >>>>> Logging the return value from rman_manage_region would be the = first step I think >>>>> to see which error value it is returning. >>>>=20 >>>> Looking at the code in sys/kern/subr_rman.c for rman_manage_region = I see >>>> the (mostly) return rv related logic only has ENONMEM (explicit = return) and >>>> EBUSY as non-0 possibilities (no other returns). >>>=20 >>> The modern code also has EINVAL via an explicit return. >>>=20 >>>> All the rv references and >>>> all the returns are shown below: >>>>=20 >>>> int rv =3D 0; >>>> . . . >>>=20 >>> The modern code also has here: >>>=20 >>> DPRINTF(("rman_manage_region: <%s> request: start %#jx, end = %#jx\n", >>> rm->rm_descr, start, end)); >>> if (start < rm->rm_start || end > rm->rm_end) >>> return EINVAL; >>>=20 >>> Adding rm->rm_start and rm->rm_end to the message might be >>> appropriate --and would also be enough additional information >>> to know if EINVAL would be returned. >>=20 >> I added such code and built, installed, and tried booting >> a separate kernel. The result was: >>=20 >> rman_manage_region: range 0..0xffffffff : = request: start 0x600000000, end 0x6000fffff >> panic: Failed to add resource to rman >>=20 >> So: >>=20 >> 0 >> vs. start: >> 0x600000000 >>=20 >> and: >>=20 >> 0xffffffff >> vs. end: >> 0x6000fffff >>=20 >> The 0x600000000..0x6000fffff range is not a range of RAM addresses >> [too large for that for the 8 GiByte RPi4B] and, so, is well above >> the 32 bit boundary too. >>=20 >> That is the only line referencing "memory window". A 32 bit initial = range >> for some reason. For reference, the source change in question was: >>=20 >> - DPRINTF(("rman_manage_region: <%s> request: start %#jx, end = %#jx\n", >> - rm->rm_descr, start, end)); >> + DPRINTF(("rman_manage_region: <%s> range %#jx..%#jx : = request: start %#jx, end %#jx\n", >> + rm->rm_descr, rm->rm_start, rm->rm_end, start, end)); >> if (start < rm->rm_start || end > rm->rm_end) >> return EINVAL; >>=20 >> I'll show a larger boot output context at the bottom of the message >> for reference. >>=20 >>>> r =3D int_alloc_resource(M_NOWAIT); >>>> if (r =3D=3D NULL) >>>> return ENOMEM; >>>> . . . >>>> /* Check for any overlap with the current region. */ >>>> if (r->r_start <=3D s->r_end && r->r_end >=3D = s->r_start) { >>>> rv =3D EBUSY; >>>> goto out; >>>> } >>>>=20 >>>> /* Check for any overlap with the next region. */ >>>> t =3D TAILQ_NEXT(s, r_link); >>>> if (t && r->r_start <=3D t->r_end && r->r_end >=3D = t->r_start) { >>>> rv =3D EBUSY; >>>> goto out; >>>> } >>>> . . . >>>> out: >>>> mtx_unlock(rm->rm_mtx); >>>> return rv; >>>>=20 >>>> int_alloc_resource failure would be failure (NULL) from the: >>>>=20 >>>> struct resource_i *r; >>>>=20 >>>> r =3D malloc(sizeof *r, M_RMAN, malloc_flag | M_ZERO); >>>>=20 >>>> (associated with the M_NOWAIT argument). >>>>=20 >>>> The malloc failure would likely go in a very different direction. >>>>=20 >>>>=20 >>>>=20 >>>> Side note: looks like the EBUSY cases leak what r references. >>>=20 >>> Still true in the newer code. >>>=20 >>>>> Probably I should fix pci_host_generic.c to handle translation = properly however. >>>>> I can work on a patch for that. >>>>=20 >>=20 >> For reference: >>=20 >> . . . >> pcib0: mem = 0x7d500000-0x7d50930f irq 80,81 on simplebus2 >> pcib0: parsing FDT for ECAM0: >> pcib0: PCI addr: 0xc0000000, CPU addr: 0x600000000, Size: 0x40000000 >> pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 >> pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 >> pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 >> pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 >> pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 >> pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 >> pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 >> pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 >> pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 >> pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 >> pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 >> pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 >> pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 >> pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 >> pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 >> pcib0: Bus is not cache-coherent >> rman_reserve_resource_bound: request: = [0xfd500000, 0xfd50930f], length 0x9310, flags 100, device pcib0 >> rman_reserve_resource_bound: trying 0x1fff <0xfd500000,0x930f> >> rman_reserve_resource_bound: tried 0x1fff <0xfd500000,0x930f> >> rman_reserve_resource_bound: tried 0x31bfffff <0xfd500000,0x930f> >> rman_reserve_resource_bound: tried 0x33298fff <0xfd500000,0x930f> >> rman_reserve_resource_bound: tried 0x39bf1fff <0xfd500000,0x930f> >> rman_reserve_resource_bound: tried 0x39c02fff <0xfd500000,0x930f> >> rman_reserve_resource_bound: tried 0x39c06fff <0xfd500000,0x930f> >> rman_reserve_resource_bound: tried 0x39c07fff <0xfd500000,0x930f> >> rman_reserve_resource_bound: tried 0x39c08fff <0xfd500000,0x930f> >> rman_reserve_resource_bound: tried 0x39c2afff <0xfd500000,0x930f> >> rman_reserve_resource_bound: tried 0x39c36fff <0xfd500000,0x930f> >> rman_reserve_resource_bound: tried 0x39c37fff <0xfd500000,0x930f> >> rman_reserve_resource_bound: tried 0x3b03ffff <0xfd500000,0x930f> >> rman_reserve_resource_bound: tried 0x3b04ffff <0xfd500000,0x930f> >> rman_reserve_resource_bound: tried 0x3b2fffff <0xfd500000,0x930f> >> rman_reserve_resource_bound: tried 0x3ee61fff <0xfd500000,0x930f> >> rman_reserve_resource_bound: tried 0x3ee63fff <0xfd500000,0x930f> >> rman_reserve_resource_bound: tried 0x3fffffff <0xfd500000,0x930f> >> rman_reserve_resource_bound: tried 0xfbffffff <0xfd500000,0x930f> >> considering [0xfc000000, 0xfd5d1fff] >> truncated region: [0xfd500000, 0xfd50930f]; size 0x9310 (requested = 0x9310) >> candidate region: [0xfd500000, 0xfd50930f], size 0x9310 >> splitting region in three parts: [0xfc000000, 0xfd4fffff]; = [0xfd500000, 0xfd50930f]; [0xfd509310, 0xfd5d1fff] >> rman_manage_region: range 0..0xffffffffffffffff : = request: start 0xc0000000, end 0xffffffff >> pcib0: hardware identifies as revision 0x304. >> pcib0: note: reported link speed is 5.0 GT/s. >> rman_reserve_resource_bound: request: [0x51, 0x51], = length 0x1, flags 0, device pcib0 >> rman_reserve_resource_bound: trying 0 <0x51,0> >> rman_reserve_resource_bound: tried 0 <0x51,0> >> rman_reserve_resource_bound: tried 0x1 <0x51,0> >> rman_reserve_resource_bound: tried 0x2 <0x51,0> >> rman_reserve_resource_bound: tried 0x3 <0x51,0> >> rman_reserve_resource_bound: tried 0x4 <0x51,0> >> rman_reserve_resource_bound: tried 0x5 <0x51,0> >> rman_reserve_resource_bound: tried 0x6 <0x51,0> >> rman_reserve_resource_bound: tried 0x7 <0x51,0> >> rman_reserve_resource_bound: tried 0xc <0x51,0> >> rman_reserve_resource_bound: tried 0xd <0x51,0> >> rman_reserve_resource_bound: tried 0xe <0x51,0> >> rman_reserve_resource_bound: tried 0xf <0x51,0> >> rman_reserve_resource_bound: tried 0x10 <0x51,0> >> rman_reserve_resource_bound: tried 0x11 <0x51,0> >> rman_reserve_resource_bound: tried 0x12 <0x51,0> >> rman_reserve_resource_bound: tried 0x17 <0x51,0> >> rman_reserve_resource_bound: tried 0x18 <0x51,0> >> rman_reserve_resource_bound: tried 0x1a <0x51,0> >> rman_reserve_resource_bound: tried 0x1b <0x51,0> >> rman_reserve_resource_bound: tried 0x22 <0x51,0> >> rman_reserve_resource_bound: tried 0x23 <0x51,0> >> rman_reserve_resource_bound: tried 0x24 <0x51,0> >> rman_reserve_resource_bound: tried 0x25 <0x51,0> >> rman_reserve_resource_bound: tried 0x26 <0x51,0> >> rman_reserve_resource_bound: tried 0x27 <0x51,0> >> rman_reserve_resource_bound: tried 0x28 <0x51,0> >> rman_reserve_resource_bound: tried 0x29 <0x51,0> >> rman_reserve_resource_bound: tried 0x4e <0x51,0> >> rman_reserve_resource_bound: tried 0x4f <0x51,0> >> considering [0x50, 0xffffffffffffffff] >> truncated region: [0x51, 0x51]; size 0x1 (requested 0x1) >> candidate region: [0x51, 0x51], size 0x1 >> splitting region in three parts: [0x50, 0x50]; [0x51, 0x51]; [0x52, = 0xffffffffffffffff] >> pci0: on pcib0 >> rman_manage_region: range 0..0xff : = request: start 0, end 0xff >> rman_reserve_resource_bound: request: [0, = 0], length 0x1, flags 0, device pci0 >> rman_reserve_resource_bound: trying 0xff <0,0> >> considering [0, 0xff] >> truncated region: [0, 0]; size 0x1 (requested 0x1) >> candidate region: [0, 0], size 0x1 >> allocating from the beginning >> pci0: domain=3D0, physical bus=3D0 >> found-> vendor=3D0x14e4, dev=3D0x2711, revid=3D0x00 >> domain=3D0, bus=3D0, slot=3D0, func=3D0 >> class=3D06-04-00, hdrtype=3D0x01, mfdev=3D0 >> cmdreg=3D0x0000, statreg=3D0x0010, cachelnsz=3D0 (dwords) >> lattimer=3D0x00 (0 ns), mingnt=3D0x00 (0 ns), maxlat=3D0x00 (0 = ns) >> intpin=3Da, irq=3D0 >> powerspec 3 supports D0 D3 current D0 >> secbus=3D1, subbus=3D1 >> rman_reserve_resource_bound: request: = [0x1, 0x1], length 0x1, flags 0, device (null) >> rman_reserve_resource_bound: trying 0 <0x1,0> >> rman_reserve_resource_bound: tried 0 <0x1,0> >> considering [0x1, 0xff] >> truncated region: [0x1, 0x1]; size 0x1 (requested 0x1) >> candidate region: [0x1, 0x1], size 0x1 >> allocating from the beginning >> pcib1: irq 91 at device 0.0 on pci0 >> rman_manage_region: range 0..0xff : request: = start 0x1, end 0x1 >> pcib0: rman_reserve_resource: start=3D0xc0000000, end=3D0xc00fffff, = count=3D0x100000 >> rman_reserve_resource_bound: request: [0xc0000000, = 0xc00fffff], length 0x100000, flags 102, device pcib1 >> rman_reserve_resource_bound: trying 0xffffffff <0xc0000000,0xfffff> >> considering [0xc0000000, 0xffffffff] >> truncated region: [0xc0000000, 0xc00fffff]; size 0x100000 (requested = 0x100000) >> candidate region: [0xc0000000, 0xc00fffff], size 0x100000 >> allocating from the beginning >> rman_manage_region: range 0..0xffffffff : = request: start 0x600000000, end 0x6000fffff >> panic: Failed to add resource to rman >> cpuid =3D 0 >> time =3D 1 >> KDB: stack backtrace: >> db_trace_self() at db_trace_self >> db_trace_self_wrapper() at db_trace_self_wrapper+0x38 >> vpanic() at vpanic+0x1a4 >> panic() at panic+0x48 >> pcib_add_window_resources() at pcib_add_window_resources+0xf4 >> pcib_alloc_window() at pcib_alloc_window+0xfc >> pcib_attach_common() at pcib_attach_common+0xa18 >> pcib_attach() at pcib_attach+0x14 >> device_attach() at device_attach+0x3fc >> device_probe_and_attach() at device_probe_and_attach+0x80 >> bus_generic_attach() at bus_generic_attach+0x1c >> pci_attach() at pci_attach+0xec >> device_attach() at device_attach+0x3fc >> device_probe_and_attach() at device_probe_and_attach+0x80 >> bus_generic_attach() at bus_generic_attach+0x1c >> device_attach() at device_attach+0x3fc >> device_probe_and_attach() at device_probe_and_attach+0x80 >> bus_generic_new_pass() at bus_generic_new_pass+0x100 >> bus_generic_new_pass() at bus_generic_new_pass+0xb0 >> bus_generic_new_pass() at bus_generic_new_pass+0xb0 >> bus_generic_new_pass() at bus_generic_new_pass+0xb0 >> bus_set_pass() at bus_set_pass+0x50 >> mi_startup() at mi_startup+0x1e0 >> virtdone() at virtdone+0x68 >> KDB: enter: panic >> [ thread pid 0 tid 100000 ] >> Stopped at kdb_enter+0x4c: str xzr, [x19, #1280] >> db>=20 >>=20 >=20 > FYI: >=20 > bcm2711-peripherals.pdf "Figure 1. BCM2711 Address Maps" > documents the PCIe address space range as (their "_" > notation): >=20 > 0x6_0000_0000..0x7_FFFF_FFFF >=20 > for both of the address maps: >=20 > 'Full 35-bit Address Map' > 'ARM view of the Address Map in "Low Peripheral" mode' >=20 > The Low Peripheral mode is what is in use. >=20 > So the 0x6_0000_0000..0x6_000f_ffff range looks > to be a valid ARM system address space for the > PCIe to map to. It looks to me like in sys/dev/pci/pci_pci.c the: static void pcib_probe_windows(struct pcib_softc *sc) { . . . pcib_alloc_window(sc, &sc->mem, SYS_RES_MEMORY, 0, 0xffffffff); . . . is just inappropriately restrictive about where in the system address space a PCIe can validly be mapped to on the high end. That, in turn, leads to the rejection on the RPi4B now that the range use is checked. More: Even the size (0xffffffff-0)+1 is smaller than the size of the range that the BCM2711 documentation indicates as reserved for PCIe (not that the fdt indicates to use the upper half of that range at all): (0x7_FFFF_FFFF - 0x6_0000_0000)+1 =3D=3D 0x1_FFFF_FFFF + 1 (I'm not so sure that this matters for the RPi4B. It is not a great example of the generality of PCIe uses. But might there be contexts where it does matter?) The lower bound being 0 allows for the actual low bound. But I wonder if use of the fdt (or ACPI) information should be possible to then indicate a actual low bound to check against instead. (A smaller high bound may be reasonable, at least for a RPi4B like context.) =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Tue Feb 13 22:43:00 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TZGZD0Lx2z59ywb for ; Tue, 13 Feb 2024 22:43:16 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic305-21.consmr.mail.gq1.yahoo.com (sonic305-21.consmr.mail.gq1.yahoo.com [98.137.64.84]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4TZGZC1CPrz4gXZ for ; Tue, 13 Feb 2024 22:43:15 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=eCSYUrWt; dmarc=pass (policy=reject) header.from=yahoo.com; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.64.84 as permitted sender) smtp.mailfrom=marklmi@yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1707864192; bh=MdAHl6ZC9yNprS7azh2dH7304E1qbJesSMAlvu3MJOc=; h=From:Subject:Date:To:References:From:Subject:Reply-To; b=eCSYUrWtYc1BDLU04egYaXXu4UTve28JjEUW9PsyWPgFKHK7U9EJwdSO1k9tdB1TQt45I2m4/b+i8KoHt5QYJXYE/lVTHqeZqLxGWQcChVdlHlZ04VUTA5FFf8cSt90H5AHr9PVUPiBiVKxoIRirEZ+Kjdr1rTNzo5TDok6rP+RBlxlVcAtNWnVHy46+IzbUit2UM18tglDtRrfBEAZOyN5g9qTNcextp0qeSPEb93RGl+6gxkoEryyGoLFGHDl/btg3cRycS7VmE1WVdPwz0zZtx8jP61WfDrf7bpr6XbkTSrQxM3/bqlV2oHnGMiMJi57zE0PD78KyWxU7f+NSBQ== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1707864192; bh=jLeASvBlZHfD4Ab7yCku2OqoZgBpzXpz/vDnGgz4EM6=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=i6eiwXfnGyAarOc7Lbi0joaCLZO0S3S0B/e0vVyx5P4XvXWav0iyJus6DEVwpcgP+zsy8BzN9NiGihJWGYHbV3+3GBq+RMREJoMau2BVOi43jKzniNDLUGfgUMKTyCXcEfcs7pDaJJXgLQaBBDQfV7OgOZIKSKvUBrmMUT/fFI9jfF23yto/IKajjmWF1DgiVX1F3okNmzKNcKSlJ+n+4mwdMOaHrWlAvFH+aTop994i66HVMcTJFTi3vawrIcX3RGrxS8Tb3Du77OCtSvCNQESN3iUmF9y5Y69gycObvxtuO67KLF7usiJTbHFs27W1USljZc79of33Fp4YU1nXLA== X-YMail-OSG: z_Dw5VgVM1nMuOhu6w.2WhNrowSPXeEN4k65seQeXvlBKVqzekJxzG_Cz_arb.c pXo3076b.7b_71Tbyx4TA36FwzxTO6qgHqQT04KP2QquR7ylfs_wQZHghuTsSCWzFP0_7wWQvpsQ 8kPwku0EY6965vvKzhMPZAHdd3PK7i1qXAlUlbx4_NKlKT5GxPd25zbwr2W80.p3sa8NPVgS7YEj BksD5tDSeGWvr7q84eAvHbkf_esiA0dUIZf8lmNjvSD_a8jHlwQiZiLOWf3K4zCW578_17i1cxIB 0L2UfgbffGrpoZcpBnqVgnsIlyhvfuFD6zuHYeVHDy3FC3KBsP_XDhvd7D0rkCRDmc4MKqHeOpv1 YmVgan0X1.JV5ZHnSx5zwpUzuaKlbWjh5JFtZ2u157XuSuabIo5.Ulg1uDm._Y6U3aZGbpwh3TYs 14b67tOPeo_GROqPvne0mYSxuteqJmLqEHU.9BKwCkRTeydRLPoSN4RyRjexRosQR7Bx4gaVvR6. iFTxvFCC3_bUB5wnQZ_WiHmdl54jmn0oaVAGt9z5otRmX1mfs9mNn6mWqqAdlyQqNSYaSzA2p2em 1Bh1lPUQnJk_2SafEGS6VNdJ_SMD0ScfHPo14PBdmvcJIXRCvqajabh5cXuooPh97yScqImpR5YY _z.Z7IAhkx7.NWbgl5p0qiGPpTO4hvu7oObST9VyT22B2nqhXRJ9jwOjCNvylq1KuxMgHcafYRL_ nBkudwBjiZBQv0TRnh5ttwSYAtn6LW4gIspIiJOd7Sld35KtouAo4dp9hkFrUNz8F0nsjaTs8oGW iQjtAMl_2vQAhJWNcd85tXZ3g.NBlp1_9Gmz1Fj__QchlXAHAGu6Kx.VG4qKRGKpdNqfrWk2FLIN b462mUIIpCA5gWJ1zFDiEFLs.rgI2wj98mxe1PWTP9TyenYbi_w_4lisiaBYmOJHsfaLRIaXYxQZ WJO9TjPshrNGURk48Wz2oHvBxHjr9tFUgE_JybgO.KvOIQ7xlVjVapXVKGE1TmGM1kz6GM0sirAY D06cxD9vq0.CPxhx7ukxTywPFJDxT8oNAq_e9CZ4Dxbk5NSSutlTeR4p45M5ucGgMK1B7icZMl6D GITk3i_K0mmIRSVCeLe53JjxwD7LBgYXyegSmBY65ZFGeoMvBv3.Z0FBYGQu8wG7MYG5myI78Hxa bmMjl3FSfaeILvs090gIV1IYVKd09345wsNR7z_NjFKHZK9czIx5qnmy8Cp7VYfxk8_IztfX4ZYF oFUZKS5CA.1MVHD3li3uGeoD41zpX40ApGPYsH7lw6gEe3NaXrw65BA7dugvm3BxnqTnP3.AOroO sLuHMZUaoxmtwq2xIsUE__VwASlKBdtIITERG9H4Cr.sfD2fKAzoVDvWUTKKgoakb9twUqQDrza5 I179vVhnAgoDWy6bvi6ejS1hLUSeHThC5mpqWhnUG3s3vpMOP.AxOfSelvyJSXTN4UkWYEfOn2wU .gMeFBEvbElIKNMvA_mkbb4NfSQJbHbDEk4AFf.drrkdybJ2lnF7TppmbjyKsXKCwzR94McIX99b 3LFXzzW8Ti6sDOqkSkeI3D8wkzTNOTWVKcQrMAdFWA8fn1HeQ2N7srDn7Su06DVMoEiPr7wZ.dxg Hh5JOrjexUYFtSWB6lpRcct0UHD..VGV28kgTAFAbfwK7EAjJPx_UnN6ZE_J9psKwe1iN7DW1aHw Dv0JdLYgmnccX1H1urdSamQyqsng9bM9cvw1.ddqFk64JIHl.GMQfGvkigY7vMDnt4NTrUNn7k8O oQA3TSgL43i3ZNCfOSyWYyjz3uQ9caZwFxNv702iLNMs2IdYIwybkjLB6GxqlO03KzDJLxrxni1G cTBcAfC6E_jNHeSmwRqX9bEjmn_IfmeRxueQiIz82pOM3VNYujd0Q914X5QSunlGuJCVRC9vZP_i a_VJZglAbG3lNIUIuq8Nh1wbfYzQA7qS2JeVVXmagLW0RayNzyEQ1jjfk6IR6PDbqoNPuz1xhPl4 mAsoQTycnv4GNL3zXM2SBE8PvKcCuB5wAJJl5outMxiMBhbaYo8w9sY2SkDzHBzeF5hGIvOeBaHV ISb1Em5bDUPzQmfh11cUVbSN4bwJTMIBFquyVPWpaR1urlzY29UoGR1Wq5QxY.QdH2ls8WD4LcnN GI5uJx7GyKGX3wSm8LCA04PngkMAjRGwWIBs2IHEjvE7Z5kC8PJ7fBOQTc3kCcEdZ2ADuxBw0R3E yq3UaTfT8X138uxb.d25TLRssz2Gges5ix0sDcEVt0DbRm9niFbleMgwTvxNIHQ-- X-Sonic-MF: X-Sonic-ID: 1fb44ae1-2855-4538-93a3-eebdfc9631c9 Received: from sonic.gate.mail.ne1.yahoo.com by sonic305.consmr.mail.gq1.yahoo.com with HTTP; Tue, 13 Feb 2024 22:43:12 +0000 Received: by hermes--production-gq1-5c57879fdf-tnlsv (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 3a15fc2828f78727da873a48e67b6a26; Tue, 13 Feb 2024 22:43:11 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.400.31\)) Subject: Rock64 snapshot on e.MMC media fails to mount root (no microsd card involved) Message-Id: <657E46C8-B4C4-4C80-BA12-D351F54D4911@yahoo.com> Date: Tue, 13 Feb 2024 14:43:00 -0800 To: FreeBSD ARM List X-Mailer: Apple Mail (2.3774.400.31) References: <657E46C8-B4C4-4C80-BA12-D351F54D4911.ref@yahoo.com> X-Spamd-Bar: --- X-Spamd-Result: default: False [-4.00 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; TO_DN_ALL(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; RCPT_COUNT_ONE(0.00)[1]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_TRACE(0.00)[0:+]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; MLMMJ_DEST(0.00)[freebsd-arm@freebsd.org]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; MID_RHS_MATCH_FROM(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.64.84:from]; APPLE_MAILER_COMMON(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.64.84:from] X-Rspamd-Queue-Id: 4TZGZC1CPrz4gXZ I downloaded and expanded (via unxz) to get: = FreeBSD-15.0-CURRENT-arm64-aarch64-ROCK64-20240208-82bebc793658-268105.img= I then did: # dd = if=3DFreeBSD-15.0-CURRENT-arm64-aarch64-ROCK64-20240208-82bebc793658-26810= 5.img of=3D/dev/da3 bs=3D1m conv=3Dsync,fsync,fdatasync status=3Dprogress onto e.MMC media. The attempted boot got to the mount root stage but hang-up: . . . WARNING: WITNESS option enabled, expect reduced performance. Unresolved linked clock found: hdmi_phy Unresolved linked clock found: usb480m_phy Trying to mount root from ufs:/dev/ufs/rootfs [rw]... GEOM: mmcsd0: the secondary GPT header is not in the last LBA. mmcsd0: Error indicated: 4 Failed uhub1: 1 port with 1 removable, self powered uhub3: 2 ports with 2 removable, self powered uhub0: 1 port with 1 removable, self powered uhub2: 1 port with 1 removable, self powered random: unblocking device. And that is all the output. This is likely (in part?) because of using a mode requiring tuning(/re-tuning?) that FreeBSD does not support for the Rock64: mmcsd0: 125GB at = mmc0 150.0MHz/8bit/1016-block mmcsd0boot0: 4MB partition 1 at mmcsd0 mmcsd0boot1: 4MB partition 2 at mmcsd0 mmcsd0rpmb: 4MB partition 3 at mmcsd0 (DDR50/DDR52 would be the fastest mode not requiring tuning. SDR104 requires tuning, as do faster modes. FreeBSD's NULL implementations of tuning that are used absent actual support for the controller for such activity lie and report success.) Tuning has controller specific definitions involved for stepping the phase and such: such is not specified by the standard. (There is a command for sending tuning data but not for how to have the controller explore the alternative phases and such and then to set good value(s).) As reported earlier, USB booting also does not complete a root mount, despite U-Boot 2024.01 supporting USB for its activities. So the Rock64 may be microsd card only for FreeBSD for now. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Wed Feb 14 16:08:37 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TZjmT2M8wz5BFhr; Wed, 14 Feb 2024 16:08:41 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TZjmT1r5hz45dx; Wed, 14 Feb 2024 16:08:41 +0000 (UTC) (envelope-from jhb@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1707926921; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=h/9057JLSucUzDn0y2NbBakKBsDjDVq0e9ZdHAvInfg=; b=eb1klHDE9V+Mrn52KhDuXMf24igekCq5PL3xerBqtNKHKwuLwoSNHopkeN71y5NR05mO9R e2p3zhk0eNQKWXmjFdc9BIWabbfoC+JhzAL3A/cRH8Kp36SPsPehNWF0ChpRMdj78NijSi zb4CQvf2nk5HSdCqI9z8l4lNHGGIHHYb3mk/JY4qBRWPkjchoandiVxbrzTR3hd26gKtB7 BH83Pm0x6sCNfG471g23qFdMS9chDRdyPZa5IXMUs2/WybF3YkjsjYQRi/LqTDkRRcCYtL +vy4O9kVr+j57WTZhwWc91L0n5BBJuhf08LlArJja4NSsaRw4euWciwfxAjyYQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1707926921; a=rsa-sha256; cv=none; b=m5QuIKaGUme/KVnjhrF9h/5uQTk6X1nXKYiJUpYR9Ogfj7x0D7icsnCKGIYBRnJd9ACdy/ zZJ10gBWXLrlyNTwnit6+ujCsqqp0VbwOLcUbCS53MKq1DS4XelDdYbeAWrjNHYktxJI+t +GwDFvTVf1l1UaPpEpNX7IpbjOGkbS7zODtqq+OXtFGNNYlWwjsA4AvWywyZ3f7EqR3G9T 8pAasI/FxgOD+Qz6kjAN9dv7bb7Ei3YunKmEudrURZpZ9eQCmsBngtutpBdbSlA5DaoEne qfOYadsmsduN+pnDppanyNjvR+V12YiYz59fhDFzl0oZ/JOYMu3U3DPzRN0+Bw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1707926921; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=h/9057JLSucUzDn0y2NbBakKBsDjDVq0e9ZdHAvInfg=; b=uAFe+XVcMYwBqBiqFB7UQ+MXuGXXOeIFC5OwODxmSeMOoiK1QIBE5GPGNJh6/le+UIQFZc hHHHc2LXaBSmYobF9IRnclZOGUGPUvWU1JxBgDns9k1SNRiZh700lWlHfpZXBPj8KrRXXX vzENceLr4xHEEYCtPJdqPHYB42ciHbMFvnK+h0sTl1DFscieciyesjDXBJAZbS2FPAQW2y XYHD7dZc9qRnuJZ4Kh+NEDPy2HST8OUl06LYchEmAUQW3LmxPcCBZYide5x39aZEs4/qz/ Em5GJd1DtD6jcwOe74gRW5QVKy8e4N2nM1huWjy0C+SGnsXia2sGaJX0fNhMgg== Received: from [IPV6:2601:644:937c:5920:4c63:23c7:5c22:d7ba] (unknown [IPv6:2601:644:937c:5920:4c63:23c7:5c22:d7ba]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: jhb) by smtp.freebsd.org (Postfix) with ESMTPSA id 4TZjmS3sx7zNsB; Wed, 14 Feb 2024 16:08:40 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Message-ID: Date: Wed, 14 Feb 2024 08:08:37 -0800 List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: Recent commits reject RPi4B booting: pcib0 vs. pcib1 "rman_manage_region: request" leads to panic Content-Language: en-US To: Mark Millard Cc: FreeBSD ARM List , Current FreeBSD , Warner Losh References: <76AB969F-5BC5-4116-8AF4-3ED2CABEBBA5.ref@yahoo.com> <76AB969F-5BC5-4116-8AF4-3ED2CABEBBA5@yahoo.com> <1F704317-FDB8-4BDA-8A67-61CF48794DFE@yahoo.com> <9AFDF067-96E4-4E67-90D2-F40DAF3F138F@yahoo.com> <4C279710-5F88-4295-B1A4-7C395F3587E5@yahoo.com> <3A145420-399D-4EBD-9FF4-18924908AB1D@yahoo.com> <1298DF9C-0F82-4567-8E81-7332A608C7FC@yahoo.com> From: John Baldwin In-Reply-To: <1298DF9C-0F82-4567-8E81-7332A608C7FC@yahoo.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit On 2/12/24 5:57 PM, Mark Millard wrote: > On Feb 12, 2024, at 16:36, Mark Millard wrote: > >> On Feb 12, 2024, at 16:10, Mark Millard wrote: >> >>> On Feb 12, 2024, at 12:00, Mark Millard wrote: >>> >>>> [Gack: I was looking at the wrong vintage of source code, predating >>>> your changes: wrong system used.] >>>> >>>> >>>> On Feb 12, 2024, at 10:41, Mark Millard wrote: >>>> >>>>> On Feb 12, 2024, at 09:32, John Baldwin wrote: >>>>> >>>>>> On 2/9/24 8:13 PM, Mark Millard wrote: >>>>>>> Summary: >>>>>>> pcib0: mem 0x7d500000-0x7d50930f irq 80,81 on simplebus2 >>>>>>> pcib0: parsing FDT for ECAM0: >>>>>>> pcib0: PCI addr: 0xc0000000, CPU addr: 0x600000000, Size: 0x40000000 >>>>>>> . . . >>>>>>> rman_manage_region: request: start 0x600000000, end 0x6000fffff >>>>>>> panic: Failed to add resource to rman >>>>>> >>>>>> Hmmm, I suspect this is due to the way that bus_translate_resource works which is >>>>>> fundamentally broken. It rewrites the start address of a resource in-situ instead >>>>>> of keeping downstream resources separate from the upstream resources. For example, >>>>>> I don't see how you could ever release a resource in this design without completely >>>>>> screwing up your rman. That is, I expect trying to detach a PCI device behind a >>>>>> translating bridge that uses the current approach should corrupt the allocated >>>>>> resource ranges in an rman long before my changes. >>>>>> >>>>>> That said, that doesn't really explain the panic. Hmm, the panic might be because >>>>>> for PCI bridge windows the driver now passes RF_ACTIVE and the bus_translate_resource >>>>>> hack only kicks in the activate_resource method of pci_host_generic.c. >>>>>> >>>>>>> Detail: >>>>>>> . . . >>>>>>> pcib0: mem 0x7d500000-0x7d50930f irq 80,81 on simplebus2 >>>>>>> pcib0: parsing FDT for ECAM0: >>>>>>> pcib0: PCI addr: 0xc0000000, CPU addr: 0x600000000, Size: 0x40000000 >>>>>> >>>>>> This indicates this is a translating bus. >>>>>> >>>>>>> pcib1: irq 91 at device 0.0 on pci0 >>>>>>> rman_manage_region: request: start 0x1, end 0x1 >>>>>>> pcib0: rman_reserve_resource: start=0xc0000000, end=0xc00fffff, count=0x100000 >>>>>>> rman_reserve_resource_bound: request: [0xc0000000, 0xc00fffff], length 0x100000, flags 102, device pcib1 >>>>>>> rman_reserve_resource_bound: trying 0xffffffff <0xc0000000,0xfffff> >>>>>>> considering [0xc0000000, 0xffffffff] >>>>>>> truncated region: [0xc0000000, 0xc00fffff]; size 0x100000 (requested 0x100000) >>>>>>> candidate region: [0xc0000000, 0xc00fffff], size 0x100000 >>>>>>> allocating from the beginning >>>>>>> rman_manage_region: request: start 0x600000000, end 0x6000fffff >>> >>> What you later typed does not match: >>> >>> 0x600000000 >>> 0x6000fffff >>> >>> You later typed: >>> >>> 0x60000000 >>> 0x600fffffff >>> >>> This seems to have lead to some confusion from using the >>> wrong figure(s). >>> >>>>>> The fact that we are trying to reserve the CPU addresses in the rman is because >>>>>> bus_translate_resource rewrote the start address in the resource after it was allocated. >>>>>> >>>>>> That said, I can't see why rman_manage_region would actually fail. At this point the >>>>>> rman is empty (this is the first call to rman_manage_region for "pcib1 memory window"), >>>>>> so only the check that should be failing are the checks against rm_start and >>>>>> rm_end. For the memory window, rm_start is always 0, and rm_end is always >>>>>> 0xffffffff, so both the old (0xc00000000 - 0xc00fffff) and new (0x60000000 - 0x600fffffff) >>>>>> ranges are within those bounds. >>> >>> No: >>> >>> 0xffffffff >>> >>> .vs (actual): >>> >>> 0x600000000 >>> 0x6000fffff Ok, then this explains the failure if the "raw" addresses are above 4G. I have access to an emag I'm currently using to test fixes to pci_host_generic.c to avoid corrupting struct resource objects. I'll post the diff once I've got something verified to work. > It looks to me like in sys/dev/pci/pci_pci.c the: > > static void > pcib_probe_windows(struct pcib_softc *sc) > { > . . . > pcib_alloc_window(sc, &sc->mem, SYS_RES_MEMORY, 0, 0xffffffff); > . . . > > is just inappropriately restrictive about where in the system > address space a PCIe can validly be mapped to on the high end. > That, in turn, leads to the rejection on the RPi4B now that > the range use is checked. No, the physical register in PCI-PCI bridges is only 32-bits. Only the prefetchable BAR supports 64-bit addresses. This is why the host bridge is doing a translation from the CPU side (0x600000000) to the PCI BAR addresses (0xc0000000) so that the BAR addresses are down in the 32-bit address range. It's also true that many PCI devices only support 32-bit addresses in memory BARs. 64-bit BARs are an optional extension not universally supported. The translation here is somewhat akin to a type of MMU where the CPU addresses are mapped to PCI addresses. The problem here is that the PCI BAR resources need to "stay" as PCI addresses since we depend on being able to use rman_get_start/end to get the PCI addresses of allocated resources, but pci_host_generic.c currently rewrites the addresses. Probably I should remove rman_set_start/end entirely (Warner added them back in 2004) as the methods don't do anything to deal with the fallout that the rman.rm_list linked-list is no longer sorted by address once some addresses get rewritten, etc. -- John Baldwin From nobody Wed Feb 14 16:42:36 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TZkWs5jXmz50xl3 for ; Wed, 14 Feb 2024 16:42:49 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-ed1-x530.google.com (mail-ed1-x530.google.com [IPv6:2a00:1450:4864:20::530]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TZkWs3dSbz49yf for ; Wed, 14 Feb 2024 16:42:49 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ed1-x530.google.com with SMTP id 4fb4d7f45d1cf-561f8b8c058so1962883a12.0 for ; Wed, 14 Feb 2024 08:42:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20230601.gappssmtp.com; s=20230601; t=1707928968; x=1708533768; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=nU/C/L6NJboRjZ/WwavYS7/vNGqdk8Pa7agypyoDG/0=; b=FrfSwvqKY97+6UXh3/7Ae1lng/09/GfKdcFjGeicRnTM8wg7PhTJFu61zTmIkY6+2w Oh8aUoXgL3amc55cDJSTvj17wENDDU27GQw3i+q/R8JgioYMFyU+V0GzDRupHp7R959q RIh+dg+j4x4M6iVn5SMfO3vurAWZhXrCRVqv4sWQnCePW6ylbiH8GJC9v0fXUz03NloA +QemISutfbIYrzHvpwFW0zkYywz9VknZacRXusGbeN9iQ2TTJe9MLFinIEO0lTZg6fN6 CH/D//2bFwc3J+rn8/JaRpS1BLSwwDDMooMjDvd9SuEof2z1AxPOECJUNPDOMzyBRxWK k1jg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707928968; x=1708533768; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=nU/C/L6NJboRjZ/WwavYS7/vNGqdk8Pa7agypyoDG/0=; b=n8CP7VyOtp4KB0A2qjWKsEyeby7t9FdwL8eFgx5JQT86biksfLM4K8aceZwJPnXSz5 KOluWzohtuvix9ys+IR48vyc1TVgcYpYsFXHPGTcXFzJOO4ymQiSJ10Sdd/A340YEQic /u0fNjVGRwnBSPuDZt+4oYqAR/jruoTIVstdeaYmc3wP+nJyVwJs6ZcT8YftpMItWAvp WhTXFT8XovOZUHbU39xoChBaA2CZFNHHDMbRPQ1taG4ks7TMlQUg6BmuJREoGJrCc+dO uslAKC+3gWoOokWfyeU09J8OLw80+ycys3jsdZXTsXRupjEMF0Z9z/nxokoVwBv7HmdE X5iA== X-Forwarded-Encrypted: i=1; AJvYcCWmfoPN+N3CiMMGR0mA05mn8bzQ7IGYEO7lBV/OfWgPJeQKnpLLWuvxim6OQFhR4hbh0f+Ixxhq8UfeeTNslp5IYv3qgOmewQ== X-Gm-Message-State: AOJu0YzTJvv+O6ONBshdgSYwf3IzVfsxTAs7dj2Ye0mUMaybFb8+uarp mxXwZJF7FNGg6NwjUJMqwViVS7sSbXo7XlgsLc+UNPJDnc3t+vu+53g9etvC0uGzxHH9ENqC6Wc bIVdxJiySEfMXapTmJOK8F2LPY4pCRz/wmnw0+w== X-Google-Smtp-Source: AGHT+IGmRp0UgqfN56Q3LCOBf4380b7Ci5HLenp16NpKSk4pV0fO/E+qlL7A4PJ6U3MZJA+d8m4nyF1vP61+TxITDno= X-Received: by 2002:aa7:c1d8:0:b0:561:f7d9:d77e with SMTP id d24-20020aa7c1d8000000b00561f7d9d77emr2223033edp.10.1707928967583; Wed, 14 Feb 2024 08:42:47 -0800 (PST) List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 References: <76AB969F-5BC5-4116-8AF4-3ED2CABEBBA5.ref@yahoo.com> <76AB969F-5BC5-4116-8AF4-3ED2CABEBBA5@yahoo.com> <1F704317-FDB8-4BDA-8A67-61CF48794DFE@yahoo.com> <9AFDF067-96E4-4E67-90D2-F40DAF3F138F@yahoo.com> <4C279710-5F88-4295-B1A4-7C395F3587E5@yahoo.com> <3A145420-399D-4EBD-9FF4-18924908AB1D@yahoo.com> <1298DF9C-0F82-4567-8E81-7332A608C7FC@yahoo.com> In-Reply-To: From: Warner Losh Date: Wed, 14 Feb 2024 09:42:36 -0700 Message-ID: Subject: Re: Recent commits reject RPi4B booting: pcib0 vs. pcib1 "rman_manage_region: request" leads to panic To: John Baldwin Cc: Mark Millard , FreeBSD ARM List , Current FreeBSD Content-Type: multipart/alternative; boundary="0000000000006a28f106115a33e8" X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] X-Rspamd-Queue-Id: 4TZkWs3dSbz49yf X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated --0000000000006a28f106115a33e8 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Wed, Feb 14, 2024 at 9:08=E2=80=AFAM John Baldwin wrot= e: > On 2/12/24 5:57 PM, Mark Millard wrote: > > On Feb 12, 2024, at 16:36, Mark Millard wrote: > > > >> On Feb 12, 2024, at 16:10, Mark Millard wrote: > >> > >>> On Feb 12, 2024, at 12:00, Mark Millard wrote: > >>> > >>>> [Gack: I was looking at the wrong vintage of source code, predating > >>>> your changes: wrong system used.] > >>>> > >>>> > >>>> On Feb 12, 2024, at 10:41, Mark Millard wrote: > >>>> > >>>>> On Feb 12, 2024, at 09:32, John Baldwin wrote: > >>>>> > >>>>>> On 2/9/24 8:13 PM, Mark Millard wrote: > >>>>>>> Summary: > >>>>>>> pcib0: mem > 0x7d500000-0x7d50930f irq 80,81 on simplebus2 > >>>>>>> pcib0: parsing FDT for ECAM0: > >>>>>>> pcib0: PCI addr: 0xc0000000, CPU addr: 0x600000000, Size: > 0x40000000 > >>>>>>> . . . > >>>>>>> rman_manage_region: request: start > 0x600000000, end 0x6000fffff > >>>>>>> panic: Failed to add resource to rman > >>>>>> > >>>>>> Hmmm, I suspect this is due to the way that bus_translate_resource > works which is > >>>>>> fundamentally broken. It rewrites the start address of a resource > in-situ instead > >>>>>> of keeping downstream resources separate from the upstream > resources. For example, > >>>>>> I don't see how you could ever release a resource in this design > without completely > >>>>>> screwing up your rman. That is, I expect trying to detach a PCI > device behind a > >>>>>> translating bridge that uses the current approach should corrupt > the allocated > >>>>>> resource ranges in an rman long before my changes. > >>>>>> > >>>>>> That said, that doesn't really explain the panic. Hmm, the panic > might be because > >>>>>> for PCI bridge windows the driver now passes RF_ACTIVE and the > bus_translate_resource > >>>>>> hack only kicks in the activate_resource method of > pci_host_generic.c. > >>>>>> > >>>>>>> Detail: > >>>>>>> . . . > >>>>>>> pcib0: mem > 0x7d500000-0x7d50930f irq 80,81 on simplebus2 > >>>>>>> pcib0: parsing FDT for ECAM0: > >>>>>>> pcib0: PCI addr: 0xc0000000, CPU addr: 0x600000000, Size: > 0x40000000 > >>>>>> > >>>>>> This indicates this is a translating bus. > >>>>>> > >>>>>>> pcib1: irq 91 at device 0.0 on pci0 > >>>>>>> rman_manage_region: request: start 0x1, end 0= x1 > >>>>>>> pcib0: rman_reserve_resource: start=3D0xc0000000, end=3D0xc00ffff= f, > count=3D0x100000 > >>>>>>> rman_reserve_resource_bound: request: [0xc0000000, > 0xc00fffff], length 0x100000, flags 102, device pcib1 > >>>>>>> rman_reserve_resource_bound: trying 0xffffffff <0xc0000000,0xffff= f> > >>>>>>> considering [0xc0000000, 0xffffffff] > >>>>>>> truncated region: [0xc0000000, 0xc00fffff]; size 0x100000 > (requested 0x100000) > >>>>>>> candidate region: [0xc0000000, 0xc00fffff], size 0x100000 > >>>>>>> allocating from the beginning > >>>>>>> rman_manage_region: request: start > 0x600000000, end 0x6000fffff > >>> > >>> What you later typed does not match: > >>> > >>> 0x600000000 > >>> 0x6000fffff > >>> > >>> You later typed: > >>> > >>> 0x60000000 > >>> 0x600fffffff > >>> > >>> This seems to have lead to some confusion from using the > >>> wrong figure(s). > >>> > >>>>>> The fact that we are trying to reserve the CPU addresses in the > rman is because > >>>>>> bus_translate_resource rewrote the start address in the resource > after it was allocated. > >>>>>> > >>>>>> That said, I can't see why rman_manage_region would actually fail. > At this point the > >>>>>> rman is empty (this is the first call to rman_manage_region for > "pcib1 memory window"), > >>>>>> so only the check that should be failing are the checks against > rm_start and > >>>>>> rm_end. For the memory window, rm_start is always 0, and rm_end i= s > always > >>>>>> 0xffffffff, so both the old (0xc00000000 - 0xc00fffff) and new > (0x60000000 - 0x600fffffff) > >>>>>> ranges are within those bounds. > >>> > >>> No: > >>> > >>> 0xffffffff > >>> > >>> .vs (actual): > >>> > >>> 0x600000000 > >>> 0x6000fffff > > Ok, then this explains the failure if the "raw" addresses are above 4G. = I > have > access to an emag I'm currently using to test fixes to pci_host_generic.c > to > avoid corrupting struct resource objects. I'll post the diff once I've g= ot > something verified to work. > > > It looks to me like in sys/dev/pci/pci_pci.c the: > > > > static void > > pcib_probe_windows(struct pcib_softc *sc) > > { > > . . . > > pcib_alloc_window(sc, &sc->mem, SYS_RES_MEMORY, 0, 0xffffffff)= ; > > . . . > > > > is just inappropriately restrictive about where in the system > > address space a PCIe can validly be mapped to on the high end. > > That, in turn, leads to the rejection on the RPi4B now that > > the range use is checked. > > No, the physical register in PCI-PCI bridges is only 32-bits. Only the > prefetchable BAR supports 64-bit addresses. This is why the host bridge > is doing a translation from the CPU side (0x600000000) to the PCI BAR > addresses (0xc0000000) so that the BAR addresses are down in the 32-bit > address range. It's also true that many PCI devices only support 32-bit > addresses in memory BARs. 64-bit BARs are an optional extension not > universally supported. > > The translation here is somewhat akin to a type of MMU where the CPU > addresses are mapped to PCI addresses. The problem here is that the > PCI BAR resources need to "stay" as PCI addresses since we depend on > being able to use rman_get_start/end to get the PCI addresses of > allocated resources, but pci_host_generic.c currently rewrites the > addresses. > > Probably I should remove rman_set_start/end entirely (Warner added them > back in 2004) as the methods don't do anything to deal with the fallout > that the rman.rm_list linked-list is no longer sorted by address once > some addresses get rewritten, etc. > At the time, they made sense. Removing it, though may take some doing since we use it in about 284 places in sys/dev today... Somewhat more pervasive than I'd have thought they'd be... Warner --0000000000006a28f106115a33e8 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Wed, Feb 14, 2024 at 9:08=E2=80=AF= AM John Baldwin <jhb@freebsd.org&= gt; wrote:
On 2/= 12/24 5:57 PM, Mark Millard wrote:
> On Feb 12, 2024, at 16:36, Mark Millard <marklmi@yahoo.com> wrote:
>
>> On Feb 12, 2024, at 16:10, Mark Millard <marklmi@yahoo.com> wrote:
>>
>>> On Feb 12, 2024, at 12:00, Mark Millard <marklmi@yahoo.com> wrote:
>>>
>>>> [Gack: I was looking at the wrong vintage of source code, = predating
>>>> your changes: wrong system used.]
>>>>
>>>>
>>>> On Feb 12, 2024, at 10:41, Mark Millard <marklmi@yahoo.com> wrote: >>>>
>>>>> On Feb 12, 2024, at 09:32, John Baldwin <jhb@freebsd.org> wrote: >>>>>
>>>>>> On 2/9/24 8:13 PM, Mark Millard wrote:
>>>>>>> Summary:
>>>>>>> pcib0: <BCM2838-compatible PCI-express cont= roller> mem 0x7d500000-0x7d50930f irq 80,81 on simplebus2
>>>>>>> pcib0: parsing FDT for ECAM0:
>>>>>>> pcib0:=C2=A0 PCI addr: 0xc0000000, CPU addr: 0= x600000000, Size: 0x40000000
>>>>>>> . . .
>>>>>>> rman_manage_region: <pcib1 memory window>= ; request: start 0x600000000, end 0x6000fffff
>>>>>>> panic: Failed to add resource to rman
>>>>>>
>>>>>> Hmmm, I suspect this is due to the way that bus_tr= anslate_resource works which is
>>>>>> fundamentally broken.=C2=A0 It rewrites the start = address of a resource in-situ instead
>>>>>> of keeping downstream resources separate from the = upstream resources.=C2=A0 =C2=A0For example,
>>>>>> I don't see how you could ever release a resou= rce in this design without completely
>>>>>> screwing up your rman.=C2=A0 That is, I expect try= ing to detach a PCI device behind a
>>>>>> translating bridge that uses the current approach = should corrupt the allocated
>>>>>> resource ranges in an rman long before my changes.=
>>>>>>
>>>>>> That said, that doesn't really explain the pan= ic.=C2=A0 Hmm, the panic might be because
>>>>>> for PCI bridge windows the driver now passes RF_AC= TIVE and the bus_translate_resource
>>>>>> hack only kicks in the activate_resource method of= pci_host_generic.c.
>>>>>>
>>>>>>> Detail:
>>>>>>> . . .
>>>>>>> pcib0: <BCM2838-compatible PCI-express cont= roller> mem 0x7d500000-0x7d50930f irq 80,81 on simplebus2
>>>>>>> pcib0: parsing FDT for ECAM0:
>>>>>>> pcib0: PCI addr: 0xc0000000, CPU addr: 0x60000= 0000, Size: 0x40000000
>>>>>>
>>>>>> This indicates this is a translating bus.
>>>>>>
>>>>>>> pcib1: <PCI-PCI bridge> irq 91 at device= 0.0 on pci0
>>>>>>> rman_manage_region: <pcib1 bus numbers> = request: start 0x1, end 0x1
>>>>>>> pcib0: rman_reserve_resource: start=3D0xc00000= 00, end=3D0xc00fffff, count=3D0x100000
>>>>>>> rman_reserve_resource_bound: <PCIe Memory&g= t; request: [0xc0000000, 0xc00fffff], length 0x100000, flags 102, device pc= ib1
>>>>>>> rman_reserve_resource_bound: trying 0xffffffff= <0xc0000000,0xfffff>
>>>>>>> considering [0xc0000000, 0xffffffff]
>>>>>>> truncated region: [0xc0000000, 0xc00fffff]; si= ze 0x100000 (requested 0x100000)
>>>>>>> candidate region: [0xc0000000, 0xc00fffff], si= ze 0x100000
>>>>>>> allocating from the beginning
>>>>>>> rman_manage_region: <pcib1 memory window>= ; request: start 0x600000000, end 0x6000fffff
>>>
>>> What you later typed does not match:
>>>
>>> 0x600000000
>>> 0x6000fffff
>>>
>>> You later typed:
>>>
>>> 0x60000000
>>> 0x600fffffff
>>>
>>> This seems to have lead to some confusion from using the
>>> wrong figure(s).
>>>
>>>>>> The fact that we are trying to reserve the CPU add= resses in the rman is because
>>>>>> bus_translate_resource rewrote the start address i= n the resource after it was allocated.
>>>>>>
>>>>>> That said, I can't see why rman_manage_region = would actually fail.=C2=A0 At this point the
>>>>>> rman is empty (this is the first call to rman_mana= ge_region for "pcib1 memory window"),
>>>>>> so only the check that should be failing are the c= hecks against rm_start and
>>>>>> rm_end.=C2=A0 For the memory window, rm_start is a= lways 0, and rm_end is always
>>>>>> 0xffffffff, so both the old (0xc00000000 - 0xc00ff= fff) and new (0x60000000 - 0x600fffffff)
>>>>>> ranges are within those bounds.
>>>
>>> No:
>>>
>>> 0xffffffff
>>>
>>> .vs (actual):
>>>
>>> 0x600000000
>>> 0x6000fffff

Ok, then this explains the failure if the "raw" addresses are abo= ve 4G.=C2=A0 I have
access to an emag I'm currently using to test fixes to pci_host_generic= .c to
avoid corrupting struct resource objects.=C2=A0 I'll post the diff once= I've got
something verified to work.

> It looks to me like in sys/dev/pci/pci_pci.c the:
>
> static void
> pcib_probe_windows(struct pcib_softc *sc)
> {
> . . .
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 pcib_alloc_window(sc, &sc->me= m, SYS_RES_MEMORY, 0, 0xffffffff);
> . . .
>
> is just inappropriately restrictive about where in the system
> address space a PCIe can validly be mapped to on the high end.
> That, in turn, leads to the rejection on the RPi4B now that
> the range use is checked.

No, the physical register in PCI-PCI bridges is only 32-bits.=C2=A0 Only th= e
prefetchable BAR supports 64-bit addresses.=C2=A0 This is why the host brid= ge
is doing a translation from the CPU side (0x600000000) to the PCI BAR
addresses (0xc0000000) so that the BAR addresses are down in the 32-bit
address range.=C2=A0 It's also true that many PCI devices only support = 32-bit
addresses in memory BARs.=C2=A0 64-bit BARs are an optional extension not universally supported.

The translation here is somewhat akin to a type of MMU where the CPU
addresses are mapped to PCI addresses.=C2=A0 The problem here is that the PCI BAR resources need to "stay" as PCI addresses since we depend= on
being able to use rman_get_start/end to get the PCI addresses of
allocated resources, but pci_host_generic.c currently rewrites the
addresses.

Probably I should remove rman_set_start/end entirely (Warner added them
back in 2004) as the methods don't do anything to deal with the fallout=
that the rman.rm_list linked-list is no longer sorted by address once
some addresses get rewritten, etc.

At t= he time, they made sense. Removing it, though may take some doing
since we use it in about 284 places=C2=A0 in sys/dev today... Somewhat mor= e
pervasive than I'd have thought they'd be...
=
Warner=C2=A0
--0000000000006a28f106115a33e8-- From nobody Wed Feb 14 17:30:46 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TZlbD66n0z514xL; Wed, 14 Feb 2024 17:30:48 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TZlbD5g25z4JC0; Wed, 14 Feb 2024 17:30:48 +0000 (UTC) (envelope-from jhb@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1707931848; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LsxTKj68riQhYjKMYsHXvMxX5HZyz+blVVNaAspRYdQ=; b=Ex/4r+jTh0fbheJXSDpf0vXJmSWa3zv0eAdtzREBVc9imqX573yYCsBPSkdtNYyFTP+y2l W8mzN6zd8JfNUivuYBYzIH3gV5TL4VAatB8VNWDzo/wr+riWLTY5V6DRNJV0mzhJdMqjqT DpnFL5+FDv+lIVZmiW6TnyBFVsRizQpxJQoxBCuuhv3VpEv8e3fh1qWHCGdJqKLSR7j31b d3fV2c9FbZvV9+BkA72QI1GkeXn7oDYZP/S3zfD4epaTnJVM4unSBlvMXSLEQX+EprhDMA xyDDq1dM824qQEO7u1BqqhBy3YmiQTn8G/UPGuvpHcw9ekpiJhWYo8C0ZtK8UA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1707931848; a=rsa-sha256; cv=none; b=bX3SP/sCs26aosB184r6RqoAkJVaED0EQ4dlOvFqNskl9MO2Yitmz89EBtYII9NSIhttqp LkEW7MWmi3Uj/ckVj0B1kbUSj7NFTguLLMBU+CzpzkDrs4DCpXQ9LFi0Zk8OY0dvdpErGw 4aq/FAJa6SrQw5cmSqA6LDrdeR8GNt7YIMEt+0zIfTPsIHigiV+E+IfUvowFHC+yVuS/MR NvKDsdU17tm5nMX8z2qa0v+E80RWVzjwwp9P5/pxXy+WXvc6mbpWE7Zhl2JIZjffHsKpZF xcDa6sVcIIJnZ8Y8B4LqM35wyEi1lrZRujzC6qlSR4PPPndJ24t3Y0hIk6kfuw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1707931848; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LsxTKj68riQhYjKMYsHXvMxX5HZyz+blVVNaAspRYdQ=; b=udUTfm/tGhI+GH+nFU1mm7tEWns6FUgC4KvKRE7Khw/2wSgHKTu0+zn5vSYSIaFEw2cFJI dzpbBAybiOFrwrUeS1uPvvMJNZcqqbxDRaDoWRBM5vk4GgNPq2tzAAgAvcV7tb2fMTkYqd o2HMD4ShtNQfLGCQrn27eq3y+rP9IjYBKXE41Hg1tyo7PxTp5Xr09etxmU/n/t9Gkg/Kz5 1lhCx1UTXMPI4sxyoxzsT+aFRkL4veO2OXOsEy2YT43nnz90+atYBgrE6jJgch3cvF53ds cLc1ILUIhjsNfcyisvEHnuCoGhQL4FnJnigxxfc3t9C8OOCwIkHsVx91U0u7DA== Received: from [IPV6:2601:644:937c:5920:5974:74f9:c690:271e] (unknown [IPv6:2601:644:937c:5920:5974:74f9:c690:271e]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: jhb) by smtp.freebsd.org (Postfix) with ESMTPSA id 4TZlbD1hB6zNJH; Wed, 14 Feb 2024 17:30:48 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Message-ID: Date: Wed, 14 Feb 2024 09:30:46 -0800 List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: Recent commits reject RPi4B booting: pcib0 vs. pcib1 "rman_manage_region: request" leads to panic Content-Language: en-US To: Warner Losh Cc: Mark Millard , FreeBSD ARM List , Current FreeBSD References: <76AB969F-5BC5-4116-8AF4-3ED2CABEBBA5.ref@yahoo.com> <76AB969F-5BC5-4116-8AF4-3ED2CABEBBA5@yahoo.com> <1F704317-FDB8-4BDA-8A67-61CF48794DFE@yahoo.com> <9AFDF067-96E4-4E67-90D2-F40DAF3F138F@yahoo.com> <4C279710-5F88-4295-B1A4-7C395F3587E5@yahoo.com> <3A145420-399D-4EBD-9FF4-18924908AB1D@yahoo.com> <1298DF9C-0F82-4567-8E81-7332A608C7FC@yahoo.com> From: John Baldwin In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit On 2/14/24 8:42 AM, Warner Losh wrote: > On Wed, Feb 14, 2024 at 9:08 AM John Baldwin wrote: > >> On 2/12/24 5:57 PM, Mark Millard wrote: >>> On Feb 12, 2024, at 16:36, Mark Millard wrote: >>> >>>> On Feb 12, 2024, at 16:10, Mark Millard wrote: >>>> >>>>> On Feb 12, 2024, at 12:00, Mark Millard wrote: >>>>> >>>>>> [Gack: I was looking at the wrong vintage of source code, predating >>>>>> your changes: wrong system used.] >>>>>> >>>>>> >>>>>> On Feb 12, 2024, at 10:41, Mark Millard wrote: >>>>>> >>>>>>> On Feb 12, 2024, at 09:32, John Baldwin wrote: >>>>>>> >>>>>>>> On 2/9/24 8:13 PM, Mark Millard wrote: >>>>>>>>> Summary: >>>>>>>>> pcib0: mem >> 0x7d500000-0x7d50930f irq 80,81 on simplebus2 >>>>>>>>> pcib0: parsing FDT for ECAM0: >>>>>>>>> pcib0: PCI addr: 0xc0000000, CPU addr: 0x600000000, Size: >> 0x40000000 >>>>>>>>> . . . >>>>>>>>> rman_manage_region: request: start >> 0x600000000, end 0x6000fffff >>>>>>>>> panic: Failed to add resource to rman >>>>>>>> >>>>>>>> Hmmm, I suspect this is due to the way that bus_translate_resource >> works which is >>>>>>>> fundamentally broken. It rewrites the start address of a resource >> in-situ instead >>>>>>>> of keeping downstream resources separate from the upstream >> resources. For example, >>>>>>>> I don't see how you could ever release a resource in this design >> without completely >>>>>>>> screwing up your rman. That is, I expect trying to detach a PCI >> device behind a >>>>>>>> translating bridge that uses the current approach should corrupt >> the allocated >>>>>>>> resource ranges in an rman long before my changes. >>>>>>>> >>>>>>>> That said, that doesn't really explain the panic. Hmm, the panic >> might be because >>>>>>>> for PCI bridge windows the driver now passes RF_ACTIVE and the >> bus_translate_resource >>>>>>>> hack only kicks in the activate_resource method of >> pci_host_generic.c. >>>>>>>> >>>>>>>>> Detail: >>>>>>>>> . . . >>>>>>>>> pcib0: mem >> 0x7d500000-0x7d50930f irq 80,81 on simplebus2 >>>>>>>>> pcib0: parsing FDT for ECAM0: >>>>>>>>> pcib0: PCI addr: 0xc0000000, CPU addr: 0x600000000, Size: >> 0x40000000 >>>>>>>> >>>>>>>> This indicates this is a translating bus. >>>>>>>> >>>>>>>>> pcib1: irq 91 at device 0.0 on pci0 >>>>>>>>> rman_manage_region: request: start 0x1, end 0x1 >>>>>>>>> pcib0: rman_reserve_resource: start=0xc0000000, end=0xc00fffff, >> count=0x100000 >>>>>>>>> rman_reserve_resource_bound: request: [0xc0000000, >> 0xc00fffff], length 0x100000, flags 102, device pcib1 >>>>>>>>> rman_reserve_resource_bound: trying 0xffffffff <0xc0000000,0xfffff> >>>>>>>>> considering [0xc0000000, 0xffffffff] >>>>>>>>> truncated region: [0xc0000000, 0xc00fffff]; size 0x100000 >> (requested 0x100000) >>>>>>>>> candidate region: [0xc0000000, 0xc00fffff], size 0x100000 >>>>>>>>> allocating from the beginning >>>>>>>>> rman_manage_region: request: start >> 0x600000000, end 0x6000fffff >>>>> >>>>> What you later typed does not match: >>>>> >>>>> 0x600000000 >>>>> 0x6000fffff >>>>> >>>>> You later typed: >>>>> >>>>> 0x60000000 >>>>> 0x600fffffff >>>>> >>>>> This seems to have lead to some confusion from using the >>>>> wrong figure(s). >>>>> >>>>>>>> The fact that we are trying to reserve the CPU addresses in the >> rman is because >>>>>>>> bus_translate_resource rewrote the start address in the resource >> after it was allocated. >>>>>>>> >>>>>>>> That said, I can't see why rman_manage_region would actually fail. >> At this point the >>>>>>>> rman is empty (this is the first call to rman_manage_region for >> "pcib1 memory window"), >>>>>>>> so only the check that should be failing are the checks against >> rm_start and >>>>>>>> rm_end. For the memory window, rm_start is always 0, and rm_end is >> always >>>>>>>> 0xffffffff, so both the old (0xc00000000 - 0xc00fffff) and new >> (0x60000000 - 0x600fffffff) >>>>>>>> ranges are within those bounds. >>>>> >>>>> No: >>>>> >>>>> 0xffffffff >>>>> >>>>> .vs (actual): >>>>> >>>>> 0x600000000 >>>>> 0x6000fffff >> >> Ok, then this explains the failure if the "raw" addresses are above 4G. I >> have >> access to an emag I'm currently using to test fixes to pci_host_generic.c >> to >> avoid corrupting struct resource objects. I'll post the diff once I've got >> something verified to work. >> >>> It looks to me like in sys/dev/pci/pci_pci.c the: >>> >>> static void >>> pcib_probe_windows(struct pcib_softc *sc) >>> { >>> . . . >>> pcib_alloc_window(sc, &sc->mem, SYS_RES_MEMORY, 0, 0xffffffff); >>> . . . >>> >>> is just inappropriately restrictive about where in the system >>> address space a PCIe can validly be mapped to on the high end. >>> That, in turn, leads to the rejection on the RPi4B now that >>> the range use is checked. >> >> No, the physical register in PCI-PCI bridges is only 32-bits. Only the >> prefetchable BAR supports 64-bit addresses. This is why the host bridge >> is doing a translation from the CPU side (0x600000000) to the PCI BAR >> addresses (0xc0000000) so that the BAR addresses are down in the 32-bit >> address range. It's also true that many PCI devices only support 32-bit >> addresses in memory BARs. 64-bit BARs are an optional extension not >> universally supported. >> >> The translation here is somewhat akin to a type of MMU where the CPU >> addresses are mapped to PCI addresses. The problem here is that the >> PCI BAR resources need to "stay" as PCI addresses since we depend on >> being able to use rman_get_start/end to get the PCI addresses of >> allocated resources, but pci_host_generic.c currently rewrites the >> addresses. >> >> Probably I should remove rman_set_start/end entirely (Warner added them >> back in 2004) as the methods don't do anything to deal with the fallout >> that the rman.rm_list linked-list is no longer sorted by address once >> some addresses get rewritten, etc. >> > > At the time, they made sense. Removing it, though may take some doing > since we use it in about 284 places in sys/dev today... Somewhat more > pervasive than I'd have thought they'd be... Eh, I only find the one that I'm now removing: % git grep -E 'rman_set_(start|end)' sys/ sys/dev/pci/pci_host_generic.c: rman_set_start(r, start); sys/dev/pci/pci_host_generic.c: rman_set_end(r, end); sys/kern/subr_rman.c:rman_set_start(struct resource *r, rman_res_t start) sys/kern/subr_rman.c:rman_set_end(struct resource *r, rman_res_t end) sys/sys/rman.h:void rman_set_end(struct resource *_r, rman_res_t _end); sys/sys/rman.h:void rman_set_start(struct resource *_r, rman_res_t _start); Also, I managed to boot the emag I have access to this morning. I had to fix a few other bugs in acpi(4) for my changes in pci_host_generic to work but will post reviews later today. -- John Baldwin From nobody Wed Feb 14 17:57:33 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TZmBR1xltz518NQ for ; Wed, 14 Feb 2024 17:57:51 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic304-24.consmr.mail.gq1.yahoo.com (sonic304-24.consmr.mail.gq1.yahoo.com [98.137.68.205]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4TZmBQ6TXkz4N1R for ; Wed, 14 Feb 2024 17:57:50 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1707933467; bh=RbzmeBWXn/60twKqejHyraerRAaQJoY0gqzcIyAj/Mk=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=Ny5WtN3ziXLVjX8fnTAiuuFAgfEVyST4d8JN80B9xKuI90J8ghqmzQJWDiHrs+f+ITTzkkBnWfqLNDTGa9zBZGr+5jpGiKwqqmb9UMe5xYIK2Y2ob2x056TCl+yvojts7uzugIoGQO4BLs4+R1jmczHIaXAPfCExlfs2CZ33RsoRAcIl08uqZtn8jmcF6BglsakiQ5rVWgPyACap8jjb+kiVzmfBPXQuKA7+O3SEDOt9yUgA6F6Z+wjsBZsCwCWwpAeK5sREo2wq6ZO7QTjA1mHISO+jwDcqmAdq/rC+opJ9sz801Z2stCQtOarZo9l1V9xGJqXv7Sa0DB2yYTG8KQ== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1707933467; bh=vtJGLELHLITbF/ZS8UJ5bB2o/yXIkArHJCzIgKYvvbZ=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=nD81xY/2/hdG0oPuB8S8YE6NNnQYVGAmBPEy50Q+gdW+4jCk6vDZ0imJb2t0D8FQSBD3GzFhYivynZ7kW13qnfMN58sNmvGwLA/9Z/PUuKpTdfdKMoX5DICBAmKFql/bZlJzLRq0V4famoCOUqQk2JCgmWsMne/4rTwv2lMB5aHT7ZYMXmWFeUDEPC9VnH1e90yZT+p8hBkZnB6UFLsz/t1W/LQim6XTBg7MibqM5MAzKP/C2SUymvIb1xHI+5CEMUIFL6xAlM3hXubFLcCFZga04IBGXqur+8yHWWtpymQ0tC7+KvC6zhL5SfbKMFFk0UB1Ry6byzDZXEnf4PBdEw== X-YMail-OSG: KO2UigkVM1nKjO9OPgctBmpFB4XReArEo1fpGCjJO8QTnid2LPBPsLYQy_Gf0tx 1QXfFDOH5H779h9aYifRQ28SUe5Gh7Kzvjl0YiKwkmBbghWw8kxhWcTbGH31tJKsjSKZO_moBxhd cRsjWOGUCL0h8tUxyMQvpK_b0JO1tfqpm0AosY6uhikz19iAGsaxsizjfY_hR.oPtMceb5PDQ6mi BtbBwWDtJ3k_WPhsbopmh5bWwLAtbGiMljTvRW6XegVGj4VSoJHi5iVEkkqqQ7w2aUqSIYt0Tiiv 5krGRBEbuq6fATmlMxlaLraUxJDJ1z84c4JY5uJBUAM361DmMaEgTJ..eBuMLG2D8WQTuy5oJLZF qMfr8JXLGgG0B0XfEcPjGaIUxbsZd2z67LWwx8YZwNmkqKjBhS2B1UhCSyjTc6VSWIEIrDI5hig5 I_ZkEc3_ureATzFAadcavcxFQbKV8d6wioRTCgQtVpUCW0v7kwCghuAPRQ3wpdUpvmdBQlE935Hq efLuRnHDXckSisoLiSVpAFEUsXkBlUuYsI3Updq7LzkjTbC38Zh7qDtgc5rJuycv6_1Fq3wPquee D1QuCwuHYbzBBqf7cR49h2ES8KaG__PynC7lVsVEoMfcphw2sYo1lY_dP_PA4oIDVzw6Uy_Th0lC 2dFsK7sqstf2kT9CtJXEetS97SqAsES2lY1Y_gH6tKAfCS_63._lBCtKTSqQQWT_RTHPSlMto5xF 3YqJcBNqg0L96JHp7WzdNe4G4L79dYjJpIuLnZw0RRggoV_ptGgxM3_6jdLUCLx7MVr1D0BoRk5k s0Fxv6Ji1L3YWDbBFSzXEFaTIFQt.hz4L.sOQofhq6iWF183ozG_13N2.nstKTouAYict4hK_aN3 nGEqtB0rENvQ7MBNLcxE2bGGVS4gmBDMa2X3FhP9012sM8GNH2hmGItIeEEDdyOLvxG3CljTQNGA kbbVTKv8b1VT9B.DOmvlNpTdmLMznFAsoiecw1kl3qjOOOCMOkq1QB8p5lFG9K_k2OoRdLtyfEEM xl3T84U8X3o54RMGF_E.iEtQKumgpTuQId3bf5l269l_Zkh0F5Lbqqt.xQkMVWVZt9DOPMPsSNIs Mku1wWMbNEh9EU.cfmSm_A0r1aKU9cXE5vN9kNcGOvH8X3Ym5FS0uamERC9BEwQYhQcaNGUie6Cs PiiBTOq9itF2yqlrLSmMeIDA3EJ5_rxNzTpp965fLz2i.sKtnmAthUhiYaBSxaU5ycCsGpQf3BQM MD2SGZMr4nywFqDj44QCpLqzfMDVlXuZjSPLrt13FWy.g.nrXn7w3AP9ucD4GWGNJLfCkjkFHW9g 0E3iYH5lL8O5DkGTXpIhuAvjIJFlg4L61w_iteakW6s5rz8E41k5FWoP._8Jeq0KAL5sprLec3Nu KGh58FkXJLqImrClNv7Kd1ffhZAdHGxo07loU0Tys1.TL8mkFjUv1pNlBnW55mqmnsuq4iJK5RLu _p_jkZBeqx5Wc6_tLA9FwceQ_ORypt4x4CtW6_uEhZ1YvKSMXGkrssbzQQzVaOn51PwWgkFY_Yuq b7JTb_AaByzmj9NkywTLIBUUeQdPbIlt1OrmOeN5qfZ1OGCYoY.8AqRuPzRm6VDFazWEwwhNVGPG 7kt5RmCNC1rF8suKyb6vU3PVKwNpE7rvNy7.EMjQY1GDEQnMgliW83TAsJwbEXkpSR5erUfYpghv PtP26AGipFnD1rDPkZJZU.4vimyTd7pJ_ejUQXhJdlmY7hiDUdxW2RvS6vdUjKbBVXNHYprgJeC0 bQ.eReO13DNrozSFgK56ZvX4I9tML0QHkaITM1OypcWxqSpqxUL9Ffkm00T2cC5T2KeaWNSbxUpr c6238u191s6Do8BwsjrIMNNPSmaWxBMp4uDdIBlPWc3ZO8iIEq3omvSUcFjHiZ5sq9N7chBxhbTe sTOPEjR5BqhrF.pJVRk_Ve6YbbiJ9VqQ9vOVrltbbpF72bFyQvPrbkHB9hRy3HuAmwrt.0V1n2vt ML2GCrTd_HEUX.5KGsAiqEzzLr65p15wZafp2DFyxvJJz3KLIx1WBbOIMVwkZM4WnsH1PcIOxUJ. dbq2f0uv4nacMLjeyFXH4G5ZNgfJx3notEQZoPDKfRQ6k1ZPpMrHMPREDGWEIhioV_Dia3x.oxuW mtTkIOEWUlB6Q4XIDuzomeZIGK1BDpbhik_wlWqDYFk4_c_m8SiItjk1_fxiiob7AaxtdRNRjF_h ZHcmPj_GxvEeXPzdBG8G8Bl1aT8ocZOKlIxt7ZgviP9N.pfpaTHwQyNa4ZZybKRS0.tfDG8mS18t E X-Sonic-MF: X-Sonic-ID: d036ca9a-cc79-47b5-8043-c2dbc5079fab Received: from sonic.gate.mail.ne1.yahoo.com by sonic304.consmr.mail.gq1.yahoo.com with HTTP; Wed, 14 Feb 2024 17:57:47 +0000 Received: by hermes--production-gq1-5c57879fdf-tnlsv (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID a26232e89c8c9752c58f7c52f7c49bc1; Wed, 14 Feb 2024 17:57:44 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.400.31\)) Subject: Re: Recent commits reject RPi4B booting: pcib0 vs. pcib1 "rman_manage_region: request" leads to panic From: Mark Millard In-Reply-To: Date: Wed, 14 Feb 2024 09:57:33 -0800 Cc: FreeBSD ARM List , Current FreeBSD , Warner Losh Content-Transfer-Encoding: quoted-printable Message-Id: <36ECB040-7E09-47A9-AF71-DE546A78E9CA@yahoo.com> References: <76AB969F-5BC5-4116-8AF4-3ED2CABEBBA5.ref@yahoo.com> <76AB969F-5BC5-4116-8AF4-3ED2CABEBBA5@yahoo.com> <1F704317-FDB8-4BDA-8A67-61CF48794DFE@yahoo.com> <9AFDF067-96E4-4E67-90D2-F40DAF3F138F@yahoo.com> <4C279710-5F88-4295-B1A4-7C395F3587E5@yahoo.com> <3A145420-399D-4EBD-9FF4-18924908AB1D@yahoo.com> <1298DF9C-0F82-4567-8E81-7332A608C7FC@yahoo.com> To: John Baldwin X-Mailer: Apple Mail (2.3774.400.31) X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US] X-Rspamd-Queue-Id: 4TZmBQ6TXkz4N1R X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated On Feb 14, 2024, at 08:08, John Baldwin wrote: > On 2/12/24 5:57 PM, Mark Millard wrote: >> On Feb 12, 2024, at 16:36, Mark Millard wrote: >>> On Feb 12, 2024, at 16:10, Mark Millard wrote: >>>=20 >>>> On Feb 12, 2024, at 12:00, Mark Millard wrote: >>>>=20 >>>>> [Gack: I was looking at the wrong vintage of source code, = predating >>>>> your changes: wrong system used.] >>>>>=20 >>>>>=20 >>>>> On Feb 12, 2024, at 10:41, Mark Millard wrote: >>>>>=20 >>>>>> On Feb 12, 2024, at 09:32, John Baldwin wrote: >>>>>>=20 >>>>>>> On 2/9/24 8:13 PM, Mark Millard wrote: >>>>>>>> Summary: >>>>>>>> pcib0: mem = 0x7d500000-0x7d50930f irq 80,81 on simplebus2 >>>>>>>> pcib0: parsing FDT for ECAM0: >>>>>>>> pcib0: PCI addr: 0xc0000000, CPU addr: 0x600000000, Size: = 0x40000000 >>>>>>>> . . . >>>>>>>> rman_manage_region: request: start = 0x600000000, end 0x6000fffff >>>>>>>> panic: Failed to add resource to rman >>>>>>>=20 >>>>>>> Hmmm, I suspect this is due to the way that = bus_translate_resource works which is >>>>>>> fundamentally broken. It rewrites the start address of a = resource in-situ instead >>>>>>> of keeping downstream resources separate from the upstream = resources. For example, >>>>>>> I don't see how you could ever release a resource in this design = without completely >>>>>>> screwing up your rman. That is, I expect trying to detach a PCI = device behind a >>>>>>> translating bridge that uses the current approach should corrupt = the allocated >>>>>>> resource ranges in an rman long before my changes. >>>>>>>=20 >>>>>>> That said, that doesn't really explain the panic. Hmm, the = panic might be because >>>>>>> for PCI bridge windows the driver now passes RF_ACTIVE and the = bus_translate_resource >>>>>>> hack only kicks in the activate_resource method of = pci_host_generic.c. >>>>>>>=20 >>>>>>>> Detail: >>>>>>>> . . . >>>>>>>> pcib0: mem = 0x7d500000-0x7d50930f irq 80,81 on simplebus2 >>>>>>>> pcib0: parsing FDT for ECAM0: >>>>>>>> pcib0: PCI addr: 0xc0000000, CPU addr: 0x600000000, Size: = 0x40000000 >>>>>>>=20 >>>>>>> This indicates this is a translating bus. >>>>>>>=20 >>>>>>>> pcib1: irq 91 at device 0.0 on pci0 >>>>>>>> rman_manage_region: request: start 0x1, end = 0x1 >>>>>>>> pcib0: rman_reserve_resource: start=3D0xc0000000, = end=3D0xc00fffff, count=3D0x100000 >>>>>>>> rman_reserve_resource_bound: request: = [0xc0000000, 0xc00fffff], length 0x100000, flags 102, device pcib1 >>>>>>>> rman_reserve_resource_bound: trying 0xffffffff = <0xc0000000,0xfffff> >>>>>>>> considering [0xc0000000, 0xffffffff] >>>>>>>> truncated region: [0xc0000000, 0xc00fffff]; size 0x100000 = (requested 0x100000) >>>>>>>> candidate region: [0xc0000000, 0xc00fffff], size 0x100000 >>>>>>>> allocating from the beginning >>>>>>>> rman_manage_region: request: start = 0x600000000, end 0x6000fffff >>>>=20 >>>> What you later typed does not match: >>>>=20 >>>> 0x600000000 >>>> 0x6000fffff >>>>=20 >>>> You later typed: >>>>=20 >>>> 0x60000000 >>>> 0x600fffffff >>>>=20 >>>> This seems to have lead to some confusion from using the >>>> wrong figure(s). >>>>=20 >>>>>>> The fact that we are trying to reserve the CPU addresses in the = rman is because >>>>>>> bus_translate_resource rewrote the start address in the resource = after it was allocated. >>>>>>>=20 >>>>>>> That said, I can't see why rman_manage_region would actually = fail. At this point the >>>>>>> rman is empty (this is the first call to rman_manage_region for = "pcib1 memory window"), >>>>>>> so only the check that should be failing are the checks against = rm_start and >>>>>>> rm_end. For the memory window, rm_start is always 0, and rm_end = is always >>>>>>> 0xffffffff, so both the old (0xc00000000 - 0xc00fffff) and new = (0x60000000 - 0x600fffffff) >>>>>>> ranges are within those bounds. >>>>=20 >>>> No: >>>>=20 >>>> 0xffffffff >>>>=20 >>>> .vs (actual): >>>>=20 >>>> 0x600000000 >>>> 0x6000fffff >=20 > Ok, then this explains the failure if the "raw" addresses are above = 4G. I have > access to an emag I'm currently using to test fixes to = pci_host_generic.c to > avoid corrupting struct resource objects. I'll post the diff once = I've got > something verified to work. >=20 >> It looks to me like in sys/dev/pci/pci_pci.c the: >> static void >> pcib_probe_windows(struct pcib_softc *sc) >> { >> . . . >> pcib_alloc_window(sc, &sc->mem, SYS_RES_MEMORY, 0, = 0xffffffff); >> . . . >> is just inappropriately restrictive about where in the system >> address space a PCIe can validly be mapped to on the high end. >> That, in turn, leads to the rejection on the RPi4B now that >> the range use is checked. >=20 > No, the physical register in PCI-PCI bridges is only 32-bits. Only = the > prefetchable BAR supports 64-bit addresses. Just for my edification . . . As I understand, SYS_RES_MEMORY for the BCM2711 means the 35 bit addressing space in the BCM2711, not a PCIe device internal address range that corresponds. Am I wrong about that? If I'm wrong, what does identify the 35 bit addressing space in the BCM2711? If I'm correct, then the 0..0xffffffff seems to be from the wrong address space up front. Or, may be, the SYS_RES_MEMORY and the 0xffffffff argments are not related as I expected and the 0xffffffff is not a SYS_RES_MEMORY value? > This is why the host bridge > is doing a translation from the CPU side (0x600000000) to the PCI BAR > addresses (0xc0000000) so that the BAR addresses are down in the = 32-bit > address range. It's also true that many PCI devices only support = 32-bit > addresses in memory BARs. 64-bit BARs are an optional extension not > universally supported. >=20 > The translation here is somewhat akin to a type of MMU where the CPU > addresses are mapped to PCI addresses. The problem here is that the > PCI BAR resources need to "stay" as PCI addresses since we depend on > being able to use rman_get_start/end to get the PCI addresses of > allocated resources, but pci_host_generic.c currently rewrites the > addresses. >=20 > Probably I should remove rman_set_start/end entirely (Warner added = them > back in 2004) as the methods don't do anything to deal with the = fallout > that the rman.rm_list linked-list is no longer sorted by address once > some addresses get rewritten, etc. >=20 =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Wed Feb 14 18:16:02 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TZmbn2ynBz51hYk for ; Wed, 14 Feb 2024 18:16:21 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic315-54.consmr.mail.gq1.yahoo.com (sonic315-54.consmr.mail.gq1.yahoo.com [98.137.65.30]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4TZmbm1rGVz4R53 for ; Wed, 14 Feb 2024 18:16:20 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=jnlLwFf9; dmarc=pass (policy=reject) header.from=yahoo.com; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.65.30 as permitted sender) smtp.mailfrom=marklmi@yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1707934578; bh=STGOR7uwlL32Djb90xxLmXVD4396T33A28E2CKIUUiA=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=jnlLwFf956VErlaD5eg2zmji0xilni35r1lTBDgpORVFrxWhGdklGgZ7GbGxpJ+2LqQqw/C08smgogeLBRBxbNg7dIhkxi2WhMnsE2VAB0yC0R+FJMdpr+LLLS4P3MH1/RBiKqST85tVWsT0nYVQsoqzfKXZKVirxpcg6TRgzx+pWZu8Lo7vChW3ld2Gyc9bG6kkG8vBz7FyMu/tQiRXxB5gternbUnc4S++ncgNyPmrOnOGO54jikus0917FFhCD9JKAXI33voDotcIQQbty5NWcQstE1OKpyJqvjGRS9fXGcl+JSeBmyHW1m1+9LVoY+t+HByrV3wp+JJwrPFynA== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1707934578; bh=YWkWVdzVaGbxA38ur6vxKcaSvb0o6IjTw2Cd08kox5V=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=kD5tTLvB0g5ld5mAnxNqAwi7a9VYzb62aBeaVRbX0Y7HYB4X7RuDou1aAQHammI7uDA98E2agLyHOqJeM9eq1SbPBhwyEnZQpu1Z6xjuafHxv+GCcMdWh6neKxTnFUj7cUuboUOugtwTD8dVfIGjdKBoigjNfZAuZzNqWzeLSJeY4tKRtI1KeNsc8O4OEjxH/qtHle+PBuqT8BKBaBRB+472Tk1E4JuW6spQ1vh14ldkoG5noJCfivIr5IZLhVrVZUac0wmrCUfw65Yfex+f+XdiQQnFxtKZzgqAcWUQG9rD6Kn2TSfSukrENPfAqyfT+Zp/nUGTxJtqdKXQqBSO4A== X-YMail-OSG: qX7yZVUVM1ki2loZzACq3_QDIMR8yV4n2qYJDLeLGqBA79ALqRiojnCCbQywhtr 9.OmEH1SIgDl_bNVIG.gyCNZ_PBJtKa1bvMqPUqJxjy4I2agSU8k1w1_xFc9pds9azyCORktWtYS ada5jVmrCos0AENmnyH4yS.21_f3px6zirSM2_KQQecPA7mitEq4AY3seKZ_3mxhPR3nFdT7tvY5 3JVsDeggwAj5jlTK7I1KJ8mJ4dYuBYiVQXUSCM_AP5JH8iAjeqw3R3nOUJfkRHMf9ykKPogm.cCM ap3mo7Go7vyoiyhfRAW0eoIgn8YjPxNy0fok4w5Yb0Yelhw8HTTNrieDHmKL0vrOeaMVlyHV7ZsJ ovqOznhyz5RtnTZOmuEveMpdsMRi3C.AeJ0ngnZBNl0o6q8Gp4Y5gLaizc_O7ECO4yJZy79w9Zbz D4hP_BGjOUsIHsWJhivgsnfP1q0E19oSuttrMW8VYBTR5CkHPrhLL6IqYLlobEXIB22rr601OoWx hq5A27.QxKfDJSme4YN7EUXqZU50nsEh2hseWWlO9zM23O.Nw0cE8306TLzfGpfalkHOmUuEwyCi qHdLfjj8xK4FFEwCGsBeZENrZsuQvB47uXPKleDD7D7dqiXF.hY3s1zDqoooWdVkJ7oH2Boqxfw5 kZY0mk2HmY8WmOthvYF.eDXLYe.a1jm3kQ4a8zatqTymj5mL2S5ZFswWDTfgdeTpwcg7BJ2Qnz4I H3shA058piGmgMWyF1tsdIit2UwZ98U99AWXAU4cN0r5GKeSECinjyLOosnVFcSjiQynru6c9uBp vKLD3HiQtwHj5tg2BgcPuYHdQTZypNZfhLzdk6AN8.mrlM1BGjXy6_UhIiACF2e7pNvNIz6PmPWT tKtfbtyOx6OAfNJlzBZUOb9rVj4k0._pHrk5on94YOgjqHIEtNHqtObs44lpZgntUrWJq33RmsF6 4ax0UivhZPRhMRpM9PYf22z5T0wzPoujfuP51.itKSZ2VKAowLg0MysNknwq_EsxeSUaieEQUenU kWEmnpEtcHIs3hONv5sDJeQc3nD53PFp4ZkndgUy3M72g7Em06rH5vI5OXWNO2sqIaufEjtecF.3 M9H6xwMxO0KfgisA1kR.QNwtunzq12tRL.En_j__oNg4JwuPFcuq_.vIBxrCVBJvgUoIpTztZZA7 9Q4eSfmUNxLp8Pder65N620UBhU5g2PgI9fRCQm5bmuj2LUHQoxyVQ6uj8roE26GifO.e201jySq crXezzuAg_3b5NkCPrDR36RmXCIZXDQGAzIbr4HkPb4Y0LWvQgAfwxggDzIk6lcQ8LbAWa1coA_A o2lSEUeoWe4RxXToLMgmN_Wr_CNZjJDgQRVet0Rv0Yl0P3fqxKBI5rDzOHmGsNBX5gFVX3op9O.z 04nkVP.NLKPMOl561SAYTTvn3KEQSKrHnPpScwfgnRncsbepYw5u46c.McRbAt3BwfNgOgexSozo dLXvoYcfuLFN3qvBHUvR7maxL8SvqKnezGxFv.D3At6WtekvbqUPrdj_kc1A49.Wp3S.8SvEqA0i bDjAICLfOLveKGx006lUmpD0ou1PCNwiwffOx3_hny.yRDAIc0oILzCH_wGSEYb2woJkHhmc4Qgq BTwXVIaU5M.W.NzZ6O_.eMunhV3TI6rJNUNiMVLnB2d_4JE0h0jY29ne1BmrovqJEJS9vbGrtiz7 p9A5h10D.oSZ3oGVBP5kCXMLS5pt8O4WkGckfkwNZSSowQT0dkSTevmrwguoPfmo..0QRce1DWf4 qrzoSD7OrLvs2lo2iu8zeIZM1.vkSDu00ANinznflOH.vmAS4_5ISwdgo_NBkywcPIKFwwqkCh43 9HLe657WJeODJ4XLkT9ZZ2o98XsXGIL7cdMhl2xBhhMQQ.wRwonmJWhaV5DIYAsZl0SyXJDfdcYs wNtYYKIRvHgscAyWP9a5oXi2B4MIcvALHRmw2KAuHmUhRnUIWASmN8kcmXVlxy58oFcWRR_zneR8 nswZvsZUAzwiDL9KNbupkWa6vHqwUQ37csE_NhRqq8XqKbec2qY3N_Rblao0zXC2kdiBe4.7uIt7 Oy3DR8a6iPdvsHH4w59iD_VaccxLxDLZTiV9UPwnQ93NxASX.WZYptyEPbNtlkqy.5csDyI..Lr0 2zzVEk4w6CljfnUJlFvLXcal4ctCmQJ96Hv1d5JSb93j_aKnM2ltq2QGdqqLTzveSztoFRqj7aMZ rAayWhVjJ4I8LotDcvIa7F5o5iDIYY76jK8i5ne0Ty7g.9.0VCKac.E6yw3TiUF5IxvfhrogjZYq q564- X-Sonic-MF: X-Sonic-ID: 0d52a574-75f6-4567-869f-8a3446ac7e69 Received: from sonic.gate.mail.ne1.yahoo.com by sonic315.consmr.mail.gq1.yahoo.com with HTTP; Wed, 14 Feb 2024 18:16:18 +0000 Received: by hermes--production-gq1-5c57879fdf-tnlsv (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 75bd2ec618e725ea0b4cd6d8312a2072; Wed, 14 Feb 2024 18:16:13 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.400.31\)) Subject: Re: Recent commits reject RPi4B booting: pcib0 vs. pcib1 "rman_manage_region: request" leads to panic From: Mark Millard In-Reply-To: <36ECB040-7E09-47A9-AF71-DE546A78E9CA@yahoo.com> Date: Wed, 14 Feb 2024 10:16:02 -0800 Cc: FreeBSD ARM List , Current FreeBSD , Warner Losh Content-Transfer-Encoding: quoted-printable Message-Id: References: <76AB969F-5BC5-4116-8AF4-3ED2CABEBBA5.ref@yahoo.com> <76AB969F-5BC5-4116-8AF4-3ED2CABEBBA5@yahoo.com> <1F704317-FDB8-4BDA-8A67-61CF48794DFE@yahoo.com> <9AFDF067-96E4-4E67-90D2-F40DAF3F138F@yahoo.com> <4C279710-5F88-4295-B1A4-7C395F3587E5@yahoo.com> <3A145420-399D-4EBD-9FF4-18924908AB1D@yahoo.com> <1298DF9C-0F82-4567-8E81-7332A608C7FC@yahoo.com> <36ECB040-7E09-47A9-AF71-DE546A78E9CA@yahoo.com> To: John Baldwin X-Mailer: Apple Mail (2.3774.400.31) X-Spamd-Bar: --- X-Spamd-Result: default: False [-4.00 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; DKIM_TRACE(0.00)[yahoo.com:+]; TO_DN_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; MIME_TRACE(0.00)[0:+]; RCPT_COUNT_THREE(0.00)[4]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; MLMMJ_DEST(0.00)[freebsd-arm@freebsd.org]; APPLE_MAILER_COMMON(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.65.30:from]; RCVD_IN_DNSWL_NONE(0.00)[98.137.65.30:from] X-Rspamd-Queue-Id: 4TZmbm1rGVz4R53 Top posting a related but separate item: I looked up some old (2022-Dec-17) lspci -v output from a Linux boot. Note the "Memory at" value 600000000 (in the 35 bit BCM2711 address space) and the "(64-bit, non-prefetchable)" (and "[size=3D4K]"). 01:00.0 USB controller: VIA Technologies, Inc. VL805/806 xHCI USB 3.0 = Controller (rev 01) (prog-if 30 [XHCI]) Subsystem: VIA Technologies, Inc. VL805/806 xHCI USB 3.0 = Controller Device tree node: = /sys/firmware/devicetree/base/scb/pcie@7d500000/pci@0,0/usb@0,0 Flags: bus master, fast devsel, latency 0, IRQ 51 Memory at 600000000 (64-bit, non-prefetchable) [size=3D4K] Capabilities: [80] Power Management version 3 Capabilities: [90] MSI: Enable+ Count=3D1/4 Maskable- 64bit+ Capabilities: [c4] Express Endpoint, MSI 00 Capabilities: [100] Advanced Error Reporting Kernel driver in use: xhci_hcd "Memory at 600000000 (64-bit, non-prefetchable)": Violation of a PCIe standard? On Feb 14, 2024, at 09:57, Mark Millard wrote: > On Feb 14, 2024, at 08:08, John Baldwin wrote: >=20 >> On 2/12/24 5:57 PM, Mark Millard wrote: >>> On Feb 12, 2024, at 16:36, Mark Millard wrote: >>>> On Feb 12, 2024, at 16:10, Mark Millard wrote: >>>>=20 >>>>> On Feb 12, 2024, at 12:00, Mark Millard wrote: >>>>>=20 >>>>>> [Gack: I was looking at the wrong vintage of source code, = predating >>>>>> your changes: wrong system used.] >>>>>>=20 >>>>>>=20 >>>>>> On Feb 12, 2024, at 10:41, Mark Millard = wrote: >>>>>>=20 >>>>>>> On Feb 12, 2024, at 09:32, John Baldwin wrote: >>>>>>>=20 >>>>>>>> On 2/9/24 8:13 PM, Mark Millard wrote: >>>>>>>>> Summary: >>>>>>>>> pcib0: mem = 0x7d500000-0x7d50930f irq 80,81 on simplebus2 >>>>>>>>> pcib0: parsing FDT for ECAM0: >>>>>>>>> pcib0: PCI addr: 0xc0000000, CPU addr: 0x600000000, Size: = 0x40000000 >>>>>>>>> . . . >>>>>>>>> rman_manage_region: request: start = 0x600000000, end 0x6000fffff >>>>>>>>> panic: Failed to add resource to rman >>>>>>>>=20 >>>>>>>> Hmmm, I suspect this is due to the way that = bus_translate_resource works which is >>>>>>>> fundamentally broken. It rewrites the start address of a = resource in-situ instead >>>>>>>> of keeping downstream resources separate from the upstream = resources. For example, >>>>>>>> I don't see how you could ever release a resource in this = design without completely >>>>>>>> screwing up your rman. That is, I expect trying to detach a = PCI device behind a >>>>>>>> translating bridge that uses the current approach should = corrupt the allocated >>>>>>>> resource ranges in an rman long before my changes. >>>>>>>>=20 >>>>>>>> That said, that doesn't really explain the panic. Hmm, the = panic might be because >>>>>>>> for PCI bridge windows the driver now passes RF_ACTIVE and the = bus_translate_resource >>>>>>>> hack only kicks in the activate_resource method of = pci_host_generic.c. >>>>>>>>=20 >>>>>>>>> Detail: >>>>>>>>> . . . >>>>>>>>> pcib0: mem = 0x7d500000-0x7d50930f irq 80,81 on simplebus2 >>>>>>>>> pcib0: parsing FDT for ECAM0: >>>>>>>>> pcib0: PCI addr: 0xc0000000, CPU addr: 0x600000000, Size: = 0x40000000 >>>>>>>>=20 >>>>>>>> This indicates this is a translating bus. >>>>>>>>=20 >>>>>>>>> pcib1: irq 91 at device 0.0 on pci0 >>>>>>>>> rman_manage_region: request: start 0x1, = end 0x1 >>>>>>>>> pcib0: rman_reserve_resource: start=3D0xc0000000, = end=3D0xc00fffff, count=3D0x100000 >>>>>>>>> rman_reserve_resource_bound: request: = [0xc0000000, 0xc00fffff], length 0x100000, flags 102, device pcib1 >>>>>>>>> rman_reserve_resource_bound: trying 0xffffffff = <0xc0000000,0xfffff> >>>>>>>>> considering [0xc0000000, 0xffffffff] >>>>>>>>> truncated region: [0xc0000000, 0xc00fffff]; size 0x100000 = (requested 0x100000) >>>>>>>>> candidate region: [0xc0000000, 0xc00fffff], size 0x100000 >>>>>>>>> allocating from the beginning >>>>>>>>> rman_manage_region: request: start = 0x600000000, end 0x6000fffff >>>>>=20 >>>>> What you later typed does not match: >>>>>=20 >>>>> 0x600000000 >>>>> 0x6000fffff >>>>>=20 >>>>> You later typed: >>>>>=20 >>>>> 0x60000000 >>>>> 0x600fffffff >>>>>=20 >>>>> This seems to have lead to some confusion from using the >>>>> wrong figure(s). >>>>>=20 >>>>>>>> The fact that we are trying to reserve the CPU addresses in the = rman is because >>>>>>>> bus_translate_resource rewrote the start address in the = resource after it was allocated. >>>>>>>>=20 >>>>>>>> That said, I can't see why rman_manage_region would actually = fail. At this point the >>>>>>>> rman is empty (this is the first call to rman_manage_region for = "pcib1 memory window"), >>>>>>>> so only the check that should be failing are the checks against = rm_start and >>>>>>>> rm_end. For the memory window, rm_start is always 0, and = rm_end is always >>>>>>>> 0xffffffff, so both the old (0xc00000000 - 0xc00fffff) and new = (0x60000000 - 0x600fffffff) >>>>>>>> ranges are within those bounds. >>>>>=20 >>>>> No: >>>>>=20 >>>>> 0xffffffff >>>>>=20 >>>>> .vs (actual): >>>>>=20 >>>>> 0x600000000 >>>>> 0x6000fffff >>=20 >> Ok, then this explains the failure if the "raw" addresses are above = 4G. I have >> access to an emag I'm currently using to test fixes to = pci_host_generic.c to >> avoid corrupting struct resource objects. I'll post the diff once = I've got >> something verified to work. >>=20 >>> It looks to me like in sys/dev/pci/pci_pci.c the: >>> static void >>> pcib_probe_windows(struct pcib_softc *sc) >>> { >>> . . . >>> pcib_alloc_window(sc, &sc->mem, SYS_RES_MEMORY, 0, = 0xffffffff); >>> . . . >>> is just inappropriately restrictive about where in the system >>> address space a PCIe can validly be mapped to on the high end. >>> That, in turn, leads to the rejection on the RPi4B now that >>> the range use is checked. >>=20 >> No, the physical register in PCI-PCI bridges is only 32-bits. Only = the >> prefetchable BAR supports 64-bit addresses. >=20 > Just for my edification . . . >=20 > As I understand, SYS_RES_MEMORY for the BCM2711 > means the 35 bit addressing space in the BCM2711, > not a PCIe device internal address range that > corresponds. Am I wrong about that? >=20 > If I'm wrong, what does identify the 35 bit > addressing space in the BCM2711? >=20 > If I'm correct, then the 0..0xffffffff > seems to be from the wrong address space up > front. Or, may be, the SYS_RES_MEMORY and the > 0xffffffff argments are not related as I > expected and the 0xffffffff is not a > SYS_RES_MEMORY value? >=20 >> This is why the host bridge >> is doing a translation from the CPU side (0x600000000) to the PCI BAR >> addresses (0xc0000000) so that the BAR addresses are down in the = 32-bit >> address range. It's also true that many PCI devices only support = 32-bit >> addresses in memory BARs. 64-bit BARs are an optional extension not >> universally supported. >>=20 >> The translation here is somewhat akin to a type of MMU where the CPU >> addresses are mapped to PCI addresses. The problem here is that the >> PCI BAR resources need to "stay" as PCI addresses since we depend on >> being able to use rman_get_start/end to get the PCI addresses of >> allocated resources, but pci_host_generic.c currently rewrites the >> addresses. >>=20 >> Probably I should remove rman_set_start/end entirely (Warner added = them >> back in 2004) as the methods don't do anything to deal with the = fallout >> that the rman.rm_list linked-list is no longer sorted by address once >> some addresses get rewritten, etc. >>=20 =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Wed Feb 14 18:17:43 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TZmdQ53SMz51hfL; Wed, 14 Feb 2024 18:17:46 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TZmdQ48Jyz4SPB; Wed, 14 Feb 2024 18:17:46 +0000 (UTC) (envelope-from jhb@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1707934666; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1DODsuObWTMBP56lJdKpk5bTPZocrl/yKB0jED9XwG8=; b=j7KF7iCdBM3q+xjFzu/qjalhYwG85ocoYiuk+CueGTuaRUUSMP0hJTSvhue8D7hd2OA+41 LqrAHqxdwS8j5f42u4+Qa6CDoS6Ad8HIcy0vPpcE2YVtoW5ghU0B24xq6H9PWiWt6wsmh7 l4nVl9PvtJNZ/kl3aWfhbtO1PEfEZc/V3UdR/W66uJnS+ndm4vdPvfInuZoV9O09OCUaqe DOD58cei3s6fr/aC+SWsSQw6z4eLEy6kzGLQH50brcNKMZ882TNEdw0GBnp5zminMbq5l9 1+U5C1vxFn752BoeccHEbHqRQ01UEadjDv89yV9vSp/xXQuUxx1zBpxLBku87w== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1707934666; a=rsa-sha256; cv=none; b=bIPdZdiYNu6LKdPUyt1+pFK1AJcftwCxx165gomfdHvVd9TNdM3BghKkeLPdBvnwDOcWYP 7mUGXgJxEU7xoKpNC4RFgmu2GJgxCCiJsn1H3Rm0noMJvTH4lH/rbwOAgxmGtfhv+WeE63 te20zMVlgGx83eIfms957h3OmQOA/8SL/+JynX5Swnym9mck72W5jpviFCGmnD6vKfZL5Z 3Sya6yTWyNKs3mc8Md9/RP3GTZm6JoYhlssjCUT3W0thc0JlaqXwPpQncotYG1pkk+W5lD grmk9LGou+e7KnVZ3cvAkTcrSGZ1h75NSkEg6oMZ3gy7EIUl7Fj3LECJqv+Qtg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1707934666; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1DODsuObWTMBP56lJdKpk5bTPZocrl/yKB0jED9XwG8=; b=B4N5VvsKwOz1iO4el6vzX58bgtLdrCLmaZ6ZsBi0f60jduASnYBVL/UiNV0Hj98dBdbr1A POi91s2jxuYOgpDWDRiyLRDIgqlDpAQnsoQT8QXJQcNaeQqqhi+A6hEUGNKYVEWPAhsDOE 31M18h2g7OvwRDzDhVbVjHpxJLcpKGETfEQf4qdjZG46rkfFu6X8+cAWX6O6nn40svsGvL vecfYpYy0tjqkZDfe/1R+ph8AwIufdyZQkZtyYtCzf2WguUUDoIwdi+M7gAfNEXzUIWKkq wlBmhaqsv21KFOlarDHOk9dHsL9LnC0UYl56NoPqetFls48SO1BnEBoWadHRhg== Received: from [IPV6:2601:644:937c:5920:5974:74f9:c690:271e] (unknown [IPv6:2601:644:937c:5920:5974:74f9:c690:271e]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: jhb) by smtp.freebsd.org (Postfix) with ESMTPSA id 4TZmdP74cJzRgh; Wed, 14 Feb 2024 18:17:45 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Message-ID: Date: Wed, 14 Feb 2024 10:17:43 -0800 List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: Recent commits reject RPi4B booting: pcib0 vs. pcib1 "rman_manage_region: request" leads to panic Content-Language: en-US To: Mark Millard Cc: FreeBSD ARM List , Current FreeBSD , Warner Losh References: <76AB969F-5BC5-4116-8AF4-3ED2CABEBBA5.ref@yahoo.com> <76AB969F-5BC5-4116-8AF4-3ED2CABEBBA5@yahoo.com> <1F704317-FDB8-4BDA-8A67-61CF48794DFE@yahoo.com> <9AFDF067-96E4-4E67-90D2-F40DAF3F138F@yahoo.com> <4C279710-5F88-4295-B1A4-7C395F3587E5@yahoo.com> <3A145420-399D-4EBD-9FF4-18924908AB1D@yahoo.com> <1298DF9C-0F82-4567-8E81-7332A608C7FC@yahoo.com> <36ECB040-7E09-47A9-AF71-DE546A78E9CA@yahoo.com> From: John Baldwin In-Reply-To: <36ECB040-7E09-47A9-AF71-DE546A78E9CA@yahoo.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit On 2/14/24 9:57 AM, Mark Millard wrote: > On Feb 14, 2024, at 08:08, John Baldwin wrote: > >> On 2/12/24 5:57 PM, Mark Millard wrote: >>> On Feb 12, 2024, at 16:36, Mark Millard wrote: >>>> On Feb 12, 2024, at 16:10, Mark Millard wrote: >>>> >>>>> On Feb 12, 2024, at 12:00, Mark Millard wrote: >>>>> >>>>>> [Gack: I was looking at the wrong vintage of source code, predating >>>>>> your changes: wrong system used.] >>>>>> >>>>>> >>>>>> On Feb 12, 2024, at 10:41, Mark Millard wrote: >>>>>> >>>>>>> On Feb 12, 2024, at 09:32, John Baldwin wrote: >>>>>>> >>>>>>>> On 2/9/24 8:13 PM, Mark Millard wrote: >>>>>>>>> Summary: >>>>>>>>> pcib0: mem 0x7d500000-0x7d50930f irq 80,81 on simplebus2 >>>>>>>>> pcib0: parsing FDT for ECAM0: >>>>>>>>> pcib0: PCI addr: 0xc0000000, CPU addr: 0x600000000, Size: 0x40000000 >>>>>>>>> . . . >>>>>>>>> rman_manage_region: request: start 0x600000000, end 0x6000fffff >>>>>>>>> panic: Failed to add resource to rman >>>>>>>> >>>>>>>> Hmmm, I suspect this is due to the way that bus_translate_resource works which is >>>>>>>> fundamentally broken. It rewrites the start address of a resource in-situ instead >>>>>>>> of keeping downstream resources separate from the upstream resources. For example, >>>>>>>> I don't see how you could ever release a resource in this design without completely >>>>>>>> screwing up your rman. That is, I expect trying to detach a PCI device behind a >>>>>>>> translating bridge that uses the current approach should corrupt the allocated >>>>>>>> resource ranges in an rman long before my changes. >>>>>>>> >>>>>>>> That said, that doesn't really explain the panic. Hmm, the panic might be because >>>>>>>> for PCI bridge windows the driver now passes RF_ACTIVE and the bus_translate_resource >>>>>>>> hack only kicks in the activate_resource method of pci_host_generic.c. >>>>>>>> >>>>>>>>> Detail: >>>>>>>>> . . . >>>>>>>>> pcib0: mem 0x7d500000-0x7d50930f irq 80,81 on simplebus2 >>>>>>>>> pcib0: parsing FDT for ECAM0: >>>>>>>>> pcib0: PCI addr: 0xc0000000, CPU addr: 0x600000000, Size: 0x40000000 >>>>>>>> >>>>>>>> This indicates this is a translating bus. >>>>>>>> >>>>>>>>> pcib1: irq 91 at device 0.0 on pci0 >>>>>>>>> rman_manage_region: request: start 0x1, end 0x1 >>>>>>>>> pcib0: rman_reserve_resource: start=0xc0000000, end=0xc00fffff, count=0x100000 >>>>>>>>> rman_reserve_resource_bound: request: [0xc0000000, 0xc00fffff], length 0x100000, flags 102, device pcib1 >>>>>>>>> rman_reserve_resource_bound: trying 0xffffffff <0xc0000000,0xfffff> >>>>>>>>> considering [0xc0000000, 0xffffffff] >>>>>>>>> truncated region: [0xc0000000, 0xc00fffff]; size 0x100000 (requested 0x100000) >>>>>>>>> candidate region: [0xc0000000, 0xc00fffff], size 0x100000 >>>>>>>>> allocating from the beginning >>>>>>>>> rman_manage_region: request: start 0x600000000, end 0x6000fffff >>>>> >>>>> What you later typed does not match: >>>>> >>>>> 0x600000000 >>>>> 0x6000fffff >>>>> >>>>> You later typed: >>>>> >>>>> 0x60000000 >>>>> 0x600fffffff >>>>> >>>>> This seems to have lead to some confusion from using the >>>>> wrong figure(s). >>>>> >>>>>>>> The fact that we are trying to reserve the CPU addresses in the rman is because >>>>>>>> bus_translate_resource rewrote the start address in the resource after it was allocated. >>>>>>>> >>>>>>>> That said, I can't see why rman_manage_region would actually fail. At this point the >>>>>>>> rman is empty (this is the first call to rman_manage_region for "pcib1 memory window"), >>>>>>>> so only the check that should be failing are the checks against rm_start and >>>>>>>> rm_end. For the memory window, rm_start is always 0, and rm_end is always >>>>>>>> 0xffffffff, so both the old (0xc00000000 - 0xc00fffff) and new (0x60000000 - 0x600fffffff) >>>>>>>> ranges are within those bounds. >>>>> >>>>> No: >>>>> >>>>> 0xffffffff >>>>> >>>>> .vs (actual): >>>>> >>>>> 0x600000000 >>>>> 0x6000fffff >> >> Ok, then this explains the failure if the "raw" addresses are above 4G. I have >> access to an emag I'm currently using to test fixes to pci_host_generic.c to >> avoid corrupting struct resource objects. I'll post the diff once I've got >> something verified to work. >> >>> It looks to me like in sys/dev/pci/pci_pci.c the: >>> static void >>> pcib_probe_windows(struct pcib_softc *sc) >>> { >>> . . . >>> pcib_alloc_window(sc, &sc->mem, SYS_RES_MEMORY, 0, 0xffffffff); >>> . . . >>> is just inappropriately restrictive about where in the system >>> address space a PCIe can validly be mapped to on the high end. >>> That, in turn, leads to the rejection on the RPi4B now that >>> the range use is checked. >> >> No, the physical register in PCI-PCI bridges is only 32-bits. Only the >> prefetchable BAR supports 64-bit addresses. > > Just for my edification . . . > > As I understand, SYS_RES_MEMORY for the BCM2711 > means the 35 bit addressing space in the BCM2711, > not a PCIe device internal address range that > corresponds. Am I wrong about that? > > If I'm wrong, what does identify the 35 bit > addressing space in the BCM2711? > > If I'm correct, then the 0..0xffffffff > seems to be from the wrong address space up > front. Or, may be, the SYS_RES_MEMORY and the > 0xffffffff argments are not related as I > expected and the 0xffffffff is not a > SYS_RES_MEMORY value? We use SYS_RES_MEMORY for both address spaces. SYS_RES_MEMORY is more of an address space "type" and doesn't necessarily name a single, unique address space. The way to think about these address spaces is instances of 'struct rman'. There's a global 'struct rman' in the arm64 nexus driver that represents the CPU physical memory address space. The pci_host_generic driver contains its own 'struct rman' instances that represent the SYS_RES_MEMORY (for memory PCI BARs) and SYS_RES_IOPORT (for I/O port PCI BARs) address spaces. Put another way, SYS_RES_MEMORY names an I/O memory address space relative to a device's given position in the tree. For a given device node in the tree, SYS_RES_MEMORY is unique, but what it maps onto is defined by a parent bus device. -- John Baldwin From nobody Wed Feb 14 18:23:56 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TZmmZ1n83z51jvq; Wed, 14 Feb 2024 18:23:58 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TZmmZ12Plz4WTr; Wed, 14 Feb 2024 18:23:58 +0000 (UTC) (envelope-from jhb@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1707935038; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TLwPYqfHBEgA07fG6sinqIBnb97j/AJBeN2GM4tRIUE=; b=dTabW5U61wyZwVxbMez5kvq7y6ddH1j3iAHf6zdmLbnqMB5v9s9f3xJODerlnCCoL+vAIZ jszsgTxLbporIC0visJZmKFpoUPDZcZAhVKgxwOLjvjUYFzpZCeTy7JktiqgEfYYTQa2iE UjCmybcPOr288ydRn/trw3LswlD9Giu6Lu2wnU8GBBByvyDLnjIjD9Q1Q9LFE66zRRCRhA r33kOoBnMhsF32JPX1jKAeILnIRh4JeShYG3YpxgAUGNKB2rRf6Bx0+obAyGO6CYzQ/tgV qS8htP/TIuogmz68GzSX4OlOWAAKahtLmJZsD9Jkj9VXTK1GDRs+ai4iVY/dWQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1707935038; a=rsa-sha256; cv=none; b=MUiAkl51VuBQEaeo8OtDQrWxGinsmA5Z91g7scOIG5xdQdmOs4Rz+ynY3RdV+CwxON5Ywe ClK88fWsA34D9AlmqBDi1yJtE2F7hLhLhBjeJ07dm45iy1TavsSBQ7J0+I8TSO261Hm9AP ezfv7j2up32Jkfsstbt69q9TioTrjnVFeRpdyvl8AuYIMY2Id7HlBgVfJXlj0Unx0lYluI VGbuuqSWTdqETIIrydaWxR8DSu2u1glem8k7igFajkf9rfH17pqir7JgJ14F9mZ8gFU5lg L1G1LEf1lU5666gNZvkXRuPH20YXU25AKpupGkqCtqMkZyKScUZWtjRsQnO6kQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1707935038; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TLwPYqfHBEgA07fG6sinqIBnb97j/AJBeN2GM4tRIUE=; b=yrIZ7eSUzbeccUiEoJAYgMCjfo3A8mw8BwIo6q8RorhBkIaItYSGR1bKwmZpokIpimg0dz PzgbaT8M/9RMEPAZJPofQsFR5gFhodWYsiLH6ubT5usOx31CxhAKjb75bcUZF0by/X5Y8J DMYH3liUl8kQLZND+60iWAclg5zj0ch5QYcW31olZKVRZB1K83NHziGuXhpTHHoJoqLJTB dcFmlnhqyuxk5XDwZl+TeJZi7oCv8pN/eaT4wMsTlpg+GUPNY3LklWg3dFW9fP4DZXw6rr e2mrYFtnTSswyx9eESDI8IP/udoPaiCBLSqaSV+ywAG/GKUtL1bGTYE+OitQMA== Received: from [IPV6:2601:644:937c:5920:4c63:23c7:5c22:d7ba] (unknown [IPv6:2601:644:937c:5920:4c63:23c7:5c22:d7ba]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: jhb) by smtp.freebsd.org (Postfix) with ESMTPSA id 4TZmmY46YWzRjN; Wed, 14 Feb 2024 18:23:57 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Message-ID: <0b0ba1b0-dee5-47e2-bb93-b44a24492abd@FreeBSD.org> Date: Wed, 14 Feb 2024 10:23:56 -0800 List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: Recent commits reject RPi4B booting: pcib0 vs. pcib1 "rman_manage_region: request" leads to panic Content-Language: en-US To: Mark Millard Cc: FreeBSD ARM List , Current FreeBSD , Warner Losh References: <76AB969F-5BC5-4116-8AF4-3ED2CABEBBA5.ref@yahoo.com> <76AB969F-5BC5-4116-8AF4-3ED2CABEBBA5@yahoo.com> <1F704317-FDB8-4BDA-8A67-61CF48794DFE@yahoo.com> <9AFDF067-96E4-4E67-90D2-F40DAF3F138F@yahoo.com> <4C279710-5F88-4295-B1A4-7C395F3587E5@yahoo.com> <3A145420-399D-4EBD-9FF4-18924908AB1D@yahoo.com> <1298DF9C-0F82-4567-8E81-7332A608C7FC@yahoo.com> <36ECB040-7E09-47A9-AF71-DE546A78E9CA@yahoo.com> From: John Baldwin In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit On 2/14/24 10:16 AM, Mark Millard wrote: > Top posting a related but separate item: > > I looked up some old (2022-Dec-17) lspci -v output from > a Linux boot. Note the "Memory at" value 600000000 (in > the 35 bit BCM2711 address space) and the "(64-bit, > non-prefetchable)" (and "[size=4K]"). > > 01:00.0 USB controller: VIA Technologies, Inc. VL805/806 xHCI USB 3.0 Controller (rev 01) (prog-if 30 [XHCI]) > Subsystem: VIA Technologies, Inc. VL805/806 xHCI USB 3.0 Controller > Device tree node: /sys/firmware/devicetree/base/scb/pcie@7d500000/pci@0,0/usb@0,0 > Flags: bus master, fast devsel, latency 0, IRQ 51 > Memory at 600000000 (64-bit, non-prefetchable) [size=4K] > Capabilities: [80] Power Management version 3 > Capabilities: [90] MSI: Enable+ Count=1/4 Maskable- 64bit+ > Capabilities: [c4] Express Endpoint, MSI 00 > Capabilities: [100] Advanced Error Reporting > Kernel driver in use: xhci_hcd > > > "Memory at 600000000 (64-bit, non-prefetchable)": > Violation of a PCIe standard? No, this is a device BAR which can be 64-bit (memory BARs can either be 32-bits or 64-bits). However, the "window" in a PCI _bridge_ for memory is only defined to be 32-bits. Windows in PCI-PCI bridges are a special type of BAR that defines the address ranges that the bridge decodes on the parent side and passes down to child devices. The prefetchable window in PCI-PCI bridges can optionally be 64-bit. BAR == a range of memory or I/O port addresses decoded by a device, usually mapped to a register bank, but sometimes mapped to internal memory (e.g. a framebuffer) Window == a range of memory or I/O port addresses decoded by a bridge for which transactions are passed across the bridge to be handled by a child device. -- John Baldwin From nobody Wed Feb 14 18:45:45 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TZnFz6Bcpz51msd for ; Wed, 14 Feb 2024 18:45:59 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-ed1-x52c.google.com (mail-ed1-x52c.google.com [IPv6:2a00:1450:4864:20::52c]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TZnFz3p88z4b0y for ; Wed, 14 Feb 2024 18:45:59 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ed1-x52c.google.com with SMTP id 4fb4d7f45d1cf-5638ae09ec1so150590a12.0 for ; Wed, 14 Feb 2024 10:45:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20230601.gappssmtp.com; s=20230601; t=1707936358; x=1708541158; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=ksiIdJwHkc+JLXgXUyQdZyacITAs7IyrYG0GJD+9dbA=; b=yyILMmYyHwVQPP62pfdJnXyJXpTIQ0ZhzD1NPezPaZ22Mcf1oWJiI6oL9CJjp0qQfZ fSGNiDgloKy6kVVLWK0COp0n1IOVa3ihXHX30F/j4A1cgo/FwpKF6Gy0jZd0WiWUb14x vze+Lme11AXfm6W7hxpPYCgCPScOnKxPTrgdJGmwyQCrOV2iFedvae+h/ejZveaz9HuQ rh5glcuCLTpKcWqNQLRs2SIiEYZ8BzFoHHMjq1K+IS7NHqbtFEG+jT0bn7iDJl8BZdpu rxZMLRRiEVEHZgk4JjgIOfBgRT18Q35zysl3jPXp0QrZtBEqcipHQLQdihOmgw8FAyJK 83qw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707936358; x=1708541158; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=ksiIdJwHkc+JLXgXUyQdZyacITAs7IyrYG0GJD+9dbA=; b=WoBIucNKUyjT3RB6md0JiaDtDNVM14l1DQyyytNe6w9gp0PLSCrEzk5pqpaXEb83JD KRnDczt/g39FxQ0XO3XhGM98cGSV4DNt+2BIOPdCeD5PGUfYqwRhRkCU+sTPOIZXwWfu WlB+MziSDjKEKNMGB4EqjM1wd3cQ1LPcn686pF9DZsFQBNbjQkvd6jHSXXszgMFl03f6 GrmeKT/faaVacgm8dZ8APuV2jlJ7qdtHeTgXEofxZ9/zlpkV+NIlwdSgqr0iQ3Pi4Z45 UVKZiMjIwsSgrXThmF7D7HCIx/tadFWw7rOGnpNRgG7k/rAUbLZlfGsIMwU3TcsHFnxd 71fA== X-Forwarded-Encrypted: i=1; AJvYcCWIXh03Ba6ZrzaCwcg2xuocpwrsUay3l+15jAYoWfUM9WP8lVtc+dJrwTngYOnXS3NSQS0LirJmGj5X/TUOI/n3crRRduN4qg== X-Gm-Message-State: AOJu0YxhryH7/N2F8oF3H492pTx2jM78hjMBLy4+OV5ynMRNWB3Sxxl+ Z2GEgzU+ONcbHw6IMnxsn/XhqN5gof5q+ihHas1PGE4yB8mGSL8+w1d6tstA8qDcJvzn8hb8Qkk Zt9t0QhyWsK/+busTxSAUj6Ra+MR1RJmqGMZH4w== X-Google-Smtp-Source: AGHT+IHJu5L8SOUlWD8AmH7X8Md6iZrTY1c2M1JmDaPGRi/pCeAMEV0zhOkXuotcoNjL0cYLyrpK367MLTJ3lHH/BGk= X-Received: by 2002:aa7:c541:0:b0:562:1a77:19a7 with SMTP id s1-20020aa7c541000000b005621a7719a7mr2696303edr.11.1707936357796; Wed, 14 Feb 2024 10:45:57 -0800 (PST) List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 References: <76AB969F-5BC5-4116-8AF4-3ED2CABEBBA5.ref@yahoo.com> <76AB969F-5BC5-4116-8AF4-3ED2CABEBBA5@yahoo.com> <1F704317-FDB8-4BDA-8A67-61CF48794DFE@yahoo.com> <9AFDF067-96E4-4E67-90D2-F40DAF3F138F@yahoo.com> <4C279710-5F88-4295-B1A4-7C395F3587E5@yahoo.com> <3A145420-399D-4EBD-9FF4-18924908AB1D@yahoo.com> <1298DF9C-0F82-4567-8E81-7332A608C7FC@yahoo.com> In-Reply-To: From: Warner Losh Date: Wed, 14 Feb 2024 11:45:45 -0700 Message-ID: Subject: Re: Recent commits reject RPi4B booting: pcib0 vs. pcib1 "rman_manage_region: request" leads to panic To: John Baldwin Cc: Mark Millard , FreeBSD ARM List , Current FreeBSD Content-Type: multipart/alternative; boundary="000000000000e7df3f06115beb28" X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] X-Rspamd-Queue-Id: 4TZnFz3p88z4b0y X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated --000000000000e7df3f06115beb28 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hey John, On Wed, Feb 14, 2024 at 10:30=E2=80=AFAM John Baldwin wro= te: > On 2/14/24 8:42 AM, Warner Losh wrote: > > On Wed, Feb 14, 2024 at 9:08=E2=80=AFAM John Baldwin = wrote: > > > >> On 2/12/24 5:57 PM, Mark Millard wrote: > >>> On Feb 12, 2024, at 16:36, Mark Millard wrote: > >>> > >>>> On Feb 12, 2024, at 16:10, Mark Millard wrote: > >>>> > >>>>> On Feb 12, 2024, at 12:00, Mark Millard wrote: > >>>>> > >>>>>> [Gack: I was looking at the wrong vintage of source code, predatin= g > >>>>>> your changes: wrong system used.] > >>>>>> > >>>>>> > >>>>>> On Feb 12, 2024, at 10:41, Mark Millard wrote: > >>>>>> > >>>>>>> On Feb 12, 2024, at 09:32, John Baldwin wrote: > >>>>>>> > >>>>>>>> On 2/9/24 8:13 PM, Mark Millard wrote: > >>>>>>>>> Summary: > >>>>>>>>> pcib0: mem > >> 0x7d500000-0x7d50930f irq 80,81 on simplebus2 > >>>>>>>>> pcib0: parsing FDT for ECAM0: > >>>>>>>>> pcib0: PCI addr: 0xc0000000, CPU addr: 0x600000000, Size: > >> 0x40000000 > >>>>>>>>> . . . > >>>>>>>>> rman_manage_region: request: start > >> 0x600000000, end 0x6000fffff > >>>>>>>>> panic: Failed to add resource to rman > >>>>>>>> > >>>>>>>> Hmmm, I suspect this is due to the way that bus_translate_resour= ce > >> works which is > >>>>>>>> fundamentally broken. It rewrites the start address of a resour= ce > >> in-situ instead > >>>>>>>> of keeping downstream resources separate from the upstream > >> resources. For example, > >>>>>>>> I don't see how you could ever release a resource in this design > >> without completely > >>>>>>>> screwing up your rman. That is, I expect trying to detach a PCI > >> device behind a > >>>>>>>> translating bridge that uses the current approach should corrupt > >> the allocated > >>>>>>>> resource ranges in an rman long before my changes. > >>>>>>>> > >>>>>>>> That said, that doesn't really explain the panic. Hmm, the pani= c > >> might be because > >>>>>>>> for PCI bridge windows the driver now passes RF_ACTIVE and the > >> bus_translate_resource > >>>>>>>> hack only kicks in the activate_resource method of > >> pci_host_generic.c. > >>>>>>>> > >>>>>>>>> Detail: > >>>>>>>>> . . . > >>>>>>>>> pcib0: mem > >> 0x7d500000-0x7d50930f irq 80,81 on simplebus2 > >>>>>>>>> pcib0: parsing FDT for ECAM0: > >>>>>>>>> pcib0: PCI addr: 0xc0000000, CPU addr: 0x600000000, Size: > >> 0x40000000 > >>>>>>>> > >>>>>>>> This indicates this is a translating bus. > >>>>>>>> > >>>>>>>>> pcib1: irq 91 at device 0.0 on pci0 > >>>>>>>>> rman_manage_region: request: start 0x1, end > 0x1 > >>>>>>>>> pcib0: rman_reserve_resource: start=3D0xc0000000, end=3D0xc00ff= fff, > >> count=3D0x100000 > >>>>>>>>> rman_reserve_resource_bound: request: [0xc0000000= , > >> 0xc00fffff], length 0x100000, flags 102, device pcib1 > >>>>>>>>> rman_reserve_resource_bound: trying 0xffffffff > <0xc0000000,0xfffff> > >>>>>>>>> considering [0xc0000000, 0xffffffff] > >>>>>>>>> truncated region: [0xc0000000, 0xc00fffff]; size 0x100000 > >> (requested 0x100000) > >>>>>>>>> candidate region: [0xc0000000, 0xc00fffff], size 0x100000 > >>>>>>>>> allocating from the beginning > >>>>>>>>> rman_manage_region: request: start > >> 0x600000000, end 0x6000fffff > >>>>> > >>>>> What you later typed does not match: > >>>>> > >>>>> 0x600000000 > >>>>> 0x6000fffff > >>>>> > >>>>> You later typed: > >>>>> > >>>>> 0x60000000 > >>>>> 0x600fffffff > >>>>> > >>>>> This seems to have lead to some confusion from using the > >>>>> wrong figure(s). > >>>>> > >>>>>>>> The fact that we are trying to reserve the CPU addresses in the > >> rman is because > >>>>>>>> bus_translate_resource rewrote the start address in the resource > >> after it was allocated. > >>>>>>>> > >>>>>>>> That said, I can't see why rman_manage_region would actually fai= l. > >> At this point the > >>>>>>>> rman is empty (this is the first call to rman_manage_region for > >> "pcib1 memory window"), > >>>>>>>> so only the check that should be failing are the checks against > >> rm_start and > >>>>>>>> rm_end. For the memory window, rm_start is always 0, and rm_end > is > >> always > >>>>>>>> 0xffffffff, so both the old (0xc00000000 - 0xc00fffff) and new > >> (0x60000000 - 0x600fffffff) > >>>>>>>> ranges are within those bounds. > >>>>> > >>>>> No: > >>>>> > >>>>> 0xffffffff > >>>>> > >>>>> .vs (actual): > >>>>> > >>>>> 0x600000000 > >>>>> 0x6000fffff > >> > >> Ok, then this explains the failure if the "raw" addresses are above > 4G. I > >> have > >> access to an emag I'm currently using to test fixes to > pci_host_generic.c > >> to > >> avoid corrupting struct resource objects. I'll post the diff once I'v= e > got > >> something verified to work. > >> > >>> It looks to me like in sys/dev/pci/pci_pci.c the: > >>> > >>> static void > >>> pcib_probe_windows(struct pcib_softc *sc) > >>> { > >>> . . . > >>> pcib_alloc_window(sc, &sc->mem, SYS_RES_MEMORY, 0, > 0xffffffff); > >>> . . . > >>> > >>> is just inappropriately restrictive about where in the system > >>> address space a PCIe can validly be mapped to on the high end. > >>> That, in turn, leads to the rejection on the RPi4B now that > >>> the range use is checked. > >> > >> No, the physical register in PCI-PCI bridges is only 32-bits. Only th= e > >> prefetchable BAR supports 64-bit addresses. This is why the host brid= ge > >> is doing a translation from the CPU side (0x600000000) to the PCI BAR > >> addresses (0xc0000000) so that the BAR addresses are down in the 32-bi= t > >> address range. It's also true that many PCI devices only support 32-b= it > >> addresses in memory BARs. 64-bit BARs are an optional extension not > >> universally supported. > >> > >> The translation here is somewhat akin to a type of MMU where the CPU > >> addresses are mapped to PCI addresses. The problem here is that the > >> PCI BAR resources need to "stay" as PCI addresses since we depend on > >> being able to use rman_get_start/end to get the PCI addresses of > >> allocated resources, but pci_host_generic.c currently rewrites the > >> addresses. > >> > >> Probably I should remove rman_set_start/end entirely (Warner added the= m > >> back in 2004) as the methods don't do anything to deal with the fallou= t > >> that the rman.rm_list linked-list is no longer sorted by address once > >> some addresses get rewritten, etc. > >> > > > > At the time, they made sense. Removing it, though may take some doing > > since we use it in about 284 places in sys/dev today... Somewhat more > > pervasive than I'd have thought they'd be... > > Eh, I only find the one that I'm now removing: > > % git grep -E 'rman_set_(start|end)' sys/ > sys/dev/pci/pci_host_generic.c: rman_set_start(r, start); > sys/dev/pci/pci_host_generic.c: rman_set_end(r, end); > sys/kern/subr_rman.c:rman_set_start(struct resource *r, rman_res_t start) > sys/kern/subr_rman.c:rman_set_end(struct resource *r, rman_res_t end) > sys/sys/rman.h:void rman_set_end(struct resource *_r, rman_res_t _end= ); > sys/sys/rman.h:void rman_set_start(struct resource *_r, rman_res_t > _start); > > Also, I managed to boot the emag I have access to this morning. I had to > fix a few other bugs in acpi(4) for my changes in pci_host_generic to wor= k > but will post reviews later today. > That's what happens when you grep for 'get' instead of 'set' before the morning caffeine kicks in .. Warner --000000000000e7df3f06115beb28 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hey John,

On Wed, Feb 14, 2024 at 10:3= 0=E2=80=AFAM John Baldwin <jhb@freebs= d.org> wrote:
On 2/14/24 8:42 AM, Warner Losh wrote:
> On Wed, Feb 14, 2024 at 9:08=E2=80=AFAM John Baldwin <jhb@freebsd.org> wrote:
>
>> On 2/12/24 5:57 PM, Mark Millard wrote:
>>> On Feb 12, 2024, at 16:36, Mark Millard <marklmi@yahoo.com> wrote:
>>>
>>>> On Feb 12, 2024, at 16:10, Mark Millard <marklmi@yahoo.com> wrote: >>>>
>>>>> On Feb 12, 2024, at 12:00, Mark Millard <marklmi@yahoo.com> wrot= e:
>>>>>
>>>>>> [Gack: I was looking at the wrong vintage of sourc= e code, predating
>>>>>> your changes: wrong system used.]
>>>>>>
>>>>>>
>>>>>> On Feb 12, 2024, at 10:41, Mark Millard <marklmi@yahoo.com> = wrote:
>>>>>>
>>>>>>> On Feb 12, 2024, at 09:32, John Baldwin <jhb@freebsd.org> = wrote:
>>>>>>>
>>>>>>>> On 2/9/24 8:13 PM, Mark Millard wrote:
>>>>>>>>> Summary:
>>>>>>>>> pcib0: <BCM2838-compatible PCI-expr= ess controller> mem
>> 0x7d500000-0x7d50930f irq 80,81 on simplebus2
>>>>>>>>> pcib0: parsing FDT for ECAM0:
>>>>>>>>> pcib0:=C2=A0 PCI addr: 0xc0000000, CPU= addr: 0x600000000, Size:
>> 0x40000000
>>>>>>>>> . . .
>>>>>>>>> rman_manage_region: <pcib1 memory w= indow> request: start
>> 0x600000000, end 0x6000fffff
>>>>>>>>> panic: Failed to add resource to rman<= br> >>>>>>>>
>>>>>>>> Hmmm, I suspect this is due to the way tha= t bus_translate_resource
>> works which is
>>>>>>>> fundamentally broken.=C2=A0 It rewrites th= e start address of a resource
>> in-situ instead
>>>>>>>> of keeping downstream resources separate f= rom the upstream
>> resources.=C2=A0 =C2=A0For example,
>>>>>>>> I don't see how you could ever release= a resource in this design
>> without completely
>>>>>>>> screwing up your rman.=C2=A0 That is, I ex= pect trying to detach a PCI
>> device behind a
>>>>>>>> translating bridge that uses the current a= pproach should corrupt
>> the allocated
>>>>>>>> resource ranges in an rman long before my = changes.
>>>>>>>>
>>>>>>>> That said, that doesn't really explain= the panic.=C2=A0 Hmm, the panic
>> might be because
>>>>>>>> for PCI bridge windows the driver now pass= es RF_ACTIVE and the
>> bus_translate_resource
>>>>>>>> hack only kicks in the activate_resource m= ethod of
>> pci_host_generic.c.
>>>>>>>>
>>>>>>>>> Detail:
>>>>>>>>> . . .
>>>>>>>>> pcib0: <BCM2838-compatible PCI-expr= ess controller> mem
>> 0x7d500000-0x7d50930f irq 80,81 on simplebus2
>>>>>>>>> pcib0: parsing FDT for ECAM0:
>>>>>>>>> pcib0: PCI addr: 0xc0000000, CPU addr:= 0x600000000, Size:
>> 0x40000000
>>>>>>>>
>>>>>>>> This indicates this is a translating bus.<= br> >>>>>>>>
>>>>>>>>> pcib1: <PCI-PCI bridge> irq 91 a= t device 0.0 on pci0
>>>>>>>>> rman_manage_region: <pcib1 bus numb= ers> request: start 0x1, end 0x1
>>>>>>>>> pcib0: rman_reserve_resource: start=3D= 0xc0000000, end=3D0xc00fffff,
>> count=3D0x100000
>>>>>>>>> rman_reserve_resource_bound: <PCIe = Memory> request: [0xc0000000,
>> 0xc00fffff], length 0x100000, flags 102, device pcib1
>>>>>>>>> rman_reserve_resource_bound: trying 0x= ffffffff <0xc0000000,0xfffff>
>>>>>>>>> considering [0xc0000000, 0xffffffff] >>>>>>>>> truncated region: [0xc0000000, 0xc00ff= fff]; size 0x100000
>> (requested 0x100000)
>>>>>>>>> candidate region: [0xc0000000, 0xc00ff= fff], size 0x100000
>>>>>>>>> allocating from the beginning
>>>>>>>>> rman_manage_region: <pcib1 memory w= indow> request: start
>> 0x600000000, end 0x6000fffff
>>>>>
>>>>> What you later typed does not match:
>>>>>
>>>>> 0x600000000
>>>>> 0x6000fffff
>>>>>
>>>>> You later typed:
>>>>>
>>>>> 0x60000000
>>>>> 0x600fffffff
>>>>>
>>>>> This seems to have lead to some confusion from using t= he
>>>>> wrong figure(s).
>>>>>
>>>>>>>> The fact that we are trying to reserve the= CPU addresses in the
>> rman is because
>>>>>>>> bus_translate_resource rewrote the start a= ddress in the resource
>> after it was allocated.
>>>>>>>>
>>>>>>>> That said, I can't see why rman_manage= _region would actually fail.
>> At this point the
>>>>>>>> rman is empty (this is the first call to r= man_manage_region for
>> "pcib1 memory window"),
>>>>>>>> so only the check that should be failing a= re the checks against
>> rm_start and
>>>>>>>> rm_end.=C2=A0 For the memory window, rm_st= art is always 0, and rm_end is
>> always
>>>>>>>> 0xffffffff, so both the old (0xc00000000 -= 0xc00fffff) and new
>> (0x60000000 - 0x600fffffff)
>>>>>>>> ranges are within those bounds.
>>>>>
>>>>> No:
>>>>>
>>>>> 0xffffffff
>>>>>
>>>>> .vs (actual):
>>>>>
>>>>> 0x600000000
>>>>> 0x6000fffff
>>
>> Ok, then this explains the failure if the "raw" addresse= s are above 4G.=C2=A0 I
>> have
>> access to an emag I'm currently using to test fixes to pci_hos= t_generic.c
>> to
>> avoid corrupting struct resource objects.=C2=A0 I'll post the = diff once I've got
>> something verified to work.
>>
>>> It looks to me like in sys/dev/pci/pci_pci.c the:
>>>
>>> static void
>>> pcib_probe_windows(struct pcib_softc *sc)
>>> {
>>> . . .
>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0pcib_alloc_window(sc, = &sc->mem, SYS_RES_MEMORY, 0, 0xffffffff);
>>> . . .
>>>
>>> is just inappropriately restrictive about where in the system<= br> >>> address space a PCIe can validly be mapped to on the high end.=
>>> That, in turn, leads to the rejection on the RPi4B now that >>> the range use is checked.
>>
>> No, the physical register in PCI-PCI bridges is only 32-bits.=C2= =A0 Only the
>> prefetchable BAR supports 64-bit addresses.=C2=A0 This is why the = host bridge
>> is doing a translation from the CPU side (0x600000000) to the PCI = BAR
>> addresses (0xc0000000) so that the BAR addresses are down in the 3= 2-bit
>> address range.=C2=A0 It's also true that many PCI devices only= support 32-bit
>> addresses in memory BARs.=C2=A0 64-bit BARs are an optional extens= ion not
>> universally supported.
>>
>> The translation here is somewhat akin to a type of MMU where the C= PU
>> addresses are mapped to PCI addresses.=C2=A0 The problem here is t= hat the
>> PCI BAR resources need to "stay" as PCI addresses since = we depend on
>> being able to use rman_get_start/end to get the PCI addresses of >> allocated resources, but pci_host_generic.c currently rewrites the=
>> addresses.
>>
>> Probably I should remove rman_set_start/end entirely (Warner added= them
>> back in 2004) as the methods don't do anything to deal with th= e fallout
>> that the rman.rm_list linked-list is no longer sorted by address o= nce
>> some addresses get rewritten, etc.
>>
>
> At the time, they made sense. Removing it, though may take some doing<= br> > since we use it in about 284 places=C2=A0 in sys/dev today... Somewhat= more
> pervasive than I'd have thought they'd be...

Eh, I only find the one that I'm now removing:

% git grep -E 'rman_set_(start|end)' sys/
sys/dev/pci/pci_host_generic.c: rman_set_start(r, start);
sys/dev/pci/pci_host_generic.c: rman_set_end(r, end);
sys/kern/subr_rman.c:rman_set_start(struct resource *r, rman_res_t start) sys/kern/subr_rman.c:rman_set_end(struct resource *r, rman_res_t end)
sys/sys/rman.h:void=C2=A0 =C2=A0 =C2=A0rman_set_end(struct resource *_r, rm= an_res_t _end);
sys/sys/rman.h:void=C2=A0 =C2=A0 =C2=A0rman_set_start(struct resource *_r, = rman_res_t _start);

Also, I managed to boot the emag I have access to this morning.=C2=A0 I had= to
fix a few other bugs in acpi(4) for my changes in pci_host_generic to work<= br> but will post reviews later today.

That= 's what happens when you grep for 'get' instead of 'set'= ; before the morning caffeine kicks in ..

Warner
--000000000000e7df3f06115beb28-- From nobody Wed Feb 14 20:04:22 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TZq0n64Fgz52k0x for ; Wed, 14 Feb 2024 20:04:41 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic312-24.consmr.mail.gq1.yahoo.com (sonic312-24.consmr.mail.gq1.yahoo.com [98.137.69.205]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4TZq0n3BqGz4nnN for ; Wed, 14 Feb 2024 20:04:41 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1707941079; bh=uQn9c2r1u6XAou+9lOv3df9TplyPdH0dHnMgHP8UgYY=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=qY4go2lUYaUzT76jq38SuSGtGIRd14015cAO5BcnZmXtVIIEB9o05VNkZhQOehABjBtTkkNcSbcGhKGEwwsZAVs2YGWxaOPp8b+ZBvhRVvKAT+khBI/gCFAmqbNA2Op9Z2TNCv2VDrG04q8yehiDvBQLVSRrUb9kyYk9zFD9tVR/+2SADJmTixRmlwVg1HaAByixmt4Wgm5pHRdkX9s0JJ6b+JSK5h7I14ba5TM5dOGvm85aqNRxoyeieAGAlIRSKilSNzENS/oGd0tRa1h5BLeZXlTUhVXK/36P3YkZZY4GCtQ2Wi/Lvwg5jqfnfmciIGsNdoDBU8Ac4sNGRAbECA== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1707941079; bh=91WMRUCvXjNLbRZh/mfvCQQXHrxC5h4QaOmuyepbDyC=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=LWQsnhWdyoFkHFvdtFntZ6zXU0u66iT9sQAhHLQO16lQ9pTCR91nhHJNvXgsxG8PKn11Nkl1i/AAY82YzArjpWEfoFJDYqChzrD/TZAs87E2KnQQwckIoehbYP5jFKurrQ+c8TJ9cCQbHTLrYbwV1/cp73DY+sZmt6Xg+H/T2wS4H/zORSi5yko/HvcpZGYTTvJPTiUKM6c6iCzDzeEA38kOPiMW4QKvtkBXfsTPyzgWYPXhFGJrYQRjIhivq0lIBKBPJEXjzuzmCqxORh+mRb3HVqbNu3P/Vj2kCZlkAs3LwyoaAaz8YCvJbgI4KZPgIcacvoxeDdhrm8xVuzzMtQ== X-YMail-OSG: Lm3eQi4VM1nqGsfifZWO8BiA._cNrdfQGyWLQ_RThvXogK.kJWr3yCjtZpuJUkC PuD9eAstissGbNb3GvKWIna5GG0VFL1mlNW3ZZukZLbmNIxzvWcEOk4oFgzsaLV6lCElpdekbG0X yU_usdugvaStrw2wKY3M00cDrIX7MkRRi_oyQwPGhoTX.Kcmyk1PYUwyFUZkUEJmpfK_BnVDqUdO 2rfYSCaAFJe12A87Q0lmerBKAyOGu7rFNmK3Su6w2i4Dygob3Z42GQ_gZFnQLQ706NyTpvL3xnlO mNwTYP2Zjz7LKAakv9z1y11vHDvJX.35Y2gpeABVZKoYqHFadN.z2RLwthHu9wtyDy21dSeGXyxm OELumNg3vZhADSKvX74IKdi3.UjeXiqri_8rGnoTOsfl5KlkTZ.xo1gnsgp19MgSxbFmmNvgFs6B o6zTIWw3BPGZ1XAbdA2x7WGLJnSnc76I5_5RWcYsyoLJoSiV.1GIuEcu5648J4hdJixDPo6T2Tk2 xpXYiOnOK4MhhLHo21mdlP45_8tkpAFY1dQTl.Rbzn8FHjMOpVbVo8FSvduntJ9WKDX56LKIB9lQ OSHy6fEZu2UHG8pgKyYWD01cHY21NeKm9mPXCXg3fZIRN2mJqw4g4xZLh5CHfax0GXmVWToYkd0W VuLVK.4DQzoNWZwfOaQp4f8zXOp6NYEJUAN_xLupzYclnHwZzqS.NUJtJjULxWagwuWJ3s29Klvv BomASwP_Y4qlvqouiBuIcAVLhquglMUXQPa_u26CW7kFyakdTwd4Dhg8dTbgM7xm1FNtBGcZITrg mUgTPT6CdDJKdzlgh1NfU1_gTL1TEcnZ9qhzlvKKdy.xftp54QURkt3HGmk3dpvINo2as1.DXb4C oJJST.2GBRTTaMHKwBlFJwDjEXTQDM6ZES_xtddrb4lmTJPH0w9nA4InTrJPxcpkEsPfNYnKv6ck tqUwUH9iUq.xmHV4vmGte4rSiJ69.Aua0Bgnd1hQBoUa_29k.v4tfDzqoLQEo6RVNIeA3_kDqEYR 0xKerjVeDoiNU82ghruPyE4vHmCfaujh_q1PQqmepwJF9evCwgkvSMQ4kl9_BmtvXBYCJ48wpuDe lGbkYkXe6.eFeGxAlKg9UhxLCUBHZbWdmdM4LO9C5Nvs5F6lGzuaWKC.1xFWz376KqFxDdUVFLui 9I_8vQJToe6s1FydI4UdKoGeKgePjpdzGu9eDoUeKg.EsK8wlgve.jI9edunTL7WbMXtLG2rgVzZ cQ_I8xTxqONH4fvqqOLW4SABrMCxi2FUMs77L8RSo9zkafn7dbS4xnhfKX7FK5uvbtj54QS08W_E NQpHwnIuKwKgUaC3kVn4glSPWdulHu0x6_.9Gdcf8ArTtzaRg6aL_O3beRk_QY5DrWhYLEt3UZGK eeHSIGHga6l4sUJOc4vRii1dIMYjIOwAlGqKuInrHu0EEt0JRy8ElgZBIGXjj5ZwEh1atLo0bzKB D86XXRffbP1P4Onfg2A5gURhLfq8o7StCDJWiDtaxIqGLxVqOvjns9KWsKP29VQRrmQteqSqKXTF eXl6cd7QqvwRowBUDKZ0DQ6vDCRL2S67F8x3wfFimZQSrL92Fx6Zv94O9ku4xHwboaepa4D7qXbG KjLzirY83dwYX7Sqy1gHe6IXUf.mfbjoybjLzp.1uXLY8dqJ2qqN_EaLQC1xc2tvr4oME0iEn0E0 AVCiIiSmECe2YhZmpws18iW62sKVd7UIsYbwK.77.WmJsyHOsPNztUKF0pQw3Xx5YIey2VgyzbBS udtoQti96Z_mtIJj7zCT8HPn_fm4eWF.Mtwsp29LMdJkAfN5RE4mmwOtBjjrTPjWjVsbvIxdRZp5 _dYzR_FK_nSSMLLW.I.feJw9OOuuKli4tVo28M1OKjT_7on1dIPHtR16sWVlNj85YD0KJMMy_1C2 5bIb4sjakoOpigndpWfPNOizKF4BSt8TzqamIT9q.jQqQe2qUbt_4bxUG9cl0yZ476pkeyi0QvvV gbOIycwOvG8t6O6hndKA4ZaqjLqJfEwLI0Dyb.CUe4dqGvhChkZIN8p_pJip0FoOoBvrmUtKSVVV sV81FlUpyT9DCdLs.vQzjruYbguXW7e2dS3kZcs0NA5Jml5eIlUpBbG5vhNBgMCxtprZSROrKanr wZ7P0I4oxSiDnmhuS7a2yJmLOdtLgoX4mp0OZdUremn7iyqaskNq7zm8nJC9O1dEkPTFrE6BMnIq maUMR2wruakdBER5oTQnROvsbnelNzpIhEgcEUDqedMV1AuJlPON0B88cTpeEoAW7kf1kEl9VhDv LYQOFEcUkki48DMJ6gWsc X-Sonic-MF: X-Sonic-ID: 0e1f2332-e8d8-4a82-8e4a-7f36db08e4be Received: from sonic.gate.mail.ne1.yahoo.com by sonic312.consmr.mail.gq1.yahoo.com with HTTP; Wed, 14 Feb 2024 20:04:39 +0000 Received: by hermes--production-gq1-5c57879fdf-7xbd4 (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID ae84ef058a9f03c91879aa66a7f75d31; Wed, 14 Feb 2024 20:04:33 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.400.31\)) Subject: Re: Recent commits reject RPi4B booting: pcib0 vs. pcib1 "rman_manage_region: request" leads to panic From: Mark Millard In-Reply-To: <0b0ba1b0-dee5-47e2-bb93-b44a24492abd@FreeBSD.org> Date: Wed, 14 Feb 2024 12:04:22 -0800 Cc: FreeBSD ARM List , Current FreeBSD , Warner Losh Content-Transfer-Encoding: quoted-printable Message-Id: <072C1A6A-85AA-4B26-9598-E5A586F9A4C2@yahoo.com> References: <76AB969F-5BC5-4116-8AF4-3ED2CABEBBA5.ref@yahoo.com> <76AB969F-5BC5-4116-8AF4-3ED2CABEBBA5@yahoo.com> <1F704317-FDB8-4BDA-8A67-61CF48794DFE@yahoo.com> <9AFDF067-96E4-4E67-90D2-F40DAF3F138F@yahoo.com> <4C279710-5F88-4295-B1A4-7C395F3587E5@yahoo.com> <3A145420-399D-4EBD-9FF4-18924908AB1D@yahoo.com> <1298DF9C-0F82-4567-8E81-7332A608C7FC@yahoo.com> <36ECB040-7E09-47A9-AF71-DE546A78E9CA@yahoo.com> <0b0ba1b0-dee5-47e2-bb93-b44a24492abd@FreeBSD.org> To: John Baldwin X-Mailer: Apple Mail (2.3774.400.31) X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US] X-Rspamd-Queue-Id: 4TZq0n3BqGz4nnN X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated [Added at bottom: EDK2 notes referencing the non-ECAM compliant PCie in the BCM2711.] On Feb 14, 2024, at 10:23, John Baldwin wrote: > On 2/14/24 10:16 AM, Mark Millard wrote: >> Top posting a related but separate item: >> I looked up some old (2022-Dec-17) lspci -v output from >> a Linux boot. Note the "Memory at" value 600000000 (in >> the 35 bit BCM2711 address space) and the "(64-bit, >> non-prefetchable)" (and "[size=3D4K]"). >> 01:00.0 USB controller: VIA Technologies, Inc. VL805/806 xHCI USB 3.0 = Controller (rev 01) (prog-if 30 [XHCI]) >> Subsystem: VIA Technologies, Inc. VL805/806 xHCI USB 3.0 = Controller >> Device tree node: = /sys/firmware/devicetree/base/scb/pcie@7d500000/pci@0,0/usb@0,0 >> Flags: bus master, fast devsel, latency 0, IRQ 51 >> Memory at 600000000 (64-bit, non-prefetchable) [size=3D4K] >> Capabilities: [80] Power Management version 3 >> Capabilities: [90] MSI: Enable+ Count=3D1/4 Maskable- 64bit+ >> Capabilities: [c4] Express Endpoint, MSI 00 >> Capabilities: [100] Advanced Error Reporting >> Kernel driver in use: xhci_hcd >> "Memory at 600000000 (64-bit, non-prefetchable)": >> Violation of a PCIe standard? >=20 > No, this is a device BAR which can be 64-bit (memory BARs can either > be 32-bits or 64-bits). However, the "window" in a PCI _bridge_ for > memory is only defined to be 32-bits. Windows in PCI-PCI bridges > are a special type of BAR that defines the address ranges that the > bridge decodes on the parent side and passes down to child devices. > The prefetchable window in PCI-PCI bridges can optionally be 64-bit. >=20 > BAR =3D=3D a range of memory or I/O port addresses decoded by a = device, > usually mapped to a register bank, but sometimes mapped to internal > memory (e.g. a framebuffer) >=20 > Window =3D=3D a range of memory or I/O port addresses decoded by a = bridge > for which transactions are passed across the bridge to be handled by > a child device. Good to know. Thanks. FYI: From: = https://github.com/tianocore/edk2-non-osi/commit/c1075e9ddd647fa7f7cb17b31= 2f6bf8246952e09 There are these notes that indicate the non-standard ECAM status: (Other details are just for reference. EDK2 UEFI/ACPI is not the official usage context.) QUOTE The RPi4 has a single nonstandard PCI config region. It is broken into = two pieces, the root port config registers and a window to a single device's config space which can move between devices. However there isn't (yet) = an authoritative public document on this since the available BCM2711 = reference notes that there is a PCIe root port in the memory map but doesn't = describe it. Considering that it's not ECAM compliant, yet relatively simple, it is however possible to make use of the newly introduced PCI SMCCC interface that was added for the RPi4 platform as part of the TF-A 2.6 release. As a result, we update the RPi4 TF-A to the 2.6 release version, and, = for good measure, the RPi3 also, using binaries that were built in an open = and verifiable manner through the GitHub Actions build script located at https://github.com/pftf/pitf. For more details on the SMCCC interface, see DEN0115 available from: https://developer.arm.com/documentation/den0115/latest END QUOTE Other notes: I remember, FreeBSD supports the SMCCC interfacing in question. By contrast, Linux refused to add such support. The default RPi4B EDK2 configuration avoids exposing the PCIe but there is an option that uses the SMCCC technique to expose it. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Thu Feb 15 00:50:46 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TZxMh3520z551QZ; Thu, 15 Feb 2024 00:51:28 +0000 (UTC) (envelope-from marietto2008@gmail.com) Received: from mail-ed1-x532.google.com (mail-ed1-x532.google.com [IPv6:2a00:1450:4864:20::532]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TZxMg4JNFz4VXd; Thu, 15 Feb 2024 00:51:27 +0000 (UTC) (envelope-from marietto2008@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20230601 header.b=ext1PDgp; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of marietto2008@gmail.com designates 2a00:1450:4864:20::532 as permitted sender) smtp.mailfrom=marietto2008@gmail.com Received: by mail-ed1-x532.google.com with SMTP id 4fb4d7f45d1cf-55f50cf2021so440017a12.1; Wed, 14 Feb 2024 16:51:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1707958283; x=1708563083; darn=freebsd.org; h=to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=QBcRte46KoGQQwVd3ukluhewgGTVRssG7yLLgW/rGLg=; b=ext1PDgp4TiAJhh4r6TA9ZXgIznf2TDEoqEK8R+XKgHrkYjlMqLS2l35vCveHtzMn2 88yPdmRpNWlXZKWfZK43niwnfb7jXZETMmXlMhgCe+G/Vtk/V6WwHF0xPNYQFmEC+CAK Rbyv1aCdcRUYrBmI+NYjiYJGxKg3sL7CYwTxegch2RXVjyZfco1ehcbtnj4RLOZSSf1t w1rQn7DT9NM8WuEtEWOdRYtN2asXdpEgOKVJterq7IXNTWoJovbbBZ6KlE1l9u0HvnPH LGTDiCYQ+0TftxgoKjLvVBU5OWEJM5QBe43YbLlVb6MFoOUo1JiL/HrimcXK4gDTaobB 22+A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707958283; x=1708563083; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=QBcRte46KoGQQwVd3ukluhewgGTVRssG7yLLgW/rGLg=; b=WS2pvddmEB/b90SQeUG+vAkMGqiCKOQd/hFty6/PAfZf3ytnDf7BkOLNRDjpAkeikm sdSOclv00jpwPy8OLc58TG7xquLFIb/SNQc5eqpPm4lWNj0clkKLYyq0OuFNs8EpbvWX UojBaVgZ6PKdz6ybd3tQXrIBXLfQgyqw1OM8vGf+aVUUZ0eNsOa99/B6yM8+l7m88qG+ cIeGO37ZdgQ7JMVe8RsS1G3fnfFBmXo4MZfkM41UFxPV7Qb0CCgsj4PJMkpdEKAwi3Yt Dq865Aa1YxS0nUHFPJ03wxwsqhXl8rVFIp2/RBv1mWgQreKFWXwd+tNi3q0pMzt+OYWo VHmw== X-Forwarded-Encrypted: i=1; AJvYcCXH9W4HGYu9gFvWGygVape5GoGyufn4WYmgeEEzanOK9pi2EQ7wS3hZr7A67YZ4imR9kfBI8usEBgxBKjuUk+QvCVS60omQ/GcovGydqPw6YqDFlhzEHJwDfn0E4KtxNB9xDouqtfdaZ0T+gcUZELfK51xSI3SYjKT5eHwsaKStKfAdbDRQkEg= X-Gm-Message-State: AOJu0YwENEJiEeG2DA0+eRpRLzGbDKhMn1zMOEshiAUCCR9r4APyI1PO x/Zm8hJw7uaelmdUfW/irM5Kam0lqwDn1BfR220VILS85pQliPup21ZX5sIlA+RJEAdHY/syjVV QoZzBfwJBk+NpUz8fjdPkkpC92pYMaGNTI8M= X-Google-Smtp-Source: AGHT+IGegySJdSfaQcO/IP17JBuRJiTnmxtDg3qr6CmOWLAP6V/et/F8fzzRLMKEF+bXuePZ3XnENeyE5KbAmBTgRlg= X-Received: by 2002:a17:906:b0d9:b0:a38:63d4:2273 with SMTP id bk25-20020a170906b0d900b00a3863d42273mr117728ejb.35.1707958282885; Wed, 14 Feb 2024 16:51:22 -0800 (PST) List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 From: Mario Marietto Date: Thu, 15 Feb 2024 01:50:46 +0100 Message-ID: Subject: files/edd962f76ea4b5869f3c6f8ee5438fb9750b802d02bb8035fe1b7bd0a8ba7401.gz not found -- snapshot corrupt. To: freebsd-arm , freebsd-hackers , FreeBSD Mailing List , FreeBSD Current Content-Type: multipart/alternative; boundary="000000000000be1e9f0611610622" X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.98 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.98)[-0.984]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20230601]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCVD_COUNT_ONE(0.00)[1]; MISSING_XM_UA(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::532:from]; ARC_NA(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; RCVD_TLS_LAST(0.00)[]; TO_DN_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MID_RHS_MATCH_FROMTLD(0.00)[]; FROM_HAS_DN(0.00)[]; MLMMJ_DEST(0.00)[freebsd-arm@freebsd.org,freebsd-hackers@freebsd.org,freebsd-questions@freebsd.org,freebsd-current@freebsd.org]; RCPT_COUNT_THREE(0.00)[4]; DKIM_TRACE(0.00)[gmail.com:+] X-Rspamd-Queue-Id: 4TZxMg4JNFz4VXd --000000000000be1e9f0611610622 Content-Type: text/plain; charset="UTF-8" Hello. After a lot of work I've been able to install FreeBSD 12.04 for armv7 on my ARM Chromebook. Now I would like to install some ports. This is what happens when I try to get a fresh ports tree : marietto@freebsd:/usr # sudo portsnap fetch extract ...... /usr/ports/databases/py-sqlalchemy10/ /usr/ports/databases/py-sqlalchemy11/ /usr/ports/databases/py-sqlalchemy12/ /usr/ports/databases/py-sqlalchemy13/ /usr/ports/databases/py-sqlalchemy14/ /usr/ports/databases/py-sqlalchemy20/ /usr/ports/databases/py-sqlcipher3/ files/edd962f76ea4b5869f3c6f8ee5438fb9750b802d02bb8035fe1b7bd0a8ba7401.gz not found -- snapshot corrupt. I repeated the "portsnap fetch extract" command,but I always get the same error. -- Mario. --000000000000be1e9f0611610622 Content-Type: text/html; charset="UTF-8"
Hello.

After a lot of work I've been able to install FreeBSD 12.04 for armv7 on my ARM Chromebook. Now I would like to install some ports. This is what happens when I try to get a fresh ports tree :

marietto@freebsd:/usr # sudo portsnap fetch extract

......
/usr/ports/databases/py-sqlalchemy10/
/usr/ports/databases/py-sqlalchemy11/
/usr/ports/databases/py-sqlalchemy12/
/usr/ports/databases/py-sqlalchemy13/
/usr/ports/databases/py-sqlalchemy14/
/usr/ports/databases/py-sqlalchemy20/
/usr/ports/databases/py-sqlcipher3/
files/edd962f76ea4b5869f3c6f8ee5438fb9750b802d02bb8035fe1b7bd0a8ba7401.gz not found -- snapshot corrupt.

I repeated the "portsnap fetch extract" command,but I always get the same error.

--
Mario.
--000000000000be1e9f0611610622-- From nobody Thu Feb 15 02:19:04 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TZzK56FTjz59hPh for ; Thu, 15 Feb 2024 02:19:21 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic316-55.consmr.mail.gq1.yahoo.com (sonic316-55.consmr.mail.gq1.yahoo.com [98.137.69.31]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4TZzK45NjWz4kgR for ; Thu, 15 Feb 2024 02:19:20 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=rwBAf1TO; dmarc=pass (policy=reject) header.from=yahoo.com; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.69.31 as permitted sender) smtp.mailfrom=marklmi@yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1707963557; bh=b+qjQjzkceIwmpemBK9JYxEdx05pf6UN1WTcBfmouEk=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=rwBAf1TOpKIXFoB/T8Y4FHom0ahJ3KNCGtyKgJ30s+HnmMF08tAds93qRMVy++brQeZNrCmbOXayk4grYIeFBS28vmU+/BpSnYmUbESuCnSRxT86sw0isqG1aeU04D2ITFdHDKWJNiIM0L84hC8UzZ9x3HTplGt19Lk9qUpLhmc8my/gETuLgcTfM7W19ka44/VqeBLy1MwRp7aMGCqiHf3PJNMlFAcx+S4c1YrELJfZCPBVmsUDCilNY538Gd7lMfS29xe2nInezgaKMNVKvcGS9SzsK6M1DtAtWsfRCH3m+tS+rEb/FBbWRa6wlqoLu0Dm1zzHodmr9CkeWlbrww== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1707963557; bh=CK6VPUa2V5nnBHpjN/a6HvQ9CWgYzhWJlsjejLixHo1=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=ei10ioVCBJ8xG/iw1ht+Ki1SaPY0naCXJnBm1N1i4dl60MWwLxGW56VqUfSFA5pmU9oBwbNY3eQDe+uq24cWjnKBwm9/dUbNl7qe+b8KTDcdNQLpBp1YkPU4CD4deIxjl6Zb6gfEJoXsg0AdqJtM9fM8EUpuFCoMf/wNSsELpst1jzfA73cqDckXmnRwyv5nbWdplEgY/bmR9YWMGjKRL0IxxENpe6fHMjBHUAykrBchsJbJfYRoO6v4B5kwZAqjJS77Tr4JgApfis8Le7I37XfrgzIUXLP/C2LUEX9oYWkXMMvaTcQupF3C+uG99XW3P8MOuOYGBbxVLVp5ah7pXA== X-YMail-OSG: eO6ZTxsVM1lFD.S4ugIkVlxRtBpOuNdwEoIUuUQ1gx7qiZAybZYfZ5A6Q4uS5dQ vutcrPdRy2.PAM9zuZKSpNToaDltLu8ixgcUXo8OqsNKOvgDeqlB8ZYip7HUzZWZJ3N0YA1jYQW7 YEfm4geNduxEl8jzBHqf4ysgbvbYh6hWxMAY9YBYxbNnG00RSk5es8SJJJpjl0DvBeAEWS1hrBd9 DshLdZq5uQJrDSs5vSweDCSt2FcSEXb5jHCATxI6ojyzPZaKEIqwWr7HiBPpD29mghudAPvsiB6a dvSa9wEKLYFbTYT4Zb5XisFslN_XincjERU.S_SKpCaqP9KOKZO2BzETseZnG9qv_Qi5B1ToalhE yPCopH48pyGY1NH8cusCBcr5EPuu4Ob.rYg5wbS80p_7eYRm1b_H1sG5VI859jj.d3m.KSxUygX0 05pZV1xx0OWwTvNoRdo2vFLxpIquH.yZQEJr0pnOGnErH.OtmrsZCaPJNvaN9sO.3RvE28p4RUFI V.OMkNM6nWJQj3Jhj9emhHuo6iiqhAcRO2CMWY46LHc0FGYZcpJICW6U_MDkaH3f3cRYDTA4v17D Bkw42f9_NYTjUnzWOiB71PPKfDdz.4B1fItTws7k16BSj4WM3akban4AGUHew3IQs49Dftvw83oB h7ZeaJc9x_71PSl8LCqVYRw64BLppCooyREbMP2wLYD4X5LcL9RvyhZY5TAR2yaFh5N7QefGxRET RLcuMXZJJgkEj8cbcsgnzBQlQ5Xav2dRRKj5WniufXGhJz5s922u5WXVYrMyRDeHq2xfiq42SSue 33WN7ZTVCBuoOJwIiqUthC_5UULxIyHdDUY7IpSuBiKGbFeW2PfyAT29UDskQCj8FMsa01oOBC67 lUNZtm9fnbVLCldFtKHJj3J6p.ZpI6zeiB12cdGEgDevZrborFMNRn7Rz_IuiBNTL7MQDB7uyUt3 hCOxIzUK2eSboqnEM3IaUmM9BZHXl19dGXTN46EDDCbhpJgYYY8D0Cvi_WJ_BK6nx_ivLMfaN1Iq Rb86wQW.TZbp1Tn.RuMh.FKVxO._72mgPb_Xes6INFjAj04CJ8LkoKzJeYNjbFL0sm0SyiYCK.Vc OX7RmPdl9pPfpXTQGHg7bS7c2ErXF42RuNEYqVHmZxns1Ete4d9cimH2Kmb7n5ou4LJDAYlaBPR2 E0nHme07ROchQ2U1ytwCOk_Q0nhWMefInRfQZA14ze2g1IsCla6UhQKlSer_U8L3YFF1tzTMwj0b H31ZL6eNpFmlwFOM46ENiBiQE2VQrI6bH2TAzDi1QGD94u_ykFMBUEhLcPWm0PYRC_HjKSEByTYZ jbKvjPahhnfy2V1h9BB5Bh3x7VSXxGVd9z8aioS8m8RMm2vMTCDxwMI6fqbA_.NCwzngt7mUjaj4 jHws_RUJzVBjXJtNRj9bUqEzyImiNArO0tD6PmwfWQm9W9ZdETvgibbqSIp5G4hEGIMNqpR8UKIz MZ8.oFdpXJZzm7OMOtCyW1sp4PAp1OaSbknFy7l7VsqsXXGDgEzqYEhxIWkht_kYEK.vs1DaOCBU Xhp59ZbqV13UkW5Ca2DLB.8gPLi.pKjUuLXQ4BM88_DSfxVJGq0azxLyM3pp.8OI0izHeS5CFeyr 7ReXUMsxUd2La_qORdQWF7hm5_.jb5rG.wAaO2AzDOPG6CslvmD6HXw6eLCTB9LBU39ESH540UPL eX53q39PEW3wtNGCF9rOlHgmvGTdy9P9vPhGNFd___tGyKDZjhNpcsS03H2Y._t_n4Ju1U7bE5Tf VuhrCgJBY_HpNvC6wHXRnG3XftqwE_dbfRov4WlTiF7q4Lv27NX5Os2e.Y9r.V8goPHQbxniDRUy gL__mOBDU9E7b1MccZdf1BKVbZ107riZsCkeJMJFp3EiY6sp4j1dEH08YdwHRnVTZ0fqwst_nqFk 6u8Vaj27FG9RTEo69QmQ1letX7pe_.5OaNK1kZ7DQj61973gOUqxPUH_xLd6.GC51c2x.r403fHs Zod3BVNFR6w_8W3TNk59VV1_f_NX7D.ODtrZt44I71OjjuV.lSNsMrBj..3Z_N_Phf0zwEGAF8AG vbPNr31wq.dCj5kPrJftQNHrY1fXKlV8CbavrxT.JWkff0m5loYGdjnCZcUMkRmjL0o3AgRU3fqF 0DR04JnZEkzqcQcx.FORSiS7PW0zmRv2dZf7E70b_SwbGBPTfx34.T3zGZCJWqBaz_zuBFiyH4jb 9KuJnTrTtxXB6UsaWyBcV9hCgyIT9oOsQXL3lCNbx25fCn20HXRfzR_cf3qt74Zv5GtDkPW8U X-Sonic-MF: X-Sonic-ID: e09eabad-ee1b-443b-9425-a66e837d3d24 Received: from sonic.gate.mail.ne1.yahoo.com by sonic316.consmr.mail.gq1.yahoo.com with HTTP; Thu, 15 Feb 2024 02:19:17 +0000 Received: by hermes--production-gq1-5c57879fdf-4h5cs (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 6c24a661158ac8166ee0dff3c467d587; Thu, 15 Feb 2024 02:19:15 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.400.31\)) Subject: Re: Recent commits reject RPi4B booting: pcib0 vs. pcib1 "rman_manage_region: request" leads to panic [now fixed] From: Mark Millard In-Reply-To: <072C1A6A-85AA-4B26-9598-E5A586F9A4C2@yahoo.com> Date: Wed, 14 Feb 2024 18:19:04 -0800 Cc: FreeBSD ARM List , Current FreeBSD , Warner Losh Content-Transfer-Encoding: quoted-printable Message-Id: <23365E1F-FB27-49C7-A691-5DE3FDE40940@yahoo.com> References: <76AB969F-5BC5-4116-8AF4-3ED2CABEBBA5.ref@yahoo.com> <76AB969F-5BC5-4116-8AF4-3ED2CABEBBA5@yahoo.com> <1F704317-FDB8-4BDA-8A67-61CF48794DFE@yahoo.com> <9AFDF067-96E4-4E67-90D2-F40DAF3F138F@yahoo.com> <4C279710-5F88-4295-B1A4-7C395F3587E5@yahoo.com> <3A145420-399D-4EBD-9FF4-18924908AB1D@yahoo.com> <1298DF9C-0F82-4567-8E81-7332A608C7FC@yahoo.com> <36ECB040-7E09-47A9-AF71-DE546A78E9CA@yahoo.com> <0b0ba1b0-dee5-47e2-bb93-b44a24492abd@FreeBSD.org> <072C1A6A-85AA-4B26-9598-E5A586F9A4C2@yahoo.com> To: John Baldwin X-Mailer: Apple Mail (2.3774.400.31) X-Spamd-Bar: --- X-Spamd-Result: default: False [-4.00 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; DKIM_TRACE(0.00)[yahoo.com:+]; TO_DN_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; MIME_TRACE(0.00)[0:+]; RCPT_COUNT_THREE(0.00)[4]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.69.31:from]; MLMMJ_DEST(0.00)[freebsd-arm@freebsd.org]; APPLE_MAILER_COMMON(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.69.31:from]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim] X-Rspamd-Queue-Id: 4TZzK45NjWz4kgR Your changes have the RPi4B that previously got the panic to boot all the way instead. Details: I have updated my pkg base environment to have the downloaded kernels (and kernel source) with your changes and have booted with each of: /boot/kernel/kernel /boot/kernel.GENERIC-NODEBUG/kernel For reference: # uname -apKU FreeBSD aarch64-main-pkgs 15.0-CURRENT FreeBSD 15.0-CURRENT = main-n268300-d79b6b8ec267 GENERIC-NODEBUG arm64 aarch64 1500014 1500012 Thanks for the fix. Now I'll update the rest of pkg base materials. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Thu Feb 15 02:23:52 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TZzQc1yLyz59j4s for ; Thu, 15 Feb 2024 02:24:08 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-ed1-x529.google.com (mail-ed1-x529.google.com [IPv6:2a00:1450:4864:20::529]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TZzQb5dF1z4mPk for ; Thu, 15 Feb 2024 02:24:07 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ed1-x529.google.com with SMTP id 4fb4d7f45d1cf-5611e54a92dso457394a12.2 for ; Wed, 14 Feb 2024 18:24:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20230601.gappssmtp.com; s=20230601; t=1707963844; x=1708568644; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=3vqM5+iVuqKR9W8y85S9lq6wDNYJwsWHb0tSin9+8jA=; b=lF6gRuvNySTwxgYo2GtwPUhzF6l9jdkyJMpxjdqm2yieJ7ScRVpjG6fq/0x0/0mS32 1bFseaSUprplpbJkdpDnkZ6sPnnLC1vl7JLntblhIin3tcmBi/XyKGzLR3r+VNbRNgQQ vjjavizl0wCfeU9Q3Vz6qPCh2L1zt8VZ87TqrwBS9224HPssVEJd59Vze8ENpabdlYfg qDOjTSZfXC8HAc5jeeosRPnZxvOCojUCHBXUCAQmHfNP0wtbRxOPHdIw3TGQkTd1LsC1 QZaDfFezXkq7XOAC14FQZr3uwa70h0rM6dzo5V6AOtwc4Es5O2ZeW+GVM6xAgO0fnrxY swnw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707963844; x=1708568644; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=3vqM5+iVuqKR9W8y85S9lq6wDNYJwsWHb0tSin9+8jA=; b=oxp0ZlDVP1KgAg/VbAxjTedEFlpmKB1kTTAEF7zXY2ygQv2cATAhXxSH9mtY1uZjpx aYySMKxElb39c26ox5+W5dPvHMUo/resJASD6ErJEfEP8sXL3AIHhc+2EWV3fNdPpzrf 4SheVt/7s/ChwMAh2lnu3qnKyJYFqvdWC4iklhqU0QsYPhrA9PUO0eppxr0gZyJbqdMg MH/51OqjwREkrJpZe6z7GwktxslRr88GkUo9vLxaHB4uQycPn+S3JwoTS3uk/jkFTH+8 QpJG9Aaaf1avJ/6WmO217UJ+hIIc0xkMQ1e68UhHvpqq1Qyiyy2wQn6qJmDFv21mRxnT 7UMw== X-Gm-Message-State: AOJu0Yw8+rx6UWHDnqe/nYG8aZAUZrc0FboU/t0TzN2Sdrx3cwWNiOpC DDh7ABJFoER6b6fFjz0YIYYMi/Z3OSc1Hkn8k/bn2nBVMRyQ9Rf+VqL2cK7/Q2bIODQBREfo2j+ Me9cxFjyBIUH3EzEKi/zoeoJgysZ/bvWNucahNg== X-Google-Smtp-Source: AGHT+IGYARZBQp+Sdj6ZOLXG5q5fIrfvnssdZkPdgmQyi71+RQbaZklybelZxBSBPEgScXgfqfgitzddW/qZFkl6ifw= X-Received: by 2002:aa7:d7d1:0:b0:560:64e2:98a5 with SMTP id e17-20020aa7d7d1000000b0056064e298a5mr276205eds.21.1707963844349; Wed, 14 Feb 2024 18:24:04 -0800 (PST) List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Warner Losh Date: Wed, 14 Feb 2024 19:23:52 -0700 Message-ID: Subject: Re: files/edd962f76ea4b5869f3c6f8ee5438fb9750b802d02bb8035fe1b7bd0a8ba7401.gz not found -- snapshot corrupt. To: Mario Marietto Cc: freebsd-arm , freebsd-hackers , FreeBSD Mailing List , FreeBSD Current Content-Type: multipart/alternative; boundary="0000000000003b6397061162521f" X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] X-Rspamd-Queue-Id: 4TZzQb5dF1z4mPk X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated --0000000000003b6397061162521f Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable You may need to grab the repo. You may have to back up to December's ports tree... Warner On Wed, Feb 14, 2024, 5:51=E2=80=AFPM Mario Marietto wrote: > Hello. > > After a lot of work I've been able to install FreeBSD 12.04 for armv7 on > my ARM Chromebook. Now I would like to install some ports. This is what > happens when I try to get a fresh ports tree : > > marietto@freebsd:/usr # sudo portsnap fetch extract > > ..... > /usr/ports/databases/py-sqlalchemy10/ > /usr/ports/databases/py-sqlalchemy11/ > /usr/ports/databases/py-sqlalchemy12/ > /usr/ports/databases/py-sqlalchemy13/ > /usr/ports/databases/py-sqlalchemy14/ > /usr/ports/databases/py-sqlalchemy20/ > /usr/ports/databases/py-sqlcipher3/ > files/edd962f76ea4b5869f3c6f8ee5438fb9750b802d02bb8035fe1b7bd0a8ba7401.gz= not found -- snapshot corrupt. > > > I repeated the "portsnap fetch extract" command,but I always get the same > error. > > -- > Mario. > --0000000000003b6397061162521f Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
You may need to grab the repo. You may have to back up to= December's ports tree...=C2=A0

Warner=C2=A0

On Wed, Feb 14, 2024, 5:51=E2=80=AFPM Mario Mariett= o <marietto2008@gmail.com&= gt; wrote:
Hello.<= br>
After a lot of work I've been able to install FreeBSD 12.04 for armv7 o= n my ARM Chromebook. Now I would like to install some ports. This is what happens when I try to get a fresh ports tree :

=09 =09 
marietto@freebsd:/usr # sudo portsnap fetch extrac=
t

.....
/usr/ports/databases/py-sqlalchemy10/
/usr/ports/databases/py-sqlalchemy11/
/usr/ports/databases/py-sqlalchemy12/
/usr/ports/databases/py-sqlalchemy13/
/usr/ports/databases/py-sqlalchemy14/
/usr/ports/databases/py-sqlalchemy20/
/usr/ports/databases/py-sqlcipher3/
files/edd962f76ea4b5869f3c6f8ee5438fb9750b802d02bb8035fe1b7bd0a8ba7401.gz n=
ot found -- snapshot corrupt.

I repeated the "portsnap fetch extract" command,but I always get = the same error.

--
Mario.
--0000000000003b6397061162521f-- From nobody Thu Feb 15 03:15:43 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Tb0ZS6NKMz59rLD for ; Thu, 15 Feb 2024 03:16:00 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic309-22.consmr.mail.gq1.yahoo.com (sonic309-22.consmr.mail.gq1.yahoo.com [98.137.65.148]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Tb0ZR4M0nz4vkK for ; Thu, 15 Feb 2024 03:15:59 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1707966956; bh=dnrTAgWr5MF+1i28AxpapS8xiHiCfQqEb/oNihHf1I0=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=CJiDYcVJbGaYGFV7FOwYmRBDj6bvyfSN2Sv8om0ZQapf6JYzVIq/gyKesVwnPCUHv/5LXpfNxcBU3Nm/mfp3gvTb3RzD2aXH9gF7oRJhSvFHu06LdfJgN/rm5NkKqCMCwz7IW1WmmoxNpmOSDce3ykCa4jzypSUUtoTqQ6ZiBOmRC/4AP+ar3w/Yk1oDPOyueO5RulAc/fX5PUgMToN/4xllpzx6jY/tR9ICWqtB4RGCRz2I778XEqUEOR7TQUEC5lHWv7uk6qBlh7FoT27DFFvGOKN4SCvpYUqgVhDdiTTEzFOO3qDKXwfEqIrD24DJAmchR3FIUe7FXHq4GZfE7Q== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1707966956; bh=mVFwqiIzo8xoApvcdYqXQ9Wg+zQjvg9FrAwkNKDGp0f=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=U9PXdlS5D+nrWOVzsfmACUvB674bVNWhB95tkBwHlf/Qy94ypwvxBD+3X5zBXWr8rHM8vW2HEihsxuQ12NWDnpt/3htDkHJ+aC8BJSQRRGfNFMnDabs5y37Y3rtpLszkYxJF4YdyofK2hbyf8SrgAH2Nu2zFkwJsZMkz7Is9sHJ5S831Lc+E6M4G7gfKKOToxkSj2g9dG+hriY94rXnzSFiwjj6tm7O3PqV/y5Xr9RtY77npgyYqDTfhlEP+lOcg0UOmXSrQC045uLX3Ja8N6viPk6mEPb/GtVsoP4pRuh+3K3t6zwZlY0YfPHnrzPqtFL6nFRAKFxSSAlL58fDItA== X-YMail-OSG: VjSLtBwVM1nbvJ_nXWby1UTb08K36h2wdBD1L0e8ovsIF6SZozu6nzwlT0ow0J8 t0jP9KjqffAy5n0N3.TX1fdWGDwvAt2vwAXSFqSPGMl1QuN2LTJ0sjoF7IdtBWHC0Hl8Y699k5yp BvuZWmTtXMcxauWlM7HJ2dH0FSLe2oUO4vtE3g6VoMlIdPcmsOy72SIz8JqQjd6cadML3IpKihlO f5yGi6OY7ew19PUVjmIK8wKENMaviZKoln3YXkcGJr.s9EZJo2wSLO4G8JKYSJj70nLv3w8R2OJ9 erlUQ49ZtYTWh2uJg6gdbeMB.59bwa.xxmO7HgsbRJBoL62oIq5fO1L6032m6j_aP9bhPka9TYck NqI3B.gAXvfMHqYZz_5OvGvRb0CAnCkHzFaeJU0WzxF3raix5n16ro2fdgjTDK7aEEyzg2yO4Yg3 Snv2nl8PVGGdMXlPN7sR4zcNDCIL35UW8repwRLyrB0xZqCIjz4tu_Zmh6aRhuaHGOWpGaLyfBOh 1yT2E73GaztBtXppRulLo_96KkDNp6qyw3NPludr5nle.4W8k2Ptyif6JLxRc7SS.07kBqsqWAXk 7t4CMro_Qnn4tTNjEmEiGLB7FfY9U9t2qMCO7ijmEdelLOO3.nlq33VH2aah1DBc2AqDn7n_8FK7 TLx6qtSdAq4xWl.p0xSNH.n4fCNI.KKfS6fTUDpHpVTV6d85fH7h_V1UPLIgl.NjQv8czfNF0HFF BcCiUv_1l6fqWji1mUd28XDZkNL8MFeXWGy3tqEASays4DYm4Utwlfean7P23SzYwV_NWNK4gcix wHbxJdqE82gsn75mrWiCJvT29Ybrheb8Zmd_FBdGkEe.XMBr2IxI0E8tl2981VYZ1dai3ZgioqDo iQHaU649AQvB1i_.XJztzbjZz2oxDdsePN4Wegit41Um4EWI.1ztJkHuiPEj558u9qyo_WpZOr9i ZB2T0hkjN37Hl_BRr3W_YSol.h6L3DUB1wyVAIuoaFPueWr0a3wB5yN6kBn9KHsDoE_Wt1UxkPIz W6ySDbfqu2b5FjtjRrhrcPMWlf44xH6hyjCeItMh324.x1di1zXekzLL.RqZQa9z5Uk2CDR6340X HrQtb5CfqzDS88tgFJ0.kmtzMbJZ.uGtZiEzwWI0l7lnrOrVEFLurPgWGEBmoDVwBsIJr2qnUGjw xMCews6_1Keuw0Kru1WAruZEi3PR2Sb9nu5FDwXYj_lAjs6lYw95HfwZR2z09rN3uNFwLn0KqDBc c__XuvUdw_TZyItMkgoEcwu_j8xN.hKexUG9KHHb3k72UQm2VrTmY4knXNxxXeUlLutAPpOYSBBo 1.jOCrWiWT.knvjgI7novtHfykOLP70XLtHHjJuWm8X3woV.DbdRRdx5.FjeEi6tPbxcn8cEQhaN .rXVYJs8KeCZeCqc2e0WmonPY6apT3xqDAG0CJRzoGR6sMdOndz7HCnbDuLFrcbUh.fTtlPFYKSN knYh.ex_tNYybVm8OVG8pQQUARaL5AqXcYcCdwrn3ge80BeBFYPVIf7ldhSN_4conRyF39ddrCxX j7IpOQNTpzJXf2Au0grOvEPcIqjEcNx8QO9Rpj1pGcx9Zkb7BaXwq8g5LX5skvUfnFMEqMHAy4X1 rzWyUEcfTu3VyFPmGQaKxjFoX6cog0DU5gpPHdx9cX7L9Yw.GHV_4a.NPYuOTIssGrE2_MUCduA_ uCyBApLGWZ4txBVHH3fqr1FhCpu8Hx_9Iw9YmUAkl2bXoVaPQIxd5Jp9HaOt9Zk3lr4ZaPqhr_E0 graUE7VxG6TEY0ruKSI7g0nmWlAmAQSwL0c9eCaInyqpwio_.2Fv0Oaj6l_hUE0NAPW.SvEWpNHo uVJleCKZ2yAvmWs1XnHFMwC1Nqv7dQmDDB5T09JCH2vWs27ut1C1WKitSxJVOZ8rdmojz9KiIoTP CYbAv5_7z0t1x3ud1MutAl8elasjlyy5WF3T1JCu3Mad_1QNbVw2oXYTvKD6CbZZVGf2LKPhfCzq TvaX3GdWbmFMhETQmUOqyqOIeyqRmfuZZc87X.uJFIaMpolFe92hJ4FkNOkM91ytXD3RMlXzZ5_J KAgZZ6_GRTGNIxGMlJVANvHK5916RAiIakVAu5yGgj53vEyg_Q4wfFXt5.wdAu030MidXEti68nY mdDwyfNZOkqdfkZmaOwNMZhB80TqCESdRvV8J9VonkTKhZgfSNgNnXEHaHW_bNRzXzyxhp37poA1 AHLGZktzNky7OHxuKB0BDRztzo6CjuANQq08cOqZPEgxlhtJqr3AyH_xa3Cbw0xYUH23juKTpotH BVFPFdYBMyf20GdqA X-Sonic-MF: X-Sonic-ID: 2a795778-1bf0-4b24-8f5f-69fe7fb48c3a Received: from sonic.gate.mail.ne1.yahoo.com by sonic309.consmr.mail.gq1.yahoo.com with HTTP; Thu, 15 Feb 2024 03:15:56 +0000 Received: by hermes--production-gq1-5c57879fdf-8lthq (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 21fc683c801ff0629383b7f8c993d52c; Thu, 15 Feb 2024 03:15:54 +0000 (UTC) Content-Type: text/plain; charset=utf-8 List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.400.31\)) Subject: Re: files/edd962f76ea4b5869f3c6f8ee5438fb9750b802d02bb8035fe1b7bd0a8ba7401.gz not found -- snapshot corrupt. From: Mark Millard In-Reply-To: Date: Wed, 14 Feb 2024 19:15:43 -0800 Cc: Mario Marietto , freebsd-arm , freebsd-hackers , FreeBSD Current Content-Transfer-Encoding: quoted-printable Message-Id: <7AA6A6B8-9BAE-4B03-9EF4-4A4D242582B4@yahoo.com> References: To: Warner Losh X-Mailer: Apple Mail (2.3774.400.31) X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US] X-Rspamd-Queue-Id: 4Tb0ZR4M0nz4vkK X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated On Feb 14, 2024, at 18:23, Warner Losh wrote: > You may need to grab the repo. You may have to back up to December's = ports tree...=20 My understanding was that portsnap was staying installed on 13.*-RELEASE until the last version is EOL and that portsnap servers would be kept operational with valid content until then. I would have guessed that this would mean that 12.4-RELEASE or the like would also be able to use the portsnap it contains over that same time frame. Am I wrong? As stands, git use on 12.4-RELEASE would require bootstrapping git somehow, possibly via portsnap, building git, and then installing git (and the other stuff required). (Not that I use portsnap.) > Warner=20 >=20 > On Wed, Feb 14, 2024, 5:51=E2=80=AFPM Mario Marietto = wrote: > Hello. >=20 > After a lot of work I've been able to install FreeBSD 12.04 for armv7 = on my ARM Chromebook. Now I would like to install some ports. This is = what happens when I try to get a fresh ports tree : >=20 > marietto@freebsd:/usr # sudo portsnap fetch extract >=20 > .... > /usr/ports/databases/py-sqlalchemy10/ > /usr/ports/databases/py-sqlalchemy11/ > /usr/ports/databases/py-sqlalchemy12/ > /usr/ports/databases/py-sqlalchemy13/ > /usr/ports/databases/py-sqlalchemy14/ > /usr/ports/databases/py-sqlalchemy20/ > /usr/ports/databases/py-sqlcipher3/ > = files/edd962f76ea4b5869f3c6f8ee5438fb9750b802d02bb8035fe1b7bd0a8ba7401.gz = not found -- snapshot corrupt. >=20 > I repeated the "portsnap fetch extract" command,but I always get the = same error. >=20 =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Thu Feb 15 07:03:36 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Tb5dM2jyHz5BQyb for ; Thu, 15 Feb 2024 07:03:51 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic316-8.consmr.mail.gq1.yahoo.com (sonic316-8.consmr.mail.gq1.yahoo.com [98.137.69.32]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Tb5dL3mhbz4Ptr for ; Thu, 15 Feb 2024 07:03:50 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=NsqkquhJ; dmarc=pass (policy=reject) header.from=yahoo.com; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.69.32 as permitted sender) smtp.mailfrom=marklmi@yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1707980627; bh=MeXQchCgb8mM0+cCyIy5FPE8IoHKb6gt5mkB4NXtxxs=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=NsqkquhJVU5CmP+Fly4l9eXMaDLQvtMMwhrJrYFMh6crHyi/p70Z76FWHbbVI/EYElT0+m7PaHvmtGhycTORQlddx0SWWoQvSlF06g+YDlLfRtexil+7Wqo+A28Z+CKLtyrbNi0Lv3UD8cHT67zLgoyPm5jCSEoLbvAq0qwGzmLrkdHI6qU5MVLkf9JazfhP1aZ5fyXGq67yTAma7Ode+Sc7sS6mG79VVIn+uRQS9e3lZMXm34bwAcIGwKiWKOoy01TPGPRtb6uo5LqP3lZr5lY+uAQLncm1QosXQApnlEjEv272NxRC3wzocmVENp87tlkCeP6R3EShc+IWnJ/mvA== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1707980627; bh=DKVJ13ryK05PfLedGgE/V+R5vZaDlcGlkrhE/o6JJlX=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=eUJROzSb9PZIrIPGT29fjLSFZdtIVRgH/vHPA/025qAYnKy3o4sPrf7YBpM/wCbMIRlczSQrjw8ESqckmvp6ZYFUo0X5N/DQpDwax2FH9p7ERGDDPavixPmCmSFt9XFu7a9ZZi+OCjiER+dBsA+wrAWLWhY+/MQ2FbAf6z/IsWPnYl1wG1fvT0rXr3dLVjQaZJgMHjx0AmpQXv1xn8VIdxAg7HSE7cQWSc4BSr0cEZ2HLxEzitzEcgM7Qrkwk0NqGTVd4pg7JWlbiHNScqMAiddMl9p05kvs9hbCX1aDEmUcu5rRDhddG4tsf1X6CYbeP5REjnej5ZzkHMmBZD649Q== X-YMail-OSG: WD4bh98VM1lhGtFTYrjqYk3BNd5Xb24cDq3hmIEmnEU..ouZyjBhhWqr3js3tRt A2F2wlpleuz0sLv.v5n_7s8ruls_fj65DxOEFSru0FZQLu9MDtAy5vVuJhHvNjq4lkhTBicPHrwd juxREbdG0GijSTPmM0JCdhh04dCncjxvTbI6aHit29ahzDVSeo44xWyueMz3frLETv6s_DzTLaFE k92tptPcu1kTeK_vtfUpIcViDfKDIVohUvy9VrvwZn29XnrUWrG.qSi1Bmg4L69BylXwSD9r75Vr KIcSIBd3LKoxtIJ.f6zGiGH4qPH165GniM3u.RBl0RtPmZWXrhGPzJsS.UttXhGGy5Oi8e5sq8OB dzyaOloDCZp3gBkqAh5N0UNoDVOamLqfuDdMd85QdZDPLmHLeKs_luIgSwPKDJi26KrfDNkBVZT3 98N6a84dS5PLxFH4zdp02T0QyxkNFl3qBMQGUdaJ_3ea7lZCSG6UjyVrkcUM8IUKnGkitT_O4r_s E48Ml61TFupbVb.xFUwq8S4tosub1oSDN.FRAoWh.xhV69xF4kpHzxp8Md0mwLiwK0XTwlUowmtg 1BiVMMgdo8jrG0TuFplSfVSl2iyI6LPeoTESN4qrE65IUPOVKU2N_Iqo9VjpmSUm3o3Ccsinfjaz 1aFxJMPV4DDVKHjgGpAc_1q7H46RHG300XvR7wdNoWvWKc1rYL7FWmQD00vc1E6s5b_XK5t0M.g7 PIoRMsHSGI6zMaLdtqyJObBpBn5Wabs0csaQ_NrAKM4Xzx6J0u76u7e70fBT6HTxhAvPrZpMgMIO DrW7RudWqGOK4yeDI28czWQLBjoHdzlJKkvUyRfIP.EinmJp9v8aqlZ6.76wM.EAJ85roRM3lq9F 8cmKpBF8C9Jp1w4o_CMMCteJGtcrFI9UPeaXRRvbhzCaBRlu.PmqdgL1kv_e6paOu.LMxeo125Y5 It0uPzQYEJWKICEirMYQYwQiFnZW6mqR_FipKno82nBDyINxYryGozx6yobZbe2RkzXeDKP9O3r_ WTRBxjsqOh6Ekktl1UPz7ZIYx6a_lcuzG6iTXKKvjr2OtT4GtZ230J9VZ6cbRzHxUOrZMxOMemJ2 lpvqOjRTXVFlqpdjRg1aPM.BChepyWMKbi7OQKUQngs8NI._7bVI07WC4eRDI2R9P7DOQPXqw5GQ GNTp0kAqTLG1GMh8uMEr9UjNg9FFDqhIzvOyJmSEX_car5vv27p210k_lO8BFsiFzpm9b9rXfMj5 J.GVv_CJTirzg8e0cvwXKQkE.P3esH6v8cQnsOqvgCMZIG9dCRsuab8Jj.Fp68MloN9.KFlT4C_x TrntBuOEFjwvmLL9gdaW2ujcAQz_WX9LEFe.1atRmST4fAxCxPgxyfsYSaOiJWhWHJaVRMQLwB63 8AARleU9RGrMyryCAPHtaNlSXbD9m9lBO1aMf0CICASuyptccPea2eCoD3bvcrEXr86XHOWBoLhd .Ep8U22mKDPlAEmisJFbwlUnHEuiLZLXeb19nmuL1vVXaZeHs2IOZz5DkGsWt6NQm2qg_E3DuE7m XAxNdCpBgGeGViwZotKtNlo.pY7CdCJsGuYrwNBl1V7ciwMzH2W6gxW8fUMDncsbQuc1ftwdSNlQ 2vkUwBwGMku6UoyThRdlHSZE0p.mr1baUQXa2Frg3htuKdaRhCUGe.hDeaTeo9jIcFsArcC_zzgj SXl.Hq5Dz5Xx2K7aJE7Y4nQI7YeD7K6Aq.KyBoyUS3i9AWptYochs4rUCw5arNmkF0M_QWeLU3g0 ta4GrwpLLaybRZYt0wCiQAvDBxHBklSPQayVv7GcNUyaNDYGp0Ogt1dJq2LwXR8vA8d13Z8KjXIn 4U5MTnwSRvh_LcdDqvJYhXe2Kuj61A1K40lafxYCBGF_A3GZqK161RHhaHj5SNc0bPwmzHdJ9k7q RA.A7Nf_nEgV56emBWQysI6emby_kKIX1zsrUJG0O3KSaBrhHiNOdTfUnyIJ5UlbgeRM9KDW1JMx lcji4grxDozCdw0IsfLXHpG3xHcX9ONSuU.iasKs75ncOIaMnNIsxJ4YNsBZyh8tVOly2khej5d. mv2dsee_KuhSvtz9OMWtrpqqCQRjMaA9oTNlytWnpBDM9aocwWm5aXzwLUqMALyIUIU8v5WeEhXR HWAHkRuCMZRk4DotYQFQD7DPYe6dDzUTcWEy1FJIKmluPCvK0.zzPYShhD2rAtmLYBk_PfD7KmaF uZygKEITxthpgqx.8fMlZQg.iiyT7EhkXdjidmHWFOQ04G_rwnsKHduxPX_roLplyJ67rQU.hLCA - X-Sonic-MF: X-Sonic-ID: 97c9ac52-1e8c-4137-9c70-4ebbec7897c9 Received: from sonic.gate.mail.ne1.yahoo.com by sonic316.consmr.mail.gq1.yahoo.com with HTTP; Thu, 15 Feb 2024 07:03:47 +0000 Received: by hermes--production-gq1-5c57879fdf-qprqq (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID f70fe77b7d2b0cd41d033cafece53803; Thu, 15 Feb 2024 07:03:46 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.400.31\)) Subject: Re: Recent commits reject RPi4B booting: pcib0 vs. pcib1 "rman_manage_region: request" leads to panic [now fixed] From: Mark Millard In-Reply-To: <23365E1F-FB27-49C7-A691-5DE3FDE40940@yahoo.com> Date: Wed, 14 Feb 2024 23:03:36 -0800 Cc: FreeBSD ARM List , Current FreeBSD , Warner Losh Content-Transfer-Encoding: quoted-printable Message-Id: <538577EB-70F0-4921-9CC6-2F12A06E9D45@yahoo.com> References: <76AB969F-5BC5-4116-8AF4-3ED2CABEBBA5.ref@yahoo.com> <76AB969F-5BC5-4116-8AF4-3ED2CABEBBA5@yahoo.com> <1F704317-FDB8-4BDA-8A67-61CF48794DFE@yahoo.com> <9AFDF067-96E4-4E67-90D2-F40DAF3F138F@yahoo.com> <4C279710-5F88-4295-B1A4-7C395F3587E5@yahoo.com> <3A145420-399D-4EBD-9FF4-18924908AB1D@yahoo.com> <1298DF9C-0F82-4567-8E81-7332A608C7FC@yahoo.com> <36ECB040-7E09-47A9-AF71-DE546A78E9CA@yahoo.com> <0b0ba1b0-dee5-47e2-bb93-b44a24492abd@FreeBSD.org> <072C1A6A-85AA-4B26-9598-E5A586F9A4C2@yahoo.com> <23365E1F-FB27-49C7-A691-5DE3FDE40940@yahoo.com> To: John Baldwin X-Mailer: Apple Mail (2.3774.400.31) X-Spamd-Bar: --- X-Spamd-Result: default: False [-4.00 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; DKIM_TRACE(0.00)[yahoo.com:+]; TO_DN_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; MIME_TRACE(0.00)[0:+]; RCPT_COUNT_THREE(0.00)[4]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; MLMMJ_DEST(0.00)[freebsd-arm@freebsd.org]; APPLE_MAILER_COMMON(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.69.32:from]; RCVD_IN_DNSWL_NONE(0.00)[98.137.69.32:from] X-Rspamd-Queue-Id: 4Tb5dL3mhbz4Ptr On Feb 14, 2024, at 18:19, Mark Millard wrote: > Your changes have the RPi4B that previously got the > panic to boot all the way instead. Details: >=20 > I have updated my pkg base environment to have the > downloaded kernels (and kernel source) with your > changes and have booted with each of: >=20 > /boot/kernel/kernel > /boot/kernel.GENERIC-NODEBUG/kernel >=20 > For reference: >=20 > # uname -apKU > FreeBSD aarch64-main-pkgs 15.0-CURRENT FreeBSD 15.0-CURRENT = main-n268300-d79b6b8ec267 GENERIC-NODEBUG arm64 aarch64 1500014 1500012 >=20 > Thanks for the fix. >=20 > Now I'll update the rest of pkg base materials. Question: Are any of the changes to be MFC'd at some point? =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Thu Feb 15 17:34:23 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TbMcw3fLZz59hmf; Thu, 15 Feb 2024 17:34:24 +0000 (UTC) (envelope-from jamie@catflap.org) Received: from donotpassgo.dyslexicfish.net (donotpassgo.dyslexicfish.net [IPv6:2001:19f0:7400:8808:123::1]) by mx1.freebsd.org (Postfix) with ESMTP id 4TbMcw2Nk9z4ffB; Thu, 15 Feb 2024 17:34:24 +0000 (UTC) (envelope-from jamie@catflap.org) Authentication-Results: mx1.freebsd.org; none X-Catflap-Envelope-From: X-Catflap-Envelope-To: freebsd-arm@FreeBSD.org Received: from donotpassgo.dyslexicfish.net (donotpassgo.dyslexicfish.net [209.250.224.51]) by donotpassgo.dyslexicfish.net (8.14.5/8.14.5) with ESMTP id 41FHYNf1061906; Thu, 15 Feb 2024 17:34:23 GMT (envelope-from jamie@donotpassgo.dyslexicfish.net) Received: (from jamie@localhost) by donotpassgo.dyslexicfish.net (8.14.5/8.14.5/Submit) id 41FHYNBj061905; Thu, 15 Feb 2024 17:34:23 GMT (envelope-from jamie) From: Jamie Landeg-Jones Message-Id: <202402151734.41FHYNBj061905@donotpassgo.dyslexicfish.net> Date: Thu, 15 Feb 2024 17:34:23 +0000 Organization: Dyslexic Fish To: marietto2008@gmail.com, freebsd-questions@FreeBSD.org, freebsd-hackers@FreeBSD.org, freebsd-current@FreeBSD.org, freebsd-arm@FreeBSD.org Subject: Re: files/edd962f76ea4b5869f3c6f8ee5438fb9750b802d02bb8035fe1b7bd0a8ba7401.gz not found -- snapshot corrupt. References: In-Reply-To: User-Agent: Heirloom mailx 12.4 7/29/08 List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (donotpassgo.dyslexicfish.net [209.250.224.51]); Thu, 15 Feb 2024 17:34:24 +0000 (GMT) X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:20473, ipnet:2001:19f0:7400::/38, country:US] X-Rspamd-Queue-Id: 4TbMcw2Nk9z4ffB X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated Mario Marietto wrote: > After a lot of work I've been able to install FreeBSD 12.04 for armv7 on my > ARM Chromebook. Now I would like to install some ports. This is what > happens when I try to get a fresh ports tree : > files/edd962f76ea4b5869f3c6f8ee5438fb9750b802d02bb8035fe1b7bd0a8ba7401.gz > not found -- snapshot corrupt. I'm not sure why the file isn't there - maybe because 12.X is EOL or portsnap is deprecated? Still, the solution is easy: Download the ports tree snapshot as a tar from https://cgit.freebsd.org/ports/ Choose a tag, and a format. I suggest 12.4-eol so just fetch https://cgit.freebsd.org/ports/snapshot/ports-12-eol.tar.gz rm -r /usr/ports then untar the downloaded tar file into place. Cheers, Jamie From nobody Thu Feb 15 17:39:50 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TbMlF09Tkz59jYB; Thu, 15 Feb 2024 17:39:53 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TbMlD6hqzz4k2f; Thu, 15 Feb 2024 17:39:52 +0000 (UTC) (envelope-from jhb@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1708018792; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=F/i6e2J+/rZijI0GgJmaVrFspSXcI5JBt68CUkYiILE=; b=sd0qYIV0ceAn9vQYdMx+cbIertMFLHEM/KxbSywfCiJ3Y0O/DsOwQewNCssPtEnjZBdpS1 14X6pLlG7fHlR1997Co8yGIQXKeG1sfDSGBezHcG+ko6RkNTBZR5DqH3E5q9UWmFejdwZJ rVYDcQZCf3lWibSQ83EHANQNwCi/OVw6n4d6qvP7lX7rnq/8TxYSURrca3++2/OOCWYIsT 3NsY4sP9oYggpc1jB+VGtrdXhVfwBcPROSJAuGjWVlwVt9Ucn7jcpbld/NFYEjwOAOsZI0 gX2OY6Q0dbzDA0t0ONEPIbIPUBi2OA3dIY0p6jEhNbI/I75bjeyMH0zRVoQ+lQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1708018792; a=rsa-sha256; cv=none; b=cDgxhK+UPtR8RbEb9M0HH9AN7HewyjfPxKsbyhX15ASQU1QxePbbLxXGqfdTZnvAa06cO2 6XvJpd/i2LXY07ioq/NkaVrosZSkLAXnmwsOybDHTmOJ9EVhscjgszmJDUENq5AA4WfTHD a0VV+nMFaU6zuhVZNv7ed38KlB+yQ3PNIdxO9zs5Aiys4G68s9D8ekMXOcWaemuQ1kMFen Bdo2Z1oTeN7/y6DANDqGfyKi61WbTeOrHIWImbpeic7/tktAY1ub452WvM6fEWRHlSMRhd e4Oin3wS4YkkrODyYoc9RnV2turYdEEOcNDXIjIvZqsp/gZyWbm6Uq3tUfgSdQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1708018792; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=F/i6e2J+/rZijI0GgJmaVrFspSXcI5JBt68CUkYiILE=; b=KVi1FpW8RQFh3PNHZWC5xa+ybE8rNj4XhKGrde6AaBr206ITnWt/u4t1a3GxYW2hcWFC6O gthQRHCdhcjZpIMVrAOAa4LVKomdCVj0Qd3BTGBlk46hpXQIAbwpmgjlrB/+jSGmtx/eM8 s490iKPzKklcA02ezdKj9DLw8f3pralJpiqlfDfZExBlUC3X/0oKJ2vqXdH+JPqOjCAch2 cwDGRE+Gl8z6XpeQeAk1Rcd4YR8KjQn/C19B8Pyz+kkCyrOK4DVx/LORr3tDTvzcylvi5e kACFI1BNAnTXLAelB7vQxOwUeCaHdXBAvmuPsPeF/3Sk5beJPJ0TiYhLHHobwQ== Received: from [IPV6:2601:644:937c:5920:581f:5b54:cee8:a782] (unknown [IPv6:2601:644:937c:5920:581f:5b54:cee8:a782]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: jhb) by smtp.freebsd.org (Postfix) with ESMTPSA id 4TbMlD2RNrz19P3; Thu, 15 Feb 2024 17:39:51 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Message-ID: Date: Thu, 15 Feb 2024 09:39:50 -0800 List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: Recent commits reject RPi4B booting: pcib0 vs. pcib1 "rman_manage_region: request" leads to panic [now fixed] Content-Language: en-US To: Mark Millard Cc: FreeBSD ARM List , Current FreeBSD , Warner Losh References: <76AB969F-5BC5-4116-8AF4-3ED2CABEBBA5.ref@yahoo.com> <76AB969F-5BC5-4116-8AF4-3ED2CABEBBA5@yahoo.com> <1F704317-FDB8-4BDA-8A67-61CF48794DFE@yahoo.com> <9AFDF067-96E4-4E67-90D2-F40DAF3F138F@yahoo.com> <4C279710-5F88-4295-B1A4-7C395F3587E5@yahoo.com> <3A145420-399D-4EBD-9FF4-18924908AB1D@yahoo.com> <1298DF9C-0F82-4567-8E81-7332A608C7FC@yahoo.com> <36ECB040-7E09-47A9-AF71-DE546A78E9CA@yahoo.com> <0b0ba1b0-dee5-47e2-bb93-b44a24492abd@FreeBSD.org> <072C1A6A-85AA-4B26-9598-E5A586F9A4C2@yahoo.com> <23365E1F-FB27-49C7-A691-5DE3FDE40940@yahoo.com> <538577EB-70F0-4921-9CC6-2F12A06E9D45@yahoo.com> From: John Baldwin In-Reply-To: <538577EB-70F0-4921-9CC6-2F12A06E9D45@yahoo.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit On 2/14/24 11:03 PM, Mark Millard wrote: > On Feb 14, 2024, at 18:19, Mark Millard wrote: > >> Your changes have the RPi4B that previously got the >> panic to boot all the way instead. Details: >> >> I have updated my pkg base environment to have the >> downloaded kernels (and kernel source) with your >> changes and have booted with each of: >> >> /boot/kernel/kernel >> /boot/kernel.GENERIC-NODEBUG/kernel >> >> For reference: >> >> # uname -apKU >> FreeBSD aarch64-main-pkgs 15.0-CURRENT FreeBSD 15.0-CURRENT main-n268300-d79b6b8ec267 GENERIC-NODEBUG arm64 aarch64 1500014 1500012 >> >> Thanks for the fix. >> >> Now I'll update the rest of pkg base materials. > > Question: Are any of the changes to be MFC'd at some point? If I do I will merge a large batch at once, and probably adjust the order. For example, I'll merge the pci_host_generic changes before pci_pci changes so that stable branches will be bisectable. -- John Baldwin From nobody Thu Feb 15 20:32:19 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TbRb01B8Cz5B7nG; Thu, 15 Feb 2024 20:33:00 +0000 (UTC) (envelope-from marietto2008@gmail.com) Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com [IPv6:2a00:1450:4864:20::62a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TbRZz6cZ2z4DBF; Thu, 15 Feb 2024 20:32:59 +0000 (UTC) (envelope-from marietto2008@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ej1-x62a.google.com with SMTP id a640c23a62f3a-a3566c0309fso161739466b.1; Thu, 15 Feb 2024 12:32:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1708029177; x=1708633977; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=CXyYJ+TWAhbtOKbEVx789CYk4pg44FBrve9Yg2bz8Pk=; b=V4Qsg2GV+teoxrgx2+n4huie0ydGEWoFinW8fu1anG4TQBkSLWz0dk9u18z46lL60y 0QF8IAlUdvZxHfItYrbW19iEJxdx+zNog0jcDFqh70upZKSO8EdZWQ3w2Yv57QHnhzZM +mqiyABkYhAG8n6NBIN3eX13C+jMt0IMTQe9dYyJXn8SLaCSz9K5G+ly3Uqm7NhEz7et TGDaVd+x+wwRNiV3EOYvnjLE4F5jzI1Idez5Ti64y9m56ZjUfdWVssRvjsS1K4b4Ovj5 flghYQjrP7PycrxqYbEMzywksGXLKNTt/wOCAfneKpadA1s+VigA5aehlCUOHhLx9mRa cXfA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708029177; x=1708633977; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=CXyYJ+TWAhbtOKbEVx789CYk4pg44FBrve9Yg2bz8Pk=; b=Doart7apug+F+3UOqXupzGHLs3RhCQb00XIBFTcvXX2AQhhluw9VRaiR6gnHodi9sC zuJt9CCVmJ4rtwKxcWXr/9yVdkwRHwMr1HgexvmFK9lBoKazNjZhkOfW5Gt8sEWYxqOT NP8z71M/IiOtX7XmB/vlF5xmhIFQ/M7QIvkdgeKNBHEvwtS0dGaYMUvqd5Ghn2hgP3jl IgaxfOrt8cGS/0doM8wcrQ+JEjQumRPMhvRNFR6fJkYc30WG9h9YHLJ6qJVVCdg3zz6c GLCbUwBMaaT2MQ5dI9F2+jJk9M+ve7f3I3ojIGBNdzJHuIy0wHAKdVPiPmgmPOIwsijA RqDQ== X-Forwarded-Encrypted: i=1; AJvYcCUVA64ABdWQonzx5cGfT5RGClVwqNQNErgXZ2hyNgTFgzofuzbGP6jlzPdqVXAR6ye7z8h5xP3dwJm4/mdUH2IHMZBKHGCg9FoRQezJPagfckXmKLBP/mM/GxnkVCP6qPlRyAdU3fhjfFKRWV3BoLEoTz6BfUj57OifepMNvaIGa6A= X-Gm-Message-State: AOJu0YygUOUAwG0QAegEZjIF3mQyga1la1oM8LTNjJs0BfPbdl5oKIUc RnW1kaE84lkk+32fXeqMIFZTeF3FfnJ5DtKNgO5c9kfTWkVRlQ9+byllyhoLSorAdYqtga0KEix xhmo64q6744WCQPKJtVYqTEcEJm1lTqsgSrNszQ== X-Google-Smtp-Source: AGHT+IFxVOyOt+MW5cGBcXU2R9bd4/Z1qVS6DdTgArAe5DkmUnC/cyShdLFkXesnlGmw+tdDJPAdrG6T+sEXfPvVJ80= X-Received: by 2002:a17:906:4c58:b0:a3d:3be6:2a7e with SMTP id d24-20020a1709064c5800b00a3d3be62a7emr2370672ejw.38.1708029176577; Thu, 15 Feb 2024 12:32:56 -0800 (PST) List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 References: <202402151734.41FHYNBj061905@donotpassgo.dyslexicfish.net> In-Reply-To: <202402151734.41FHYNBj061905@donotpassgo.dyslexicfish.net> From: Mario Marietto Date: Thu, 15 Feb 2024 21:32:19 +0100 Message-ID: Subject: Re: files/edd962f76ea4b5869f3c6f8ee5438fb9750b802d02bb8035fe1b7bd0a8ba7401.gz not found -- snapshot corrupt. To: Jamie Landeg-Jones Cc: freebsd-questions@freebsd.org, freebsd-hackers@freebsd.org, freebsd-current@freebsd.org, freebsd-arm@freebsd.org Content-Type: multipart/alternative; boundary="00000000000056059806117188fc" X-Spamd-Bar: ---- X-Rspamd-Queue-Id: 4TbRZz6cZ2z4DBF X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] --00000000000056059806117188fc Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hello. What's the correct port tree for FreeBSD 12.04 for arm 32 bit ? A or B ? A) https://cgit.freebsd.org/ports/snapshot/ports-12.4-eol.tar.gz B) https://cgit.freebsd.org/ports/snapshot/ports-release/12.4.0.tar.gz thanks. On Thu, Feb 15, 2024 at 6:34=E2=80=AFPM Jamie Landeg-Jones wrote: > Mario Marietto wrote: > > > After a lot of work I've been able to install FreeBSD 12.04 for armv7 o= n > my > > ARM Chromebook. Now I would like to install some ports. This is what > > happens when I try to get a fresh ports tree : > > files/edd962f76ea4b5869f3c6f8ee5438fb9750b802d02bb8035fe1b7bd0a8ba7401.= gz > > not found -- snapshot corrupt. > > I'm not sure why the file isn't there - maybe because 12.X is EOL or > portsnap > is deprecated? > > Still, the solution is easy: > > Download the ports tree snapshot as a tar from > https://cgit.freebsd.org/ports/ > > Choose a tag, and a format. I suggest 12.4-eol so just fetch > > https://cgit.freebsd.org/ports/snapshot/ports-12-eol.tar.gz > > rm -r /usr/ports > then untar the downloaded tar file into place. > > Cheers, Jamie > > --=20 Mario. --00000000000056059806117188fc Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hello.

What's the correc= t port tree for FreeBSD 12.04 for arm 32 bit ? A or B ?

= On Thu, Feb 15, 2024 at 6:34=E2=80=AFPM Jamie Landeg-Jones <jamie@catflap.org> wrote:
Mario Marietto <marietto2008@gmail.com>= ; wrote:

> After a lot of work I've been able to install FreeBSD 12.04 for ar= mv7 on my
> ARM Chromebook. Now I would like to install some ports. This is what > happens when I try to get a fresh ports tree :
> files/edd962f76ea4b5869f3c6f8ee5438fb9750b802d02bb8035fe1b7bd0a8ba7401= .gz
> not found -- snapshot corrupt.

I'm not sure why the file isn't there - maybe because 12.X is EOL o= r portsnap
is deprecated?

Still, the solution is easy:

Download the ports tree snapshot as a tar from https://cgit.freebsd.o= rg/ports/

Choose a tag, and a format. I suggest 12.4-eol so just fetch

https://cgit.freebsd.org/ports/snapshot/p= orts-12-eol.tar.gz

rm -r /usr/ports
then untar the downloaded tar file into place.

Cheers, Jamie



--
Mario.
--00000000000056059806117188fc-- From nobody Thu Feb 15 20:47:44 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TbRwK6KJRz5B9ls for ; Thu, 15 Feb 2024 20:48:01 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic314-20.consmr.mail.gq1.yahoo.com (sonic314-20.consmr.mail.gq1.yahoo.com [98.137.69.83]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4TbRwK3qYyz4JK0 for ; Thu, 15 Feb 2024 20:48:01 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1708030079; bh=4qKGG+MSXbrIiD5DPqZlxtgJLWticdhZifwIwYbpPD4=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=LNnhJr46HkiKy5NmESDobTdPKQp1OEeNs+KEzaBHljZDkqfXkQ8r3eLwuF83LWVQnCwHw/7AuOv2fpLQnKp8BMmoJ/UReUDYrfH3y0onF6IqzmhplHz6nx8DCFm5Udlz9rDmVN/WZSsD5NNI2tevDabj5rDn0ovGLu0k8fWfNHH4f0dzqKLqd7voaEkpJNdnBAtcgLR70uDZnoQMO8BRFcrnB39S7JW00RNNZZdYSuBRaqgVnTIiUTaOv54yImWxAjGXLehPi3EYdmB9e7FfvlBdIPVjJSBQ4mw7hY1E6mSo2HmXD7aoyc9t/0d/9toLDLM8f1oQBjaEA4QTbSAPlA== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1708030079; bh=gY9uDz+JDyV1B2hPJeot1XXcJo8+PExKA3uusY03wno=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=kGENsrDFLV8Rg1saFDFwn5pR5BT5DDAWX75j0OLg0OHIptWYABpkTfAwo1iu/gKKLTAVJcBa9YnSxaxAP76ouU2nrW/O67YcrELeJtMTzhcqU7HvAZ8BhWvX7gp2dl320aex4KT+Qvzf8THPjISwOfsuAfChSXY/3hAogsi/49DhCTIqkxwUhQLmziT6TaZtS/+qjWGPSctIN+eyxQ/iEVPvjQQ/LSecGjPlweyWcAeUJgpmIFC0wzm4vdIe1ER3Ezl+zkGiG3VILly9cSbZT4fOCCnleJDsg6BjBk8XBEhv74CQZy++IoKOiOlMkbC3WeSfaNyrXjMaSSt5qs+Vow== X-YMail-OSG: RnFkW2YVM1kJm01VQ4r7IRrpHX8_R4YPDELZN0mM4zbW1.c7wOeHrDmBiVdYlBd 3gKztPHZXanfCoG5HltMhjvNNPbeFOAdhXEgduMZTh71AEi3RrIBq2EPIl.4OWczaCJNUS10Uz5J 2vbPT1b92rpRqNP7n31z64cLWvsDpJmtOM82luqg13ngiWsmdzN4UKUrOcxTA06.BkDh_Qkyc7OZ vobuGhZ.L44DluftmJuccT7pVyaIOFLJf5gAmMX_rSMixeT6Dm6n40DwsckGZIptJ3sWqSPAnYr0 vszW9dwUbvHi9TeMYFu2kxGD69X76e.xgDtt1q0R0NnnlqwgKr_whFMVwSvS0hiAGPpoxE1mJICN U6klG91WFRz3YYGpAA0M.GI6N16w8ucE69QeNfxgOAhI_4424ztKl2ZyPNcS7dmokVJGxwguaVgT iR6JSNKnoAQmtMG.KDSQnSllw9gARHsYMXO_zZJ6ZhhHLYlF3KP33EH8UzdP6clNC9FNGO8MAwfY VrvL1g.qNMysZ3SRORwk48gRxyCSLcC.ufQcH6WHododLUbzat.W3_oX4AIxbv58qhIXiJEDRySk BqEg6oJrZxS_taZlPoHcgv8YtE3NgkK8G0rRmi7JubRXNA_BNxobtwE6C_0GzMlfDGxY6KJplojO fNazSVRyTx0WkEy.JRn0iG0g7a4VtYiHWxPwO0GApj0rmeHxwTUPRl_bO6IuS.Z2y3BHozvZ7tGK vN7jsJPoHlO6Qb.m2JAjHoKshOnd0JTxvt3fZrtdPBQfa3Msny01rG1Dx7h77kUYB1RgbQNYMCCv umJ_lG9KaUSmFx5rST7UI2R9qBaAFASnjcQYQRZis0beFvkZxgzxu3OI2cZi9NxmzhPw2l1v_Xsj 7_0KpuJKFCsPOjA2ZoKgVZ1DRQO3cHPP1VLjFUYlUHnARghoONFC9bS26A.kZkcaBRaQlzo2IkJi HEP0.Q_BpN8nLTtvZJls5XeXoi6VXoefVKPWAonB8fIdXvHmqytnYk0wB52.BjkTL8Ug48gfUL_i g1cpRQwB97aYve69OtMPvAp2LwpkZrZ7BizOL6MjCPj.f8G.LJUXevhQWhiIotHOQ54X3r3Y9QYO gN4DDcbAPFxAUOyb0ZtVIl3uXZzCecpQQ8U0HI4DJO4i3L3qCbcSxJWWLc.l7j.7MaVSQ3LXewSS m.Uag8xr8p0kga8kil6NI_hsey9847tV1dt0beL_w_d3O1sHx54VGroWqKmL2UzlDndQfekO_scJ Ha6byJXGvBvtGHRaMOwl5AoA4Mu_N.R47KgFhJ_D9Hu8L8WUqA9Q.rFfHMDtq_JwhuzltIPubDHb EOVQpOCLO2bQLeYAPipVCVk3TgPOzoPbltljZwDhpKGJ_jiBfSFJCDExUnS8FPKTsxc4keDUo_5Y azSqKFYs.w_S6qF_rX5njL4s.lTnIYuCPZVhTiVF6CLYoJ.vldkwk_vAhzBRBvWBa9OO3uvyIRis 8OMZXWhtDnEe4ePy8xv4RCRDAVvFo._MO00YKjwwBb32_WKShH3E1llhLbtoY0hmcuzs5Cy2Fb_S s5eCoCIp.cP1iYnO5uxgqjmLgBBaULmPtXHwPW.235MNGcAGqEK_q.AW8_UVmv5xZK3BjeTWv_Bj 0FNvUTLpi3mefGKe2QETr26FICE.Fijte27P6.XlNWweatTMyA7PfyfvXNbGiswjQcKHzVwbYpe8 vloNYEdpeWV6_Z6H7VpXMJtgRLsPUd2nwnAMlP0bXPxMVyP31sg544o2VjVFT3roDRJsTi1vYgWJ J416b8IIeLBuk00aigJT.EUQbyq8p..choMk7Fb3tTVK29J03M_JRU.wQhtSR_p07D4bE.zHfAOs 3LNNRp2sILHe5rWNBo4ciOi6Nhw7tx617CPhMfSlSmxutofg_mqFV.3PjiaK8psM9nMM21HCkyxk et20fG.j9p_7HN8ms5sOO7DQWnHtqg8unmssp8Qyt8CEF39ALMe1AUJpT7vO55ulxOq7nNNKWzcf EwKxdm_wIl3U1KTaUXdxAiG.SZN8AoF16Z7jNv1U73FTzYN7lnfXKCZuyjLKROc7F3akMdPVTpWM d9R3lxuX54rfbsZaxS4ONG982eHtum9W0pF2USu6zICQNua7BsKbzGeIiRPiPBGrF0JGa5olZIRN k561OC6NCeysFaMFByE.GdhJf9Z86uS07JBjirrV6SptCzVS1wEfwCeGmMwVMmjVqwKt9tUirwJ4 o._0BTZoS028FIw2AD021WHDN.UhW45EcQJV.nFhxQKszwo.qFaxw2VkKdMbzdUHP8P3VnACREGS RGkv5DiWRecDrMm88qpM- X-Sonic-MF: X-Sonic-ID: b8863f13-c3bf-42a0-a0d4-1a850d6a52aa Received: from sonic.gate.mail.ne1.yahoo.com by sonic314.consmr.mail.gq1.yahoo.com with HTTP; Thu, 15 Feb 2024 20:47:59 +0000 Received: by hermes--production-gq1-5c57879fdf-jv4v7 (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID ab63eb978d9def609a59fb9b6b6b8a28; Thu, 15 Feb 2024 20:47:55 +0000 (UTC) Content-Type: text/plain; charset=utf-8 List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.400.31\)) Subject: Re: files/edd962f76ea4b5869f3c6f8ee5438fb9750b802d02bb8035fe1b7bd0a8ba7401.gz not found -- snapshot corrupt. From: Mark Millard In-Reply-To: Date: Thu, 15 Feb 2024 12:47:44 -0800 Cc: Jamie Landeg-Jones , freebsd-hackers , freebsd-arm Content-Transfer-Encoding: quoted-printable Message-Id: References: <202402151734.41FHYNBj061905@donotpassgo.dyslexicfish.net> To: Mario Marietto X-Mailer: Apple Mail (2.3774.400.31) X-Spamd-Bar: ---- X-Rspamd-Queue-Id: 4TbRwK3qYyz4JK0 X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US] On Feb 15, 2024, at 12:32, Mario Marietto = wrote: > What's the correct port tree for FreeBSD 12.04 for arm 32 bit ? A or B = ? >=20 > A) https://cgit.freebsd.org/ports/snapshot/ports-12.4-eol.tar.gz The above is the newer one, with more security updates, updated ports, and such: it is from when 12.4-RELEASE went EOL. > B) https://cgit.freebsd.org/ports/snapshot/ports-release/12.4.0.tar.gz This is the older one from when 12.4-RELEASE was first built, long before it went EOL. > thanks. >=20 > On Thu, Feb 15, 2024 at 6:34=E2=80=AFPM Jamie Landeg-Jones = wrote: > Mario Marietto wrote: >=20 > > After a lot of work I've been able to install FreeBSD 12.04 for = armv7 on my > > ARM Chromebook. Now I would like to install some ports. This is what > > happens when I try to get a fresh ports tree : > > = files/edd962f76ea4b5869f3c6f8ee5438fb9750b802d02bb8035fe1b7bd0a8ba7401gz > > not found -- snapshot corrupt. >=20 > I'm not sure why the file isn't there - maybe because 12.X is EOL or = portsnap > is deprecated? >=20 > Still, the solution is easy: >=20 > Download the ports tree snapshot as a tar from = https://cgit.freebsd.org/ports/ Cool. I'd not explored there. > Choose a tag, and a format. I suggest 12.4-eol so just fetch >=20 > https://cgit.freebsd.org/ports/snapshot/ports-12-eol.tar.gz >=20 > rm -r /usr/ports > then untar the downloaded tar file into place. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Thu Feb 15 21:27:18 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TbSnw5Qtqz5BGx5 for ; Thu, 15 Feb 2024 21:27:32 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-ed1-x52e.google.com (mail-ed1-x52e.google.com [IPv6:2a00:1450:4864:20::52e]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TbSnv6Vykz4PKr for ; Thu, 15 Feb 2024 21:27:31 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ed1-x52e.google.com with SMTP id 4fb4d7f45d1cf-55a5e7fa471so104094a12.1 for ; Thu, 15 Feb 2024 13:27:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20230601.gappssmtp.com; s=20230601; t=1708032450; x=1708637250; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=QMjxRvsl+qiM5W6RWrLajeCn8pHAymIm439C+l12YUA=; b=rDn/anZ/crL77WZ1kG5f731cIgzIQkN0y2vrjB3UM/gqJhjFTo8wPb5eJpTxnpkZfH aXsKkKrtoBxCRtOfB4Ax6XpQV2kac8cWgAtY5cvAyPJEYyer9EqFB//B80Q9uYpbn/9W AE+J0JvXihUpfZ+b82qCoztE7NzK2mCYd6uR8PkxZ/V8Gveg2YdVYKGCvK7EvO87yCl0 ye4oZ1SkSw+zHMGG/kvb4Ndl33Ppx5P54OHQHcoqllFJf54pYWeZ0fOFqdjcxFyBiZEX xEFxUZvtqWZHjxrhdKnSPeMUjZBbmK9TWzu6KjP8gglK/6/wZA+IVRPe7rNc/fZHWHO8 G3CQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708032450; x=1708637250; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=QMjxRvsl+qiM5W6RWrLajeCn8pHAymIm439C+l12YUA=; b=PiMhtzUfcMHh9OsozfudD/0scJgTdZcfwJXUcn3/1fnjOW9L26d5WME9mVKLQTbiXu vd08+tGqdhYW4NdbWr+J5XTJ8uWZC9BWmpoeHsEUemyVxc51ALfyDXi337Kyxb9wTnJz P+ruZQN04mM2I8bhuUrpgreZ8qiGN7046Nv0tM0Qs7cTxt1aPVITm65+J1tf7RYcvASD fMlKw5HkFetvmjNGvjHnVNurtRmgcT6r1sD+XZqsX19UDHfHwWj8H4raBlE9fNIzeaWH avv5kzfe2mfeHBefGi50NWpMdy2XhjYi6i4xXqtAc9R0PWFqzBdKyIGZ/6y8qYRDQFiN ekVA== X-Forwarded-Encrypted: i=1; AJvYcCXP+vpIF/guNIkaTsaSjzgc4oMkVXKs2Nym74rWoSbLQC8WwlnvHcRvnRYPRK+CLh7daj2RmHsmN1X9fJziKKH58T+xsQNFxQ== X-Gm-Message-State: AOJu0Yy7ci6v9GxJRfiRw6ccexKzKjzvID3jNyxRT5E9Afzllvfsz7AD W8oh5oCTb/WQkFb8YT5n9qdLd/vvFHbOe2GLn8RANm2TQoi4VeBNPriWBMRKjuzpelm6O22FDVF fdHJ/3liyzYjRnpwEarxzsDqCEZFkQVULSWWx+A== X-Google-Smtp-Source: AGHT+IGO8LCIpFlEaM9QH4uFjQeU+TEAKF5YAe8f7aSZELnUW7LQ/Foi3IPkJvXCbi27d4s0rxlW1eTLurPZiIIPObk= X-Received: by 2002:a05:6402:646:b0:560:1652:e7cb with SMTP id u6-20020a056402064600b005601652e7cbmr2235970edx.16.1708032450496; Thu, 15 Feb 2024 13:27:30 -0800 (PST) List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 References: <202402151734.41FHYNBj061905@donotpassgo.dyslexicfish.net> In-Reply-To: From: Warner Losh Date: Thu, 15 Feb 2024 14:27:18 -0700 Message-ID: Subject: Re: files/edd962f76ea4b5869f3c6f8ee5438fb9750b802d02bb8035fe1b7bd0a8ba7401.gz not found -- snapshot corrupt. To: Mario Marietto Cc: Jamie Landeg-Jones , FreeBSD questions , FreeBSD Hackers , FreeBSD Current , "freebsd-arm@freebsd.org" Content-Type: multipart/alternative; boundary="0000000000007a2e2d0611724b4b" X-Spamd-Bar: ---- X-Rspamd-Queue-Id: 4TbSnv6Vykz4PKr X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] --0000000000007a2e2d0611724b4b Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Thu, Feb 15, 2024, 1:33=E2=80=AFPM Mario Marietto wrote: > Hello. > > What's the correct port tree for FreeBSD 12.04 for arm 32 bit ? A or B ? > > A) https://cgit.freebsd.org/ports/snapshot/ports-12.4-eol.tar.gz > > B) https://cgit.freebsd.org/ports/snapshot/ports-release/12.4.0.tar.gz > A is your best bet. Warner thanks. > > On Thu, Feb 15, 2024 at 6:34=E2=80=AFPM Jamie Landeg-Jones > wrote: > >> Mario Marietto wrote: >> >> > After a lot of work I've been able to install FreeBSD 12.04 for armv7 >> on my >> > ARM Chromebook. Now I would like to install some ports. This is what >> > happens when I try to get a fresh ports tree : >> > files/edd962f76ea4b5869f3c6f8ee5438fb9750b802d02bb8035fe1b7bd0a8ba7401= gz >> > not found -- snapshot corrupt. >> >> I'm not sure why the file isn't there - maybe because 12.X is EOL or >> portsnap >> is deprecated? >> >> Still, the solution is easy: >> >> Download the ports tree snapshot as a tar from >> https://cgit.freebsd.org/ports/ >> >> Choose a tag, and a format. I suggest 12.4-eol so just fetch >> >> https://cgit.freebsd.org/ports/snapshot/ports-12-eol.tar.gz >> >> rm -r /usr/ports >> then untar the downloaded tar file into place. >> >> Cheers, Jamie >> >> > > -- > Mario. > --0000000000007a2e2d0611724b4b Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Thu, Feb 15, 2024, 1:33=E2=80=AFPM Mario Marietto &= lt;marietto2008@gmail.com>= wrote:
Hello= .

What's the correct port tree for FreeBSD 12.= 04 for arm 32 bit ? A or B ?



A is your best bet.

Warner

th= anks.

On Thu, Feb 15, 2024 at 6:34=E2=80=AFPM Jamie Landeg-Jones = <jamie@catflap.org> wrote:
Mario Marietto <marietto2008@gmail.com> w= rote:

> After a lot of work I've been able to install FreeBSD 12.04 for ar= mv7 on my
> ARM Chromebook. Now I would like to install some ports. This is what > happens when I try to get a fresh ports tree :
> files/edd962f76ea4b5869f3c6f8ee5438fb9750b802d02bb8035fe1b7bd0a8ba7401= gz
> not found -- snapshot corrupt.

I'm not sure why the file isn't there - maybe because 12.X is EOL o= r portsnap
is deprecated?

Still, the solution is easy:

Download the ports tree snapshot as a tar from https://cgi= t.freebsd.org/ports/

Choose a tag, and a format. I suggest 12.4-eol so just fetch

https://cgit.freebsd.org/ports= /snapshot/ports-12-eol.tar.gz

rm -r /usr/ports
then untar the downloaded tar file into place.

Cheers, Jamie



--
Mario.
--0000000000007a2e2d0611724b4b-- From nobody Thu Feb 15 21:37:26 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TbT1k1DYFz5BJ1r for ; Thu, 15 Feb 2024 21:37:46 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic305-21.consmr.mail.gq1.yahoo.com (sonic305-21.consmr.mail.gq1.yahoo.com [98.137.64.84]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4TbT1g2rgqz4Vsj for ; Thu, 15 Feb 2024 21:37:43 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=cQWDHP+h; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.64.84 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1708033062; bh=FKVtFU/rh+NlM8DtM7UFJRWxHeXnKLv/enJyV6IjqD4=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=cQWDHP+hGlG3Kx1G2738f1MlfWs+SkZAwOQxSkCJDgw4nOg7os4AcVZGgaJUHpfakcWUuIqYK2AhLkkjyLGRVB07DtYLCDJQbF+rcRvrBiEifC6Hc17YBQ2ktKaJ3sivlyABcDpYmf9SLXym9XyBYJxvhZZHxb9PY69buIlr3gVgirFnBS4UxlL7L/awgnCclGxIgIRBxe0oXMJmBmKCre82PlbyBKDOcHmRb2l3vofbbLP6xsXo5ikKgnkBn+XrP+FT84UCPXWCq2NtXAhxc+r9z7WZW02gVIfoOCzUbeqR7Vbv3GcmcQ3HtHKuMMPT5ZxWMoaPSyM+jsSDZmLJvw== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1708033062; bh=6RMyh8CxlZasyjg6VYiWwaed2F6WfB83a64s8S19Xnn=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=bQ2dQ6qAdh3fFd7wpAuDEjh1FvJZBRc0qinJuO83lz2bPCzjTwUCgEmp1De0NeTktSulVp942EPiLqZbUwuLumQy7xL251/AuScZ5poSI1PTh430uiluO/HRw7ByUEr2M0xNlrnfyQhQ/Mzx807ZG2mGKg/y3Tt627Gwc9spQPzJ1tjRco08xGmGELoa+ou6BLH+amQiBECyPjahRdVOWSjCGe3oxg8xi8/nKDnyrnn6wOKs9v7/OwJwMuIIuv9bSlA+9WpjDxuNeg8WF6YB7B4A75WvjL/KaJTMkVaLDtAePs5eycLMLjekliZsb+yu47Fdxn6aktjbIrNpCjfTPw== X-YMail-OSG: 2jkczWEVM1kFVcSbaHxShZLqiGFeZwHcMmUAwpOiiyL2obWmfCLayAQSuBvgtw8 wz6lq1N469I.m5yemHZRip5_9bNgZNfN6sOS5qKaMBa8COZ3sY5WAkbM14HRgrekcxYsW.TXeM_c MNwk6TA_cpqlayJmttFuZLTf7YPOLw4sCFrMGsYp8Gmcglrf8O.t4ZNar1WKg0BS6gkPelMm1nCL 8r9GqEz4PByqVmaQ22tmxgpvnj2zQsexu75UGKRN.thJ1EqU917Q8a.ED.yy6_KnQjIUMOiv9S59 YcfOXFCFZGMbT5jh.9g_Bht5js_tSzJ48OPeU6FYevmG4.lUO_.J1vH0eilhRHQa7c.NwMNsoC.v EAvYWbHLyDcdQuw.FkKgeKf_5OWJ38QNN_TsHnisH83tR9mVPLNjBuOzXM51rjKpfpbqGYcb1BOF 3Lon.jjqZuGS7miHlTPcCYQb2Vy82TnRcSdB6.T_K50u9OaRO9EAmbeJuvnLBzuwO5XFC4kaCQSV 9zKYo2N4zIHfOfGflp68q36FbeWTm3VpT.PUpWpx9smGhXpzyrgm3nQon62QvJTFzHHCPmdsdnLH HClnYrx9pf3EX4ZHanWtgRSh4s1IRdiAPtgM9BT95cCj6hW3wdKGYoXDVvgIJGAcnEl2C2V9uZ_r HTp3Mgn_aCBwLY8XqBh.U0Xnc3DK10ciYINiBwyLaFIcwhVykdhWTMWugib15STL6u8yQhICuReh hRofPMYRyCwgIcCAJTHwfREJU1L_NWXjw1ocLpwBlYaQPpghyN8YyMLMJFDnPydGeNL.vnVgJBHj evdcdxZyh.TjjOxf3xUJ9bEtG78kbmEgeuMY2lioRHn50I7FFefGqvLhAh2aCzOASk2hIiR32Tff 9sTBDZh0pYcix_UPN4lltA3xlM6LJTaMMJ8DmGGJZKGOyQMp66ISgrZ7MI1oHS6mhP48um1FDKsC JETn6TtRUj4iDuI9XLAo1mbK1zGpvsjbX3Ef2Q.ajiZ2pSdBY4jYvugu8Pnt2TCSYzXVPGc13ZA0 pKTCAGC2RcsE7KSawFvY1BXXvcIENPjbd9qaGPU4E2QAnL7hJmxckGQqBqVH4o37l.kdGGZdoHqY t4ECyw03gqueFw8TB8FFWgVGStsS_x1GhWmTBcQJwTynbKZQEZx1hNiTld_lI_GG.jmi.cAdtBik f2L5NFNVDQf4ikyKPEn0s2qjsjzGHnKN6M96cNeFxXNO8rlc9Gnv9Q6BvRygOFxfOSNaIcWIOPfy lVJN3VYFgQhv.6xHjKJ2tcKcbOZm0uwEc0g976S0.ejmprCiuhiXo9cRVwGC1IjSQC7_uyMhG1hI yazzT6Kw_XpIM0HxcI0g13MHm8.IdTFeO4nkbJfBAWCYeyqGruCgynRa3YVVhnkUIXclfc_KBlGY ZUJtLKawCnjLGLeF5zzty6qHGaYHJw1Grxhw88Y5RKRSungSNFfBZ1VVc4DbAjecpbnSjMu8UzED oEO6aUWkQRdVKiJYq3YCJ859aujXrgp9rGpNp1xYetz_DIGGmBwah3sgKc.ouv5ys.JTKdqGqn2k _rhHTLRecL6gxd_MObMhWQb859Hurnb_mYd2tJiIKBXP8VtYLjMwrQoqqLQvvIogax.tUckVEh0E K13I9Xqrm3tRCLhCoyvJDdmH.0Gm3wl.eGz_2MCWOaHHrr_HIEVXDPmkwC3RqdwPuzyYLX7L7TKV l7wUaOE3LDeOltGtBKrZQ4RVM0MOhVvSuOtCJGDBRGGNsHoxthLYhnZ5_6IW9g4ZPi9D0hCagkB9 867bsWa_vatgKoAe3WI9IzCeCCP1Hl6lzfWzfEy0AoVDLZAXquuzqdSInXauSXJQ0KbsZdVkuDER JyOMbummWTCP4GriMT_Kna33sOY1248z6jhYAhXxsg3osiCVKf5SQCtY0n.KAGwigetYnsW4pR.T CEU5zjvb4Yw14jNUPhC5UoUIKNcFkSIfrkW17mq8UQObGg.afNL.kbk7tSWqmYPdixcv.3tgE.qC UZwVCWWBFXlffoDfroSWmJ_7WkI_5lW93Yeez.uiCleRUXtSq5vOLqF9avHtoR5SxFkHDG7yapR0 1MJdWttKOlqy0nZCakdRMXHzWzrtR7b4Qz9Bb9txJ1fValEFFG3ESZXEcICYO1D5ofx35Wnushud TqO1f2_7_D_GXTVUphJ_sd5QU8UqzK5Wq4OP4n0NQD2sIp8ZseFWlvRwCzvJ760NW6cQv0ZTToff Hmwf872GIM.pC0wYjf8XxM0dbkpKOYHtFj.TrAbpggoPJhkvogGKvLPy4ePTWHUqvn0wMsBAGc9T 3.w-- X-Sonic-MF: X-Sonic-ID: cc1cecf4-c09e-476f-84ac-f1b0a1ae0f7d Received: from sonic.gate.mail.ne1.yahoo.com by sonic305.consmr.mail.gq1.yahoo.com with HTTP; Thu, 15 Feb 2024 21:37:42 +0000 Received: by hermes--production-gq1-5c57879fdf-kht2b (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID feb34b8322bd7143fc17524dcea6c662; Thu, 15 Feb 2024 21:37:37 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.400.31\)) Subject: Rock64 USB3 booting actually works when using a microsd card for just U-Boot 2024.01, given no e.MMC present From: Mark Millard In-Reply-To: Date: Thu, 15 Feb 2024 13:37:26 -0800 Cc: Emmanuel Vadot Content-Transfer-Encoding: quoted-printable Message-Id: References: <170680759205.8.12637019708519018262.259259510@qoruscant.com> <8346B40C-AE52-4A3D-9C4C-5D3971C7091F@yahoo.com> <170681711563.7.10357820412414458874.259369299@qoruscant.com> <28E5ED9B-957D-4C56-A682-6273E96581CE@yahoo.com> <170776238774.7.5044812714107165058.265966159@qoruscant.com> <342AF419-EA46-4256-971D-EC0403E9D3F8@yahoo.com> <3F5501BC-29E6-4D72-913F-85181B47EA19@yahoo.com> <97D27B5A-9817-4825-BF7D-E2750FCB4AC5@yahoo.com> To: freebsd-arm X-Mailer: Apple Mail (2.3774.400.31) X-Rspamd-Queue-Id: 4TbT1g2rgqz4Vsj X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.68 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.68)[-0.680]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; TO_DN_ALL(0.00)[]; RCVD_TLS_LAST(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; MIME_TRACE(0.00)[0:+]; ARC_NA(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; FREEMAIL_FROM(0.00)[yahoo.com]; DKIM_TRACE(0.00)[yahoo.com:+]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; MLMMJ_DEST(0.00)[freebsd-arm@freebsd.org]; APPLE_MAILER_COMMON(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.64.84:from]; RCVD_IN_DNSWL_NONE(0.00)[98.137.64.84:from] I discovered that using a microsd card for what holds the U-Boot and not having an e.MMC at all allowed me to boot a stock aarch64 pkg base based USB3 media that I have around when plugged into the USB3 port: no custom kernels or installation of materials in odd places. U-Boot 2024.01 is what is on the microsd card.=20 What had stopped my earlier attempts was the e.MMC speed handling issue (lack of tuning/retuning support in FreeBSD's kernel) when the e.MMC media was present. I did not have to move the microsd/E.MMC jumper from where it has been for years. This, of course, means no use of the e.MMC media that I have around, but, with USB3 and such working, I'm happy with that. No more odd procedures for booting USB3 media after all these years. What had been missing was just U-Boot being able to deal with the USB3 media for its stage (and the FreeBSD loader stage that depends on U-Boot's services). Sorry for the earlier noise where I'd guessed that FreeBSD had not tracked some Linux .dtb update. It turned out to be just the old e.MMMC speed handling issue. For reference: # uname -apKU FreeBSD aarch64-main-pkgs 15.0-CURRENT FreeBSD 15.0-CURRENT = main-n268300-d79b6b8ec267 GENERIC-NODEBUG arm64 aarch64 1500014 1500014 # sysctl hw.fdt hw.fdt.serial-number: 0000000000000000 hw.fdt.freebsd-version: 6.4 hw.fdt.compatible: pine64,rock64 rockchip,rk3328 hw.fdt.model: Pine64 Rock64 # df -m Filesystem 1M-blocks Used Avail Capacity Mounted on /dev/ufs/rootfs 213682 21043 175543 11% / devfs 0 0 0 0% /dev /dev/msdosfs/EFI 49 25 24 52% /boot/efi (Note: This media also boots RPi4B's and such, which is why it has a /dev/msdosfs/EFI present --but unused on the Rock64.) # gpart show -pl =3D> 40 62333872 mmcsd0 GPT (30G) 40 32728 - free - (16M) 32768 524288 mmcsd0p1 Rock64Empty (256M) 557056 61776856 - free - (29G) =3D> 40 468862048 da0 GPT (224G) 40 32728 - free - (16M) 32768 102400 da0p1 efi (50M) 135168 451809280 da0p2 rootfs (215G) 451944448 16916480 da0p3 (null) (8.1G) 468860928 1160 - free - (580K) Note: da0p3 is of type freebsd-swap . =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Thu Feb 15 23:29:36 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TbWYw5fpmz50wZR; Thu, 15 Feb 2024 23:32:20 +0000 (UTC) (envelope-from warlock@phouka1.phouka.net) Received: from phouka1.phouka.net (phouka1.phouka.net [107.170.196.116]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "phouka.net", Issuer "Go Daddy Secure Certificate Authority - G2" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TbWYw2CBjz4hNd; Thu, 15 Feb 2024 23:32:20 +0000 (UTC) (envelope-from warlock@phouka1.phouka.net) Authentication-Results: mx1.freebsd.org; none Received: from phouka1.phouka.net (localhost [127.0.0.1]) by phouka1.phouka.net (8.17.1/8.17.1) with ESMTPS id 41FNTc7P073658 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Thu, 15 Feb 2024 15:29:39 -0800 (PST) (envelope-from warlock@phouka1.phouka.net) Received: (from warlock@localhost) by phouka1.phouka.net (8.17.1/8.17.1/Submit) id 41FNTbNc073655; Thu, 15 Feb 2024 15:29:37 -0800 (PST) (envelope-from warlock) Date: Thu, 15 Feb 2024 15:29:36 -0800 From: John Kennedy To: Mark Millard Cc: John Baldwin , FreeBSD ARM List , Current FreeBSD , Warner Losh Subject: Re: Recent commits reject RPi4B booting: pcib0 vs. pcib1 "rman_manage_region: request" leads to panic [now fixed] Message-ID: References: <9AFDF067-96E4-4E67-90D2-F40DAF3F138F@yahoo.com> <4C279710-5F88-4295-B1A4-7C395F3587E5@yahoo.com> <3A145420-399D-4EBD-9FF4-18924908AB1D@yahoo.com> <1298DF9C-0F82-4567-8E81-7332A608C7FC@yahoo.com> <36ECB040-7E09-47A9-AF71-DE546A78E9CA@yahoo.com> <0b0ba1b0-dee5-47e2-bb93-b44a24492abd@FreeBSD.org> <072C1A6A-85AA-4B26-9598-E5A586F9A4C2@yahoo.com> <23365E1F-FB27-49C7-A691-5DE3FDE40940@yahoo.com> List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <23365E1F-FB27-49C7-A691-5DE3FDE40940@yahoo.com> X-Spamd-Bar: ---- X-Rspamd-Queue-Id: 4TbWYw2CBjz4hNd X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:14061, ipnet:107.170.192.0/18, country:US] On Wed, Feb 14, 2024 at 06:19:04PM -0800, Mark Millard wrote: > Your changes have the RPi4B that previously got the > panic to boot all the way instead. Details: > > I have updated my pkg base environment to have the > downloaded kernels (and kernel source) with your > changes and have booted with each of: > > /boot/kernel/kernel > /boot/kernel.GENERIC-NODEBUG/kernel > > For reference: > > # uname -apKU > FreeBSD aarch64-main-pkgs 15.0-CURRENT FreeBSD 15.0-CURRENT main-n268300-d79b6b8ec267 GENERIC-NODEBUG arm64 aarch64 1500014 1500012 > > Thanks for the fix. > > Now I'll update the rest of pkg base materials. The recent changes resolved my boot issues as well. FreeBSD 15.0-CURRENT #245 main-n268300-d79b6b8ec26 (GENERIC-NODEBUG arm64 1500014) From nobody Fri Feb 16 03:19:00 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Tbcbd04mHz542Vf; Fri, 16 Feb 2024 03:19:09 +0000 (UTC) (envelope-from ordinarybit@proton.me) Received: from mail-40137.protonmail.ch (mail-40137.protonmail.ch [185.70.40.137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "protonmail.com", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Tbcbc0Xcwz49ry; Fri, 16 Feb 2024 03:19:08 +0000 (UTC) (envelope-from ordinarybit@proton.me) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=proton.me header.s=esn2m4x7mzfgde7y45ye74elhi.protonmail header.b=bWAUhbIl; spf=pass (mx1.freebsd.org: domain of ordinarybit@proton.me designates 185.70.40.137 as permitted sender) smtp.mailfrom=ordinarybit@proton.me; dmarc=pass (policy=quarantine) header.from=proton.me DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=proton.me; s=esn2m4x7mzfgde7y45ye74elhi.protonmail; t=1708053545; x=1708312745; bh=5p1Ni9S0KpRt9nKZzUCfM1X5NR9ctzZ/sNnguBQz7AU=; h=Date:To:From:Subject:Message-ID:Feedback-ID:From:To:Cc:Date: Subject:Reply-To:Feedback-ID:Message-ID:BIMI-Selector; b=bWAUhbIlmh8eCS6d8PoE+3MvV/5UdwBQZHY5vjMOt0WoVZFrSPOVD3gyxQV7UkQa5 ttH3DdAVLpOEHFXrNJMqakQGs8Sbf1o03Gi1ubFN1e/+75qHaQINqrC8fO/cCIBHR5 eWgtWPwfn8CheYqB3N5wT7so/n5D+dFdhIe0zP0M62z32TRNLeLeNUrfQadKC36ZdW +DlXvSbMoibjda1PHt9OGShHMnMUNiDqnFTG6jeZFYjUmrUCY4bg/coesL8QG/6CqF hjo5RAj+kO3XSNIOZnw1bfpPT5ZQPPfZgL8JVnuHoXe50ldrw99qLilKrzLTZis6gx eti6Wba6uIuWg== Date: Fri, 16 Feb 2024 03:19:00 +0000 To: "freebsd-questions@FreeBSD.org" , "freebsd-arm@freebsd.org" From: Ordinary Bit Subject: newfs TRIM flag device support Message-ID: Feedback-ID: 100108111:user:proton List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="b1_L30lbpn4GAbLMrxSzcoacoifqoDxiyyyuFUuwDwLWVM" X-Rspamd-Queue-Id: 4Tbcbc0Xcwz49ry X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.29 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; URI_COUNT_ODD(1.00)[1]; MIME_BASE64_TEXT_BOGUS(1.00)[]; NEURAL_HAM_SHORT(-0.99)[-0.989]; DMARC_POLICY_ALLOW(-0.50)[proton.me,quarantine]; RWL_MAILSPIKE_EXCELLENT(-0.40)[185.70.40.137:from]; R_DKIM_ALLOW(-0.20)[proton.me:s=esn2m4x7mzfgde7y45ye74elhi.protonmail]; R_SPF_ALLOW(-0.20)[+ip4:185.70.40.0/24:c]; MIME_BASE64_TEXT(0.10)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; MISSING_XM_UA(0.00)[]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:62371, ipnet:185.70.40.0/24, country:CH]; MID_RHS_MATCH_FROM(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCVD_COUNT_ZERO(0.00)[0]; TO_DN_EQ_ADDR_ALL(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; HAS_PHPMAILER_SIG(0.00)[]; MLMMJ_DEST(0.00)[freebsd-arm@freebsd.org,freebsd-questions@FreeBSD.org]; DKIM_TRACE(0.00)[proton.me:+] This is a multi-part message in MIME format. --b1_L30lbpn4GAbLMrxSzcoacoifqoDxiyyyuFUuwDwLWVM Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: base64 SGksCgpJJ20gcmVhZGluZyB0aGUgbmV3ZnMgbWFudWFsIGh0dHBzOi8vbWFuLmZyZWVic2Qub3Jn L2NnaS9tYW4uY2dpP25ld2ZzKDgpIHRvIGJlIGFibGUgdG8ga25vdyBhYm91dCB0aGUgVFJJTSBm bGFnLiBJbiB0aGUgbWFudWFsIHVuZGVyIC10IHBhcmFtZXRlciwgaXQgbWVudGlvbmVkIGFib3V0 ICJ1bmRlcmx5aW5nIGRldmljZSBzdXBwb3J0Iiwgd2hhdCBleGFjdGx5IGlzIHRoaXMgZGV2aWNl PyBJcyBpdCB0aGUgaG9zdCAoZm9yIGV4YW1wbGUsIFJhc3BiZXJyeSBQaSBTRC9lTU1DIGhvc3Qg cmVhZGVyKSBvciB0aGUgU0QvZU1NQyBjYXJkIChjb250cm9sbGVyKSBvciBib3RoPwoKLXQKClR1 cm4gIG9uCXRoZSBUUklNIGVuYWJsZQlmbGFnLiAgSWYgZW5hYmxlZCwgYW5kIGlmIHRoZSB1bmRl cmx5LQoJaW5nIGRldmljZSBzdXBwb3J0cyB0aGUgQklPX0RFTEVURSAgY29tbWFuZCwgIHRoZSAg ZmlsZQlzeXN0ZW0KCXdpbGwgIHNlbmQgIGEgIGRlbGV0ZSByZXF1ZXN0IHRvCXRoZSB1bmRlcmx5 aW5nIGRldmljZSBmb3IgZWFjaAoJZnJlZWQgYmxvY2suICBUaGUgdHJpbSBlbmFibGUgZmxhZyBp cyB0eXBpY2FsbHkgc2V0IGZvcglmbGFzaC0KCW1lbW9yeSBkZXZpY2VzIHRvIHJlZHVjZQl3cml0 ZSBhbXBsaWZpY2F0aW9uIHdoaWNoIHJlZHVjZXMgd2VhcgoJb24gd3JpdGUtbGltaXRlZAlmbGFz aC1tZW1vcnkgYW5kIG9mdGVuIGltcHJvdmVzCWxvbmctdGVybSBwZXItCglmb3JtYW5jZS4gICBU aGlubHkgcHJvdmlzaW9uZWQgc3RvcmFnZSBhbHNvIGJlbmVmaXRzIGJ5IHJldHVybi0KCWluZyB1 bnVzZWQgYmxvY2tzIHRvIHRoZQlnbG9iYWwgcG9vbC4KCkJSLAoKb3JiaXQ= --b1_L30lbpn4GAbLMrxSzcoacoifqoDxiyyyuFUuwDwLWVM Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: base64 PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6IEFyaWFsLCBzYW5zLXNlcmlmOyBmb250LXNpemU6IDE0 cHg7Ij48YnI+PC9kaXY+PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6IEFyaWFsLCBzYW5zLXNlcmlm OyBmb250LXNpemU6IDE0cHg7IGNvbG9yOiByZ2IoMCwgMCwgMCk7IGJhY2tncm91bmQtY29sb3I6 IHJnYigyNTUsIDI1NSwgMjU1KTsiPkhpLDwvZGl2PjxkaXYgc3R5bGU9ImZvbnQtZmFtaWx5OiBB cmlhbCwgc2Fucy1zZXJpZjsgZm9udC1zaXplOiAxNHB4OyBjb2xvcjogcmdiKDAsIDAsIDApOyBi YWNrZ3JvdW5kLWNvbG9yOiByZ2IoMjU1LCAyNTUsIDI1NSk7Ij48YnI+PC9kaXY+PGRpdiBzdHls ZT0iZm9udC1mYW1pbHk6IEFyaWFsLCBzYW5zLXNlcmlmOyBmb250LXNpemU6IDE0cHg7IGNvbG9y OiByZ2IoMCwgMCwgMCk7IGJhY2tncm91bmQtY29sb3I6IHJnYigyNTUsIDI1NSwgMjU1KTsiPkkn bSByZWFkaW5nIHRoZSBuZXdmcyBtYW51YWwmbmJzcDs8c3Bhbj48c3Bhbj48YSB0YXJnZXQ9Il9i bGFuayIgcmVsPSJub3JlZmVycmVyIG5vZm9sbG93IG5vb3BlbmVyIiBocmVmPSJodHRwczovL21h bi5mcmVlYnNkLm9yZy9jZ2kvbWFuLmNnaT9uZXdmcyg4KSI+aHR0cHM6Ly9tYW4uZnJlZWJzZC5v cmcvY2dpL21hbi5jZ2k/bmV3ZnMoOCk8L2E+PC9zcGFuPiB0byBiZSBhYmxlIHRvIGtub3cgYWJv dXQgdGhlIFRSSU0gZmxhZy4gSW4gdGhlIG1hbnVhbCB1bmRlciAtdCBwYXJhbWV0ZXIsIGl0IG1l bnRpb25lZCBhYm91dCAidW5kZXJseWluZyBkZXZpY2Ugc3VwcG9ydCIsIHdoYXQgZXhhY3RseSBp cyB0aGlzIGRldmljZT8gSXMgaXQgdGhlIGhvc3QgKGZvciBleGFtcGxlLCBSYXNwYmVycnkgUGkg U0QvZU1NQyBob3N0IHJlYWRlcikgb3IgdGhlIFNEL2VNTUMgY2FyZCAoY29udHJvbGxlcikgb3Ig Ym90aD88L3NwYW4+PC9kaXY+PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6IEFyaWFsLCBzYW5zLXNl cmlmOyBmb250LXNpemU6IDE0cHg7IGNvbG9yOiByZ2IoMCwgMCwgMCk7IGJhY2tncm91bmQtY29s b3I6IHJnYigyNTUsIDI1NSwgMjU1KTsiPjxzcGFuPjxwcmU+Jm5ic3A7ICZuYnNwOyAmbmJzcDsg Jm5ic3A7PGI+LXQ8L2I+Jm5ic3A7ICZuYnNwOyAmbmJzcDsgVHVybiAmbmJzcDtvbgl0aGUgVFJJ TSBlbmFibGUJZmxhZy4gJm5ic3A7SWYgZW5hYmxlZCwgYW5kIGlmIHRoZSB1bmRlcmx5LQ0KCSAm bmJzcDsgJm5ic3A7ICZuYnNwOyBpbmcgZGV2aWNlIHN1cHBvcnRzIHRoZSBCSU9fREVMRVRFICZu YnNwO2NvbW1hbmQsICZuYnNwO3RoZSAmbmJzcDtmaWxlCXN5c3RlbQ0KCSAmbmJzcDsgJm5ic3A7 ICZuYnNwOyB3aWxsICZuYnNwO3NlbmQgJm5ic3A7YSAmbmJzcDtkZWxldGUgcmVxdWVzdCB0bwl0 aGUgdW5kZXJseWluZyBkZXZpY2UgZm9yIGVhY2gNCgkgJm5ic3A7ICZuYnNwOyAmbmJzcDsgZnJl ZWQgYmxvY2suICZuYnNwO1RoZSB0cmltIGVuYWJsZSBmbGFnIGlzIHR5cGljYWxseSBzZXQgZm9y CWZsYXNoLQ0KCSAmbmJzcDsgJm5ic3A7ICZuYnNwOyBtZW1vcnkgZGV2aWNlcyB0byByZWR1Y2UJ d3JpdGUgYW1wbGlmaWNhdGlvbiB3aGljaCByZWR1Y2VzIHdlYXINCgkgJm5ic3A7ICZuYnNwOyAm bmJzcDsgb24gd3JpdGUtbGltaXRlZAlmbGFzaC1tZW1vcnkgYW5kIG9mdGVuIGltcHJvdmVzCWxv bmctdGVybSBwZXItDQoJICZuYnNwOyAmbmJzcDsgJm5ic3A7IGZvcm1hbmNlLiAmbmJzcDsgVGhp bmx5IHByb3Zpc2lvbmVkIHN0b3JhZ2UgYWxzbyBiZW5lZml0cyBieSByZXR1cm4tDQoJICZuYnNw OyAmbmJzcDsgJm5ic3A7IGluZyB1bnVzZWQgYmxvY2tzIHRvIHRoZQlnbG9iYWwgcG9vbC48YnI+ PGJyPjxzcGFuIHN0eWxlPSJmb250LWZhbWlseTogQXJpYWwsIHNhbnMtc2VyaWY7IGJhY2tncm91 bmQtY29sb3I6IHJnYigyNTUsIDI1NSwgMjU1KTsiPjxzcGFuPkJSLDxicj5vcmJpdDwvc3Bhbj48 L3NwYW4+PGJyPjxicj48YnI+PGJyPjwvcHJlPjxicj48L3NwYW4+IDwvZGl2Pg== --b1_L30lbpn4GAbLMrxSzcoacoifqoDxiyyyuFUuwDwLWVM-- From nobody Fri Feb 16 03:32:40 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TbcvF4HFBz544GW for ; Fri, 16 Feb 2024 03:32:41 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TbcvD6rppz4DkQ for ; Fri, 16 Feb 2024 03:32:40 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1708054361; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=SnhFTm8wfFkHI/IK+8IPjsMY6AF10+bcMgJaAHMZQKI=; b=hubh6nDJVVhNbnaOn80oPtS1Px5JmWGzH3LLXybEZpJ9D00UxHllpoSQJ7ozhxiTpbUaXL oG41+pNctEx3lvKbh4Tv5XzKzrz1bqxz8BQNgpim6lKJlXC7jOcI8lj+eo8DCiMi7sVe3i JlJLQxJVcI0UfIMyq61QjoHzCEXOJi8RTeL5X6ctkYuy+R+G5oXAUto+o9pDE0EujgHpPa fvfpr5loXAYnlBdDCtoCWXtvgJT5XEvu1M0IpKd0E7imYMRqFCwb63LR7leH4Xqlvo5sia I/BU5ZdIIO8OH9D3XrYSPfQ6OPOp7E9KRWyDhbTyxz/SL6J6/hgaHYlKcEZBZQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1708054361; a=rsa-sha256; cv=none; b=kSk2g4FxYNNwVeJSW6OoxtUQoZNXso5Dp9Pi9Cuw/dToj54MP4gCsGeRrBy9BPsuh/8TcV TxtvWcPm6+Oo+LanzXfUoDedvk/+KtfkeU2TidXFXAx5cS4O89tFqhexYhHalN1FBmGdxc Wv9kmxAju+FFeQ/OgNVCQkO1jGF0KkAL9dsEtzeQsBsv1VyA3YSPryLmUz7iXxPV7fc/Kk Km5tXONxu7KXt9k8RMMc7mP8qpEefIJzDr+zDKqRARyS6kDxd0E88gt0t6+rhDCraKLzim OZcfMa9p1oEYzBKNZkngewHzpwxqKpGDBE6HjK4J0EUYZfgd8GifGU+yVw+Jiw== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4TbcvD5tj1z185f for ; Fri, 16 Feb 2024 03:32:40 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 41G3WeIw011881 for ; Fri, 16 Feb 2024 03:32:40 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 41G3Webg011880 for freebsd-arm@FreeBSD.org; Fri, 16 Feb 2024 03:32:40 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: freebsd-arm@FreeBSD.org Subject: [Bug 277083] Boot countdown takes too long Date: Fri, 16 Feb 2024 03:32:40 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: arm X-Bugzilla-Version: 14.0-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: bc979@lafn.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-arm@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform op_sys bug_status bug_severity priority component assigned_to reporter Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D277083 Bug ID: 277083 Summary: Boot countdown takes too long Product: Base System Version: 14.0-RELEASE Hardware: arm64 OS: Any Status: New Severity: Affects Many People Priority: --- Component: arm Assignee: freebsd-arm@FreeBSD.org Reporter: bc979@lafn.org During the boot sequence for a Raspberry PI 4B, there is the sequence where= it starts a 10 second countdown before booting the OS. However, the time betw= een each second of the countdown is actually 6 seconds so that instead of a 10 second delay (like is seen on AMD64), it actually takes 60 seconds. This m= ay be related to 242732 although that is a much older PR for an earlier Raspbe= rry Pi. --=20 You are receiving this mail because: You are the assignee for the bug.= From nobody Fri Feb 16 03:41:25 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Tbd5c6K1Yz545Dk for ; Fri, 16 Feb 2024 03:41:40 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic313-21.consmr.mail.gq1.yahoo.com (sonic313-21.consmr.mail.gq1.yahoo.com [98.137.65.84]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Tbd5c3fH7z4FTl for ; Fri, 16 Feb 2024 03:41:40 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1708054897; bh=58onuahrg+2+7yp3AYifQPqFUieViKvWvJyRlbZo+X0=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=RaoapLSuX61w0ppUUPUOrkqECWG7UcpJBiVLzpwYOz3094Nh7usRZZ6iY8uTvZwjQAeTufXcn/1hqs/cfijiqbwMYs3APp5ynoveTCH2XQfGpVY0U/oSfR8JjCF44MYyjY4Jm6VlEDBa/JhuW7KS6m69cUxpzrli28NE3YukZGZRYTnaxyPVVBgpc1x1Xv7nk6jmFmn1BK5aY/vQjzfbmQETEn7mgcup9vGF4xUxgf/B3XL8sdOVFWm7JK3ritd7jVleFK6iHIPAZ+BIGiE9slUDmAhIbXN8jKL2G66k/ZmYN5kk+ErUZlWWuaHIhXvuwTBoeXlHq9SfNGs8BTGr4g== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1708054897; bh=nl9vQuy11qSc5dYwiYjsSb8KUREbUxjgCBl/7CHJ4Af=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=K0cTYga/4sU5n9UDIk73gPfkdZdL7JSJ64b5UbDapjJCAf+u8J9myCLZSpmSQC50dE/+9lhBqkHiqssES21HlrTLxxk776OioCx/2s0CK9Onm5jSUqjQN7yzGJ2H65ZOy7lb5TGF/l9TVauKgRJ8IQuVEspa0ujGVEvdgQQc9UVEFZIKF7PM2NNYzZwQ3D9V5KNS/DHTHq8BTEKYZVHjaphI94G2Pp16jQnA+h9FupjkKsYRpGNCWfS4j7G5+DXHAzZj0F3lRtV/imT4bhmHKp19p92r9vmWNUz9BjZ2+1QSZwyrUB9ajDxx9Ak6WtH9fefsgYg9WW1A4gTVBLqj9w== X-YMail-OSG: trktUjEVM1muHMtoP4.A3Fx3N5PVYXxR9OuDFO0P05sjxDBXHwRI4lH1KScBACW ATHrQ0u4RnV3uUJKN8bl2d1ZJFMpay2sJEK2G_wc0eoasWIMPOIrjR66ybuqTCHcPoNuj9Nmemz. VcdHQ49AALj59cDmRUivzOEuODsENhgrvI7pCBf0H7U4cEIxW2XSTCFa2fby2sB4Qp8hJsG.Sgwz S5JMY8190jLKxjaQ7jWFpKlssSRcYot_7L391DP79P0hzvnNIriiPtw4.hb_l4Cn6XHPUMuAAy4P _Z4n.9910IvsQNhlWgmTsFL03.K4xW.9CT8KHgWMAaMdK5emwfcPPr_Zk04ZTbFYa4bllgN5VaB. C64myMtTtBsvZT3YNUjPY6HZKPJEM1mD1vsA1BDAN_e207e3QgC92bJZE17hp5.kKRERcRQIDzW8 0oDTpCXMnBnEW9LTe6DGELL0aXYdDDH_Oa8T6byJRmfuUSvqQxKte89rbMq9pPINEXKZWcO0G0de tnvP6MeyGdL4LfJ3AWOEJWCezOA.j0eTWFfabio.LZqGTcpRFUdQRgAx7lPwXsQNFWgmFdVxLSnG yxoT4jR6cMC0RRwd6dbJ.9CoTD.euSQZYNq5i5H8xa_dTSoW2SHJcR.cS9IT4YGZpbbFK6vz5mJa Aur_RKOHCiK0LkMpQIMAxFiUGOANLa5Dpa5cL5u0jckA.UdbhyaL_hfpZ1nHyWpxThdb4VxaTqOU FG.Hfs7mFDoosTo0mw8t4H2kSynR2hfFDmoxIdnqgzIQtHWQW94wW8YnUToQqvxkeEWlFfKi7CPK J1xxAGWyKO.1ll0rOdob0uxRoruSmBkNWu3FnOpK87ycj.5nJMvIXOvev4a4yAoo0XO8fWeAM97g R7oTsdVzzTLuPtHVcTuElhr66bBh8eA1Bc07vxN4uxgLI5m24Sij7B2eIAPgr.GXWEcHN0XiUUCt COKyIKFBTY7ytstqbXOZgdho7g0aSmYa6gAKjI8wXz33v9ommLcDQzHjXJBpNdV2e_wZJz4h.Acd 8FRCQLr0c6wHw0QDQEIrogz7YYlWOnNwP9VSd_PQ7pvFvHykzneozMI7EJn8ZK5SFqonYhrjdt05 1dUUfZtfSuWT5Ps49u2DIIenatc49Bhj4chafCjadQtt2oEhlVhu3JYMh13RhKY3QQI75ZIjvPWl sSMEx6AO.JVcxbBB8LHshtLM_BShiBxPKlIHtb5URjzKY7LxIn6eYmihm2BwQLOuEtUOq7J29G8v gHFfyNOSd0thFz_QLWCW9.eBrapDpIrK9z3R7x5y96i955Ea5GNFLe5NYVQCZG8JATw_aYxufmw8 Ij4HtC_Y0q7MGsVI865kUW34Wgh_7.riyldfZ5mlkMDdZCnsyXxR8vWasjR4MjScJQmqvKPlQMgp oLz6YDK2c6R5b5uuHb9jHmE88lzyErKAq7p9jhG9e7zVNJIqtWeqPWnM1tflcfOLfDusAvkFtXJY Cz5HGMLdfYif0soSAtF4Yzv0kS5HpXWY5YFDITubKKfaxOHoMq9Od.whmVF94hWhIVlxua.tLNNx _GKqvOwFsPiZkSbkoALauQC42fG1nwbWwGwgcWsbzwjsN9GZ4wlwRBE1wMCupiesQrGOcpTHY2EB fU3G9jw0gLN1EIqrBcr79IOAZJqZhkResYPfPJQ0pC1fOzV0pe5QVK4ArV615lo9QbbWjM27a_xH Lz1zj9xZG5sTIQgrJ4BNLAseoN5nw7WXymaO5LIp6L9zAb_VyqAMMOy3EEiAu8Dg5nb6Wg_z5ngq rMYkJ_McgnfMBi98nyJhmcDfIn44TBJ2keyLZqEKL6iXg4lewvSgMG8gdwJC.G0jRnkaIu4Gf4ic 73wP2JfFCjPdQyILL3UoY6Iojwon1wtnxxEDwcbGstqcq4kofMSDygj9cXcVNO6szh1JXFNW5s8_ clRatvs7Qyuhx.bI0SutAEEG.cO0ZK37auHoodE8NCGAy6xRFjBx2afyrDSB5eI4KnVQ3ech3eKC .1uHsR7M_KbHrk1TCenGURMW13YR_xNKSHAmIdsdH1Q2FkXv5VYDA3lzfCukKpb6mhGF_cMP97Tn 5IX_SXWbLqLjVKCMDvhPm4ZExZK9pTIvBt5_zd8LqNkA8LakphL_vkWDwHQv634ZnkJfIRNJ5Fjz AktIYfZBqPAq9BW9L2QfCyeSFeC.a_Hhz9owHQh.IbtZyK6.IgTbW5..3sI_2EV7ujs_bG.EoyoE XkA_I0wl0pES8cZTnPcE.OkBZTNs8R5ohOZZyoF8FKsxxzFvMqKRsj5UMJYZt8lGKaCpkCPO82YW KNAE- X-Sonic-MF: X-Sonic-ID: 35fb7996-d460-4181-918c-91f7c29c80eb Received: from sonic.gate.mail.ne1.yahoo.com by sonic313.consmr.mail.gq1.yahoo.com with HTTP; Fri, 16 Feb 2024 03:41:37 +0000 Received: by hermes--production-gq1-5c57879fdf-vxz7c (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID d36458d22f445acb5badd4ba15b91fcc; Fri, 16 Feb 2024 03:41:36 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.400.31\)) Subject: Re: newfs TRIM flag device support From: Mark Millard In-Reply-To: Date: Thu, 15 Feb 2024 19:41:25 -0800 Cc: "freebsd-arm@freebsd.org" Content-Transfer-Encoding: quoted-printable Message-Id: <6AD380D4-9B42-4511-9E02-94EB0005D278@yahoo.com> References: To: Ordinary Bit X-Mailer: Apple Mail (2.3774.400.31) X-Spamd-Bar: ---- X-Rspamd-Queue-Id: 4Tbd5c3fH7z4FTl X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US] [Only replying to lists I subscribe to.] On Feb 15, 2024, at 19:19, Ordinary Bit wrote: > I'm reading the newfs manual = https://man.freebsd.org/cgi/man.cgi?newfs(8) to be able to know about = the TRIM flag. In the manual under -t parameter, it mentioned about = "underlying device support", what exactly is this device? 2 contrasting examples: Example 0: Optane NVMe media (PCIe card or U.2, for example) Optane has no need of TRIM and, so, never supports TRIM. Example 1: microsd card media usage A microsd card in the normal type of microsd card slot on Small Board Computers (normally) supports TRIM. Take the same card and put it in a USB reader/writer and use it via USB on the same system: no TRIM is supported by FreeBSD over USB. FYI: When the file system has TRIM enabled, FreeBSD put out a notice if TRIM will not actually be used in the actual context in use. > Is it the host (for example, Raspberry Pi SD/eMMC host reader) or the = SD/eMMC card (controller) or both? > -t Turn on the TRIM enable flag. If enabled, and if the = underly- > ing device supports the BIO_DELETE command, the file system > will send a delete request to the underlying device for each > freed block. The trim enable flag is typically set for flash- > memory devices to reduce write amplification which reduces wear > on write-limited flash-memory and often improves long-term per- > formance. Thinly provisioned storage also benefits by return- > ing unused blocks to the global pool. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Fri Feb 16 04:08:09 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TbdhV1rV4z59b4G for ; Fri, 16 Feb 2024 04:08:26 +0000 (UTC) (envelope-from ordinarybit@proton.me) Received: from mail-40135.protonmail.ch (mail-40135.protonmail.ch [185.70.40.135]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "protonmail.com", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TbdhT6VLxz4N2X for ; Fri, 16 Feb 2024 04:08:25 +0000 (UTC) (envelope-from ordinarybit@proton.me) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=proton.me; s=protonmail; t=1708056502; x=1708315702; bh=EmVqAlGPa7T8jVDPOVlPvy99UrHaJUkWVA4fQpxPIrE=; h=Date:To:From:Cc:Subject:Message-ID:In-Reply-To:References: Feedback-ID:From:To:Cc:Date:Subject:Reply-To:Feedback-ID: Message-ID:BIMI-Selector; b=e4qz2catv2Zz+PEKQqIlJxnuAd0j7PXgSd0UWL901bm029iWOwZIY64aIhf2iX+OB 2lKH75S5recEJHL4igs9JEwDK0CWb+UUHnB+Z60HGqSl6qlK/u+yfe9k6n0NY/alPd XGXy5aQS+/AKkDzynHRhuXB4eaYXW02wXCMFBYNAGyJSoB3hcWSi1qQ7miWKN4vTBN 819J4w279+7QUje/Ejw6zRRJaPLdszbA1xJh/1LjMvK9DwEOBhx+UmhpfTn19S+slR /xppig/jtTpT09qInKofQ9xUddJY7LNldeSuDU6tmHwWTkOgn+W30h8XBuyOIrT7fZ kABgPQZ8bVFNg== Date: Fri, 16 Feb 2024 04:08:09 +0000 To: Mark Millard From: Ordinary Bit Cc: "freebsd-arm@freebsd.org" Subject: Re: newfs TRIM flag device support Message-ID: In-Reply-To: <6AD380D4-9B42-4511-9E02-94EB0005D278@yahoo.com> References: <6AD380D4-9B42-4511-9E02-94EB0005D278@yahoo.com> Feedback-ID: 100108111:user:proton List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: ---- X-Rspamd-Queue-Id: 4TbdhT6VLxz4N2X X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:62371, ipnet:185.70.40.0/24, country:CH] Hi! On Friday, 16 February 2024 at 11:41, Mark Millard wrot= e: > [Only replying to lists I subscribe to.] >=20 > On Feb 15, 2024, at 19:19, Ordinary Bit ordinarybit@proton.me wrote: >=20 > > I'm reading the newfs manual https://man.freebsd.org/cgi/man.cgi?newfs(= 8) to be able to know about the TRIM flag. In the manual under -t parameter= , it mentioned about "underlying device support", what exactly is this devi= ce? >=20 >=20 > 2 contrasting examples: >=20 >=20 > Example 0: Optane NVMe media (PCIe card or U.2, for example) >=20 > Optane has no need of TRIM and, so, never supports TRIM. >=20 >=20 > Example 1: microsd card media usage >=20 > A microsd card in the normal type of microsd card slot on > Small Board Computers (normally) supports TRIM. Take the > same card and put it in a USB reader/writer and use it > via USB on the same system: no TRIM is supported by > FreeBSD over USB. > So you mean to say that if I have a Rasperry Pi 3 or 4 now and then have my= FreeBSD installed in a microSD card (for example, SanDisk Extreme card) wi= th UFS/FFS filesytem in it with TRIM enabled parameter then is it going to = recognize it? How to verify? >=20 > FYI: >=20 > When the file system has TRIM enabled, FreeBSD put out a > notice if TRIM will not actually be used in the actual > context in use. > Ok, got it. How to check this as well? BR, orbit From nobody Fri Feb 16 06:02:44 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TbhDh0lQ2z59sjD for ; Fri, 16 Feb 2024 06:03:00 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic303-23.consmr.mail.gq1.yahoo.com (sonic303-23.consmr.mail.gq1.yahoo.com [98.137.64.204]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4TbhDg5MQPz4cG4 for ; Fri, 16 Feb 2024 06:02:59 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1708063376; bh=AND9qyD7+uuVqa7Ya+2umu2Hyof8rbu3wP4FRvbKBNs=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=XXp05RDn6YsbbnLcb5sJm2WJTnj5btNzgQ0cFAdBm/YtykYTEXtc/DTlJrZqm0+GX7MLzwXOBKrETmAnKvVS8xloiJ3CW3hsx4dnzKXG1VC/SbknoSJrNoTuXyGfR1apsm9WC+JLUsXs+pHed5ZlGmX7APziKSwBG3lZlatDmkbMz1CYCG7PaRY4DLTsNq0y8nXNV4o+6AKS/234hV8y77lBeY025aYHKHD4VPc6LcgnshGWDFsbOqGfSa3vecRESXBqIowS5QCWDo9p4b3CtN1uNLlOy2X+hJ/51cN1Rh2C9ZiR5MK3X/MUfKINTXvi3efwO9pO/Frc6zN6OdeBNA== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1708063376; bh=ROez+IfQByXJR9U5j1tO8RDk1yKCLKTq/8ZRhR6XYl+=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=i4oFmySnjA4C81kyitjkiqcrj6hwB+mSo17tf2IbELEHdU/3Z954MHj2jvWAt0/MiWYBx1YsUCdFV+nt7ZsiML4OyrMGpDTOrmm6wJyJaoUQzdPw95IiY7CukqHTSmprSEwGIcLXRs2XcxUdIoCpdY0W4twP8N4z6n9OqR3Q/WGFl7sqYXOGHENxFE9lSFTLG5vNfKT7Bu8LPjc3GL+z2Ms8XhCr5xu4nTqO5pmJ4vB3dmmo49LL1YG3BiriQ7P4eXZcXEm8QHxigH+pFZviFW1/DATeSrpePXCEQIbCPtDTrIgz6jDuI7r3txc/2aWC0c170IQW4BVb2YiNKAJt6Q== X-YMail-OSG: t7577D8VM1naaGZTmd4heG.8QbyksT4frz21NsZCu4Pj9EPztAreT3bWwEPf8OU wud2zynPnfDv503pCNLHzdOrZ.SjwZV5ezdLrpvsgoDeRr3zhIn2O6MI1fH91VCu2EEUrmMR7ODO FiE_MNwcX3TKl9m8wbfhFYuRow.2Ogx9J8WYDniII8e1IEUvRkTLdoZk.dDV8yrfEIB_ahOZSKKs KEarkciVw.ytOPwAMXdkljB68LbR_.9pvtbJJjOFnwZ2XPpZS_XuMrhmBPVIW.vFuuW6TXEwlPPU QfOguhEpHI2eElaGdveqH_IbnaLBNEgCWJrrcOzTETgzGfxSrr9qBTJF1LvofwzUh6KqmIdLMwK5 MrXYGTsRKL_v4WC7FD.vAYLOHzGOsFtHthxvgjFD5odLqHOcizmGNltcIW6iI1qMoIE1cU_fp_Qk nHIlbpHEZs9o.Pan32uFSObslOThM1xWJrXRxi4bE_CnMMNxUl.00qDmSKTKvfimxs8ny0LFGLvK 2J4JdgA4XIE_WD032mx3jOf_zR.QG57vkIZ.Mbn8hQ3qok.82tm60oRPTlydPjgzWJ_Y04ZOkt.p TOa6gs7JYPIq1_qLQSc7uAD_xO2OUT_zYJiJFufC7tva6nNiD0uznNDjO47Y0S2.nwCAx9PMYxEw Xx6EH03cao5b9knwJlnPiUmALs1OVA5fjiH15V.qKWZcWews8TzNB6FtEZF.UFtQYoIBfUy9Uaa2 3PSNFADxAv4B6KN9ruB2iE18DVPz9La9SnFL4OogHsvgSVkYQpj9tAmg36UxnfV4qObE5i57ofyy JDMCz05K76yxD1aFsl5uOP1ZAH2fLMTRtDuibT1vDY_Q3nmHVGiEoUhlOOFrbpnKnod7fF5O4kKl m93Cn0Qlr5gNf_hj2hbQcYby5i.2D4axf_jpTWTAvPevA1Pc0VBxyZffJtiOZHbcWJeb8aufTTzZ DB6EvLBBD2XkjZ9gFvU7rFYKNY0IQGoR9uH7PzW51kdtJQ4ZYmFfLG3OxnkQJZggZiAvnaToCpTQ 32.4p9U3IK4QwLmmxMznaDYvGcWA6SLZoEYLFkyq5INZgLg0qcK4.eBt_7ohrYbhZWqfXVfAJ8vi wh8_EXP2fGAlCX5RZjfWhWAEPre.hEETTk4Wph1J8BMs5tbl59HAw1WrOex3WlZsfLl0yl9XvmNc C1NvvNgif9EQbNrqsehzNYk2_d.a3wWk0vNLg.DvHtW_jdtRMcVJAewnL9ByIqWN1en0hG2PzAi. vkC0cb8s3VeD4wvyhA0yAhVUnHp3DAeYb3bjRLMQRUSlJrvztkZWbVYmyv3YDfntkOHXzK5.KV7B bDlsdLIBMNLSiMZzQdnE1Tp_JNlETjPDmTYCgZaoO4dNLQkJtP8WD.pMueI0yCxsvDxdGqoETM8b QSTf8AvkcFSnRDucV6WrzWmpkzirj.e2QJmZyyRbt2Hrd2Mx2VUMpEv6HPCff.rKs4mGWfB5FdkU UQWb2hP71MQGUH1mVxzNGxhtO8mBGWpsphUEQzmouMS_z9qmAxYE3dV..K_6GUbc008l2wmH0D.q Bpop6AX1SjKL3pVHAiN3Kq1Iz9eRV3WfEs2G0UrAxXvr9ds1BA82.Oc2caknDfLWPwmdTwH.i09D 6wPMqpLMeRdg3v_KqYoDMMGtgGAeIA_rRlFb9dvHWXUCeZYc6eNQmTfceaMx3La1yrsWtANwbR3_ Rrbu9fzYn6r4qsJ.n1XpslRQvbKZymXH9j1ROwwshyNk54J5JL5SrOeX1t3DoovWpQeZl0JfHaeM VB0TEGLHegRze2YF6ncfyRN.mPZiwCnYF9k1L6FoJPsPmPtsUNHyisPLl3WA2APsdZxYGMCvqMJw 9wOSzIxC5dzEfNdGhr46JLHgim6IXDushRs28dJsgaczG5oiSKe8GKX.jJxVmpC6mnN.2jpRSwTQ SuCymBxHRwgyruqN7eg3yFoQZ1bHQsUs6QcB0N3zxzSA..9THAXc5tc78Q5EmCJNlbYXUZUBxHmq L9.dEHRGtjlkpw5VVZHes.BRBRLzm.JtR9bWCdGtFj7ejIs0Tc36hDPrhiJWk4Eah2E.XdZVHdos 5gZgjKUrBuaL2_0.7no96D_QYbQSxnITjY.DKLMkq56ssz2YsdgHQBGwDY.XMWL8k7s2TeBknJSW 07GtDw6PMjgbAgT7sDGF9k.mFMgR1WCa2K4pcDm0RNKtohFHPW8cI72gabgHZTFf5b.Zbgp3xLvd bbP0w94mWfG_8QYNZ8_bvB0LCeKSLjU3KGIwrPbvvPR1i0AgfECF1TNafAXtCK33GTONTcJ8M X-Sonic-MF: X-Sonic-ID: 5702ccdd-ce8d-489a-ab2f-0c60a81068c9 Received: from sonic.gate.mail.ne1.yahoo.com by sonic303.consmr.mail.gq1.yahoo.com with HTTP; Fri, 16 Feb 2024 06:02:56 +0000 Received: by hermes--production-gq1-5c57879fdf-7pjsc (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID ccbd2472ecb7f0c2c7704e4058af097c; Fri, 16 Feb 2024 06:02:55 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.400.31\)) Subject: Re: newfs TRIM flag device support From: Mark Millard In-Reply-To: Date: Thu, 15 Feb 2024 22:02:44 -0800 Cc: "freebsd-arm@freebsd.org" Content-Transfer-Encoding: quoted-printable Message-Id: <8D8AA3F5-74A3-4AFE-A560-F52C7C55CF12@yahoo.com> References: <6AD380D4-9B42-4511-9E02-94EB0005D278@yahoo.com> To: Ordinary Bit X-Mailer: Apple Mail (2.3774.400.31) X-Spamd-Bar: ---- X-Rspamd-Queue-Id: 4TbhDg5MQPz4cG4 X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US] On Feb 15, 2024, at 20:08, Ordinary Bit wrote: > Hi! Hello. > On Friday, 16 February 2024 at 11:41, Mark Millard = wrote: >=20 >> [Only replying to lists I subscribe to.] >>=20 >> On Feb 15, 2024, at 19:19, Ordinary Bit ordinarybit@proton.me wrote: >>=20 >>> I'm reading the newfs manual = https://man.freebsd.org/cgi/man.cgi?newfs(8) to be able to know about = the TRIM flag. In the manual under -t parameter, it mentioned about = "underlying device support", what exactly is this device? >>=20 >>=20 >> 2 contrasting examples: >>=20 >>=20 >> Example 0: Optane NVMe media (PCIe card or U.2, for example) >>=20 >> Optane has no need of TRIM and, so, never supports TRIM. >>=20 >>=20 >> Example 1: microsd card media usage >>=20 >> A microsd card in the normal type of microsd card slot on >> Small Board Computers (normally) supports TRIM. Take the >> same card and put it in a USB reader/writer and use it >> via USB on the same system: no TRIM is supported by >> FreeBSD over USB. >>=20 >=20 > So you mean to say that if I have a Rasperry Pi 3 or 4 now and then = have my FreeBSD installed in a microSD card (for example, SanDisk = Extreme card) with UFS/FFS filesytem in it with TRIM enabled parameter = then is it going to recognize it? How to verify? >=20 >>=20 >> FYI: >>=20 >> When the file system has TRIM enabled, FreeBSD put out a >> notice if TRIM will not actually be used in the actual >> context in use. >>=20 >=20 > Ok, got it. How to check this as well? The console gets a message like: WARNING: /mnt: TRIM flag on fs but disk does not support TRIM when a mount is attempted (automatic or manually) for a file system with the trim flag enabled but trim does not end up active. So, for example: # dmesg -a | grep TRIM WARNING: /mnt: TRIM flag on fs but disk does not support TRIM (This was a microsd card in a USB reader/writer that was not used as the boot media: a separate manual mount was used.) The file system's TRIM flag status can be checked via use of "tunefs -p . . .": # tunefs -p /mnt tunefs: POSIX.1e ACLs: (-a) disabled tunefs: NFSv4 ACLs: (-N) disabled tunefs: MAC multilabel: (-l) disabled tunefs: soft updates: (-n) enabled tunefs: soft update journaling: (-j) disabled tunefs: gjournal: (-J) disabled tunefs: trim: (-t) enabled tunefs: maximum blocks per file in a cylinder group: (-e) 4096 tunefs: average file size: (-f) 16384 tunefs: average number of files in a directory: (-s) 64 tunefs: minimum percentage of free space: (-m) 8% tunefs: space to hold for metadata blocks: (-k) 6400 tunefs: optimization preference: (-o) time tunefs: volume label: (-L) =20 If the trim flag is enabled but the mount does not produce the console message, then TRIM is active. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Fri Feb 16 09:18:47 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TbmZp2r4Bz5BLgr for ; Fri, 16 Feb 2024 09:18:58 +0000 (UTC) (envelope-from ordinarybit@proton.me) Received: from mail-4319.protonmail.ch (mail-4319.protonmail.ch [185.70.43.19]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "protonmail.com", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TbmZn6Mmvz41m5 for ; Fri, 16 Feb 2024 09:18:57 +0000 (UTC) (envelope-from ordinarybit@proton.me) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=proton.me; s=protonmail; t=1708075134; x=1708334334; bh=qncEK/y41TSwaklunGySvNohHAQdbqgg44rIy4WKJj0=; h=Date:To:From:Cc:Subject:Message-ID:In-Reply-To:References: Feedback-ID:From:To:Cc:Date:Subject:Reply-To:Feedback-ID: Message-ID:BIMI-Selector; b=XUPsivIETLCxC94rBzxJz2aPQ+TgDGoShceosHdxIWLq4ZZ0u4WyXxI+nWfCXHT6t jDk+/vXM524GQct18JI7u53vFhbdScLTz8Jh2kq1afNfwPbSwNcCKvkxO0/XJRcT1z 1NFQNwcGVtFj20olxaJr1SjjwQdMK0VjveWL9olSFAAQ1J4DEuPKaDy5MSr9H6VGeZ naJfY0iRlvcS4ZlQMdRIbbpOaGyg2w7Eh+Uv5x91BpKSZOm+oILR6v3aiWDcijITJI PtR/twRGO++bZOpTrCrysbCeIeomjxhmKYy8pdisLmYPVUx9SyHlyPNB+mUSHHlsgg Pg/VAuJ/bOv3w== Date: Fri, 16 Feb 2024 09:18:47 +0000 To: Mark Millard From: Ordinary Bit Cc: "freebsd-arm@freebsd.org" Subject: Re: newfs TRIM flag device support Message-ID: <7okx4eDPse2uF1n3aX4W8RIOdbHXziUhqgvHxqqxpYN-7NsJfEGrfHbDxLgNS3PdOwZt7j9aOXp-0MHDjQAryL4t1Gd2QHUG1sFNmd5dY0k=@proton.me> In-Reply-To: <8D8AA3F5-74A3-4AFE-A560-F52C7C55CF12@yahoo.com> References: <6AD380D4-9B42-4511-9E02-94EB0005D278@yahoo.com> <8D8AA3F5-74A3-4AFE-A560-F52C7C55CF12@yahoo.com> Feedback-ID: 100108111:user:proton List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: ---- X-Rspamd-Queue-Id: 4TbmZn6Mmvz41m5 X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:62371, ipnet:185.70.43.0/24, country:CH] Sent with Proton Mail secure email. On Friday, 16 February 2024 at 14:02, Mark Millard wrot= e: > On Feb 15, 2024, at 20:08, Ordinary Bit ordinarybit@proton.me wrote: >=20 > > Hi! >=20 >=20 > Hello. >=20 > > On Friday, 16 February 2024 at 11:41, Mark Millard marklmi@yahoo.com wr= ote: > >=20 > > > [Only replying to lists I subscribe to.] > > >=20 > > > On Feb 15, 2024, at 19:19, Ordinary Bit ordinarybit@proton.me wrote: > > >=20 > > > > I'm reading the newfs manual https://man.freebsd.org/cgi/man.cgi?ne= wfs(8) to be able to know about the TRIM flag. In the manual under -t param= eter, it mentioned about "underlying device support", what exactly is this = device? > > >=20 > > > 2 contrasting examples: > > >=20 > > > Example 0: Optane NVMe media (PCIe card or U.2, for example) > > >=20 > > > Optane has no need of TRIM and, so, never supports TRIM. > > >=20 > > > Example 1: microsd card media usage > > >=20 > > > A microsd card in the normal type of microsd card slot on > > > Small Board Computers (normally) supports TRIM. Take the > > > same card and put it in a USB reader/writer and use it > > > via USB on the same system: no TRIM is supported by > > > FreeBSD over USB. > >=20 > > So you mean to say that if I have a Rasperry Pi 3 or 4 now and then hav= e my FreeBSD installed in a microSD card (for example, SanDisk Extreme card= ) with UFS/FFS filesytem in it with TRIM enabled parameter then is it going= to recognize it? How to verify? > >=20 > > > FYI: > > >=20 > > > When the file system has TRIM enabled, FreeBSD put out a > > > notice if TRIM will not actually be used in the actual > > > context in use. > >=20 > > Ok, got it. How to check this as well? >=20 >=20 > The console gets a message like: >=20 > WARNING: /mnt: TRIM flag on fs but disk does not support TRIM >=20 > when a mount is attempted (automatic or manually) for a > file system with the trim flag enabled but trim does not > end up active. >=20 > So, for example: >=20 > # dmesg -a | grep TRIM > WARNING: /mnt: TRIM flag on fs but disk does not support TRIM >=20 > (This was a microsd card in a USB reader/writer that was not > used as the boot media: a separate manual mount was used.) >=20 > The file system's TRIM flag status can be checked via use of > "tunefs -p . . .": >=20 > # tunefs -p /mnt > tunefs: POSIX.1e ACLs: (-a) disabled > tunefs: NFSv4 ACLs: (-N) disabled > tunefs: MAC multilabel: (-l) disabled > tunefs: soft updates: (-n) enabled > tunefs: soft update journaling: (-j) disabled > tunefs: gjournal: (-J) disabled > tunefs: trim: (-t) enabled > tunefs: maximum blocks per file in a cylinder group: (-e) 4096 > tunefs: average file size: (-f) 16384 > tunefs: average number of files in a directory: (-s) 64 > tunefs: minimum percentage of free space: (-m) 8% > tunefs: space to hold for metadata blocks: (-k) 6400 > tunefs: optimization preference: (-o) time > tunefs: volume label: (-L) >=20 > If the trim flag is enabled but the mount does not produce the > console message, then TRIM is active. >=20 Oh, that's amazing! I try it with my SanDisk Ultra microSDXC 64GB and mount= it in a USB reader and it shows enabled in the tunefs and is detected in t= he dmesg, the same as yours. Therefore, SanDisk Ultra microSD supports it! root@sd-extreme:~ # /sbin/newfs -U -t /dev/da0s2a /dev/da0s2a: 59000.9MB (120833920 sectors) block size 32768, fragment size = 4096 using 95 cylinder groups of 625.22MB, 20007 blks, 80128 inodes. with soft updates super-block backups (for fsck_ffs -b #) at: 192, 1280640, 2561088, 3841536, 5121984, 6402432, 7682880, 8963328, 102437= 76, 11524224, 12804672, 14085120, 15365568, 16646016, 17926464, 19206912, 20487360, 21767808, 23048256, 2432= 8704, 25609152, 26889600, 28170048, 29450496, 30730944, 32011392, 33291840, 34572288, 35852736, 37133184, 3841= 3632, 39694080, 40974528, 42254976, 43535424, 44815872, 46096320, 47376768, 48657216, 49937664, 51218112, 5249= 8560, 53779008, 55059456, 56339904, 57620352, 58900800, 60181248, 61461696, 62742144, 64022592, 65303040, 6658= 3488, 67863936, 69144384, 70424832, 71705280, 72985728, 74266176, 75546624, 76827072, 78107520, 79387968, 8066= 8416, 81948864, 83229312, 84509760, 85790208, 87070656, 88351104, 89631552, 90912000, 92192448, 93472896, 9475= 3344, 96033792, 97314240, 98594688, 99875136, 101155584, 102436032, 103716480, 104996928, 106277376, 107557824= , 108838272, 110118720, 111399168, 112679616, 113960064, 115240512, 116520960, 117801408, 119081856, 12036230= 4 root@sd-extreme:~ # dmesg | grep TRIM WARNING: /mnt: TRIM flag on fs but disk does not support TRIM root@sd-extreme:~ # tunefs -p /mnt tunefs: POSIX.1e ACLs: (-a) disabled tunefs: NFSv4 ACLs: (-N) disabled tunefs: MAC multilabel: (-l) disabled tunefs: soft updates: (-n) enabled tunefs: soft update journaling: (-j) disabled tunefs: gjournal: (-J) disabled tunefs: trim: (-t) enabled tunefs: maximum blocks per file in a cylinder group: (-e) 4096 tunefs: average file size: (-f) 16384 tunefs: average number of files in a directory: (-s) 64 tunefs: minimum percentage of free space: (-m) 8% tunefs: space to hold for metadata blocks: (-k) 6400 tunefs: optimization preference: (-o) time tunefs: volume label: (-L) I plan to have TRIM flag enabled in the rootfs partition (/dev/ufs/rootfs) = of my Raspberry Pi. Do you think of any implications when enabled? As shown= below, it is disabled. root@sd-extreme:~ # tunefs -p /dev/ufs/rootfs tunefs: POSIX.1e ACLs: (-a) disabled tunefs: NFSv4 ACLs: (-N) disabled tunefs: MAC multilabel: (-l) disabled tunefs: soft updates: (-n) enabled tunefs: soft update journaling: (-j) disabled tunefs: gjournal: (-J) disabled tunefs: trim: (-t) disabled tunefs: maximum blocks per file in a cylinder group: (-e) 4096 tunefs: average file size: (-f) 16384 tunefs: average number of files in a directory: (-s) 64 tunefs: minimum percentage of free space: (-m) 8% tunefs: space to hold for metadata blocks: (-k) 6400 tunefs: optimization preference: (-o) time tunefs: volume label: (-L) rootfs BR, orbit =20 From nobody Fri Feb 16 09:52:41 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TbnLB0Vyyz5BQsB for ; Fri, 16 Feb 2024 09:53:06 +0000 (UTC) (envelope-from ordinarybit@proton.me) Received: from mail-40141.protonmail.ch (mail-40141.protonmail.ch [185.70.40.141]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "protonmail.com", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TbnL93btnz44X2 for ; Fri, 16 Feb 2024 09:53:05 +0000 (UTC) (envelope-from ordinarybit@proton.me) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=proton.me; s=protonmail; t=1708077182; x=1708336382; bh=leS2ygNkdTObGeKiQ1babyT6IQdsrHJZOxOd8YkjMZk=; h=Date:To:From:Cc:Subject:Message-ID:In-Reply-To:References: Feedback-ID:From:To:Cc:Date:Subject:Reply-To:Feedback-ID: Message-ID:BIMI-Selector; b=NJtU8c4You/KYAG66BTvczbf85AO6kXRFZJFKQR1cIlA/NGOYm5G8igRpfEWEOpCQ Be9FTGhf68SL/Z2XOc5vbbbdEZ1e/MeeG1Uobj/k0jzBtxAKTpfw/tOdofB6DkOLz/ IetT0TY1nnMdzw37aJeeLE6BNnXxG6Vti5e6R8QlHIfSx/DMfcvmtDzbYyNTOP5fHy jXc0Zg2aPbQnxkeJE8f1ArG25kr7xKxZppHEdPRmhnZPf/nkk9d2F78aXR8C/y5bMe b8ACY02EAe4Lz6zkVD8igeokBJ/TurY6wKa8O0HgsNx6sJ+WwoFeJ0u3jr8W4E3R0K fQC15es1jE86Q== Date: Fri, 16 Feb 2024 09:52:41 +0000 To: Kevin Oberman From: Ordinary Bit Cc: "freebsd-questions@FreeBSD.org" , "freebsd-arm@freebsd.org" Subject: Re: newfs TRIM flag device support Message-ID: In-Reply-To: References: Feedback-ID: 100108111:user:proton List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="b1_062cPWRS8AQ9VA4jvUrSeywWG5XPZzkRbeZDHdbQPzE" X-Spamd-Bar: ---- X-Rspamd-Queue-Id: 4TbnL93btnz44X2 X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:62371, ipnet:185.70.40.0/24, country:CH] This is a multi-part message in MIME format. --b1_062cPWRS8AQ9VA4jvUrSeywWG5XPZzkRbeZDHdbQPzE Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: base64 T24gRnJpZGF5LCAxNiBGZWJydWFyeSAyMDI0IGF0IDEyOjMzLCBLZXZpbiBPYmVybWFuIDxya29i ZXJtYW5AZ21haWwuY29tPiB3cm90ZToKCj4gT24gVGh1LCBGZWIgMTUsIDIwMjQgYXQgNzoxOeKA r1BNIE9yZGluYXJ5IEJpdCA8b3JkaW5hcnliaXRAcHJvdG9uLm1lPiB3cm90ZToKPgo+PiBIaSwK Pj4KPj4gSSdtIHJlYWRpbmcgdGhlIG5ld2ZzIG1hbnVhbCBodHRwczovL21hbi5mcmVlYnNkLm9y Zy9jZ2kvbWFuLmNnaT9uZXdmcyg4KSB0byBiZSBhYmxlIHRvIGtub3cgYWJvdXQgdGhlIFRSSU0g ZmxhZy4gSW4gdGhlIG1hbnVhbCB1bmRlciAtdCBwYXJhbWV0ZXIsIGl0IG1lbnRpb25lZCBhYm91 dCAidW5kZXJseWluZyBkZXZpY2Ugc3VwcG9ydCIsIHdoYXQgZXhhY3RseSBpcyB0aGlzIGRldmlj ZT8gSXMgaXQgdGhlIGhvc3QgKGZvciBleGFtcGxlLCBSYXNwYmVycnkgUGkgU0QvZU1NQyBob3N0 IHJlYWRlcikgb3IgdGhlIFNEL2VNTUMgY2FyZCAoY29udHJvbGxlcikgb3IgYm90aD8KPj4KPj4g LXQKPj4KPj4gVHVybiAgb24JdGhlIFRSSU0gZW5hYmxlCWZsYWcuICBJZiBlbmFibGVkLCBhbmQg aWYgdGhlIHVuZGVybHktCj4+CWluZyBkZXZpY2Ugc3VwcG9ydHMgdGhlIEJJT19ERUxFVEUgIGNv bW1hbmQsICB0aGUgIGZpbGUJc3lzdGVtCj4+CXdpbGwgIHNlbmQgIGEgIGRlbGV0ZSByZXF1ZXN0 IHRvCXRoZSB1bmRlcmx5aW5nIGRldmljZSBmb3IgZWFjaAo+PglmcmVlZCBibG9jay4gIFRoZSB0 cmltIGVuYWJsZSBmbGFnIGlzIHR5cGljYWxseSBzZXQgZm9yCWZsYXNoLQo+PgltZW1vcnkgZGV2 aWNlcyB0byByZWR1Y2UJd3JpdGUgYW1wbGlmaWNhdGlvbiB3aGljaCByZWR1Y2VzIHdlYXIKPj4J b24gd3JpdGUtbGltaXRlZAlmbGFzaC1tZW1vcnkgYW5kIG9mdGVuIGltcHJvdmVzCWxvbmctdGVy bSBwZXItCj4+CWZvcm1hbmNlLiAgIFRoaW5seSBwcm92aXNpb25lZCBzdG9yYWdlIGFsc28gYmVu ZWZpdHMgYnkgcmV0dXJuLQo+PglpbmcgdW51c2VkIGJsb2NrcyB0byB0aGUJZ2xvYmFsIHBvb2wu Cj4+Cj4+IEJSLAo+Pgo+PiBvcmJpdAoKSGkhCgo+IFRSSU0gaXMgZm9yIFNTRHMuIEl0IGlzIHRp ZWQgdG8gdGhlIGRyaXZlLCBidXQgdGhlIGNvbnRyb2xsZXIgb3Igc3lzdGVtLiBJIHRoaW5rIExp bnV4IGVuYWJsZXMgaXQgYXV0b21hdGljYWxseSwgYnV0IEknbSBub3Qgc3VyZS4gSW4gdGhlIGNv bnRleHQgb2YgdGhlIGRlc2NyaXB0aW9uIGFib3ZlLCB0aGUgZHJpdmUgaXMgdGhlIGRldmljZS4K ClRoYW5rcyBmb3Igc2hhcmluZyEgSSBjaGVja2VkIGhlcmUgaHR0cHM6Ly9lbi53aWtpcGVkaWEu b3JnL3dpa2kvVHJpbV8oY29tcHV0aW5nKSNTRC9NTUMgd2hpY2ggc2F5cyB0aGF0IHRoZXJlIGlz IGEgc2ltaWxhciBmdW5jdGlvbmFsaXR5IG9mIEFUQSBUUklNIHRvIE1NQyBhbmQgU0Qgd2hpY2gg aXMgdGhlIEVSQVNFIChDTUQzOCksIHRoaXMgbWFrZSBtZSBjdXJpb3VzIGFib3V0IFRSSU0gc3Vw cG9ydCBhdmFpbGFibGUgaW4gdGhlIFVGUy9GRlMuIEkgaGF2ZSB0cmllZCBhIHdoaWxlIGFnbyBl bmFibGluZyBUUklNIGluIG15IFNhbkRpc2sgVWx0cmEgbWljcm9TRFhDIDY0R0IgYW5kIHdoZW4g bW91bnRlZCwgaXQgc2hvd3MgZW5hYmxlZCBpbiB0aGUgdHVuZWZzLiBIb3dldmVyLCBJIGFtIHN0 aWxsIHRoaW5raW5nIGlmIGl0IHdvcmtzIHdoZW4gVFJJTSBpcyBlbmFibGVkIGluIHRoZSByb290 ZnMgYW5kIHVzZWQgaXQgYXMgbXkgYm9vdCBtZWRpYSBpbiB0aGUgUmFzcGJlcnJ5IFBpIG1pY3Jv U0QgY2FyZCBzbG90LiBBbnl3YXksIHRoaXMgaXMgd29ydGggYSB0cnkuCgpCUiwKb3JiaXQKCj4g LS0KPgo+IEtldmluIE9iZXJtYW4sIFBhcnQgdGltZSBraWQgaGVyZGVyIGFuZCByZXRpcmVkIE5l dHdvcmsgRW5naW5lZXIKPiBFLW1haWw6IHJrb2Jlcm1hbkBnbWFpbC5jb20KPiBQR1AgRmluZ2Vy cHJpbnQ6IEQwM0ZCOThBRkE3OEUzQjc4QzE2OTRCMzE4QUIzOUVGMUIwNTU2ODM= --b1_062cPWRS8AQ9VA4jvUrSeywWG5XPZzkRbeZDHdbQPzE Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: base64 T24gRnJpZGF5LCAxNiBGZWJydWFyeSAyMDI0IGF0IDEyOjMzLCBLZXZpbiBPYmVybWFuICZsdDty a29iZXJtYW5AZ21haWwuY29tJmd0OyB3cm90ZToNCiAgICAgICAgPGRpdiBjbGFzcz0icHJvdG9u bWFpbF9xdW90ZSI+PGJsb2NrcXVvdGUgY2xhc3M9InByb3Rvbm1haWxfcXVvdGUiIHR5cGU9ImNp dGUiPg0KICAgICAgICAgICAgPGRpdiBkaXI9Imx0ciI+PGRpdiBkaXI9Imx0ciI+PGRpdiBzdHls ZT0iZm9udC1mYW1pbHk6dGFob21hLHNhbnMtc2VyaWY7Zm9udC1zaXplOnNtYWxsIiBjbGFzcz0i Z21haWxfZGVmYXVsdCI+T24gVGh1LCBGZWIgMTUsIDIwMjQgYXQgNzoxOeKAr1BNIE9yZGluYXJ5 IEJpdCAmbHQ7PGEgaHJlZj0ibWFpbHRvOm9yZGluYXJ5Yml0QHByb3Rvbi5tZSIgcmVsPSJub3Jl ZmVycmVyIG5vZm9sbG93IG5vb3BlbmVyIiB0YXJnZXQ9Il9ibGFuayI+b3JkaW5hcnliaXRAcHJv dG9uLm1lPC9hPiZndDsgd3JvdGU6PC9kaXY+PC9kaXY+PGRpdiBjbGFzcz0iZ21haWxfcXVvdGUi PjxibG9ja3F1b3RlIHN0eWxlPSJtYXJnaW46MHB4IDBweCAwcHggMC44ZXg7Ym9yZGVyLWxlZnQ6 MXB4IHNvbGlkIHJnYigyMDQsMjA0LDIwNCk7cGFkZGluZy1sZWZ0OjFleCIgY2xhc3M9ImdtYWls X3F1b3RlIj48ZGl2IHN0eWxlPSJmb250LWZhbWlseTpBcmlhbCxzYW5zLXNlcmlmO2ZvbnQtc2l6 ZToxNHB4Ij48YnI+PC9kaXY+PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6IEFyaWFsLCBzYW5zLXNl cmlmOyBmb250LXNpemU6IDE0cHg7IGNvbG9yOiByZ2IoMCwgMCwgMCk7IGJhY2tncm91bmQtY29s b3I6IHJnYigyNTUsIDI1NSwgMjU1KTsiPkhpLDwvZGl2PjxkaXYgc3R5bGU9ImZvbnQtZmFtaWx5 OiBBcmlhbCwgc2Fucy1zZXJpZjsgZm9udC1zaXplOiAxNHB4OyBjb2xvcjogcmdiKDAsIDAsIDAp OyBiYWNrZ3JvdW5kLWNvbG9yOiByZ2IoMjU1LCAyNTUsIDI1NSk7Ij48YnI+PC9kaXY+PGRpdiBz dHlsZT0iZm9udC1mYW1pbHk6IEFyaWFsLCBzYW5zLXNlcmlmOyBmb250LXNpemU6IDE0cHg7IGNv bG9yOiByZ2IoMCwgMCwgMCk7IGJhY2tncm91bmQtY29sb3I6IHJnYigyNTUsIDI1NSwgMjU1KTsi PkknbSByZWFkaW5nIHRoZSBuZXdmcyBtYW51YWwgPHNwYW4+PHNwYW4+PGEgdGFyZ2V0PSJfYmxh bmsiIGhyZWY9Imh0dHBzOi8vbWFuLmZyZWVic2Qub3JnL2NnaS9tYW4uY2dpP25ld2ZzKDgpIiBy ZWw9Im5vcmVmZXJyZXIgbm9mb2xsb3cgbm9vcGVuZXIiPmh0dHBzOi8vbWFuLmZyZWVic2Qub3Jn L2NnaS9tYW4uY2dpP25ld2ZzKDgpPC9hPjwvc3Bhbj4gdG8gYmUgYWJsZSB0byBrbm93IGFib3V0 IHRoZSBUUklNIGZsYWcuIEluIHRoZSBtYW51YWwgdW5kZXIgLXQgcGFyYW1ldGVyLCBpdCBtZW50 aW9uZWQgYWJvdXQgInVuZGVybHlpbmcgZGV2aWNlIHN1cHBvcnQiLCB3aGF0IGV4YWN0bHkgaXMg dGhpcyBkZXZpY2U/IElzIGl0IHRoZSBob3N0IChmb3IgZXhhbXBsZSwgUmFzcGJlcnJ5IFBpIFNE L2VNTUMgaG9zdCByZWFkZXIpIG9yIHRoZSBTRC9lTU1DIGNhcmQgKGNvbnRyb2xsZXIpIG9yIGJv dGg/PC9zcGFuPjwvZGl2PjxkaXYgc3R5bGU9ImZvbnQtZmFtaWx5OiBBcmlhbCwgc2Fucy1zZXJp ZjsgZm9udC1zaXplOiAxNHB4OyBjb2xvcjogcmdiKDAsIDAsIDApOyBiYWNrZ3JvdW5kLWNvbG9y OiByZ2IoMjU1LCAyNTUsIDI1NSk7Ij48c3Bhbj48cHJlPiAgICAgICA8Yj4tdDwvYj4gICAgICBU dXJuICBvbgl0aGUgVFJJTSBlbmFibGUJZmxhZy4gIElmIGVuYWJsZWQsIGFuZCBpZiB0aGUgdW5k ZXJseS0NCgkgICAgICAgaW5nIGRldmljZSBzdXBwb3J0cyB0aGUgQklPX0RFTEVURSAgY29tbWFu ZCwgIHRoZSAgZmlsZQlzeXN0ZW0NCgkgICAgICAgd2lsbCAgc2VuZCAgYSAgZGVsZXRlIHJlcXVl c3QgdG8JdGhlIHVuZGVybHlpbmcgZGV2aWNlIGZvciBlYWNoDQoJICAgICAgIGZyZWVkIGJsb2Nr LiAgVGhlIHRyaW0gZW5hYmxlIGZsYWcgaXMgdHlwaWNhbGx5IHNldCBmb3IJZmxhc2gtDQoJICAg ICAgIG1lbW9yeSBkZXZpY2VzIHRvIHJlZHVjZQl3cml0ZSBhbXBsaWZpY2F0aW9uIHdoaWNoIHJl ZHVjZXMgd2Vhcg0KCSAgICAgICBvbiB3cml0ZS1saW1pdGVkCWZsYXNoLW1lbW9yeSBhbmQgb2Z0 ZW4gaW1wcm92ZXMJbG9uZy10ZXJtIHBlci0NCgkgICAgICAgZm9ybWFuY2UuICAgVGhpbmx5IHBy b3Zpc2lvbmVkIHN0b3JhZ2UgYWxzbyBiZW5lZml0cyBieSByZXR1cm4tDQoJICAgICAgIGluZyB1 bnVzZWQgYmxvY2tzIHRvIHRoZQlnbG9iYWwgcG9vbC48YnI+PGJyPjxzcGFuIHN0eWxlPSJmb250 LWZhbWlseTogQXJpYWwsIHNhbnMtc2VyaWY7IGJhY2tncm91bmQtY29sb3I6IHJnYigyNTUsIDI1 NSwgMjU1KTsiPjxzcGFuPkJSLDxicj48L3NwYW4+PC9zcGFuPjwvcHJlPjxwcmU+PHNwYW4gc3R5 bGU9ImZvbnQtZmFtaWx5OiBBcmlhbCwgc2Fucy1zZXJpZjsgYmFja2dyb3VuZC1jb2xvcjogcmdi KDI1NSwgMjU1LCAyNTUpOyI+PHNwYW4+b3JiaXQ8YnI+PC9zcGFuPjwvc3Bhbj48L3ByZT48L3Nw YW4+PC9kaXY+PC9ibG9ja3F1b3RlPjwvZGl2PjwvZGl2PjwvYmxvY2txdW90ZT48ZGl2IHN0eWxl PSJmb250LWZhbWlseTogQXJpYWwsIHNhbnMtc2VyaWY7IGZvbnQtc2l6ZTogMTRweDsgY29sb3I6 IHJnYigwLCAwLCAwKTsgYmFja2dyb3VuZC1jb2xvcjogcmdiKDI1NSwgMjU1LCAyNTUpOyI+SGkh PC9kaXY+PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6IEFyaWFsLCBzYW5zLXNlcmlmOyBmb250LXNp emU6IDE0cHg7IGNvbG9yOiByZ2IoMCwgMCwgMCk7IGJhY2tncm91bmQtY29sb3I6IHJnYigyNTUs IDI1NSwgMjU1KTsiPjxicj48L2Rpdj48YmxvY2txdW90ZSBjbGFzcz0icHJvdG9ubWFpbF9xdW90 ZSIgdHlwZT0iY2l0ZSI+PGRpdiBkaXI9Imx0ciI+PGRpdiBjbGFzcz0iZ21haWxfcXVvdGUiPjxk aXYgY2xhc3M9ImdtYWlsX2RlZmF1bHQiIHN0eWxlPSJmb250LWZhbWlseTp0YWhvbWEsc2Fucy1z ZXJpZjtmb250LXNpemU6c21hbGwiPlRSSU0gaXMgZm9yIFNTRHMuIEl0IGlzIHRpZWQgdG8gdGhl IGRyaXZlLCBidXQgdGhlIGNvbnRyb2xsZXIgb3Igc3lzdGVtLiBJIHRoaW5rIExpbnV4IGVuYWJs ZXMgaXQgYXV0b21hdGljYWxseSwgYnV0IEknbSBub3Qgc3VyZS4gSW4gdGhlIGNvbnRleHQgb2Yg dGhlIGRlc2NyaXB0aW9uIGFib3ZlLCB0aGUgZHJpdmUgaXMgdGhlIGRldmljZS48L2Rpdj48L2Rp dj48L2Rpdj48L2Jsb2NrcXVvdGU+PGRpdiBjbGFzcz0iZ21haWxfZGVmYXVsdCIgc3R5bGU9ImZv bnQtZmFtaWx5OnRhaG9tYSxzYW5zLXNlcmlmO2ZvbnQtc2l6ZTpzbWFsbCI+PGJyPjwvZGl2Pjxk aXYgY2xhc3M9ImdtYWlsX2RlZmF1bHQiIHN0eWxlPSJmb250LWZhbWlseTp0YWhvbWEsc2Fucy1z ZXJpZjtmb250LXNpemU6c21hbGwiPlRoYW5rcyBmb3Igc2hhcmluZyEgSSBjaGVja2VkIGhlcmUg PHNwYW4+PGEgdGFyZ2V0PSJfYmxhbmsiIHJlbD0ibm9yZWZlcnJlciBub2ZvbGxvdyBub29wZW5l ciIgaHJlZj0iaHR0cHM6Ly9lbi53aWtpcGVkaWEub3JnL3dpa2kvVHJpbV8oY29tcHV0aW5nKSNT RC9NTUMiPmh0dHBzOi8vZW4ud2lraXBlZGlhLm9yZy93aWtpL1RyaW1fKGNvbXB1dGluZykjU0Qv TU1DPC9hPjwvc3Bhbj4gd2hpY2ggc2F5cyB0aGF0IHRoZXJlIGlzIGEgc2ltaWxhciBmdW5jdGlv bmFsaXR5IG9mIEFUQSBUUklNIHRvIE1NQyBhbmQgU0Qgd2hpY2ggaXMgdGhlIEVSQVNFIChDTUQz OCksIHRoaXMgbWFrZSBtZSBjdXJpb3VzIGFib3V0IFRSSU0gc3VwcG9ydCBhdmFpbGFibGUgaW4g dGhlIFVGUy9GRlMuIEkgaGF2ZSB0cmllZCBhIHdoaWxlIGFnbyBlbmFibGluZyBUUklNIGluIG15 IFNhbkRpc2sgVWx0cmEgbWljcm9TRFhDIDY0R0IgYW5kIHdoZW4gbW91bnRlZCwgaXQgc2hvd3Mg ZW5hYmxlZCBpbiB0aGUgdHVuZWZzLiBIb3dldmVyLCBJIGFtIHN0aWxsIHRoaW5raW5nIGlmIGl0 IHdvcmtzIHdoZW4gVFJJTSBpcyBlbmFibGVkIGluIHRoZSByb290ZnMgYW5kIHVzZWQgaXQgYXMg bXkgYm9vdCBtZWRpYSBpbiB0aGUgUmFzcGJlcnJ5IFBpIG1pY3JvU0QgY2FyZCBzbG90LiBBbnl3 YXksIHRoaXMgaXMgd29ydGggYSB0cnkuPC9kaXY+PGRpdiBjbGFzcz0iZ21haWxfZGVmYXVsdCIg c3R5bGU9ImZvbnQtZmFtaWx5OnRhaG9tYSxzYW5zLXNlcmlmO2ZvbnQtc2l6ZTpzbWFsbCI+PGJy PjwvZGl2PjxkaXYgY2xhc3M9ImdtYWlsX2RlZmF1bHQiIHN0eWxlPSJmb250LWZhbWlseTp0YWhv bWEsc2Fucy1zZXJpZjtmb250LXNpemU6c21hbGwiPkJSLDwvZGl2PjxkaXYgY2xhc3M9ImdtYWls X2RlZmF1bHQiIHN0eWxlPSJmb250LWZhbWlseTp0YWhvbWEsc2Fucy1zZXJpZjtmb250LXNpemU6 c21hbGwiPm9yYml0PGJyPjwvZGl2PjxkaXYgY2xhc3M9ImdtYWlsX2RlZmF1bHQiIHN0eWxlPSJm b250LWZhbWlseTp0YWhvbWEsc2Fucy1zZXJpZjtmb250LXNpemU6c21hbGwiPjxicj48L2Rpdj48 YmxvY2txdW90ZSBjbGFzcz0icHJvdG9ubWFpbF9xdW90ZSIgdHlwZT0iY2l0ZSI+PGRpdiBkaXI9 Imx0ciI+PHNwYW4gY2xhc3M9ImdtYWlsX3NpZ25hdHVyZV9wcmVmaXgiPi0tIDwvc3Bhbj48YnI+ PGRpdiBjbGFzcz0iZ21haWxfc2lnbmF0dXJlIiBkaXI9Imx0ciI+PGRpdiBkaXI9Imx0ciI+PGRp dj48ZGl2IGRpcj0ibHRyIj48ZGl2PjxkaXYgZGlyPSJsdHIiPjxkaXY+PGRpdiBkaXI9Imx0ciI+ S2V2aW4gT2Jlcm1hbiwgUGFydCB0aW1lIGtpZCBoZXJkZXIgYW5kIHJldGlyZWQgTmV0d29yayBF bmdpbmVlcjxicj5FLW1haWw6IDxhIHRhcmdldD0iX2JsYW5rIiBocmVmPSJtYWlsdG86cmtvYmVy bWFuQGdtYWlsLmNvbSIgcmVsPSJub3JlZmVycmVyIG5vZm9sbG93IG5vb3BlbmVyIj5ya29iZXJt YW5AZ21haWwuY29tPC9hPjxicj48L2Rpdj48ZGl2PlBHUCBGaW5nZXJwcmludDogRDAzRkI5OEFG QTc4RTNCNzhDMTY5NEIzMThBQjM5RUYxQjA1NTY4MzwvZGl2PjwvZGl2PjwvZGl2PjwvZGl2Pjwv ZGl2PjwvZGl2PjwvZGl2PjwvZGl2PjwvZGl2Pg0KDQogICAgICAgIDwvYmxvY2txdW90ZT48YnI+ DQogICAgPC9kaXY+ --b1_062cPWRS8AQ9VA4jvUrSeywWG5XPZzkRbeZDHdbQPzE-- From nobody Fri Feb 16 10:31:29 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TbpBX6HDpz5BVyT for ; Fri, 16 Feb 2024 10:31:32 +0000 (UTC) (envelope-from SRS0=qAai=JZ=klop.ws=ronald-lists@realworks.nl) Received: from smtp-relay-int.realworks.nl (smtp-relay-int.realworks.nl [194.109.157.24]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4TbpBX3SYjz49ZX for ; Fri, 16 Feb 2024 10:31:32 +0000 (UTC) (envelope-from SRS0=qAai=JZ=klop.ws=ronald-lists@realworks.nl) Authentication-Results: mx1.freebsd.org; none Date: Fri, 16 Feb 2024 11:31:29 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=klop.ws; s=rw2; t=1708079489; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=QQwRNEEBOVDAEa8MVDRbqbZbypQ0fOVwkf2svDOXC/I=; b=yy5GjYEAEWQjdSLcbWotLV/x5g1O5DpwytAFbkaTsHd/1L4RW2KTH/Wl83HFphDjnsfkjr 9cfLC2+MkH9ewbH8hvMlJbWVCZTAEyzWFjYY78UDY0mASEFaDXSM6Er3t/6sRl/g5hfNwz hrOkpwpfwHNMExnYerWJnrhstgcHClq8Zo6RW/a8x3oZJQEPHcVyogAvfh6Oengtunhq2d LePwZwvV5nObnNmOywCkgWMu86pKrwh+mxXVnesRarWnXRI7rMAL3tKbMOtv42ixSCaWS9 uZiHL4zz/M6Hg58OCjw6BxLDbp250xx/rpKKdkgr6CpxvEBf1+IncMh1c7OxgA== From: Ronald Klop To: Ordinary Bit Cc: "freebsd-arm@freebsd.org" , Mark Millard Message-ID: <454941689.1180.1708079489598@localhost> In-Reply-To: <7okx4eDPse2uF1n3aX4W8RIOdbHXziUhqgvHxqqxpYN-7NsJfEGrfHbDxLgNS3PdOwZt7j9aOXp-0MHDjQAryL4t1Gd2QHUG1sFNmd5dY0k=@proton.me> References: <6AD380D4-9B42-4511-9E02-94EB0005D278@yahoo.com> <8D8AA3F5-74A3-4AFE-A560-F52C7C55CF12@yahoo.com> <7okx4eDPse2uF1n3aX4W8RIOdbHXziUhqgvHxqqxpYN-7NsJfEGrfHbDxLgNS3PdOwZt7j9aOXp-0MHDjQAryL4t1Gd2QHUG1sFNmd5dY0k=@proton.me> Subject: Re: newfs TRIM flag device support List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_1179_1625227637.1708079489560" X-Mailer: Realworks (690.28) Importance: Normal X-Priority: 3 (Normal) X-Spamd-Bar: ---- X-Rspamd-Queue-Id: 4TbpBX3SYjz49ZX X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:3265, ipnet:194.109.0.0/16, country:NL] ------=_Part_1179_1625227637.1708079489560 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Van: Ordinary Bit Datum: vrijdag, 16 februari 2024 10:18 Aan: Mark Millard CC: "freebsd-arm@freebsd.org" Onderwerp: Re: newfs TRIM flag device support > > > > > > Sent with Proton Mail secure email. > > On Friday, 16 February 2024 at 14:02, Mark Millard wrote: > > > On Feb 15, 2024, at 20:08, Ordinary Bit ordinarybit@proton.me wrote: > > > > > Hi! > > > > > > Hello. > > > > > On Friday, 16 February 2024 at 11:41, Mark Millard marklmi@yahoo.com wrote: > > > > > > > [Only replying to lists I subscribe to.] > > > > > > > > On Feb 15, 2024, at 19:19, Ordinary Bit ordinarybit@proton.me wrote: > > > > > > > > > I'm reading the newfs manual https://man.freebsd.org/cgi/man.cgi?newfs(8) to be able to know about the TRIM flag. In the manual under -t parameter, it mentioned about "underlying device support", what exactly is this device? > > > > > > > > 2 contrasting examples: > > > > > > > > Example 0: Optane NVMe media (PCIe card or U.2, for example) > > > > > > > > Optane has no need of TRIM and, so, never supports TRIM. > > > > > > > > Example 1: microsd card media usage > > > > > > > > A microsd card in the normal type of microsd card slot on > > > > Small Board Computers (normally) supports TRIM. Take the > > > > same card and put it in a USB reader/writer and use it > > > > via USB on the same system: no TRIM is supported by > > > > FreeBSD over USB. > > > > > > So you mean to say that if I have a Rasperry Pi 3 or 4 now and then have my FreeBSD installed in a microSD card (for example, SanDisk Extreme card) with UFS/FFS filesytem in it with TRIM enabled parameter then is it going to recognize it? How to verify? > > > > > > > FYI: > > > > > > > > When the file system has TRIM enabled, FreeBSD put out a > > > > notice if TRIM will not actually be used in the actual > > > > context in use. > > > > > > Ok, got it. How to check this as well? > > > > > > The console gets a message like: > > > > WARNING: /mnt: TRIM flag on fs but disk does not support TRIM > > > > when a mount is attempted (automatic or manually) for a > > file system with the trim flag enabled but trim does not > > end up active. > > > > So, for example: > > > > # dmesg -a | grep TRIM > > WARNING: /mnt: TRIM flag on fs but disk does not support TRIM > > > > (This was a microsd card in a USB reader/writer that was not > > used as the boot media: a separate manual mount was used.) > > > > The file system's TRIM flag status can be checked via use of > > "tunefs -p . . .": > > > > # tunefs -p /mnt > > tunefs: POSIX.1e ACLs: (-a) disabled > > tunefs: NFSv4 ACLs: (-N) disabled > > tunefs: MAC multilabel: (-l) disabled > > tunefs: soft updates: (-n) enabled > > tunefs: soft update journaling: (-j) disabled > > tunefs: gjournal: (-J) disabled > > tunefs: trim: (-t) enabled > > tunefs: maximum blocks per file in a cylinder group: (-e) 4096 > > tunefs: average file size: (-f) 16384 > > tunefs: average number of files in a directory: (-s) 64 > > tunefs: minimum percentage of free space: (-m) 8% > > tunefs: space to hold for metadata blocks: (-k) 6400 > > tunefs: optimization preference: (-o) time > > tunefs: volume label: (-L) > > > > If the trim flag is enabled but the mount does not produce the > > console message, then TRIM is active. > > > > Oh, that's amazing! I try it with my SanDisk Ultra microSDXC 64GB and mount it in a USB reader and it shows enabled in the tunefs and is detected in the dmesg, the same as yours. Therefore, SanDisk Ultra microSD supports it! > > root@sd-extreme:~ # /sbin/newfs -U -t /dev/da0s2a > /dev/da0s2a: 59000.9MB (120833920 sectors) block size 32768, fragment size 4096 > using 95 cylinder groups of 625.22MB, 20007 blks, 80128 inodes. > with soft updates > super-block backups (for fsck_ffs -b #) at: > 192, 1280640, 2561088, 3841536, 5121984, 6402432, 7682880, 8963328, 10243776, 11524224, 12804672, 14085120, > 15365568, 16646016, 17926464, 19206912, 20487360, 21767808, 23048256, 24328704, 25609152, 26889600, 28170048, > 29450496, 30730944, 32011392, 33291840, 34572288, 35852736, 37133184, 38413632, 39694080, 40974528, 42254976, > 43535424, 44815872, 46096320, 47376768, 48657216, 49937664, 51218112, 52498560, 53779008, 55059456, 56339904, > 57620352, 58900800, 60181248, 61461696, 62742144, 64022592, 65303040, 66583488, 67863936, 69144384, 70424832, > 71705280, 72985728, 74266176, 75546624, 76827072, 78107520, 79387968, 80668416, 81948864, 83229312, 84509760, > 85790208, 87070656, 88351104, 89631552, 90912000, 92192448, 93472896, 94753344, 96033792, 97314240, 98594688, > 99875136, 101155584, 102436032, 103716480, 104996928, 106277376, 107557824, 108838272, 110118720, 111399168, > 112679616, 113960064, 115240512, 116520960, 117801408, 119081856, 120362304 > > root@sd-extreme:~ # dmesg | grep TRIM > WARNING: /mnt: TRIM flag on fs but disk does not support TRIM > > root@sd-extreme:~ # tunefs -p /mnt > tunefs: POSIX.1e ACLs: (-a) disabled > tunefs: NFSv4 ACLs: (-N) disabled > tunefs: MAC multilabel: (-l) disabled > tunefs: soft updates: (-n) enabled > tunefs: soft update journaling: (-j) disabled > tunefs: gjournal: (-J) disabled > tunefs: trim: (-t) enabled > tunefs: maximum blocks per file in a cylinder group: (-e) 4096 > tunefs: average file size: (-f) 16384 > tunefs: average number of files in a directory: (-s) 64 > tunefs: minimum percentage of free space: (-m) 8% > tunefs: space to hold for metadata blocks: (-k) 6400 > tunefs: optimization preference: (-o) time > tunefs: volume label: (-L) > > I plan to have TRIM flag enabled in the rootfs partition (/dev/ufs/rootfs) of my Raspberry Pi. Do you think of any implications when enabled? As shown below, it is disabled. > > root@sd-extreme:~ # tunefs -p /dev/ufs/rootfs > tunefs: POSIX.1e ACLs: (-a) disabled > tunefs: NFSv4 ACLs: (-N) disabled > tunefs: MAC multilabel: (-l) disabled > tunefs: soft updates: (-n) enabled > tunefs: soft update journaling: (-j) disabled > tunefs: gjournal: (-J) disabled > tunefs: trim: (-t) disabled > tunefs: maximum blocks per file in a cylinder group: (-e) 4096 > tunefs: average file size: (-f) 16384 > tunefs: average number of files in a directory: (-s) 64 > tunefs: minimum percentage of free space: (-m) 8% > tunefs: space to hold for metadata blocks: (-k) 6400 > tunefs: optimization preference: (-o) time > tunefs: volume label: (-L) rootfs > > BR, > orbit > > > > > > > Hi, I'm sorry to spoil the fun but this message says the device does *not* support trim. "root@sd-extreme:~ # dmesg | grep TRIM WARNING: /mnt: TRIM flag on fs but disk does not support TRIM" But to give some hope. As Mark pointed out it is not only about the disk. The controller also needs to support it. As a real world example: I have an RPI3 with an SSD (supporting trim) connected via USB-to-SATA (not supporting trim). So the total system does not support trim. $ cat /var/run/dmesg.boot | grep -E "umass0|da0|TRIM" umass0 on uhub1 umass0: on usbus1 umass0: SCSI over Bulk-Only; quirks = 0x0100 umass0:0:0: Attached to scbus0 da0 at umass-sim0 bus 0 scbus0 target 0 lun 0 da0: Fixed Direct Access SPC-4 SCSI device da0: Serial Number 2015100000D6 da0: 40.000MB/s transfers da0: 244198MB (500118192 512 byte sectors) da0: quirks=0x2 WARNING: /: TRIM flag on fs but disk does not support TRIM $ sysctl kern.cam.da.0 | grep -E "trim|delete" kern.cam.da.0.trim_ticks: 0 kern.cam.da.0.trim_goal: 0 kern.cam.da.0.trim_lbas: 0 kern.cam.da.0.trim_ranges: 0 kern.cam.da.0.trim_count: 0 kern.cam.da.0.delete_max: 65536 kern.cam.da.0.delete_method: NONE On an RPI4B I have an SSD (supporting trim) connected via USB-to-SATA (which does support trim also). Here trim works fine. (NB: this uses ZFS, but that does not matter) $ dmesg | grep -E "umass0|da0" umass0 on uhub0 umass0: on usbus0 da0 at umass-sim0 bus 0 scbus0 target 0 lun 0 da0: Fixed Direct Access SPC-4 SCSI device da0: Serial Number 000000004BA8 da0: 400.000MB/s transfers da0: 244198MB (500118192 512 byte sectors) da0: quirks=0x2 $ sysctl kern.cam.da.0 | grep -E "trim|delete" kern.cam.da.0.trim_ticks: 0 kern.cam.da.0.trim_goal: 0 kern.cam.da.0.trim_lbas: 546841704 kern.cam.da.0.trim_ranges: 317280 kern.cam.da.0.trim_count: 301424 kern.cam.da.0.delete_max: 4294901760 kern.cam.da.0.delete_method: UNMAP BTW: enabling trim on the fs while the device does not support it does not matter and does no harm. Using the sysctl above you can see if the device actually uses trim. Running "gstat -d" while deleting a lot of files will also show if trim/delete actions are sent to the disk. So the combination of your USB reader + microSD do not support TRIM, but you can just try to put the microSD directly in the RPI4 and see what that does. Regards, Ronald. ------=_Part_1179_1625227637.1708079489560 Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit

Van: Ordinary Bit <ordinarybit@proton.me>
Datum: vrijdag, 16 februari 2024 10:18
Aan: Mark Millard <marklmi@yahoo.com>
CC: "freebsd-arm@freebsd.org" <freebsd-arm@freebsd.org>
Onderwerp: Re: newfs TRIM flag device support






Sent with Proton Mail secure email.

On Friday, 16 February 2024 at 14:02, Mark Millard <marklmi@yahoo.com> wrote:

> On Feb 15, 2024, at 20:08, Ordinary Bit ordinarybit@proton.me wrote:
>
> > Hi!
>
>
> Hello.
>
> > On Friday, 16 February 2024 at 11:41, Mark Millard marklmi@yahoo.com wrote:
> >
> > > [Only replying to lists I subscribe to.]
> > >
> > > On Feb 15, 2024, at 19:19, Ordinary Bit ordinarybit@proton.me wrote:
> > >
> > > > I'm reading the newfs manual https://man.freebsd.org/cgi/man.cgi?newfs(8) to be able to know about the TRIM flag. In the manual under -t parameter, it mentioned about "underlying device support", what exactly is this device?
> > >
> > > 2 contrasting examples:
> > >
> > > Example 0: Optane NVMe media (PCIe card or U.2, for example)
> > >
> > > Optane has no need of TRIM and, so, never supports TRIM.
> > >
> > > Example 1: microsd card media usage
> > >
> > > A microsd card in the normal type of microsd card slot on
> > > Small Board Computers (normally) supports TRIM. Take the
> > > same card and put it in a USB reader/writer and use it
> > > via USB on the same system: no TRIM is supported by
> > > FreeBSD over USB.
> >
> > So you mean to say that if I have a Rasperry Pi 3 or 4 now and then have my FreeBSD installed in a microSD card (for example, SanDisk Extreme card) with UFS/FFS filesytem in it with TRIM enabled parameter then is it going to recognize it? How to verify?
> >
> > > FYI:
> > >
> > > When the file system has TRIM enabled, FreeBSD put out a
> > > notice if TRIM will not actually be used in the actual
> > > context in use.
> >
> > Ok, got it. How to check this as well?
>
>
> The console gets a message like:
>
> WARNING: /mnt: TRIM flag on fs but disk does not support TRIM
>
> when a mount is attempted (automatic or manually) for a
> file system with the trim flag enabled but trim does not
> end up active.
>
> So, for example:
>
> # dmesg -a | grep TRIM
> WARNING: /mnt: TRIM flag on fs but disk does not support TRIM
>
> (This was a microsd card in a USB reader/writer that was not
> used as the boot media: a separate manual mount was used.)
>
> The file system's TRIM flag status can be checked via use of
> "tunefs -p . . .":
>
> # tunefs -p /mnt
> tunefs: POSIX.1e ACLs: (-a) disabled
> tunefs: NFSv4 ACLs: (-N) disabled
> tunefs: MAC multilabel: (-l) disabled
> tunefs: soft updates: (-n) enabled
> tunefs: soft update journaling: (-j) disabled
> tunefs: gjournal: (-J) disabled
> tunefs: trim: (-t) enabled
> tunefs: maximum blocks per file in a cylinder group: (-e) 4096
> tunefs: average file size: (-f) 16384
> tunefs: average number of files in a directory: (-s) 64
> tunefs: minimum percentage of free space: (-m) 8%
> tunefs: space to hold for metadata blocks: (-k) 6400
> tunefs: optimization preference: (-o) time
> tunefs: volume label: (-L)
>
> If the trim flag is enabled but the mount does not produce the
> console message, then TRIM is active.
>

Oh, that's amazing! I try it with my SanDisk Ultra microSDXC 64GB and mount it in a USB reader and it shows enabled in the tunefs and is detected in the dmesg, the same as yours. Therefore, SanDisk Ultra microSD supports it!

root@sd-extreme:~ # /sbin/newfs -U -t /dev/da0s2a
/dev/da0s2a: 59000.9MB (120833920 sectors) block size 32768, fragment size 4096
        using 95 cylinder groups of 625.22MB, 20007 blks, 80128 inodes.
        with soft updates
super-block backups (for fsck_ffs -b #) at:
 192, 1280640, 2561088, 3841536, 5121984, 6402432, 7682880, 8963328, 10243776, 11524224, 12804672, 14085120,
 15365568, 16646016, 17926464, 19206912, 20487360, 21767808, 23048256, 24328704, 25609152, 26889600, 28170048,
 29450496, 30730944, 32011392, 33291840, 34572288, 35852736, 37133184, 38413632, 39694080, 40974528, 42254976,
 43535424, 44815872, 46096320, 47376768, 48657216, 49937664, 51218112, 52498560, 53779008, 55059456, 56339904,
 57620352, 58900800, 60181248, 61461696, 62742144, 64022592, 65303040, 66583488, 67863936, 69144384, 70424832,
 71705280, 72985728, 74266176, 75546624, 76827072, 78107520, 79387968, 80668416, 81948864, 83229312, 84509760,
 85790208, 87070656, 88351104, 89631552, 90912000, 92192448, 93472896, 94753344, 96033792, 97314240, 98594688,
 99875136, 101155584, 102436032, 103716480, 104996928, 106277376, 107557824, 108838272, 110118720, 111399168,
 112679616, 113960064, 115240512, 116520960, 117801408, 119081856, 120362304

root@sd-extreme:~ # dmesg | grep TRIM
WARNING: /mnt: TRIM flag on fs but disk does not support TRIM

root@sd-extreme:~ # tunefs -p /mnt
tunefs: POSIX.1e ACLs: (-a)                                disabled
tunefs: NFSv4 ACLs: (-N)                                   disabled
tunefs: MAC multilabel: (-l)                               disabled
tunefs: soft updates: (-n)                                 enabled
tunefs: soft update journaling: (-j)                       disabled
tunefs: gjournal: (-J)                                     disabled
tunefs: trim: (-t)                                         enabled
tunefs: maximum blocks per file in a cylinder group: (-e)  4096
tunefs: average file size: (-f)                            16384
tunefs: average number of files in a directory: (-s)       64
tunefs: minimum percentage of free space: (-m)             8%
tunefs: space to hold for metadata blocks: (-k)            6400
tunefs: optimization preference: (-o)                      time
tunefs: volume label: (-L)

I plan to have TRIM flag enabled in the rootfs partition (/dev/ufs/rootfs) of my Raspberry Pi. Do you think of any implications when enabled? As shown below, it is disabled.

root@sd-extreme:~ # tunefs -p /dev/ufs/rootfs
tunefs: POSIX.1e ACLs: (-a)                                disabled
tunefs: NFSv4 ACLs: (-N)                                   disabled
tunefs: MAC multilabel: (-l)                               disabled
tunefs: soft updates: (-n)                                 enabled
tunefs: soft update journaling: (-j)                       disabled
tunefs: gjournal: (-J)                                     disabled
tunefs: trim: (-t)                                         disabled
tunefs: maximum blocks per file in a cylinder group: (-e)  4096
tunefs: average file size: (-f)                            16384
tunefs: average number of files in a directory: (-s)       64
tunefs: minimum percentage of free space: (-m)             8%
tunefs: space to hold for metadata blocks: (-k)            6400
tunefs: optimization preference: (-o)                      time
tunefs: volume label: (-L)                                 rootfs

BR,
orbit
 


 

Hi,

I'm sorry to spoil the fun but this message says the device does *not* support trim.

"root@sd-extreme:~ # dmesg | grep TRIM
WARNING: /mnt: TRIM flag on fs but disk does not support TRIM"


But to give some hope. As Mark pointed out it is not only about the disk. The controller also needs to support it.

As a real world example:

I have an RPI3 with an SSD (supporting trim) connected via USB-to-SATA (not supporting trim). So the total system does not support trim.
$ cat /var/run/dmesg.boot | grep -E "umass0|da0|TRIM"
umass0 on uhub1
umass0: <GODO USB3.0 to SATA adapter, class 0/0, rev 2.10/1.00, addr 5> on usbus1
umass0:  SCSI over Bulk-Only; quirks = 0x0100
umass0:0:0: Attached to scbus0
da0 at umass-sim0 bus 0 scbus0 target 0 lun 0
da0: <LITEON C V3-DE256 0> Fixed Direct Access SPC-4 SCSI device
da0: Serial Number 2015100000D6
da0: 40.000MB/s transfers
da0: 244198MB (500118192 512 byte sectors)
da0: quirks=0x2<NO_6_BYTE>
WARNING: /: TRIM flag on fs but disk does not support TRIM

$ sysctl kern.cam.da.0 | grep -E "trim|delete"
kern.cam.da.0.trim_ticks: 0
kern.cam.da.0.trim_goal: 0
kern.cam.da.0.trim_lbas: 0
kern.cam.da.0.trim_ranges: 0
kern.cam.da.0.trim_count: 0
kern.cam.da.0.delete_max: 65536
kern.cam.da.0.delete_method: NONE


On an RPI4B I have an SSD (supporting trim) connected via USB-to-SATA (which does support trim also). Here trim works fine. (NB: this uses ZFS, but that does not matter)
$ dmesg | grep -E "umass0|da0"
umass0 on uhub0
umass0: <USB 3.0 Device USB 3.0 Device, class 0/0, rev 3.00/3.01, addr 3> on usbus0
da0 at umass-sim0 bus 0 scbus0 target 0 lun 0
da0: <Micron 1 100 SATA 256GB 0301> Fixed Direct Access SPC-4 SCSI device
da0: Serial Number 000000004BA8
da0: 400.000MB/s transfers
da0: 244198MB (500118192 512 byte sectors)
da0: quirks=0x2<NO_6_BYTE>

$ sysctl kern.cam.da.0 | grep -E "trim|delete"
kern.cam.da.0.trim_ticks: 0
kern.cam.da.0.trim_goal: 0
kern.cam.da.0.trim_lbas: 546841704
kern.cam.da.0.trim_ranges: 317280
kern.cam.da.0.trim_count: 301424
kern.cam.da.0.delete_max: 4294901760
kern.cam.da.0.delete_method: UNMAP


BTW: enabling trim on the fs while the device does not support it does not matter and does no harm. Using the sysctl above you can see if the device actually uses trim.

Running "gstat -d" while deleting a lot of files will also show if trim/delete actions are sent to the disk.

So the combination of your USB reader + microSD do not support TRIM, but you can just try to put the microSD directly in the RPI4 and see what that does.

Regards,
Ronald.
  ------=_Part_1179_1625227637.1708079489560-- From nobody Fri Feb 16 10:54:12 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Tbpj64bvXz5BYsN for ; Fri, 16 Feb 2024 10:54:34 +0000 (UTC) (envelope-from ordinarybit@proton.me) Received: from mail-40132.protonmail.ch (mail-40132.protonmail.ch [185.70.40.132]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "protonmail.com", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Tbpj60nK6z4FPX for ; Fri, 16 Feb 2024 10:54:34 +0000 (UTC) (envelope-from ordinarybit@proton.me) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=proton.me; s=protonmail; t=1708080869; x=1708340069; bh=2mkjxt1qlJaT57nE8+Xh0wpA7wK/NH2iJoSrmqx2zuI=; h=Date:To:From:Cc:Subject:Message-ID:In-Reply-To:References: Feedback-ID:From:To:Cc:Date:Subject:Reply-To:Feedback-ID: Message-ID:BIMI-Selector; b=K+j6cWhl18e62JjBEjwqiP9H6uDk09aqbZIByhfJ0Cp7t4jjSsoYr/2SscMRqel4W XdqXXxzoXlI/Sdnd1u1ZCdEdENYaFrV4Ov6H7X16c2cpJbcgMQeljOk1vAzdbs0FB0 B4tqlVz9wdM9+9OOdZJI+7guFQLSNAX5Tq428KJBFWLRGdMYGnUMGAnO4sRk8koPmX aStKRxnNedv0UAzyufjw+D9LbqN+3k4KlC1vbL0G75KRBvTzq6fl9WQXtXBCmfMjT4 zpJ6eCteu5BNk0JgxtFThy6SAfaWFFesnagMb0eA7/Jntg67IjL+4l2dpEaTIgoEoP TD/iThFbJH1fg== Date: Fri, 16 Feb 2024 10:54:12 +0000 To: Ronald Klop From: Ordinary Bit Cc: "freebsd-arm@freebsd.org" , Mark Millard Subject: Re: newfs TRIM flag device support Message-ID: In-Reply-To: <454941689.1180.1708079489598@localhost> References: <6AD380D4-9B42-4511-9E02-94EB0005D278@yahoo.com> <8D8AA3F5-74A3-4AFE-A560-F52C7C55CF12@yahoo.com> <7okx4eDPse2uF1n3aX4W8RIOdbHXziUhqgvHxqqxpYN-7NsJfEGrfHbDxLgNS3PdOwZt7j9aOXp-0MHDjQAryL4t1Gd2QHUG1sFNmd5dY0k=@proton.me> <454941689.1180.1708079489598@localhost> Feedback-ID: 100108111:user:proton List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="b1_i4TVXaKbceUP6PQinOcTun6iLjvwKymRhlhbJDABQI" X-Spamd-Bar: ---- X-Rspamd-Queue-Id: 4Tbpj60nK6z4FPX X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:62371, ipnet:185.70.40.0/24, country:CH] This is a multi-part message in MIME format. --b1_i4TVXaKbceUP6PQinOcTun6iLjvwKymRhlhbJDABQI Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: base64 T24gRnJpZGF5LCAxNiBGZWJydWFyeSAyMDI0IGF0IDE4OjMxLCBSb25hbGQgS2xvcCA8cm9uYWxk LWxpc3RzQGtsb3Aud3M+IHdyb3RlOgoKPiBWYW46IE9yZGluYXJ5IEJpdCA8b3JkaW5hcnliaXRA cHJvdG9uLm1lPgo+IERhdHVtOiB2cmlqZGFnLCAxNiBmZWJydWFyaSAyMDI0IDEwOjE4Cj4gQWFu OiBNYXJrIE1pbGxhcmQgPG1hcmtsbWlAeWFob28uY29tPgo+IENDOiAiZnJlZWJzZC1hcm1AZnJl ZWJzZC5vcmciIDxmcmVlYnNkLWFybUBmcmVlYnNkLm9yZz4KPiBPbmRlcndlcnA6IFJlOiBuZXdm cyBUUklNIGZsYWcgZGV2aWNlIHN1cHBvcnQKPgo+PiBTZW50IHdpdGggUHJvdG9uIE1haWwgc2Vj dXJlIGVtYWlsLgo+Pgo+PiBPbiBGcmlkYXksIDE2IEZlYnJ1YXJ5IDIwMjQgYXQgMTQ6MDIsIE1h cmsgTWlsbGFyZCA8bWFya2xtaUB5YWhvby5jb20+IHdyb3RlOgo+Pgo+Pj4gT24gRmViIDE1LCAy MDI0LCBhdCAyMDowOCwgT3JkaW5hcnkgQml0IG9yZGluYXJ5Yml0QHByb3Rvbi5tZSB3cm90ZToK Pj4+Cj4+PiA+IEhpIQo+Pj4KPj4+Cj4+PiBIZWxsby4KPj4+Cj4+PiA+IE9uIEZyaWRheSwgMTYg RmVicnVhcnkgMjAyNCBhdCAxMTo0MSwgTWFyayBNaWxsYXJkIG1hcmtsbWlAeWFob28uY29tIHdy b3RlOgo+Pj4gPgo+Pj4gPiA+IFtPbmx5IHJlcGx5aW5nIHRvIGxpc3RzIEkgc3Vic2NyaWJlIHRv Ll0KPj4+ID4gPgo+Pj4gPiA+IE9uIEZlYiAxNSwgMjAyNCwgYXQgMTk6MTksIE9yZGluYXJ5IEJp dCBvcmRpbmFyeWJpdEBwcm90b24ubWUgd3JvdGU6Cj4+PiA+ID4KPj4+ID4gPiA+IEknbSByZWFk aW5nIHRoZSBuZXdmcyBtYW51YWwgaHR0cHM6Ly9tYW4uZnJlZWJzZC5vcmcvY2dpL21hbi5jZ2k/ bmV3ZnMoOCkgdG8gYmUgYWJsZSB0byBrbm93IGFib3V0IHRoZSBUUklNIGZsYWcuIEluIHRoZSBt YW51YWwgdW5kZXIgLXQgcGFyYW1ldGVyLCBpdCBtZW50aW9uZWQgYWJvdXQgInVuZGVybHlpbmcg ZGV2aWNlIHN1cHBvcnQiLCB3aGF0IGV4YWN0bHkgaXMgdGhpcyBkZXZpY2U/Cj4+PiA+ID4KPj4+ ID4gPiAyIGNvbnRyYXN0aW5nIGV4YW1wbGVzOgo+Pj4gPiA+Cj4+PiA+ID4gRXhhbXBsZSAwOiBP cHRhbmUgTlZNZSBtZWRpYSAoUENJZSBjYXJkIG9yIFUuMiwgZm9yIGV4YW1wbGUpCj4+PiA+ID4K Pj4+ID4gPiBPcHRhbmUgaGFzIG5vIG5lZWQgb2YgVFJJTSBhbmQsIHNvLCBuZXZlciBzdXBwb3J0 cyBUUklNLgo+Pj4gPiA+Cj4+PiA+ID4gRXhhbXBsZSAxOiBtaWNyb3NkIGNhcmQgbWVkaWEgdXNh Z2UKPj4+ID4gPgo+Pj4gPiA+IEEgbWljcm9zZCBjYXJkIGluIHRoZSBub3JtYWwgdHlwZSBvZiBt aWNyb3NkIGNhcmQgc2xvdCBvbgo+Pj4gPiA+IFNtYWxsIEJvYXJkIENvbXB1dGVycyAobm9ybWFs bHkpIHN1cHBvcnRzIFRSSU0uIFRha2UgdGhlCj4+PiA+ID4gc2FtZSBjYXJkIGFuZCBwdXQgaXQg aW4gYSBVU0IgcmVhZGVyL3dyaXRlciBhbmQgdXNlIGl0Cj4+PiA+ID4gdmlhIFVTQiBvbiB0aGUg c2FtZSBzeXN0ZW06IG5vIFRSSU0gaXMgc3VwcG9ydGVkIGJ5Cj4+PiA+ID4gRnJlZUJTRCBvdmVy IFVTQi4KPj4+ID4KPj4+ID4gU28geW91IG1lYW4gdG8gc2F5IHRoYXQgaWYgSSBoYXZlIGEgUmFz cGVycnkgUGkgMyBvciA0IG5vdyBhbmQgdGhlbiBoYXZlIG15IEZyZWVCU0QgaW5zdGFsbGVkIGlu IGEgbWljcm9TRCBjYXJkIChmb3IgZXhhbXBsZSwgU2FuRGlzayBFeHRyZW1lIGNhcmQpIHdpdGgg VUZTL0ZGUyBmaWxlc3l0ZW0gaW4gaXQgd2l0aCBUUklNIGVuYWJsZWQgcGFyYW1ldGVyIHRoZW4g aXMgaXQgZ29pbmcgdG8gcmVjb2duaXplIGl0PyBIb3cgdG8gdmVyaWZ5Pwo+Pj4gPgo+Pj4gPiA+ IEZZSToKPj4+ID4gPgo+Pj4gPiA+IFdoZW4gdGhlIGZpbGUgc3lzdGVtIGhhcyBUUklNIGVuYWJs ZWQsIEZyZWVCU0QgcHV0IG91dCBhCj4+PiA+ID4gbm90aWNlIGlmIFRSSU0gd2lsbCBub3QgYWN0 dWFsbHkgYmUgdXNlZCBpbiB0aGUgYWN0dWFsCj4+PiA+ID4gY29udGV4dCBpbiB1c2UuCj4+PiA+ Cj4+PiA+IE9rLCBnb3QgaXQuIEhvdyB0byBjaGVjayB0aGlzIGFzIHdlbGw/Cj4+Pgo+Pj4KPj4+ IFRoZSBjb25zb2xlIGdldHMgYSBtZXNzYWdlIGxpa2U6Cj4+Pgo+Pj4gV0FSTklORzogL21udDog VFJJTSBmbGFnIG9uIGZzIGJ1dCBkaXNrIGRvZXMgbm90IHN1cHBvcnQgVFJJTQo+Pj4KPj4+IHdo ZW4gYSBtb3VudCBpcyBhdHRlbXB0ZWQgKGF1dG9tYXRpYyBvciBtYW51YWxseSkgZm9yIGEKPj4+ IGZpbGUgc3lzdGVtIHdpdGggdGhlIHRyaW0gZmxhZyBlbmFibGVkIGJ1dCB0cmltIGRvZXMgbm90 Cj4+PiBlbmQgdXAgYWN0aXZlLgo+Pj4KPj4+IFNvLCBmb3IgZXhhbXBsZToKPj4+Cj4+PiAjIGRt ZXNnIC1hIHwgZ3JlcCBUUklNCj4+PiBXQVJOSU5HOiAvbW50OiBUUklNIGZsYWcgb24gZnMgYnV0 IGRpc2sgZG9lcyBub3Qgc3VwcG9ydCBUUklNCj4+Pgo+Pj4gKFRoaXMgd2FzIGEgbWljcm9zZCBj YXJkIGluIGEgVVNCIHJlYWRlci93cml0ZXIgdGhhdCB3YXMgbm90Cj4+PiB1c2VkIGFzIHRoZSBi b290IG1lZGlhOiBhIHNlcGFyYXRlIG1hbnVhbCBtb3VudCB3YXMgdXNlZC4pCj4+Pgo+Pj4gVGhl IGZpbGUgc3lzdGVtJ3MgVFJJTSBmbGFnIHN0YXR1cyBjYW4gYmUgY2hlY2tlZCB2aWEgdXNlIG9m Cj4+PiAidHVuZWZzIC1wIC4gLiAuIjoKPj4+Cj4+PiAjIHR1bmVmcyAtcCAvbW50Cj4+PiB0dW5l ZnM6IFBPU0lYLjFlIEFDTHM6ICgtYSkgZGlzYWJsZWQKPj4+IHR1bmVmczogTkZTdjQgQUNMczog KC1OKSBkaXNhYmxlZAo+Pj4gdHVuZWZzOiBNQUMgbXVsdGlsYWJlbDogKC1sKSBkaXNhYmxlZAo+ Pj4gdHVuZWZzOiBzb2Z0IHVwZGF0ZXM6ICgtbikgZW5hYmxlZAo+Pj4gdHVuZWZzOiBzb2Z0IHVw ZGF0ZSBqb3VybmFsaW5nOiAoLWopIGRpc2FibGVkCj4+PiB0dW5lZnM6IGdqb3VybmFsOiAoLUop IGRpc2FibGVkCj4+PiB0dW5lZnM6IHRyaW06ICgtdCkgZW5hYmxlZAo+Pj4gdHVuZWZzOiBtYXhp bXVtIGJsb2NrcyBwZXIgZmlsZSBpbiBhIGN5bGluZGVyIGdyb3VwOiAoLWUpIDQwOTYKPj4+IHR1 bmVmczogYXZlcmFnZSBmaWxlIHNpemU6ICgtZikgMTYzODQKPj4+IHR1bmVmczogYXZlcmFnZSBu dW1iZXIgb2YgZmlsZXMgaW4gYSBkaXJlY3Rvcnk6ICgtcykgNjQKPj4+IHR1bmVmczogbWluaW11 bSBwZXJjZW50YWdlIG9mIGZyZWUgc3BhY2U6ICgtbSkgOCUKPj4+IHR1bmVmczogc3BhY2UgdG8g aG9sZCBmb3IgbWV0YWRhdGEgYmxvY2tzOiAoLWspIDY0MDAKPj4+IHR1bmVmczogb3B0aW1pemF0 aW9uIHByZWZlcmVuY2U6ICgtbykgdGltZQo+Pj4gdHVuZWZzOiB2b2x1bWUgbGFiZWw6ICgtTCkK Pj4+Cj4+PiBJZiB0aGUgdHJpbSBmbGFnIGlzIGVuYWJsZWQgYnV0IHRoZSBtb3VudCBkb2VzIG5v dCBwcm9kdWNlIHRoZQo+Pj4gY29uc29sZSBtZXNzYWdlLCB0aGVuIFRSSU0gaXMgYWN0aXZlLgo+ Pj4KPj4KPj4gT2gsIHRoYXQncyBhbWF6aW5nISBJIHRyeSBpdCB3aXRoIG15IFNhbkRpc2sgVWx0 cmEgbWljcm9TRFhDIDY0R0IgYW5kIG1vdW50IGl0IGluIGEgVVNCIHJlYWRlciBhbmQgaXQgc2hv d3MgZW5hYmxlZCBpbiB0aGUgdHVuZWZzIGFuZCBpcyBkZXRlY3RlZCBpbiB0aGUgZG1lc2csIHRo ZSBzYW1lIGFzIHlvdXJzLiBUaGVyZWZvcmUsIFNhbkRpc2sgVWx0cmEgbWljcm9TRCBzdXBwb3J0 cyBpdCEKPj4KPj4gcm9vdEBzZC1leHRyZW1lOn4gIyAvc2Jpbi9uZXdmcyAtVSAtdCAvZGV2L2Rh MHMyYQo+PiAvZGV2L2RhMHMyYTogNTkwMDAuOU1CICgxMjA4MzM5MjAgc2VjdG9ycykgYmxvY2sg c2l6ZSAzMjc2OCwgZnJhZ21lbnQgc2l6ZSA0MDk2Cj4+IHVzaW5nIDk1IGN5bGluZGVyIGdyb3Vw cyBvZiA2MjUuMjJNQiwgMjAwMDcgYmxrcywgODAxMjggaW5vZGVzLgo+PiB3aXRoIHNvZnQgdXBk YXRlcwo+PiBzdXBlci1ibG9jayBiYWNrdXBzIChmb3IgZnNja19mZnMgLWIgIykgYXQ6Cj4+IDE5 MiwgMTI4MDY0MCwgMjU2MTA4OCwgMzg0MTUzNiwgNTEyMTk4NCwgNjQwMjQzMiwgNzY4Mjg4MCwg ODk2MzMyOCwgMTAyNDM3NzYsIDExNTI0MjI0LCAxMjgwNDY3MiwgMTQwODUxMjAsCj4+IDE1MzY1 NTY4LCAxNjY0NjAxNiwgMTc5MjY0NjQsIDE5MjA2OTEyLCAyMDQ4NzM2MCwgMjE3Njc4MDgsIDIz MDQ4MjU2LCAyNDMyODcwNCwgMjU2MDkxNTIsIDI2ODg5NjAwLCAyODE3MDA0OCwKPj4gMjk0NTA0 OTYsIDMwNzMwOTQ0LCAzMjAxMTM5MiwgMzMyOTE4NDAsIDM0NTcyMjg4LCAzNTg1MjczNiwgMzcx MzMxODQsIDM4NDEzNjMyLCAzOTY5NDA4MCwgNDA5NzQ1MjgsIDQyMjU0OTc2LAo+PiA0MzUzNTQy NCwgNDQ4MTU4NzIsIDQ2MDk2MzIwLCA0NzM3Njc2OCwgNDg2NTcyMTYsIDQ5OTM3NjY0LCA1MTIx ODExMiwgNTI0OTg1NjAsIDUzNzc5MDA4LCA1NTA1OTQ1NiwgNTYzMzk5MDQsCj4+IDU3NjIwMzUy LCA1ODkwMDgwMCwgNjAxODEyNDgsIDYxNDYxNjk2LCA2Mjc0MjE0NCwgNjQwMjI1OTIsIDY1MzAz MDQwLCA2NjU4MzQ4OCwgNjc4NjM5MzYsIDY5MTQ0Mzg0LCA3MDQyNDgzMiwKPj4gNzE3MDUyODAs IDcyOTg1NzI4LCA3NDI2NjE3NiwgNzU1NDY2MjQsIDc2ODI3MDcyLCA3ODEwNzUyMCwgNzkzODc5 NjgsIDgwNjY4NDE2LCA4MTk0ODg2NCwgODMyMjkzMTIsIDg0NTA5NzYwLAo+PiA4NTc5MDIwOCwg ODcwNzA2NTYsIDg4MzUxMTA0LCA4OTYzMTU1MiwgOTA5MTIwMDAsIDkyMTkyNDQ4LCA5MzQ3Mjg5 NiwgOTQ3NTMzNDQsIDk2MDMzNzkyLCA5NzMxNDI0MCwgOTg1OTQ2ODgsCj4+IDk5ODc1MTM2LCAx MDExNTU1ODQsIDEwMjQzNjAzMiwgMTAzNzE2NDgwLCAxMDQ5OTY5MjgsIDEwNjI3NzM3NiwgMTA3 NTU3ODI0LCAxMDg4MzgyNzIsIDExMDExODcyMCwgMTExMzk5MTY4LAo+PiAxMTI2Nzk2MTYsIDEx Mzk2MDA2NCwgMTE1MjQwNTEyLCAxMTY1MjA5NjAsIDExNzgwMTQwOCwgMTE5MDgxODU2LCAxMjAz NjIzMDQKPj4KPj4gcm9vdEBzZC1leHRyZW1lOn4gIyBkbWVzZyB8IGdyZXAgVFJJTQo+PiBXQVJO SU5HOiAvbW50OiBUUklNIGZsYWcgb24gZnMgYnV0IGRpc2sgZG9lcyBub3Qgc3VwcG9ydCBUUklN Cj4+Cj4+IHJvb3RAc2QtZXh0cmVtZTp+ICMgdHVuZWZzIC1wIC9tbnQKPj4gdHVuZWZzOiBQT1NJ WC4xZSBBQ0xzOiAoLWEpIGRpc2FibGVkCj4+IHR1bmVmczogTkZTdjQgQUNMczogKC1OKSBkaXNh YmxlZAo+PiB0dW5lZnM6IE1BQyBtdWx0aWxhYmVsOiAoLWwpIGRpc2FibGVkCj4+IHR1bmVmczog c29mdCB1cGRhdGVzOiAoLW4pIGVuYWJsZWQKPj4gdHVuZWZzOiBzb2Z0IHVwZGF0ZSBqb3VybmFs aW5nOiAoLWopIGRpc2FibGVkCj4+IHR1bmVmczogZ2pvdXJuYWw6ICgtSikgZGlzYWJsZWQKPj4g dHVuZWZzOiB0cmltOiAoLXQpIGVuYWJsZWQKPj4gdHVuZWZzOiBtYXhpbXVtIGJsb2NrcyBwZXIg ZmlsZSBpbiBhIGN5bGluZGVyIGdyb3VwOiAoLWUpIDQwOTYKPj4gdHVuZWZzOiBhdmVyYWdlIGZp bGUgc2l6ZTogKC1mKSAxNjM4NAo+PiB0dW5lZnM6IGF2ZXJhZ2UgbnVtYmVyIG9mIGZpbGVzIGlu IGEgZGlyZWN0b3J5OiAoLXMpIDY0Cj4+IHR1bmVmczogbWluaW11bSBwZXJjZW50YWdlIG9mIGZy ZWUgc3BhY2U6ICgtbSkgOCUKPj4gdHVuZWZzOiBzcGFjZSB0byBob2xkIGZvciBtZXRhZGF0YSBi bG9ja3M6ICgtaykgNjQwMAo+PiB0dW5lZnM6IG9wdGltaXphdGlvbiBwcmVmZXJlbmNlOiAoLW8p IHRpbWUKPj4gdHVuZWZzOiB2b2x1bWUgbGFiZWw6ICgtTCkKPj4KPj4gSSBwbGFuIHRvIGhhdmUg VFJJTSBmbGFnIGVuYWJsZWQgaW4gdGhlIHJvb3RmcyBwYXJ0aXRpb24gKC9kZXYvdWZzL3Jvb3Rm cykgb2YgbXkgUmFzcGJlcnJ5IFBpLiBEbyB5b3UgdGhpbmsgb2YgYW55IGltcGxpY2F0aW9ucyB3 aGVuIGVuYWJsZWQ/IEFzIHNob3duIGJlbG93LCBpdCBpcyBkaXNhYmxlZC4KPj4KPj4gcm9vdEBz ZC1leHRyZW1lOn4gIyB0dW5lZnMgLXAgL2Rldi91ZnMvcm9vdGZzCj4+IHR1bmVmczogUE9TSVgu MWUgQUNMczogKC1hKSBkaXNhYmxlZAo+PiB0dW5lZnM6IE5GU3Y0IEFDTHM6ICgtTikgZGlzYWJs ZWQKPj4gdHVuZWZzOiBNQUMgbXVsdGlsYWJlbDogKC1sKSBkaXNhYmxlZAo+PiB0dW5lZnM6IHNv ZnQgdXBkYXRlczogKC1uKSBlbmFibGVkCj4+IHR1bmVmczogc29mdCB1cGRhdGUgam91cm5hbGlu ZzogKC1qKSBkaXNhYmxlZAo+PiB0dW5lZnM6IGdqb3VybmFsOiAoLUopIGRpc2FibGVkCj4+IHR1 bmVmczogdHJpbTogKC10KSBkaXNhYmxlZAo+PiB0dW5lZnM6IG1heGltdW0gYmxvY2tzIHBlciBm aWxlIGluIGEgY3lsaW5kZXIgZ3JvdXA6ICgtZSkgNDA5Ngo+PiB0dW5lZnM6IGF2ZXJhZ2UgZmls ZSBzaXplOiAoLWYpIDE2Mzg0Cj4+IHR1bmVmczogYXZlcmFnZSBudW1iZXIgb2YgZmlsZXMgaW4g YSBkaXJlY3Rvcnk6ICgtcykgNjQKPj4gdHVuZWZzOiBtaW5pbXVtIHBlcmNlbnRhZ2Ugb2YgZnJl ZSBzcGFjZTogKC1tKSA4JQo+PiB0dW5lZnM6IHNwYWNlIHRvIGhvbGQgZm9yIG1ldGFkYXRhIGJs b2NrczogKC1rKSA2NDAwCj4+IHR1bmVmczogb3B0aW1pemF0aW9uIHByZWZlcmVuY2U6ICgtbykg dGltZQo+PiB0dW5lZnM6IHZvbHVtZSBsYWJlbDogKC1MKSByb290ZnMKPj4KPj4gQlIsCj4+IG9y Yml0Cj4+Cj4+IC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLQo+Cj4gSGksCj4KPiBJJ20gc29ycnkgdG8gc3BvaWwgdGhlIGZ1biBi dXQgdGhpcyBtZXNzYWdlIHNheXMgdGhlIGRldmljZSBkb2VzICpub3QqIHN1cHBvcnQgdHJpbS4K Pgo+ICJyb290QHNkLWV4dHJlbWU6fiAjIGRtZXNnIHwgZ3JlcCBUUklNCj4gV0FSTklORzogL21u dDogVFJJTSBmbGFnIG9uIGZzIGJ1dCBkaXNrIGRvZXMgbm90IHN1cHBvcnQgVFJJTSIKPgo+IEJ1 dCB0byBnaXZlIHNvbWUgaG9wZS4gQXMgTWFyayBwb2ludGVkIG91dCBpdCBpcyBub3Qgb25seSBh Ym91dCB0aGUgZGlzay4gVGhlIGNvbnRyb2xsZXIgYWxzbyBuZWVkcyB0byBzdXBwb3J0IGl0Lgo+ Cj4gQXMgYSByZWFsIHdvcmxkIGV4YW1wbGU6Cj4KPiBJIGhhdmUgYW4gUlBJMyB3aXRoIGFuIFNT RCAoc3VwcG9ydGluZyB0cmltKSBjb25uZWN0ZWQgdmlhIFVTQi10by1TQVRBIChub3Qgc3VwcG9y dGluZyB0cmltKS4gU28gdGhlIHRvdGFsIHN5c3RlbSBkb2VzIG5vdCBzdXBwb3J0IHRyaW0uCj4g JCBjYXQgL3Zhci9ydW4vZG1lc2cuYm9vdCB8IGdyZXAgLUUgInVtYXNzMHxkYTB8VFJJTSIKPiB1 bWFzczAgb24gdWh1YjEKPiB1bWFzczA6IDxHT0RPIFVTQjMuMCB0byBTQVRBIGFkYXB0ZXIsIGNs YXNzIDAvMCwgcmV2IDIuMTAvMS4wMCwgYWRkciA1PiBvbiB1c2J1czEKPiB1bWFzczA6IFNDU0kg b3ZlciBCdWxrLU9ubHk7IHF1aXJrcyA9IDB4MDEwMAo+IHVtYXNzMDowOjA6IEF0dGFjaGVkIHRv IHNjYnVzMAo+IGRhMCBhdCB1bWFzcy1zaW0wIGJ1cyAwIHNjYnVzMCB0YXJnZXQgMCBsdW4gMAo+ IGRhMDogPExJVEVPTiBDIFYzLURFMjU2IDA+IEZpeGVkIERpcmVjdCBBY2Nlc3MgU1BDLTQgU0NT SSBkZXZpY2UKPiBkYTA6IFNlcmlhbCBOdW1iZXIgMjAxNTEwMDAwMEQ2Cj4gZGEwOiA0MC4wMDBN Qi9zIHRyYW5zZmVycwo+IGRhMDogMjQ0MTk4TUIgKDUwMDExODE5MiA1MTIgYnl0ZSBzZWN0b3Jz KQo+IGRhMDogcXVpcmtzPTB4MjxOT182X0JZVEU+Cj4gV0FSTklORzogLzogVFJJTSBmbGFnIG9u IGZzIGJ1dCBkaXNrIGRvZXMgbm90IHN1cHBvcnQgVFJJTQo+Cj4gJCBzeXNjdGwga2Vybi5jYW0u ZGEuMCB8IGdyZXAgLUUgInRyaW18ZGVsZXRlIgo+IGtlcm4uY2FtLmRhLjAudHJpbV90aWNrczog MAo+IGtlcm4uY2FtLmRhLjAudHJpbV9nb2FsOiAwCj4ga2Vybi5jYW0uZGEuMC50cmltX2xiYXM6 IDAKPiBrZXJuLmNhbS5kYS4wLnRyaW1fcmFuZ2VzOiAwCj4ga2Vybi5jYW0uZGEuMC50cmltX2Nv dW50OiAwCj4ga2Vybi5jYW0uZGEuMC5kZWxldGVfbWF4OiA2NTUzNgo+IGtlcm4uY2FtLmRhLjAu ZGVsZXRlX21ldGhvZDogTk9ORQo+Cj4gT24gYW4gUlBJNEIgSSBoYXZlIGFuIFNTRCAoc3VwcG9y dGluZyB0cmltKSBjb25uZWN0ZWQgdmlhIFVTQi10by1TQVRBICh3aGljaCBkb2VzIHN1cHBvcnQg dHJpbSBhbHNvKS4gSGVyZSB0cmltIHdvcmtzIGZpbmUuIChOQjogdGhpcyB1c2VzIFpGUywgYnV0 IHRoYXQgZG9lcyBub3QgbWF0dGVyKQo+ICQgZG1lc2cgfCBncmVwIC1FICJ1bWFzczB8ZGEwIgo+ IHVtYXNzMCBvbiB1aHViMAo+IHVtYXNzMDogPFVTQiAzLjAgRGV2aWNlIFVTQiAzLjAgRGV2aWNl LCBjbGFzcyAwLzAsIHJldiAzLjAwLzMuMDEsIGFkZHIgMz4gb24gdXNidXMwCj4gZGEwIGF0IHVt YXNzLXNpbTAgYnVzIDAgc2NidXMwIHRhcmdldCAwIGx1biAwCj4gZGEwOiA8TWljcm9uIDEgMTAw IFNBVEEgMjU2R0IgMDMwMT4gRml4ZWQgRGlyZWN0IEFjY2VzcyBTUEMtNCBTQ1NJIGRldmljZQo+ IGRhMDogU2VyaWFsIE51bWJlciAwMDAwMDAwMDRCQTgKPiBkYTA6IDQwMC4wMDBNQi9zIHRyYW5z ZmVycwo+IGRhMDogMjQ0MTk4TUIgKDUwMDExODE5MiA1MTIgYnl0ZSBzZWN0b3JzKQo+IGRhMDog cXVpcmtzPTB4MjxOT182X0JZVEU+Cj4KPiAkIHN5c2N0bCBrZXJuLmNhbS5kYS4wIHwgZ3JlcCAt RSAidHJpbXxkZWxldGUiCj4ga2Vybi5jYW0uZGEuMC50cmltX3RpY2tzOiAwCj4ga2Vybi5jYW0u ZGEuMC50cmltX2dvYWw6IDAKPiBrZXJuLmNhbS5kYS4wLnRyaW1fbGJhczogNTQ2ODQxNzA0Cj4g a2Vybi5jYW0uZGEuMC50cmltX3JhbmdlczogMzE3MjgwCj4ga2Vybi5jYW0uZGEuMC50cmltX2Nv dW50OiAzMDE0MjQKPiBrZXJuLmNhbS5kYS4wLmRlbGV0ZV9tYXg6IDQyOTQ5MDE3NjAKPiBrZXJu LmNhbS5kYS4wLmRlbGV0ZV9tZXRob2Q6IFVOTUFQCj4KPiBCVFc6IGVuYWJsaW5nIHRyaW0gb24g dGhlIGZzIHdoaWxlIHRoZSBkZXZpY2UgZG9lcyBub3Qgc3VwcG9ydCBpdCBkb2VzIG5vdCBtYXR0 ZXIgYW5kIGRvZXMgbm8gaGFybS4gVXNpbmcgdGhlIHN5c2N0bCBhYm92ZSB5b3UgY2FuIHNlZSBp ZiB0aGUgZGV2aWNlIGFjdHVhbGx5IHVzZXMgdHJpbS4KPgo+IFJ1bm5pbmcgImdzdGF0IC1kIiB3 aGlsZSBkZWxldGluZyBhIGxvdCBvZiBmaWxlcyB3aWxsIGFsc28gc2hvdyBpZiB0cmltL2RlbGV0 ZSBhY3Rpb25zIGFyZSBzZW50IHRvIHRoZSBkaXNrLgo+Cj4gU28gdGhlIGNvbWJpbmF0aW9uIG9m IHlvdXIgVVNCIHJlYWRlciArIG1pY3JvU0QgZG8gbm90IHN1cHBvcnQgVFJJTSwgYnV0IHlvdSBj YW4ganVzdCB0cnkgdG8gcHV0IHRoZSBtaWNyb1NEIGRpcmVjdGx5IGluIHRoZSBSUEk0IGFuZCBz ZWUgd2hhdCB0aGF0IGRvZXMuCj4KPiBSZWdhcmRzLAo+IFJvbmFsZC4KClRoYW5rcyBmb3IgdGhl IGNhbGxvdXQhIEdvdCB3YXJtZWQtdXAgd2hlbiBJIHNhdyB0aGUgVFJJTSBpcyBlbmFibGVkIDot KSBidXQgbXkgdWx0aW1hdGUgZ29hbCBpcyByZWFsbHkgdGhlIG1pY3JvU0QgY2FyZCBzbG90IG9m IG15IFJhc3BiZXJyeSBQaSAzIG9yIDQgYW5kIG5vdCB3aXRoIHRoZSBVU0IgcmVhZGVyLiBJJ2xs IGNyZWF0ZSBhbiBpbWFnZSB0aGVuIHNlZSBob3cgaXQgZG9lcy4KCkJSLApvcmJpdA== --b1_i4TVXaKbceUP6PQinOcTun6iLjvwKymRhlhbJDABQI Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: base64 T24gRnJpZGF5LCAxNiBGZWJydWFyeSAyMDI0IGF0IDE4OjMxLCBSb25hbGQgS2xvcCAmbHQ7cm9u YWxkLWxpc3RzQGtsb3Aud3MmZ3Q7IHdyb3RlOg0KICAgICAgICA8ZGl2IGNsYXNzPSJwcm90b25t YWlsX3F1b3RlIj48YmxvY2txdW90ZSBjbGFzcz0icHJvdG9ubWFpbF9xdW90ZSIgdHlwZT0iY2l0 ZSI+DQogICAgICAgICAgICA8YnI+DQo8cD48c3Ryb25nPlZhbjo8L3N0cm9uZz4gT3JkaW5hcnkg Qml0ICZsdDtvcmRpbmFyeWJpdEBwcm90b24ubWUmZ3Q7PGJyPg0KPHN0cm9uZz5EYXR1bTo8L3N0 cm9uZz4gdnJpamRhZywgMTYgZmVicnVhcmkgMjAyNCAxMDoxODxicj4NCjxzdHJvbmc+QWFuOjwv c3Ryb25nPiBNYXJrIE1pbGxhcmQgJmx0O21hcmtsbWlAeWFob28uY29tJmd0Ozxicj4NCjxzdHJv bmc+Q0M6PC9zdHJvbmc+ICJmcmVlYnNkLWFybUBmcmVlYnNkLm9yZyIgJmx0O2ZyZWVic2QtYXJt QGZyZWVic2Qub3JnJmd0Ozxicj4NCjxzdHJvbmc+T25kZXJ3ZXJwOjwvc3Ryb25nPiBSZTogbmV3 ZnMgVFJJTSBmbGFnIGRldmljZSBzdXBwb3J0PC9wPg0KDQo8YmxvY2txdW90ZSBzdHlsZT0icGFk ZGluZy1yaWdodDogMHB4OyBwYWRkaW5nLWxlZnQ6IDVweDsgbWFyZ2luLWxlZnQ6IDVweDsgYm9y ZGVyLWxlZnQ6ICMwMDAwMDAgMnB4IHNvbGlkOyBtYXJnaW4tcmlnaHQ6IDBweCI+DQo8ZGl2IGlk PSJQIiBjbGFzcz0iTWVzc2FnZVJGQzgyMlZpZXdlciI+DQo8ZGl2IGlkPSJQLlAiIGNsYXNzPSJU ZXh0UGxhaW5WaWV3ZXIiPjxicj4NCjxicj4NCjxicj4NCjxicj4NCjxicj4NClNlbnQgd2l0aCBQ cm90b24gTWFpbCBzZWN1cmUgZW1haWwuPGJyPg0KPGJyPg0KT24gRnJpZGF5LCAxNiBGZWJydWFy eSAyMDI0IGF0IDE0OjAyLCBNYXJrIE1pbGxhcmQgJmx0O21hcmtsbWlAeWFob28uY29tJmd0OyB3 cm90ZTo8YnI+DQo8YnI+DQomZ3Q7IE9uIEZlYiAxNSwgMjAyNCwgYXQgMjA6MDgsIE9yZGluYXJ5 IEJpdCBvcmRpbmFyeWJpdEBwcm90b24ubWUgd3JvdGU6PGJyPg0KJmd0Ozxicj4NCiZndDsgJmd0 OyBIaSE8YnI+DQomZ3Q7PGJyPg0KJmd0Ozxicj4NCiZndDsgSGVsbG8uPGJyPg0KJmd0Ozxicj4N CiZndDsgJmd0OyBPbiBGcmlkYXksIDE2IEZlYnJ1YXJ5IDIwMjQgYXQgMTE6NDEsIE1hcmsgTWls bGFyZCBtYXJrbG1pQHlhaG9vLmNvbSB3cm90ZTo8YnI+DQomZ3Q7ICZndDs8YnI+DQomZ3Q7ICZn dDsgJmd0OyBbT25seSByZXBseWluZyB0byBsaXN0cyBJIHN1YnNjcmliZSB0by5dPGJyPg0KJmd0 OyAmZ3Q7ICZndDs8YnI+DQomZ3Q7ICZndDsgJmd0OyBPbiBGZWIgMTUsIDIwMjQsIGF0IDE5OjE5 LCBPcmRpbmFyeSBCaXQgb3JkaW5hcnliaXRAcHJvdG9uLm1lIHdyb3RlOjxicj4NCiZndDsgJmd0 OyAmZ3Q7PGJyPg0KJmd0OyAmZ3Q7ICZndDsgJmd0OyBJJ20gcmVhZGluZyB0aGUgbmV3ZnMgbWFu dWFsIDxhIGhyZWY9Imh0dHBzOi8vbWFuLmZyZWVic2Qub3JnL2NnaS9tYW4uY2dpP25ld2ZzKDgi IHJlbD0ibm9yZWZlcnJlciBub2ZvbGxvdyBub29wZW5lciIgdGFyZ2V0PSJfYmxhbmsiPmh0dHBz Oi8vbWFuLmZyZWVic2Qub3JnL2NnaS9tYW4uY2dpP25ld2ZzKDg8L2E+KSB0byBiZSBhYmxlIHRv IGtub3cgYWJvdXQgdGhlIFRSSU0gZmxhZy4gSW4gdGhlIG1hbnVhbCB1bmRlciAtdCBwYXJhbWV0 ZXIsIGl0IG1lbnRpb25lZCBhYm91dCAidW5kZXJseWluZyBkZXZpY2Ugc3VwcG9ydCIsIHdoYXQg ZXhhY3RseSBpcyB0aGlzIGRldmljZT88YnI+DQomZ3Q7ICZndDsgJmd0Ozxicj4NCiZndDsgJmd0 OyAmZ3Q7IDIgY29udHJhc3RpbmcgZXhhbXBsZXM6PGJyPg0KJmd0OyAmZ3Q7ICZndDs8YnI+DQom Z3Q7ICZndDsgJmd0OyBFeGFtcGxlIDA6IE9wdGFuZSBOVk1lIG1lZGlhIChQQ0llIGNhcmQgb3Ig VS4yLCBmb3IgZXhhbXBsZSk8YnI+DQomZ3Q7ICZndDsgJmd0Ozxicj4NCiZndDsgJmd0OyAmZ3Q7 IE9wdGFuZSBoYXMgbm8gbmVlZCBvZiBUUklNIGFuZCwgc28sIG5ldmVyIHN1cHBvcnRzIFRSSU0u PGJyPg0KJmd0OyAmZ3Q7ICZndDs8YnI+DQomZ3Q7ICZndDsgJmd0OyBFeGFtcGxlIDE6IG1pY3Jv c2QgY2FyZCBtZWRpYSB1c2FnZTxicj4NCiZndDsgJmd0OyAmZ3Q7PGJyPg0KJmd0OyAmZ3Q7ICZn dDsgQSBtaWNyb3NkIGNhcmQgaW4gdGhlIG5vcm1hbCB0eXBlIG9mIG1pY3Jvc2QgY2FyZCBzbG90 IG9uPGJyPg0KJmd0OyAmZ3Q7ICZndDsgU21hbGwgQm9hcmQgQ29tcHV0ZXJzIChub3JtYWxseSkg c3VwcG9ydHMgVFJJTS4gVGFrZSB0aGU8YnI+DQomZ3Q7ICZndDsgJmd0OyBzYW1lIGNhcmQgYW5k IHB1dCBpdCBpbiBhIFVTQiByZWFkZXIvd3JpdGVyIGFuZCB1c2UgaXQ8YnI+DQomZ3Q7ICZndDsg Jmd0OyB2aWEgVVNCIG9uIHRoZSBzYW1lIHN5c3RlbTogbm8gVFJJTSBpcyBzdXBwb3J0ZWQgYnk8 YnI+DQomZ3Q7ICZndDsgJmd0OyBGcmVlQlNEIG92ZXIgVVNCLjxicj4NCiZndDsgJmd0Ozxicj4N CiZndDsgJmd0OyBTbyB5b3UgbWVhbiB0byBzYXkgdGhhdCBpZiBJIGhhdmUgYSBSYXNwZXJyeSBQ aSAzIG9yIDQgbm93IGFuZCB0aGVuIGhhdmUgbXkgRnJlZUJTRCBpbnN0YWxsZWQgaW4gYSBtaWNy b1NEIGNhcmQgKGZvciBleGFtcGxlLCBTYW5EaXNrIEV4dHJlbWUgY2FyZCkgd2l0aCBVRlMvRkZT IGZpbGVzeXRlbSBpbiBpdCB3aXRoIFRSSU0gZW5hYmxlZCBwYXJhbWV0ZXIgdGhlbiBpcyBpdCBn b2luZyB0byByZWNvZ25pemUgaXQ/IEhvdyB0byB2ZXJpZnk/PGJyPg0KJmd0OyAmZ3Q7PGJyPg0K Jmd0OyAmZ3Q7ICZndDsgRllJOjxicj4NCiZndDsgJmd0OyAmZ3Q7PGJyPg0KJmd0OyAmZ3Q7ICZn dDsgV2hlbiB0aGUgZmlsZSBzeXN0ZW0gaGFzIFRSSU0gZW5hYmxlZCwgRnJlZUJTRCBwdXQgb3V0 IGE8YnI+DQomZ3Q7ICZndDsgJmd0OyBub3RpY2UgaWYgVFJJTSB3aWxsIG5vdCBhY3R1YWxseSBi ZSB1c2VkIGluIHRoZSBhY3R1YWw8YnI+DQomZ3Q7ICZndDsgJmd0OyBjb250ZXh0IGluIHVzZS48 YnI+DQomZ3Q7ICZndDs8YnI+DQomZ3Q7ICZndDsgT2ssIGdvdCBpdC4gSG93IHRvIGNoZWNrIHRo aXMgYXMgd2VsbD88YnI+DQomZ3Q7PGJyPg0KJmd0Ozxicj4NCiZndDsgVGhlIGNvbnNvbGUgZ2V0 cyBhIG1lc3NhZ2UgbGlrZTo8YnI+DQomZ3Q7PGJyPg0KJmd0OyBXQVJOSU5HOiAvbW50OiBUUklN IGZsYWcgb24gZnMgYnV0IGRpc2sgZG9lcyBub3Qgc3VwcG9ydCBUUklNPGJyPg0KJmd0Ozxicj4N CiZndDsgd2hlbiBhIG1vdW50IGlzIGF0dGVtcHRlZCAoYXV0b21hdGljIG9yIG1hbnVhbGx5KSBm b3IgYTxicj4NCiZndDsgZmlsZSBzeXN0ZW0gd2l0aCB0aGUgdHJpbSBmbGFnIGVuYWJsZWQgYnV0 IHRyaW0gZG9lcyBub3Q8YnI+DQomZ3Q7IGVuZCB1cCBhY3RpdmUuPGJyPg0KJmd0Ozxicj4NCiZn dDsgU28sIGZvciBleGFtcGxlOjxicj4NCiZndDs8YnI+DQomZ3Q7ICMgZG1lc2cgLWEgfCBncmVw IFRSSU08YnI+DQomZ3Q7IFdBUk5JTkc6IC9tbnQ6IFRSSU0gZmxhZyBvbiBmcyBidXQgZGlzayBk b2VzIG5vdCBzdXBwb3J0IFRSSU08YnI+DQomZ3Q7PGJyPg0KJmd0OyAoVGhpcyB3YXMgYSBtaWNy b3NkIGNhcmQgaW4gYSBVU0IgcmVhZGVyL3dyaXRlciB0aGF0IHdhcyBub3Q8YnI+DQomZ3Q7IHVz ZWQgYXMgdGhlIGJvb3QgbWVkaWE6IGEgc2VwYXJhdGUgbWFudWFsIG1vdW50IHdhcyB1c2VkLik8 YnI+DQomZ3Q7PGJyPg0KJmd0OyBUaGUgZmlsZSBzeXN0ZW0ncyBUUklNIGZsYWcgc3RhdHVzIGNh biBiZSBjaGVja2VkIHZpYSB1c2Ugb2Y8YnI+DQomZ3Q7ICJ0dW5lZnMgLXAgLiAuIC4iOjxicj4N CiZndDs8YnI+DQomZ3Q7ICMgdHVuZWZzIC1wIC9tbnQ8YnI+DQomZ3Q7IHR1bmVmczogUE9TSVgu MWUgQUNMczogKC1hKSBkaXNhYmxlZDxicj4NCiZndDsgdHVuZWZzOiBORlN2NCBBQ0xzOiAoLU4p IGRpc2FibGVkPGJyPg0KJmd0OyB0dW5lZnM6IE1BQyBtdWx0aWxhYmVsOiAoLWwpIGRpc2FibGVk PGJyPg0KJmd0OyB0dW5lZnM6IHNvZnQgdXBkYXRlczogKC1uKSBlbmFibGVkPGJyPg0KJmd0OyB0 dW5lZnM6IHNvZnQgdXBkYXRlIGpvdXJuYWxpbmc6ICgtaikgZGlzYWJsZWQ8YnI+DQomZ3Q7IHR1 bmVmczogZ2pvdXJuYWw6ICgtSikgZGlzYWJsZWQ8YnI+DQomZ3Q7IHR1bmVmczogdHJpbTogKC10 KSBlbmFibGVkPGJyPg0KJmd0OyB0dW5lZnM6IG1heGltdW0gYmxvY2tzIHBlciBmaWxlIGluIGEg Y3lsaW5kZXIgZ3JvdXA6ICgtZSkgNDA5Njxicj4NCiZndDsgdHVuZWZzOiBhdmVyYWdlIGZpbGUg c2l6ZTogKC1mKSAxNjM4NDxicj4NCiZndDsgdHVuZWZzOiBhdmVyYWdlIG51bWJlciBvZiBmaWxl cyBpbiBhIGRpcmVjdG9yeTogKC1zKSA2NDxicj4NCiZndDsgdHVuZWZzOiBtaW5pbXVtIHBlcmNl bnRhZ2Ugb2YgZnJlZSBzcGFjZTogKC1tKSA4JTxicj4NCiZndDsgdHVuZWZzOiBzcGFjZSB0byBo b2xkIGZvciBtZXRhZGF0YSBibG9ja3M6ICgtaykgNjQwMDxicj4NCiZndDsgdHVuZWZzOiBvcHRp bWl6YXRpb24gcHJlZmVyZW5jZTogKC1vKSB0aW1lPGJyPg0KJmd0OyB0dW5lZnM6IHZvbHVtZSBs YWJlbDogKC1MKTxicj4NCiZndDs8YnI+DQomZ3Q7IElmIHRoZSB0cmltIGZsYWcgaXMgZW5hYmxl ZCBidXQgdGhlIG1vdW50IGRvZXMgbm90IHByb2R1Y2UgdGhlPGJyPg0KJmd0OyBjb25zb2xlIG1l c3NhZ2UsIHRoZW4gVFJJTSBpcyBhY3RpdmUuPGJyPg0KJmd0Ozxicj4NCjxicj4NCk9oLCB0aGF0 J3MgYW1hemluZyEgSSB0cnkgaXQgd2l0aCBteSBTYW5EaXNrIFVsdHJhIG1pY3JvU0RYQyA2NEdC IGFuZCBtb3VudCBpdCBpbiBhIFVTQiByZWFkZXIgYW5kIGl0IHNob3dzIGVuYWJsZWQgaW4gdGhl IHR1bmVmcyBhbmQgaXMgZGV0ZWN0ZWQgaW4gdGhlIGRtZXNnLCB0aGUgc2FtZSBhcyB5b3Vycy4g VGhlcmVmb3JlLCBTYW5EaXNrIFVsdHJhIG1pY3JvU0Qgc3VwcG9ydHMgaXQhPGJyPg0KPGJyPg0K cm9vdEBzZC1leHRyZW1lOn4gIyAvc2Jpbi9uZXdmcyAtVSAtdCAvZGV2L2RhMHMyYTxicj4NCi9k ZXYvZGEwczJhOiA1OTAwMC45TUIgKDEyMDgzMzkyMCBzZWN0b3JzKSBibG9jayBzaXplIDMyNzY4 LCBmcmFnbWVudCBzaXplIDQwOTY8YnI+DQombmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsm bmJzcDsmbmJzcDsmbmJzcDt1c2luZyA5NSBjeWxpbmRlciBncm91cHMgb2YgNjI1LjIyTUIsIDIw MDA3IGJsa3MsIDgwMTI4IGlub2Rlcy48YnI+DQombmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJz cDsmbmJzcDsmbmJzcDsmbmJzcDt3aXRoIHNvZnQgdXBkYXRlczxicj4NCnN1cGVyLWJsb2NrIGJh Y2t1cHMgKGZvciBmc2NrX2ZmcyAtYiAjKSBhdDo8YnI+DQombmJzcDsxOTIsIDEyODA2NDAsIDI1 NjEwODgsIDM4NDE1MzYsIDUxMjE5ODQsIDY0MDI0MzIsIDc2ODI4ODAsIDg5NjMzMjgsIDEwMjQz Nzc2LCAxMTUyNDIyNCwgMTI4MDQ2NzIsIDE0MDg1MTIwLDxicj4NCiZuYnNwOzE1MzY1NTY4LCAx NjY0NjAxNiwgMTc5MjY0NjQsIDE5MjA2OTEyLCAyMDQ4NzM2MCwgMjE3Njc4MDgsIDIzMDQ4MjU2 LCAyNDMyODcwNCwgMjU2MDkxNTIsIDI2ODg5NjAwLCAyODE3MDA0OCw8YnI+DQombmJzcDsyOTQ1 MDQ5NiwgMzA3MzA5NDQsIDMyMDExMzkyLCAzMzI5MTg0MCwgMzQ1NzIyODgsIDM1ODUyNzM2LCAz NzEzMzE4NCwgMzg0MTM2MzIsIDM5Njk0MDgwLCA0MDk3NDUyOCwgNDIyNTQ5NzYsPGJyPg0KJm5i c3A7NDM1MzU0MjQsIDQ0ODE1ODcyLCA0NjA5NjMyMCwgNDczNzY3NjgsIDQ4NjU3MjE2LCA0OTkz NzY2NCwgNTEyMTgxMTIsIDUyNDk4NTYwLCA1Mzc3OTAwOCwgNTUwNTk0NTYsIDU2MzM5OTA0LDxi cj4NCiZuYnNwOzU3NjIwMzUyLCA1ODkwMDgwMCwgNjAxODEyNDgsIDYxNDYxNjk2LCA2Mjc0MjE0 NCwgNjQwMjI1OTIsIDY1MzAzMDQwLCA2NjU4MzQ4OCwgNjc4NjM5MzYsIDY5MTQ0Mzg0LCA3MDQy NDgzMiw8YnI+DQombmJzcDs3MTcwNTI4MCwgNzI5ODU3MjgsIDc0MjY2MTc2LCA3NTU0NjYyNCwg NzY4MjcwNzIsIDc4MTA3NTIwLCA3OTM4Nzk2OCwgODA2Njg0MTYsIDgxOTQ4ODY0LCA4MzIyOTMx MiwgODQ1MDk3NjAsPGJyPg0KJm5ic3A7ODU3OTAyMDgsIDg3MDcwNjU2LCA4ODM1MTEwNCwgODk2 MzE1NTIsIDkwOTEyMDAwLCA5MjE5MjQ0OCwgOTM0NzI4OTYsIDk0NzUzMzQ0LCA5NjAzMzc5Miwg OTczMTQyNDAsIDk4NTk0Njg4LDxicj4NCiZuYnNwOzk5ODc1MTM2LCAxMDExNTU1ODQsIDEwMjQz NjAzMiwgMTAzNzE2NDgwLCAxMDQ5OTY5MjgsIDEwNjI3NzM3NiwgMTA3NTU3ODI0LCAxMDg4Mzgy NzIsIDExMDExODcyMCwgMTExMzk5MTY4LDxicj4NCiZuYnNwOzExMjY3OTYxNiwgMTEzOTYwMDY0 LCAxMTUyNDA1MTIsIDExNjUyMDk2MCwgMTE3ODAxNDA4LCAxMTkwODE4NTYsIDEyMDM2MjMwNDxi cj4NCjxicj4NCnJvb3RAc2QtZXh0cmVtZTp+ICMgZG1lc2cgfCBncmVwIFRSSU08YnI+DQpXQVJO SU5HOiAvbW50OiBUUklNIGZsYWcgb24gZnMgYnV0IGRpc2sgZG9lcyBub3Qgc3VwcG9ydCBUUklN PGJyPg0KPGJyPg0Kcm9vdEBzZC1leHRyZW1lOn4gIyB0dW5lZnMgLXAgL21udDxicj4NCnR1bmVm czogUE9TSVguMWUgQUNMczogKC1hKSAmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJz cDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsm bmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJz cDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDtkaXNhYmxlZDxicj4NCnR1bmVm czogTkZTdjQgQUNMczogKC1OKSAmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsm bmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJz cDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsm bmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDtkaXNh YmxlZDxicj4NCnR1bmVmczogTUFDIG11bHRpbGFiZWw6ICgtbCkgJm5ic3A7Jm5ic3A7Jm5ic3A7 Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5i c3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7 Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7ZGlzYWJsZWQ8 YnI+DQp0dW5lZnM6IHNvZnQgdXBkYXRlczogKC1uKSAmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsm bmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJz cDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsm bmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDtlbmFi bGVkPGJyPg0KdHVuZWZzOiBzb2Z0IHVwZGF0ZSBqb3VybmFsaW5nOiAoLWopICZuYnNwOyZuYnNw OyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZu YnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNw OyZuYnNwO2Rpc2FibGVkPGJyPg0KdHVuZWZzOiBnam91cm5hbDogKC1KKSAmbmJzcDsmbmJzcDsm bmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJz cDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsm bmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJz cDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDtkaXNhYmxlZDxicj4NCnR1bmVmczogdHJp bTogKC10KSAmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsm bmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJz cDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsm bmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJz cDsmbmJzcDsmbmJzcDsmbmJzcDtlbmFibGVkPGJyPg0KdHVuZWZzOiBtYXhpbXVtIGJsb2NrcyBw ZXIgZmlsZSBpbiBhIGN5bGluZGVyIGdyb3VwOiAoLWUpICZuYnNwOzQwOTY8YnI+DQp0dW5lZnM6 IGF2ZXJhZ2UgZmlsZSBzaXplOiAoLWYpICZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZu YnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNw OyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZu YnNwOyZuYnNwOyZuYnNwOzE2Mzg0PGJyPg0KdHVuZWZzOiBhdmVyYWdlIG51bWJlciBvZiBmaWxl cyBpbiBhIGRpcmVjdG9yeTogKC1zKSAmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJz cDs2NDxicj4NCnR1bmVmczogbWluaW11bSBwZXJjZW50YWdlIG9mIGZyZWUgc3BhY2U6ICgtbSkg Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5i c3A7Jm5ic3A7Jm5ic3A7OCU8YnI+DQp0dW5lZnM6IHNwYWNlIHRvIGhvbGQgZm9yIG1ldGFkYXRh IGJsb2NrczogKC1rKSAmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsm bmJzcDsmbmJzcDsmbmJzcDsmbmJzcDs2NDAwPGJyPg0KdHVuZWZzOiBvcHRpbWl6YXRpb24gcHJl ZmVyZW5jZTogKC1vKSAmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsm bmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJz cDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDt0aW1lPGJyPg0KdHVuZWZzOiB2b2x1bWUgbGFiZWw6 ICgtTCk8YnI+DQo8YnI+DQpJIHBsYW4gdG8gaGF2ZSBUUklNIGZsYWcgZW5hYmxlZCBpbiB0aGUg cm9vdGZzIHBhcnRpdGlvbiAoL2Rldi91ZnMvcm9vdGZzKSBvZiBteSBSYXNwYmVycnkgUGkuIERv IHlvdSB0aGluayBvZiBhbnkgaW1wbGljYXRpb25zIHdoZW4gZW5hYmxlZD8gQXMgc2hvd24gYmVs b3csIGl0IGlzIGRpc2FibGVkLjxicj4NCjxicj4NCnJvb3RAc2QtZXh0cmVtZTp+ICMgdHVuZWZz IC1wIC9kZXYvdWZzL3Jvb3Rmczxicj4NCnR1bmVmczogUE9TSVguMWUgQUNMczogKC1hKSAmbmJz cDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsm bmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJz cDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsm bmJzcDsmbmJzcDtkaXNhYmxlZDxicj4NCnR1bmVmczogTkZTdjQgQUNMczogKC1OKSAmbmJzcDsm bmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJz cDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsm bmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJz cDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDtkaXNhYmxlZDxicj4NCnR1bmVmczogTUFDIG11bHRp bGFiZWw6ICgtbCkgJm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5i c3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7 Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5i c3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7ZGlzYWJsZWQ8YnI+DQp0dW5lZnM6IHNvZnQgdXBkYXRlczog KC1uKSAmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJz cDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsm bmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJz cDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDtlbmFibGVkPGJyPg0KdHVuZWZzOiBzb2Z0IHVwZGF0 ZSBqb3VybmFsaW5nOiAoLWopICZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZu YnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNw OyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwO2Rpc2FibGVkPGJyPg0KdHVuZWZz OiBnam91cm5hbDogKC1KKSAmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJz cDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsm bmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJz cDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsm bmJzcDtkaXNhYmxlZDxicj4NCnR1bmVmczogdHJpbTogKC10KSAmbmJzcDsmbmJzcDsmbmJzcDsm bmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJz cDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsm bmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJz cDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDtkaXNhYmxl ZDxicj4NCnR1bmVmczogbWF4aW11bSBibG9ja3MgcGVyIGZpbGUgaW4gYSBjeWxpbmRlciBncm91 cDogKC1lKSAmbmJzcDs0MDk2PGJyPg0KdHVuZWZzOiBhdmVyYWdlIGZpbGUgc2l6ZTogKC1mKSAm bmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJz cDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsm bmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsxNjM4NDxicj4N CnR1bmVmczogYXZlcmFnZSBudW1iZXIgb2YgZmlsZXMgaW4gYSBkaXJlY3Rvcnk6ICgtcykgJm5i c3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7NjQ8YnI+DQp0dW5lZnM6IG1pbmltdW0g cGVyY2VudGFnZSBvZiBmcmVlIHNwYWNlOiAoLW0pICZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZu YnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOzglPGJyPg0KdHVu ZWZzOiBzcGFjZSB0byBob2xkIGZvciBtZXRhZGF0YSBibG9ja3M6ICgtaykgJm5ic3A7Jm5ic3A7 Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7NjQw MDxicj4NCnR1bmVmczogb3B0aW1pemF0aW9uIHByZWZlcmVuY2U6ICgtbykgJm5ic3A7Jm5ic3A7 Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5i c3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7 dGltZTxicj4NCnR1bmVmczogdm9sdW1lIGxhYmVsOiAoLUwpICZuYnNwOyZuYnNwOyZuYnNwOyZu YnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNw OyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZu YnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNw O3Jvb3Rmczxicj4NCjxicj4NCkJSLDxicj4NCm9yYml0PGJyPg0KJm5ic3A7PGJyPg0KPGJyPg0K PGJyPg0KJm5ic3A7PC9kaXY+DQoNCjxocj48L2Rpdj4NCjwvYmxvY2txdW90ZT4NCjxicj4NCkhp LDxicj4NCjxicj4NCkknbSBzb3JyeSB0byBzcG9pbCB0aGUgZnVuIGJ1dCB0aGlzIG1lc3NhZ2Ug c2F5cyB0aGUgZGV2aWNlIGRvZXMgKm5vdCogc3VwcG9ydCB0cmltLjxicj4NCjxicj4NCiJyb290 QHNkLWV4dHJlbWU6fiAjIGRtZXNnIHwgZ3JlcCBUUklNPGJyPg0KV0FSTklORzogL21udDogVFJJ TSBmbGFnIG9uIGZzIGJ1dCBkaXNrIGRvZXMgbm90IHN1cHBvcnQgVFJJTSI8YnI+DQo8YnI+DQo8 YnI+DQpCdXQgdG8gZ2l2ZSBzb21lIGhvcGUuIEFzIE1hcmsgcG9pbnRlZCBvdXQgaXQgaXMgbm90 IG9ubHkgYWJvdXQgdGhlIGRpc2suIFRoZSBjb250cm9sbGVyIGFsc28gbmVlZHMgdG8gc3VwcG9y dCBpdC48YnI+DQo8YnI+DQpBcyBhIHJlYWwgd29ybGQgZXhhbXBsZTo8YnI+DQo8YnI+DQpJIGhh dmUgYW4gUlBJMyB3aXRoIGFuIFNTRCAoc3VwcG9ydGluZyB0cmltKSBjb25uZWN0ZWQgdmlhIFVT Qi10by1TQVRBIChub3Qgc3VwcG9ydGluZyB0cmltKS4gU28gdGhlIHRvdGFsIHN5c3RlbSBkb2Vz IG5vdCBzdXBwb3J0IHRyaW0uPGJyPg0KJCBjYXQgL3Zhci9ydW4vZG1lc2cuYm9vdCB8IGdyZXAg LUUgInVtYXNzMHxkYTB8VFJJTSI8YnI+DQp1bWFzczAgb24gdWh1YjE8YnI+DQp1bWFzczA6ICZs dDtHT0RPIFVTQjMuMCB0byBTQVRBIGFkYXB0ZXIsIGNsYXNzIDAvMCwgcmV2IDIuMTAvMS4wMCwg YWRkciA1Jmd0OyBvbiB1c2J1czE8YnI+DQp1bWFzczA6Jm5ic3A7IFNDU0kgb3ZlciBCdWxrLU9u bHk7IHF1aXJrcyA9IDB4MDEwMDxicj4NCnVtYXNzMDowOjA6IEF0dGFjaGVkIHRvIHNjYnVzMDxi cj4NCmRhMCBhdCB1bWFzcy1zaW0wIGJ1cyAwIHNjYnVzMCB0YXJnZXQgMCBsdW4gMDxicj4NCmRh MDogJmx0O0xJVEVPTiBDIFYzLURFMjU2IDAmZ3Q7IEZpeGVkIERpcmVjdCBBY2Nlc3MgU1BDLTQg U0NTSSBkZXZpY2U8YnI+DQpkYTA6IFNlcmlhbCBOdW1iZXIgMjAxNTEwMDAwMEQ2PGJyPg0KZGEw OiA0MC4wMDBNQi9zIHRyYW5zZmVyczxicj4NCmRhMDogMjQ0MTk4TUIgKDUwMDExODE5MiA1MTIg Ynl0ZSBzZWN0b3JzKTxicj4NCmRhMDogcXVpcmtzPTB4MiZsdDtOT182X0JZVEUmZ3Q7PGJyPg0K V0FSTklORzogLzogVFJJTSBmbGFnIG9uIGZzIGJ1dCBkaXNrIGRvZXMgbm90IHN1cHBvcnQgVFJJ TTxicj4NCjxicj4NCiQgc3lzY3RsIGtlcm4uY2FtLmRhLjAgfCBncmVwIC1FICJ0cmltfGRlbGV0 ZSI8YnI+DQprZXJuLmNhbS5kYS4wLnRyaW1fdGlja3M6IDA8YnI+DQprZXJuLmNhbS5kYS4wLnRy aW1fZ29hbDogMDxicj4NCmtlcm4uY2FtLmRhLjAudHJpbV9sYmFzOiAwPGJyPg0Ka2Vybi5jYW0u ZGEuMC50cmltX3JhbmdlczogMDxicj4NCmtlcm4uY2FtLmRhLjAudHJpbV9jb3VudDogMDxicj4N Cmtlcm4uY2FtLmRhLjAuZGVsZXRlX21heDogNjU1MzY8YnI+DQprZXJuLmNhbS5kYS4wLmRlbGV0 ZV9tZXRob2Q6IE5PTkU8YnI+DQo8YnI+DQo8YnI+DQpPbiBhbiBSUEk0QiBJIGhhdmUgYW4gU1NE IChzdXBwb3J0aW5nIHRyaW0pIGNvbm5lY3RlZCB2aWEgVVNCLXRvLVNBVEEgKHdoaWNoIGRvZXMg c3VwcG9ydCB0cmltIGFsc28pLiBIZXJlIHRyaW0gd29ya3MgZmluZS4gKE5COiB0aGlzIHVzZXMg WkZTLCBidXQgdGhhdCBkb2VzIG5vdCBtYXR0ZXIpPGJyPg0KJCBkbWVzZyB8IGdyZXAgLUUgInVt YXNzMHxkYTAiPGJyPg0KdW1hc3MwIG9uIHVodWIwPGJyPg0KdW1hc3MwOiAmbHQ7VVNCIDMuMCBE ZXZpY2UgVVNCIDMuMCBEZXZpY2UsIGNsYXNzIDAvMCwgcmV2IDMuMDAvMy4wMSwgYWRkciAzJmd0 OyBvbiB1c2J1czA8YnI+DQpkYTAgYXQgdW1hc3Mtc2ltMCBidXMgMCBzY2J1czAgdGFyZ2V0IDAg bHVuIDA8YnI+DQpkYTA6ICZsdDtNaWNyb24gMSAxMDAgU0FUQSAyNTZHQiAwMzAxJmd0OyBGaXhl ZCBEaXJlY3QgQWNjZXNzIFNQQy00IFNDU0kgZGV2aWNlPGJyPg0KZGEwOiBTZXJpYWwgTnVtYmVy IDAwMDAwMDAwNEJBODxicj4NCmRhMDogNDAwLjAwME1CL3MgdHJhbnNmZXJzPGJyPg0KZGEwOiAy NDQxOThNQiAoNTAwMTE4MTkyIDUxMiBieXRlIHNlY3RvcnMpPGJyPg0KZGEwOiBxdWlya3M9MHgy Jmx0O05PXzZfQllURSZndDs8YnI+DQo8YnI+DQokIHN5c2N0bCBrZXJuLmNhbS5kYS4wIHwgZ3Jl cCAtRSAidHJpbXxkZWxldGUiPGJyPg0Ka2Vybi5jYW0uZGEuMC50cmltX3RpY2tzOiAwPGJyPg0K a2Vybi5jYW0uZGEuMC50cmltX2dvYWw6IDA8YnI+DQprZXJuLmNhbS5kYS4wLnRyaW1fbGJhczog NTQ2ODQxNzA0PGJyPg0Ka2Vybi5jYW0uZGEuMC50cmltX3JhbmdlczogMzE3MjgwPGJyPg0Ka2Vy bi5jYW0uZGEuMC50cmltX2NvdW50OiAzMDE0MjQ8YnI+DQprZXJuLmNhbS5kYS4wLmRlbGV0ZV9t YXg6IDQyOTQ5MDE3NjA8YnI+DQprZXJuLmNhbS5kYS4wLmRlbGV0ZV9tZXRob2Q6IFVOTUFQPGJy Pg0KPGJyPg0KPGJyPg0KQlRXOiBlbmFibGluZyB0cmltIG9uIHRoZSBmcyB3aGlsZSB0aGUgZGV2 aWNlIGRvZXMgbm90IHN1cHBvcnQgaXQgZG9lcyBub3QgbWF0dGVyIGFuZCBkb2VzIG5vIGhhcm0u IFVzaW5nIHRoZSBzeXNjdGwgYWJvdmUgeW91IGNhbiBzZWUgaWYgdGhlIGRldmljZSBhY3R1YWxs eSB1c2VzIHRyaW0uPGJyPg0KPGJyPg0KUnVubmluZyAiZ3N0YXQgLWQiIHdoaWxlIGRlbGV0aW5n IGEgbG90IG9mIGZpbGVzIHdpbGwgYWxzbyBzaG93IGlmIHRyaW0vZGVsZXRlIGFjdGlvbnMgYXJl IHNlbnQgdG8gdGhlIGRpc2suPGJyPg0KPGJyPg0KU28gdGhlIGNvbWJpbmF0aW9uIG9mIHlvdXIg VVNCIHJlYWRlciArIG1pY3JvU0QgZG8gbm90IHN1cHBvcnQgVFJJTSwgYnV0IHlvdSBjYW4ganVz dCB0cnkgdG8gcHV0IHRoZSBtaWNyb1NEIGRpcmVjdGx5IGluIHRoZSBSUEk0IGFuZCBzZWUgd2hh dCB0aGF0IGRvZXMuPGJyPg0KPGJyPg0KUmVnYXJkcyw8YnI+DQpSb25hbGQuPGJyPjwvYmxvY2tx dW90ZT48L2Rpdj48ZGl2IGNsYXNzPSJwcm90b25tYWlsX3F1b3RlIiBzdHlsZT0iZm9udC1mYW1p bHk6IEFyaWFsLCBzYW5zLXNlcmlmOyBmb250LXNpemU6IDE0cHg7IGNvbG9yOiByZ2IoMCwgMCwg MCk7IGJhY2tncm91bmQtY29sb3I6IHJnYigyNTUsIDI1NSwgMjU1KTsiPjxicj48L2Rpdj48ZGl2 IGNsYXNzPSJwcm90b25tYWlsX3F1b3RlIiBzdHlsZT0iZm9udC1mYW1pbHk6IEFyaWFsLCBzYW5z LXNlcmlmOyBmb250LXNpemU6IDE0cHg7IGNvbG9yOiByZ2IoMCwgMCwgMCk7IGJhY2tncm91bmQt Y29sb3I6IHJnYigyNTUsIDI1NSwgMjU1KTsiPlRoYW5rcyBmb3IgdGhlIGNhbGxvdXQhIEdvdCB3 YXJtZWQtdXAgd2hlbiBJIHNhdyB0aGUgVFJJTSBpcyBlbmFibGVkIDotKSBidXQgbXkgdWx0aW1h dGUgZ29hbCBpcyByZWFsbHkgdGhlIG1pY3JvU0QgY2FyZCBzbG90IG9mIG15IFJhc3BiZXJyeSBQ aSAzIG9yIDQgYW5kIG5vdCB3aXRoIHRoZSBVU0IgcmVhZGVyLiBJJ2xsIGNyZWF0ZSBhbiBpbWFn ZSB0aGVuIHNlZSBob3cgaXQgZG9lcy48L2Rpdj48ZGl2IGNsYXNzPSJwcm90b25tYWlsX3F1b3Rl IiBzdHlsZT0iZm9udC1mYW1pbHk6IEFyaWFsLCBzYW5zLXNlcmlmOyBmb250LXNpemU6IDE0cHg7 IGNvbG9yOiByZ2IoMCwgMCwgMCk7IGJhY2tncm91bmQtY29sb3I6IHJnYigyNTUsIDI1NSwgMjU1 KTsiPjxicj48L2Rpdj48ZGl2IGNsYXNzPSJwcm90b25tYWlsX3F1b3RlIiBzdHlsZT0iZm9udC1m YW1pbHk6IEFyaWFsLCBzYW5zLXNlcmlmOyBmb250LXNpemU6IDE0cHg7IGNvbG9yOiByZ2IoMCwg MCwgMCk7IGJhY2tncm91bmQtY29sb3I6IHJnYigyNTUsIDI1NSwgMjU1KTsiPkJSLDwvZGl2Pjxk aXYgY2xhc3M9InByb3Rvbm1haWxfcXVvdGUiIHN0eWxlPSJmb250LWZhbWlseTogQXJpYWwsIHNh bnMtc2VyaWY7IGZvbnQtc2l6ZTogMTRweDsgY29sb3I6IHJnYigwLCAwLCAwKTsgYmFja2dyb3Vu ZC1jb2xvcjogcmdiKDI1NSwgMjU1LCAyNTUpOyI+b3JiaXQmbmJzcDsgPGJyPjwvZGl2PjxkaXYg Y2xhc3M9InByb3Rvbm1haWxfcXVvdGUiIHN0eWxlPSJmb250LWZhbWlseTogQXJpYWwsIHNhbnMt c2VyaWY7IGZvbnQtc2l6ZTogMTRweDsgY29sb3I6IHJnYigwLCAwLCAwKTsgYmFja2dyb3VuZC1j b2xvcjogcmdiKDI1NSwgMjU1LCAyNTUpOyI+PGJyPjwvZGl2PjxkaXYgY2xhc3M9InByb3Rvbm1h aWxfcXVvdGUiPg0KICAgIDwvZGl2Pg== --b1_i4TVXaKbceUP6PQinOcTun6iLjvwKymRhlhbJDABQI-- From nobody Sat Feb 17 02:40:46 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TcCj43g7yz5BSBG for ; Sat, 17 Feb 2024 02:40:56 +0000 (UTC) (envelope-from ordinarybit@proton.me) Received: from mail-40138.protonmail.ch (mail-40138.protonmail.ch [185.70.40.138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "protonmail.com", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TcCj375XHz46Bv for ; Sat, 17 Feb 2024 02:40:55 +0000 (UTC) (envelope-from ordinarybit@proton.me) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=proton.me; s=protonmail; t=1708137652; x=1708396852; bh=mp9ttb2EFVTJ3y6o+x+/TxBp2rp2XOOs26ffSTzatJc=; h=Date:To:From:Cc:Subject:Message-ID:In-Reply-To:References: Feedback-ID:From:To:Cc:Date:Subject:Reply-To:Feedback-ID: Message-ID:BIMI-Selector; b=BsUnWjPPtrd+BYHHviqY4OXQa1QFlR/Cwd2qjlSsb/vpgC5Zh7W6IDv5t/Ot5KdzJ oenDQbGcATpe5BNKxoeSpND75XccMuHckgUL8WBQUol9WbGlkz1DXvTRp5V+2CiD3P pb5d3g68zVoJDNnpikKzFAsEeVxIS7Z/VjzSlwfTPROVwmvMwzSAFfbSm8LSG5Gqwy AMAZcmNBUA6+wAj8bo0YpzBxVqhGE1cSPM+I1Y0FhHScAPeCDI7+WkRZN9yceHIaRm AFVkoM9+pLP9e6v+z0q+VfntDQUUE8mQJZWmk6sJkQfFxwy+jgWxEoyVLAq6GUtboX fDd50/GOOX0fA== Date: Sat, 17 Feb 2024 02:40:46 +0000 To: Ordinary Bit From: Ordinary Bit Cc: Ronald Klop , "freebsd-arm@freebsd.org" , Mark Millard Subject: Re: newfs TRIM flag device support Message-ID: In-Reply-To: References: <6AD380D4-9B42-4511-9E02-94EB0005D278@yahoo.com> <8D8AA3F5-74A3-4AFE-A560-F52C7C55CF12@yahoo.com> <7okx4eDPse2uF1n3aX4W8RIOdbHXziUhqgvHxqqxpYN-7NsJfEGrfHbDxLgNS3PdOwZt7j9aOXp-0MHDjQAryL4t1Gd2QHUG1sFNmd5dY0k=@proton.me> <454941689.1180.1708079489598@localhost> Feedback-ID: 100108111:user:proton List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="b1_hp6lbu7uCCqtZcnP3HWOX4Pb94jbjUYIGrM6EMDv3o" X-Spamd-Bar: ---- X-Rspamd-Queue-Id: 4TcCj375XHz46Bv X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:62371, ipnet:185.70.40.0/24, country:CH] This is a multi-part message in MIME format. --b1_hp6lbu7uCCqtZcnP3HWOX4Pb94jbjUYIGrM6EMDv3o Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: base64 T24gRnJpZGF5LCAxNiBGZWJydWFyeSAyMDI0IGF0IDE4OjU0LCBPcmRpbmFyeSBCaXQgPG9yZGlu YXJ5Yml0QHByb3Rvbi5tZT4gd3JvdGU6Cgo+IE9uIEZyaWRheSwgMTYgRmVicnVhcnkgMjAyNCBh dCAxODozMSwgUm9uYWxkIEtsb3AgPHJvbmFsZC1saXN0c0BrbG9wLndzPiB3cm90ZToKPgo+PiBW YW46IE9yZGluYXJ5IEJpdCA8b3JkaW5hcnliaXRAcHJvdG9uLm1lPgo+PiBEYXR1bTogdnJpamRh ZywgMTYgZmVicnVhcmkgMjAyNCAxMDoxOAo+PiBBYW46IE1hcmsgTWlsbGFyZCA8bWFya2xtaUB5 YWhvby5jb20+Cj4+IENDOiAiZnJlZWJzZC1hcm1AZnJlZWJzZC5vcmciIDxmcmVlYnNkLWFybUBm cmVlYnNkLm9yZz4KPj4gT25kZXJ3ZXJwOiBSZTogbmV3ZnMgVFJJTSBmbGFnIGRldmljZSBzdXBw b3J0Cj4+Cj4+PiBTZW50IHdpdGggUHJvdG9uIE1haWwgc2VjdXJlIGVtYWlsLgo+Pj4KPj4+IE9u IEZyaWRheSwgMTYgRmVicnVhcnkgMjAyNCBhdCAxNDowMiwgTWFyayBNaWxsYXJkIDxtYXJrbG1p QHlhaG9vLmNvbT4gd3JvdGU6Cj4+Pgo+Pj4+IE9uIEZlYiAxNSwgMjAyNCwgYXQgMjA6MDgsIE9y ZGluYXJ5IEJpdCBvcmRpbmFyeWJpdEBwcm90b24ubWUgd3JvdGU6Cj4+Pj4KPj4+PiA+IEhpIQo+ Pj4+Cj4+Pj4KPj4+PiBIZWxsby4KPj4+Pgo+Pj4+ID4gT24gRnJpZGF5LCAxNiBGZWJydWFyeSAy MDI0IGF0IDExOjQxLCBNYXJrIE1pbGxhcmQgbWFya2xtaUB5YWhvby5jb20gd3JvdGU6Cj4+Pj4g Pgo+Pj4+ID4gPiBbT25seSByZXBseWluZyB0byBsaXN0cyBJIHN1YnNjcmliZSB0by5dCj4+Pj4g PiA+Cj4+Pj4gPiA+IE9uIEZlYiAxNSwgMjAyNCwgYXQgMTk6MTksIE9yZGluYXJ5IEJpdCBvcmRp bmFyeWJpdEBwcm90b24ubWUgd3JvdGU6Cj4+Pj4gPiA+Cj4+Pj4gPiA+ID4gSSdtIHJlYWRpbmcg dGhlIG5ld2ZzIG1hbnVhbCBodHRwczovL21hbi5mcmVlYnNkLm9yZy9jZ2kvbWFuLmNnaT9uZXdm cyg4KSB0byBiZSBhYmxlIHRvIGtub3cgYWJvdXQgdGhlIFRSSU0gZmxhZy4gSW4gdGhlIG1hbnVh bCB1bmRlciAtdCBwYXJhbWV0ZXIsIGl0IG1lbnRpb25lZCBhYm91dCAidW5kZXJseWluZyBkZXZp Y2Ugc3VwcG9ydCIsIHdoYXQgZXhhY3RseSBpcyB0aGlzIGRldmljZT8KPj4+PiA+ID4KPj4+PiA+ ID4gMiBjb250cmFzdGluZyBleGFtcGxlczoKPj4+PiA+ID4KPj4+PiA+ID4gRXhhbXBsZSAwOiBP cHRhbmUgTlZNZSBtZWRpYSAoUENJZSBjYXJkIG9yIFUuMiwgZm9yIGV4YW1wbGUpCj4+Pj4gPiA+ Cj4+Pj4gPiA+IE9wdGFuZSBoYXMgbm8gbmVlZCBvZiBUUklNIGFuZCwgc28sIG5ldmVyIHN1cHBv cnRzIFRSSU0uCj4+Pj4gPiA+Cj4+Pj4gPiA+IEV4YW1wbGUgMTogbWljcm9zZCBjYXJkIG1lZGlh IHVzYWdlCj4+Pj4gPiA+Cj4+Pj4gPiA+IEEgbWljcm9zZCBjYXJkIGluIHRoZSBub3JtYWwgdHlw ZSBvZiBtaWNyb3NkIGNhcmQgc2xvdCBvbgo+Pj4+ID4gPiBTbWFsbCBCb2FyZCBDb21wdXRlcnMg KG5vcm1hbGx5KSBzdXBwb3J0cyBUUklNLiBUYWtlIHRoZQo+Pj4+ID4gPiBzYW1lIGNhcmQgYW5k IHB1dCBpdCBpbiBhIFVTQiByZWFkZXIvd3JpdGVyIGFuZCB1c2UgaXQKPj4+PiA+ID4gdmlhIFVT QiBvbiB0aGUgc2FtZSBzeXN0ZW06IG5vIFRSSU0gaXMgc3VwcG9ydGVkIGJ5Cj4+Pj4gPiA+IEZy ZWVCU0Qgb3ZlciBVU0IuCj4+Pj4gPgo+Pj4+ID4gU28geW91IG1lYW4gdG8gc2F5IHRoYXQgaWYg SSBoYXZlIGEgUmFzcGVycnkgUGkgMyBvciA0IG5vdyBhbmQgdGhlbiBoYXZlIG15IEZyZWVCU0Qg aW5zdGFsbGVkIGluIGEgbWljcm9TRCBjYXJkIChmb3IgZXhhbXBsZSwgU2FuRGlzayBFeHRyZW1l IGNhcmQpIHdpdGggVUZTL0ZGUyBmaWxlc3l0ZW0gaW4gaXQgd2l0aCBUUklNIGVuYWJsZWQgcGFy YW1ldGVyIHRoZW4gaXMgaXQgZ29pbmcgdG8gcmVjb2duaXplIGl0PyBIb3cgdG8gdmVyaWZ5Pwo+ Pj4+ID4KPj4+PiA+ID4gRllJOgo+Pj4+ID4gPgo+Pj4+ID4gPiBXaGVuIHRoZSBmaWxlIHN5c3Rl bSBoYXMgVFJJTSBlbmFibGVkLCBGcmVlQlNEIHB1dCBvdXQgYQo+Pj4+ID4gPiBub3RpY2UgaWYg VFJJTSB3aWxsIG5vdCBhY3R1YWxseSBiZSB1c2VkIGluIHRoZSBhY3R1YWwKPj4+PiA+ID4gY29u dGV4dCBpbiB1c2UuCj4+Pj4gPgo+Pj4+ID4gT2ssIGdvdCBpdC4gSG93IHRvIGNoZWNrIHRoaXMg YXMgd2VsbD8KPj4+Pgo+Pj4+Cj4+Pj4gVGhlIGNvbnNvbGUgZ2V0cyBhIG1lc3NhZ2UgbGlrZToK Pj4+Pgo+Pj4+IFdBUk5JTkc6IC9tbnQ6IFRSSU0gZmxhZyBvbiBmcyBidXQgZGlzayBkb2VzIG5v dCBzdXBwb3J0IFRSSU0KPj4+Pgo+Pj4+IHdoZW4gYSBtb3VudCBpcyBhdHRlbXB0ZWQgKGF1dG9t YXRpYyBvciBtYW51YWxseSkgZm9yIGEKPj4+PiBmaWxlIHN5c3RlbSB3aXRoIHRoZSB0cmltIGZs YWcgZW5hYmxlZCBidXQgdHJpbSBkb2VzIG5vdAo+Pj4+IGVuZCB1cCBhY3RpdmUuCj4+Pj4KPj4+ PiBTbywgZm9yIGV4YW1wbGU6Cj4+Pj4KPj4+PiAjIGRtZXNnIC1hIHwgZ3JlcCBUUklNCj4+Pj4g V0FSTklORzogL21udDogVFJJTSBmbGFnIG9uIGZzIGJ1dCBkaXNrIGRvZXMgbm90IHN1cHBvcnQg VFJJTQo+Pj4+Cj4+Pj4gKFRoaXMgd2FzIGEgbWljcm9zZCBjYXJkIGluIGEgVVNCIHJlYWRlci93 cml0ZXIgdGhhdCB3YXMgbm90Cj4+Pj4gdXNlZCBhcyB0aGUgYm9vdCBtZWRpYTogYSBzZXBhcmF0 ZSBtYW51YWwgbW91bnQgd2FzIHVzZWQuKQo+Pj4+Cj4+Pj4gVGhlIGZpbGUgc3lzdGVtJ3MgVFJJ TSBmbGFnIHN0YXR1cyBjYW4gYmUgY2hlY2tlZCB2aWEgdXNlIG9mCj4+Pj4gInR1bmVmcyAtcCAu IC4gLiI6Cj4+Pj4KPj4+PiAjIHR1bmVmcyAtcCAvbW50Cj4+Pj4gdHVuZWZzOiBQT1NJWC4xZSBB Q0xzOiAoLWEpIGRpc2FibGVkCj4+Pj4gdHVuZWZzOiBORlN2NCBBQ0xzOiAoLU4pIGRpc2FibGVk Cj4+Pj4gdHVuZWZzOiBNQUMgbXVsdGlsYWJlbDogKC1sKSBkaXNhYmxlZAo+Pj4+IHR1bmVmczog c29mdCB1cGRhdGVzOiAoLW4pIGVuYWJsZWQKPj4+PiB0dW5lZnM6IHNvZnQgdXBkYXRlIGpvdXJu YWxpbmc6ICgtaikgZGlzYWJsZWQKPj4+PiB0dW5lZnM6IGdqb3VybmFsOiAoLUopIGRpc2FibGVk Cj4+Pj4gdHVuZWZzOiB0cmltOiAoLXQpIGVuYWJsZWQKPj4+PiB0dW5lZnM6IG1heGltdW0gYmxv Y2tzIHBlciBmaWxlIGluIGEgY3lsaW5kZXIgZ3JvdXA6ICgtZSkgNDA5Ngo+Pj4+IHR1bmVmczog YXZlcmFnZSBmaWxlIHNpemU6ICgtZikgMTYzODQKPj4+PiB0dW5lZnM6IGF2ZXJhZ2UgbnVtYmVy IG9mIGZpbGVzIGluIGEgZGlyZWN0b3J5OiAoLXMpIDY0Cj4+Pj4gdHVuZWZzOiBtaW5pbXVtIHBl cmNlbnRhZ2Ugb2YgZnJlZSBzcGFjZTogKC1tKSA4JQo+Pj4+IHR1bmVmczogc3BhY2UgdG8gaG9s ZCBmb3IgbWV0YWRhdGEgYmxvY2tzOiAoLWspIDY0MDAKPj4+PiB0dW5lZnM6IG9wdGltaXphdGlv biBwcmVmZXJlbmNlOiAoLW8pIHRpbWUKPj4+PiB0dW5lZnM6IHZvbHVtZSBsYWJlbDogKC1MKQo+ Pj4+Cj4+Pj4gSWYgdGhlIHRyaW0gZmxhZyBpcyBlbmFibGVkIGJ1dCB0aGUgbW91bnQgZG9lcyBu b3QgcHJvZHVjZSB0aGUKPj4+PiBjb25zb2xlIG1lc3NhZ2UsIHRoZW4gVFJJTSBpcyBhY3RpdmUu Cj4+Pj4KPj4+Cj4+PiBPaCwgdGhhdCdzIGFtYXppbmchIEkgdHJ5IGl0IHdpdGggbXkgU2FuRGlz ayBVbHRyYSBtaWNyb1NEWEMgNjRHQiBhbmQgbW91bnQgaXQgaW4gYSBVU0IgcmVhZGVyIGFuZCBp dCBzaG93cyBlbmFibGVkIGluIHRoZSB0dW5lZnMgYW5kIGlzIGRldGVjdGVkIGluIHRoZSBkbWVz ZywgdGhlIHNhbWUgYXMgeW91cnMuIFRoZXJlZm9yZSwgU2FuRGlzayBVbHRyYSBtaWNyb1NEIHN1 cHBvcnRzIGl0IQo+Pj4KPj4+IHJvb3RAc2QtZXh0cmVtZTp+ICMgL3NiaW4vbmV3ZnMgLVUgLXQg L2Rldi9kYTBzMmEKPj4+IC9kZXYvZGEwczJhOiA1OTAwMC45TUIgKDEyMDgzMzkyMCBzZWN0b3Jz KSBibG9jayBzaXplIDMyNzY4LCBmcmFnbWVudCBzaXplIDQwOTYKPj4+IHVzaW5nIDk1IGN5bGlu ZGVyIGdyb3VwcyBvZiA2MjUuMjJNQiwgMjAwMDcgYmxrcywgODAxMjggaW5vZGVzLgo+Pj4gd2l0 aCBzb2Z0IHVwZGF0ZXMKPj4+IHN1cGVyLWJsb2NrIGJhY2t1cHMgKGZvciBmc2NrX2ZmcyAtYiAj KSBhdDoKPj4+IDE5MiwgMTI4MDY0MCwgMjU2MTA4OCwgMzg0MTUzNiwgNTEyMTk4NCwgNjQwMjQz MiwgNzY4Mjg4MCwgODk2MzMyOCwgMTAyNDM3NzYsIDExNTI0MjI0LCAxMjgwNDY3MiwgMTQwODUx MjAsCj4+PiAxNTM2NTU2OCwgMTY2NDYwMTYsIDE3OTI2NDY0LCAxOTIwNjkxMiwgMjA0ODczNjAs IDIxNzY3ODA4LCAyMzA0ODI1NiwgMjQzMjg3MDQsIDI1NjA5MTUyLCAyNjg4OTYwMCwgMjgxNzAw NDgsCj4+PiAyOTQ1MDQ5NiwgMzA3MzA5NDQsIDMyMDExMzkyLCAzMzI5MTg0MCwgMzQ1NzIyODgs IDM1ODUyNzM2LCAzNzEzMzE4NCwgMzg0MTM2MzIsIDM5Njk0MDgwLCA0MDk3NDUyOCwgNDIyNTQ5 NzYsCj4+PiA0MzUzNTQyNCwgNDQ4MTU4NzIsIDQ2MDk2MzIwLCA0NzM3Njc2OCwgNDg2NTcyMTYs IDQ5OTM3NjY0LCA1MTIxODExMiwgNTI0OTg1NjAsIDUzNzc5MDA4LCA1NTA1OTQ1NiwgNTYzMzk5 MDQsCj4+PiA1NzYyMDM1MiwgNTg5MDA4MDAsIDYwMTgxMjQ4LCA2MTQ2MTY5NiwgNjI3NDIxNDQs IDY0MDIyNTkyLCA2NTMwMzA0MCwgNjY1ODM0ODgsIDY3ODYzOTM2LCA2OTE0NDM4NCwgNzA0MjQ4 MzIsCj4+PiA3MTcwNTI4MCwgNzI5ODU3MjgsIDc0MjY2MTc2LCA3NTU0NjYyNCwgNzY4MjcwNzIs IDc4MTA3NTIwLCA3OTM4Nzk2OCwgODA2Njg0MTYsIDgxOTQ4ODY0LCA4MzIyOTMxMiwgODQ1MDk3 NjAsCj4+PiA4NTc5MDIwOCwgODcwNzA2NTYsIDg4MzUxMTA0LCA4OTYzMTU1MiwgOTA5MTIwMDAs IDkyMTkyNDQ4LCA5MzQ3Mjg5NiwgOTQ3NTMzNDQsIDk2MDMzNzkyLCA5NzMxNDI0MCwgOTg1OTQ2 ODgsCj4+PiA5OTg3NTEzNiwgMTAxMTU1NTg0LCAxMDI0MzYwMzIsIDEwMzcxNjQ4MCwgMTA0OTk2 OTI4LCAxMDYyNzczNzYsIDEwNzU1NzgyNCwgMTA4ODM4MjcyLCAxMTAxMTg3MjAsIDExMTM5OTE2 OCwKPj4+IDExMjY3OTYxNiwgMTEzOTYwMDY0LCAxMTUyNDA1MTIsIDExNjUyMDk2MCwgMTE3ODAx NDA4LCAxMTkwODE4NTYsIDEyMDM2MjMwNAo+Pj4KPj4+IHJvb3RAc2QtZXh0cmVtZTp+ICMgZG1l c2cgfCBncmVwIFRSSU0KPj4+IFdBUk5JTkc6IC9tbnQ6IFRSSU0gZmxhZyBvbiBmcyBidXQgZGlz ayBkb2VzIG5vdCBzdXBwb3J0IFRSSU0KPj4+Cj4+PiByb290QHNkLWV4dHJlbWU6fiAjIHR1bmVm cyAtcCAvbW50Cj4+PiB0dW5lZnM6IFBPU0lYLjFlIEFDTHM6ICgtYSkgZGlzYWJsZWQKPj4+IHR1 bmVmczogTkZTdjQgQUNMczogKC1OKSBkaXNhYmxlZAo+Pj4gdHVuZWZzOiBNQUMgbXVsdGlsYWJl bDogKC1sKSBkaXNhYmxlZAo+Pj4gdHVuZWZzOiBzb2Z0IHVwZGF0ZXM6ICgtbikgZW5hYmxlZAo+ Pj4gdHVuZWZzOiBzb2Z0IHVwZGF0ZSBqb3VybmFsaW5nOiAoLWopIGRpc2FibGVkCj4+PiB0dW5l ZnM6IGdqb3VybmFsOiAoLUopIGRpc2FibGVkCj4+PiB0dW5lZnM6IHRyaW06ICgtdCkgZW5hYmxl ZAo+Pj4gdHVuZWZzOiBtYXhpbXVtIGJsb2NrcyBwZXIgZmlsZSBpbiBhIGN5bGluZGVyIGdyb3Vw OiAoLWUpIDQwOTYKPj4+IHR1bmVmczogYXZlcmFnZSBmaWxlIHNpemU6ICgtZikgMTYzODQKPj4+ IHR1bmVmczogYXZlcmFnZSBudW1iZXIgb2YgZmlsZXMgaW4gYSBkaXJlY3Rvcnk6ICgtcykgNjQK Pj4+IHR1bmVmczogbWluaW11bSBwZXJjZW50YWdlIG9mIGZyZWUgc3BhY2U6ICgtbSkgOCUKPj4+ IHR1bmVmczogc3BhY2UgdG8gaG9sZCBmb3IgbWV0YWRhdGEgYmxvY2tzOiAoLWspIDY0MDAKPj4+ IHR1bmVmczogb3B0aW1pemF0aW9uIHByZWZlcmVuY2U6ICgtbykgdGltZQo+Pj4gdHVuZWZzOiB2 b2x1bWUgbGFiZWw6ICgtTCkKPj4+Cj4+PiBJIHBsYW4gdG8gaGF2ZSBUUklNIGZsYWcgZW5hYmxl ZCBpbiB0aGUgcm9vdGZzIHBhcnRpdGlvbiAoL2Rldi91ZnMvcm9vdGZzKSBvZiBteSBSYXNwYmVy cnkgUGkuIERvIHlvdSB0aGluayBvZiBhbnkgaW1wbGljYXRpb25zIHdoZW4gZW5hYmxlZD8gQXMg c2hvd24gYmVsb3csIGl0IGlzIGRpc2FibGVkLgo+Pj4KPj4+IHJvb3RAc2QtZXh0cmVtZTp+ICMg dHVuZWZzIC1wIC9kZXYvdWZzL3Jvb3Rmcwo+Pj4gdHVuZWZzOiBQT1NJWC4xZSBBQ0xzOiAoLWEp IGRpc2FibGVkCj4+PiB0dW5lZnM6IE5GU3Y0IEFDTHM6ICgtTikgZGlzYWJsZWQKPj4+IHR1bmVm czogTUFDIG11bHRpbGFiZWw6ICgtbCkgZGlzYWJsZWQKPj4+IHR1bmVmczogc29mdCB1cGRhdGVz OiAoLW4pIGVuYWJsZWQKPj4+IHR1bmVmczogc29mdCB1cGRhdGUgam91cm5hbGluZzogKC1qKSBk aXNhYmxlZAo+Pj4gdHVuZWZzOiBnam91cm5hbDogKC1KKSBkaXNhYmxlZAo+Pj4gdHVuZWZzOiB0 cmltOiAoLXQpIGRpc2FibGVkCj4+PiB0dW5lZnM6IG1heGltdW0gYmxvY2tzIHBlciBmaWxlIGlu IGEgY3lsaW5kZXIgZ3JvdXA6ICgtZSkgNDA5Ngo+Pj4gdHVuZWZzOiBhdmVyYWdlIGZpbGUgc2l6 ZTogKC1mKSAxNjM4NAo+Pj4gdHVuZWZzOiBhdmVyYWdlIG51bWJlciBvZiBmaWxlcyBpbiBhIGRp cmVjdG9yeTogKC1zKSA2NAo+Pj4gdHVuZWZzOiBtaW5pbXVtIHBlcmNlbnRhZ2Ugb2YgZnJlZSBz cGFjZTogKC1tKSA4JQo+Pj4gdHVuZWZzOiBzcGFjZSB0byBob2xkIGZvciBtZXRhZGF0YSBibG9j a3M6ICgtaykgNjQwMAo+Pj4gdHVuZWZzOiBvcHRpbWl6YXRpb24gcHJlZmVyZW5jZTogKC1vKSB0 aW1lCj4+PiB0dW5lZnM6IHZvbHVtZSBsYWJlbDogKC1MKSByb290ZnMKPj4+Cj4+PiBCUiwKPj4+ IG9yYml0Cj4+Pgo+Pj4gLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tCj4+Cj4+IEhpLAo+Pgo+PiBJJ20gc29ycnkgdG8gc3BvaWwg dGhlIGZ1biBidXQgdGhpcyBtZXNzYWdlIHNheXMgdGhlIGRldmljZSBkb2VzICpub3QqIHN1cHBv cnQgdHJpbS4KPj4KPj4gInJvb3RAc2QtZXh0cmVtZTp+ICMgZG1lc2cgfCBncmVwIFRSSU0KPj4g V0FSTklORzogL21udDogVFJJTSBmbGFnIG9uIGZzIGJ1dCBkaXNrIGRvZXMgbm90IHN1cHBvcnQg VFJJTSIKPj4KPj4gQnV0IHRvIGdpdmUgc29tZSBob3BlLiBBcyBNYXJrIHBvaW50ZWQgb3V0IGl0 IGlzIG5vdCBvbmx5IGFib3V0IHRoZSBkaXNrLiBUaGUgY29udHJvbGxlciBhbHNvIG5lZWRzIHRv IHN1cHBvcnQgaXQuCj4+Cj4+IEFzIGEgcmVhbCB3b3JsZCBleGFtcGxlOgo+Pgo+PiBJIGhhdmUg YW4gUlBJMyB3aXRoIGFuIFNTRCAoc3VwcG9ydGluZyB0cmltKSBjb25uZWN0ZWQgdmlhIFVTQi10 by1TQVRBIChub3Qgc3VwcG9ydGluZyB0cmltKS4gU28gdGhlIHRvdGFsIHN5c3RlbSBkb2VzIG5v dCBzdXBwb3J0IHRyaW0uCj4+ICQgY2F0IC92YXIvcnVuL2RtZXNnLmJvb3QgfCBncmVwIC1FICJ1 bWFzczB8ZGEwfFRSSU0iCj4+IHVtYXNzMCBvbiB1aHViMQo+PiB1bWFzczA6IDxHT0RPIFVTQjMu MCB0byBTQVRBIGFkYXB0ZXIsIGNsYXNzIDAvMCwgcmV2IDIuMTAvMS4wMCwgYWRkciA1PiBvbiB1 c2J1czEKPj4gdW1hc3MwOiBTQ1NJIG92ZXIgQnVsay1Pbmx5OyBxdWlya3MgPSAweDAxMDAKPj4g dW1hc3MwOjA6MDogQXR0YWNoZWQgdG8gc2NidXMwCj4+IGRhMCBhdCB1bWFzcy1zaW0wIGJ1cyAw IHNjYnVzMCB0YXJnZXQgMCBsdW4gMAo+PiBkYTA6IDxMSVRFT04gQyBWMy1ERTI1NiAwPiBGaXhl ZCBEaXJlY3QgQWNjZXNzIFNQQy00IFNDU0kgZGV2aWNlCj4+IGRhMDogU2VyaWFsIE51bWJlciAy MDE1MTAwMDAwRDYKPj4gZGEwOiA0MC4wMDBNQi9zIHRyYW5zZmVycwo+PiBkYTA6IDI0NDE5OE1C ICg1MDAxMTgxOTIgNTEyIGJ5dGUgc2VjdG9ycykKPj4gZGEwOiBxdWlya3M9MHgyPE5PXzZfQllU RT4KPj4gV0FSTklORzogLzogVFJJTSBmbGFnIG9uIGZzIGJ1dCBkaXNrIGRvZXMgbm90IHN1cHBv cnQgVFJJTQo+Pgo+PiAkIHN5c2N0bCBrZXJuLmNhbS5kYS4wIHwgZ3JlcCAtRSAidHJpbXxkZWxl dGUiCj4+IGtlcm4uY2FtLmRhLjAudHJpbV90aWNrczogMAo+PiBrZXJuLmNhbS5kYS4wLnRyaW1f Z29hbDogMAo+PiBrZXJuLmNhbS5kYS4wLnRyaW1fbGJhczogMAo+PiBrZXJuLmNhbS5kYS4wLnRy aW1fcmFuZ2VzOiAwCj4+IGtlcm4uY2FtLmRhLjAudHJpbV9jb3VudDogMAo+PiBrZXJuLmNhbS5k YS4wLmRlbGV0ZV9tYXg6IDY1NTM2Cj4+IGtlcm4uY2FtLmRhLjAuZGVsZXRlX21ldGhvZDogTk9O RQo+Pgo+PiBPbiBhbiBSUEk0QiBJIGhhdmUgYW4gU1NEIChzdXBwb3J0aW5nIHRyaW0pIGNvbm5l Y3RlZCB2aWEgVVNCLXRvLVNBVEEgKHdoaWNoIGRvZXMgc3VwcG9ydCB0cmltIGFsc28pLiBIZXJl IHRyaW0gd29ya3MgZmluZS4gKE5COiB0aGlzIHVzZXMgWkZTLCBidXQgdGhhdCBkb2VzIG5vdCBt YXR0ZXIpCj4+ICQgZG1lc2cgfCBncmVwIC1FICJ1bWFzczB8ZGEwIgo+PiB1bWFzczAgb24gdWh1 YjAKPj4gdW1hc3MwOiA8VVNCIDMuMCBEZXZpY2UgVVNCIDMuMCBEZXZpY2UsIGNsYXNzIDAvMCwg cmV2IDMuMDAvMy4wMSwgYWRkciAzPiBvbiB1c2J1czAKPj4gZGEwIGF0IHVtYXNzLXNpbTAgYnVz IDAgc2NidXMwIHRhcmdldCAwIGx1biAwCj4+IGRhMDogPE1pY3JvbiAxIDEwMCBTQVRBIDI1NkdC IDAzMDE+IEZpeGVkIERpcmVjdCBBY2Nlc3MgU1BDLTQgU0NTSSBkZXZpY2UKPj4gZGEwOiBTZXJp YWwgTnVtYmVyIDAwMDAwMDAwNEJBOAo+PiBkYTA6IDQwMC4wMDBNQi9zIHRyYW5zZmVycwo+PiBk YTA6IDI0NDE5OE1CICg1MDAxMTgxOTIgNTEyIGJ5dGUgc2VjdG9ycykKPj4gZGEwOiBxdWlya3M9 MHgyPE5PXzZfQllURT4KPj4KPj4gJCBzeXNjdGwga2Vybi5jYW0uZGEuMCB8IGdyZXAgLUUgInRy aW18ZGVsZXRlIgo+PiBrZXJuLmNhbS5kYS4wLnRyaW1fdGlja3M6IDAKPj4ga2Vybi5jYW0uZGEu MC50cmltX2dvYWw6IDAKPj4ga2Vybi5jYW0uZGEuMC50cmltX2xiYXM6IDU0Njg0MTcwNAo+PiBr ZXJuLmNhbS5kYS4wLnRyaW1fcmFuZ2VzOiAzMTcyODAKPj4ga2Vybi5jYW0uZGEuMC50cmltX2Nv dW50OiAzMDE0MjQKPj4ga2Vybi5jYW0uZGEuMC5kZWxldGVfbWF4OiA0Mjk0OTAxNzYwCj4+IGtl cm4uY2FtLmRhLjAuZGVsZXRlX21ldGhvZDogVU5NQVAKPj4KPj4gQlRXOiBlbmFibGluZyB0cmlt IG9uIHRoZSBmcyB3aGlsZSB0aGUgZGV2aWNlIGRvZXMgbm90IHN1cHBvcnQgaXQgZG9lcyBub3Qg bWF0dGVyIGFuZCBkb2VzIG5vIGhhcm0uIFVzaW5nIHRoZSBzeXNjdGwgYWJvdmUgeW91IGNhbiBz ZWUgaWYgdGhlIGRldmljZSBhY3R1YWxseSB1c2VzIHRyaW0uCj4+Cj4+IFJ1bm5pbmcgImdzdGF0 IC1kIiB3aGlsZSBkZWxldGluZyBhIGxvdCBvZiBmaWxlcyB3aWxsIGFsc28gc2hvdyBpZiB0cmlt L2RlbGV0ZSBhY3Rpb25zIGFyZSBzZW50IHRvIHRoZSBkaXNrLgo+Pgo+PiBTbyB0aGUgY29tYmlu YXRpb24gb2YgeW91ciBVU0IgcmVhZGVyICsgbWljcm9TRCBkbyBub3Qgc3VwcG9ydCBUUklNLCBi dXQgeW91IGNhbiBqdXN0IHRyeSB0byBwdXQgdGhlIG1pY3JvU0QgZGlyZWN0bHkgaW4gdGhlIFJQ STQgYW5kIHNlZSB3aGF0IHRoYXQgZG9lcy4KPj4KPj4gUmVnYXJkcywKPj4gUm9uYWxkLgo+Cj4g VGhhbmtzIGZvciB0aGUgY2FsbG91dCEgR290IHdhcm1lZC11cCB3aGVuIEkgc2F3IHRoZSBUUklN IGlzIGVuYWJsZWQgOi0pIGJ1dCBteSB1bHRpbWF0ZSBnb2FsIGlzIHJlYWxseSB0aGUgbWljcm9T RCBjYXJkIHNsb3Qgb2YgbXkgUmFzcGJlcnJ5IFBpIDMgb3IgNCBhbmQgbm90IHdpdGggdGhlIFVT QiByZWFkZXIuIEknbGwgY3JlYXRlIGFuIGltYWdlIHRoZW4gc2VlIGhvdyBpdCBkb2VzLgo+Cj4g QlIsCj4gb3JiaXQKCkNvbmZpcm1lZCwgdGhlIFNhbkRpc2sgVWx0cmEgbWljcm9TRCBkb2VzIG5v dCBzdXBwb3J0IFRSSU0gdXNpbmcgdGhlIG1pY3JvU0Qgc2xvdCBpbiBib3RoIFJhc3BiZXJyeSBQ aSAzIGFuZCA0LiBJbnN0ZWFkIG9mIGJ1aWxkaW5nIGFuIGltYWdlIGZyb20gc2NyYXRjaCwgSSBk b3dubG9hZGVkIGEgRnJlZUJTRCAxNC4wIGltYWdlIGFuZCBjcmVhdGVkIGEgbmV3IHBhcnRpdGlv biAoOEdCIGluIHNpemUpIHdpdGggVUZTL0ZGUyB3aXRoIFRSSU0gZW5hYmxlZCBhbmQgbW91bnQg aXQuIFRoZSBwYXJ0aXRpb24gaXMgL2Rldi9tbWNzZDBzMy4gVGhlcmUncyBubyBkZWxldGUgYWN0 aXZpdHkgb2JzZXJ2ZWQgd2l0aCAiZ3N0YXQgLWQiIHdoZW4gSSBkZWxldGVkIHNvbWUgZmlsZXMg aW4gaXQuCgpyb290QHNkLWV4dHJlbWU6fiAjIC9zYmluL2dwYXJ0IGFkZCAtcyA4RyAtdCBmcmVl YnNkIGRhMApkYTBzMyBhZGRlZApyb290QHNkLWV4dHJlbWU6fiAjIGdwYXJ0IHNob3cKPT4gNjMg MTI0NzM1NDI1IG1tY3NkMCBNQlIgKDU5RykKNjMgMTk4NSAtIGZyZWUgLSAoOTkzSykKMjA0OCAx MDI0MDAgMSBmYXQzMmxiYSBbYWN0aXZlXSAoNTBNKQoxMDQ0NDggMTI0NjI2OTQ0IDIgZnJlZWJz ZCAoNTlHKQoxMjQ3MzEzOTIgNDA5NiAtIGZyZWUgLSAoMi4wTSkKCj0+IDAgMTI0NjI2OTQ0IG1t Y3NkMHMyIEJTRCAoNTlHKQowIDEyOCAtIGZyZWUgLSAoNjRLKQoxMjggMTIwODMzOTIwIDEgZnJl ZWJzZC11ZnMgKDU4RykKMTIwODM0MDQ4IDM3OTI4OTYgMiBmcmVlYnNkLXN3YXAgKDEuOEcpCgo9 PiA2MyAxMjQ3MzU0MjUgZGEwIE1CUiAoNTlHKQo2MyAxOTg1IC0gZnJlZSAtICg5OTNLKQoyMDQ4 IDEwMjQwMCAxIGZhdDMybGJhIFthY3RpdmVdICg1ME0pCjEwNDQ0OCAxMDM4MTMxMiAyIGZyZWVi c2QgKDUuMEcpCjEwNDg1NzYwIDE2Nzc3MjE2IDMgZnJlZWJzZCAoOC4wRykKMjcyNjI5NzYgOTc0 NzI1MTIgLSBmcmVlIC0gKDQ2RykKCj0+IDAgMTAzODEzMTIgZGEwczIgQlNEICg1LjBHKQowIDEy OCAtIGZyZWUgLSAoNjRLKQoxMjggMTAzODExODQgMSBmcmVlYnNkLXVmcyAoNC45RykKCj0+IDYz IDEyNDczNTQyNSBkaXNraWQvRElTSy0xMjEyMjAxNjAyMDQgTUJSICg1OUcpCjYzIDE5ODUgLSBm cmVlIC0gKDk5M0spCjIwNDggMTAyNDAwIDEgZmF0MzJsYmEgW2FjdGl2ZV0gKDUwTSkKMTA0NDQ4 IDEwMzgxMzEyIDIgZnJlZWJzZCAoNS4wRykKMTA0ODU3NjAgMTY3NzcyMTYgMyBmcmVlYnNkICg4 LjBHKQoyNzI2Mjk3NiA5NzQ3MjUxMiAtIGZyZWUgLSAoNDZHKQoKPT4gMCAxMDM4MTMxMiBkaXNr aWQvRElTSy0xMjEyMjAxNjAyMDRzMiBCU0QgKDUuMEcpCjAgMTI4IC0gZnJlZSAtICg2NEspCjEy OCAxMDM4MTE4NCAxIGZyZWVic2QtdWZzICg0LjlHKQoKcm9vdEBzZC1leHRyZW1lOn4gIyBscyAt bGFoIC9kZXYvZGEwCi9kZXYvZGEwIC9kZXYvZGEwczEgL2Rldi9kYTBzMiAvZGV2L2RhMHMyYSAv ZGV2L2RhMHMzCgpyb290QHNkLWV4dHJlbWU6fiAjIG5ld2ZzIC1VIC10IC9kZXYvZGEwczMKL2Rl di9kYTBzMzogODE5Mi4wTUIgKDE2Nzc3MjE2IHNlY3RvcnMpIGJsb2NrIHNpemUgMzI3NjgsIGZy YWdtZW50IHNpemUgNDA5Ngp1c2luZyAxNCBjeWxpbmRlciBncm91cHMgb2YgNjI1LjIyTUIsIDIw MDA3IGJsa3MsIDgwMTI4IGlub2Rlcy4Kd2l0aCBzb2Z0IHVwZGF0ZXMKc3VwZXItYmxvY2sgYmFj a3VwcyAoZm9yIGZzY2tfZmZzIC1iICMpIGF0OgoxOTIsIDEyODA2NDAsIDI1NjEwODgsIDM4NDE1 MzYsIDUxMjE5ODQsIDY0MDI0MzIsIDc2ODI4ODAsIDg5NjMzMjgsIDEwMjQzNzc2LCAxMTUyNDIy NCwgMTI4MDQ2NzIsIDE0MDg1MTIwLAoxNTM2NTU2OCwgMTY2NDYwMTYKCnJvb3RAc2QtdWx0cmE6 fiAjIGdwYXJ0IHNob3cgLWwKPT4gNjMgMTI0NzM1NDI1IG1tY3NkMCBNQlIgKDU5RykKNjMgMTk4 NSAtIGZyZWUgLSAoOTkzSykKMjA0OCAxMDI0MDAgMSAobnVsbCkgW2FjdGl2ZV0gKDUwTSkKMTA0 NDQ4IDEwMzgxMzEyIDIgKG51bGwpICg1LjBHKQoxMDQ4NTc2MCAxNjc3NzIxNiAzIChudWxsKSAo OC4wRykKMjcyNjI5NzYgOTc0NzI1MTIgLSBmcmVlIC0gKDQ2RykKCj0+IDAgMTAzODEzMTIgbW1j c2QwczIgQlNEICg1LjBHKQowIDEyOCAtIGZyZWUgLSAoNjRLKQoxMjggMTAzODExODQgMSAobnVs bCkgKDQuOUcpCgpyb290QHNkLXVsdHJhOn4gIyBtb3VudCAtbAovZGV2L3Vmcy9yb290ZnMgb24g LyAodWZzLCBsb2NhbCwgc29mdC11cGRhdGVzKQpkZXZmcyBvbiAvZGV2IChkZXZmcykKL2Rldi9t c2Rvc2ZzL0VGSSBvbiAvYm9vdC9lZmkgKG1zZG9zZnMsIGxvY2FsLCBub2F0aW1lKQp0bXBmcyBv biAvdG1wICh0bXBmcywgbG9jYWwpCi9kZXYvbW1jc2QwczMgb24gL21udCAodWZzLCBsb2NhbCwg c29mdC11cGRhdGVzKQoKcm9vdEBzZC11bHRyYTp+ICMgZ3N0YXQgLWQKZFQ6IDEuMDY1cyB3OiAx LjAwMHMKTChxKSBvcHMvcyByL3Mga0JwcyBtcy9yIHcvcyBrQnBzIG1zL3cgZC9zIGtCcHMgbXMv ZCAlYnVzeSBOYW1lCjAgMCAwIDAgMC4wIDAgMCAwLjAgMCAwIDAuMCAwLjB8IG1tY3NkMAowIDAg MCAwIDAuMCAwIDAgMC4wIDAgMCAwLjAgMC4wfCBtbWNzZDBzMQowIDAgMCAwIDAuMCAwIDAgMC4w IDAgMCAwLjAgMC4wfCBtbWNzZDBzMgowIDAgMCAwIDAuMCAwIDAgMC4wIDAgMCAwLjAgMC4wfCBt bWNzZDBzMwowIDAgMCAwIDAuMCAwIDAgMC4wIDAgMCAwLjAgMC4wfCBtc2Rvc2ZzL0VGSQowIDAg MCAwIDAuMCAwIDAgMC4wIDAgMCAwLjAgMC4wfCBtbWNzZDBzMmEgMCAwIDAgMCAwLjAgMCAwIDAu MCAwIDAgMC4wIDAuMHwgdWZzL3Jvb3Rmcwpyb290QHNkLXVsdHJhOn4gIyBtb3VudCAtbCAtdgov ZGV2L3Vmcy9yb290ZnMgb24gLyAodWZzLCBsb2NhbCwgc29mdC11cGRhdGVzLCB3cml0ZXM6IHN5 bmMgODM4IGFzeW5jIDQzMTYsIHJlYWRzOiBzeW5jIDIyNDEgYXN5bmMgMjY5NywgZnNpZCA4OWZj NGQ2NTJhYmExN2NkLCB2bm9kZXM6IGNvdW50IDEyNjQgKQpkZXZmcyBvbiAvZGV2IChkZXZmcywg ZnNpZCAwMGZmMDA3MTcxMDAwMDAwLCB2bm9kZXM6IGNvdW50IDM0ICkKL2Rldi9tc2Rvc2ZzL0VG SSBvbiAvYm9vdC9lZmkgKG1zZG9zZnMsIGxvY2FsLCBub2F0aW1lLCB3cml0ZXM6IHN5bmMgMSBh c3luYyAwLCByZWFkczogc3luYyA4IGFzeW5jIDAsIGZzaWQgNWIwMDAwMDAzMjAwMDAwMCwgdm5v ZGVzOiBjb3VudCAxICkKdG1wZnMgb24gL3RtcCAodG1wZnMsIGxvY2FsLCBmc2lkIDAxZmYwMDg3 ODcwMDAwMDAsIHZub2RlczogY291bnQgNiApCi9kZXYvbW1jc2QwczMgb24gL21udCAodWZzLCBs b2NhbCwgc29mdC11cGRhdGVzLCB3cml0ZXM6IHN5bmMgMiBhc3luYyA3ODY2LCByZWFkczogc3lu YyAxMjE0OTEgYXN5bmMgMCwgZnNpZCA0ZWUwYzg2NTA2MzMwODkwLCB2bm9kZXM6IGNvdW50IDMg KQoKcm9vdEBzZC11bHRyYTp+ICMgc3lzY3RsIC1hIHwgZ3JlcCB0cmltCjwxMTg+Q3JlYXRpbmcg YW5kL29yIHRyaW1taW5nIGxvZyBmaWxlcy4Ka2Vybi5jYW0ubmRhLm1heF90cmltOiAyNTZ2ZnMu ZmZzLmRvdHJpbWNvbnM6IDEKCkknbGwgcHJvY2VlZCB3aXRoIHBsYW4gQiwgdG8gYnV5IGluZHVz dHJpYWwgZ3JhZGUgbWljcm9TRCBjYXJkIGhhdmluZyBhIGdhcmJhZ2UgY29sbGVjdGlvbiBmZWF0 dXJlIGluIHRoZSBjb250cm9sbGVyIGluc3RlYWQgb2YgdGhpcyBjb25zdW1lciBncmFkZSBjYXJk LiBFaXRoZXIgVFJJTSB3aWxsIHdvcmsgb3Igbm90IHRoZXJlJ3Mgc3RpbGwgZ2FyYmFnZSBjb2xs ZWN0aW9uIEkgY291bGQgYmVuZWZpdC4KCkJSLApvcmJpdAoKPg== --b1_hp6lbu7uCCqtZcnP3HWOX4Pb94jbjUYIGrM6EMDv3o Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: base64 T24gRnJpZGF5LCAxNiBGZWJydWFyeSAyMDI0IGF0IDE4OjU0LCBPcmRpbmFyeSBCaXQgJmx0O29y ZGluYXJ5Yml0QHByb3Rvbi5tZSZndDsgd3JvdGU6DQogICAgICAgIDxkaXYgY2xhc3M9InByb3Rv bm1haWxfcXVvdGUiPjxibG9ja3F1b3RlIGNsYXNzPSJwcm90b25tYWlsX3F1b3RlIiB0eXBlPSJj aXRlIj4NCiAgICAgICAgICAgIE9uIEZyaWRheSwgMTYgRmVicnVhcnkgMjAyNCBhdCAxODozMSwg Um9uYWxkIEtsb3AgJmx0O3JvbmFsZC1saXN0c0BrbG9wLndzJmd0OyB3cm90ZToNCiAgICAgICAg PGRpdiBjbGFzcz0icHJvdG9ubWFpbF9xdW90ZSI+PGJsb2NrcXVvdGUgdHlwZT0iY2l0ZSIgY2xh c3M9InByb3Rvbm1haWxfcXVvdGUiPg0KICAgICAgICAgICAgPGJyPg0KPHA+PHN0cm9uZz5WYW46 PC9zdHJvbmc+IE9yZGluYXJ5IEJpdCAmbHQ7b3JkaW5hcnliaXRAcHJvdG9uLm1lJmd0Ozxicj4N CjxzdHJvbmc+RGF0dW06PC9zdHJvbmc+IHZyaWpkYWcsIDE2IGZlYnJ1YXJpIDIwMjQgMTA6MTg8 YnI+DQo8c3Ryb25nPkFhbjo8L3N0cm9uZz4gTWFyayBNaWxsYXJkICZsdDttYXJrbG1pQHlhaG9v LmNvbSZndDs8YnI+DQo8c3Ryb25nPkNDOjwvc3Ryb25nPiAiZnJlZWJzZC1hcm1AZnJlZWJzZC5v cmciICZsdDtmcmVlYnNkLWFybUBmcmVlYnNkLm9yZyZndDs8YnI+DQo8c3Ryb25nPk9uZGVyd2Vy cDo8L3N0cm9uZz4gUmU6IG5ld2ZzIFRSSU0gZmxhZyBkZXZpY2Ugc3VwcG9ydDwvcD4NCg0KPGJs b2NrcXVvdGUgc3R5bGU9InBhZGRpbmctcmlnaHQ6IDBweDsgcGFkZGluZy1sZWZ0OiA1cHg7IG1h cmdpbi1sZWZ0OiA1cHg7IGJvcmRlci1sZWZ0OiAjMDAwMDAwIDJweCBzb2xpZDsgbWFyZ2luLXJp Z2h0OiAwcHgiPg0KPGRpdiBjbGFzcz0iTWVzc2FnZVJGQzgyMlZpZXdlciIgaWQ9IlAiPg0KPGRp diBjbGFzcz0iVGV4dFBsYWluVmlld2VyIiBpZD0iUC5QIj48YnI+DQo8YnI+DQo8YnI+DQo8YnI+ DQo8YnI+DQpTZW50IHdpdGggUHJvdG9uIE1haWwgc2VjdXJlIGVtYWlsLjxicj4NCjxicj4NCk9u IEZyaWRheSwgMTYgRmVicnVhcnkgMjAyNCBhdCAxNDowMiwgTWFyayBNaWxsYXJkICZsdDttYXJr bG1pQHlhaG9vLmNvbSZndDsgd3JvdGU6PGJyPg0KPGJyPg0KJmd0OyBPbiBGZWIgMTUsIDIwMjQs IGF0IDIwOjA4LCBPcmRpbmFyeSBCaXQgb3JkaW5hcnliaXRAcHJvdG9uLm1lIHdyb3RlOjxicj4N CiZndDs8YnI+DQomZ3Q7ICZndDsgSGkhPGJyPg0KJmd0Ozxicj4NCiZndDs8YnI+DQomZ3Q7IEhl bGxvLjxicj4NCiZndDs8YnI+DQomZ3Q7ICZndDsgT24gRnJpZGF5LCAxNiBGZWJydWFyeSAyMDI0 IGF0IDExOjQxLCBNYXJrIE1pbGxhcmQgbWFya2xtaUB5YWhvby5jb20gd3JvdGU6PGJyPg0KJmd0 OyAmZ3Q7PGJyPg0KJmd0OyAmZ3Q7ICZndDsgW09ubHkgcmVwbHlpbmcgdG8gbGlzdHMgSSBzdWJz Y3JpYmUgdG8uXTxicj4NCiZndDsgJmd0OyAmZ3Q7PGJyPg0KJmd0OyAmZ3Q7ICZndDsgT24gRmVi IDE1LCAyMDI0LCBhdCAxOToxOSwgT3JkaW5hcnkgQml0IG9yZGluYXJ5Yml0QHByb3Rvbi5tZSB3 cm90ZTo8YnI+DQomZ3Q7ICZndDsgJmd0Ozxicj4NCiZndDsgJmd0OyAmZ3Q7ICZndDsgSSdtIHJl YWRpbmcgdGhlIG5ld2ZzIG1hbnVhbCA8YSB0YXJnZXQ9Il9ibGFuayIgcmVsPSJub3JlZmVycmVy IG5vZm9sbG93IG5vb3BlbmVyIiBocmVmPSJodHRwczovL21hbi5mcmVlYnNkLm9yZy9jZ2kvbWFu LmNnaT9uZXdmcyg4Ij5odHRwczovL21hbi5mcmVlYnNkLm9yZy9jZ2kvbWFuLmNnaT9uZXdmcyg4 PC9hPikgdG8gYmUgYWJsZSB0byBrbm93IGFib3V0IHRoZSBUUklNIGZsYWcuIEluIHRoZSBtYW51 YWwgdW5kZXIgLXQgcGFyYW1ldGVyLCBpdCBtZW50aW9uZWQgYWJvdXQgInVuZGVybHlpbmcgZGV2 aWNlIHN1cHBvcnQiLCB3aGF0IGV4YWN0bHkgaXMgdGhpcyBkZXZpY2U/PGJyPg0KJmd0OyAmZ3Q7 ICZndDs8YnI+DQomZ3Q7ICZndDsgJmd0OyAyIGNvbnRyYXN0aW5nIGV4YW1wbGVzOjxicj4NCiZn dDsgJmd0OyAmZ3Q7PGJyPg0KJmd0OyAmZ3Q7ICZndDsgRXhhbXBsZSAwOiBPcHRhbmUgTlZNZSBt ZWRpYSAoUENJZSBjYXJkIG9yIFUuMiwgZm9yIGV4YW1wbGUpPGJyPg0KJmd0OyAmZ3Q7ICZndDs8 YnI+DQomZ3Q7ICZndDsgJmd0OyBPcHRhbmUgaGFzIG5vIG5lZWQgb2YgVFJJTSBhbmQsIHNvLCBu ZXZlciBzdXBwb3J0cyBUUklNLjxicj4NCiZndDsgJmd0OyAmZ3Q7PGJyPg0KJmd0OyAmZ3Q7ICZn dDsgRXhhbXBsZSAxOiBtaWNyb3NkIGNhcmQgbWVkaWEgdXNhZ2U8YnI+DQomZ3Q7ICZndDsgJmd0 Ozxicj4NCiZndDsgJmd0OyAmZ3Q7IEEgbWljcm9zZCBjYXJkIGluIHRoZSBub3JtYWwgdHlwZSBv ZiBtaWNyb3NkIGNhcmQgc2xvdCBvbjxicj4NCiZndDsgJmd0OyAmZ3Q7IFNtYWxsIEJvYXJkIENv bXB1dGVycyAobm9ybWFsbHkpIHN1cHBvcnRzIFRSSU0uIFRha2UgdGhlPGJyPg0KJmd0OyAmZ3Q7 ICZndDsgc2FtZSBjYXJkIGFuZCBwdXQgaXQgaW4gYSBVU0IgcmVhZGVyL3dyaXRlciBhbmQgdXNl IGl0PGJyPg0KJmd0OyAmZ3Q7ICZndDsgdmlhIFVTQiBvbiB0aGUgc2FtZSBzeXN0ZW06IG5vIFRS SU0gaXMgc3VwcG9ydGVkIGJ5PGJyPg0KJmd0OyAmZ3Q7ICZndDsgRnJlZUJTRCBvdmVyIFVTQi48 YnI+DQomZ3Q7ICZndDs8YnI+DQomZ3Q7ICZndDsgU28geW91IG1lYW4gdG8gc2F5IHRoYXQgaWYg SSBoYXZlIGEgUmFzcGVycnkgUGkgMyBvciA0IG5vdyBhbmQgdGhlbiBoYXZlIG15IEZyZWVCU0Qg aW5zdGFsbGVkIGluIGEgbWljcm9TRCBjYXJkIChmb3IgZXhhbXBsZSwgU2FuRGlzayBFeHRyZW1l IGNhcmQpIHdpdGggVUZTL0ZGUyBmaWxlc3l0ZW0gaW4gaXQgd2l0aCBUUklNIGVuYWJsZWQgcGFy YW1ldGVyIHRoZW4gaXMgaXQgZ29pbmcgdG8gcmVjb2duaXplIGl0PyBIb3cgdG8gdmVyaWZ5Pzxi cj4NCiZndDsgJmd0Ozxicj4NCiZndDsgJmd0OyAmZ3Q7IEZZSTo8YnI+DQomZ3Q7ICZndDsgJmd0 Ozxicj4NCiZndDsgJmd0OyAmZ3Q7IFdoZW4gdGhlIGZpbGUgc3lzdGVtIGhhcyBUUklNIGVuYWJs ZWQsIEZyZWVCU0QgcHV0IG91dCBhPGJyPg0KJmd0OyAmZ3Q7ICZndDsgbm90aWNlIGlmIFRSSU0g d2lsbCBub3QgYWN0dWFsbHkgYmUgdXNlZCBpbiB0aGUgYWN0dWFsPGJyPg0KJmd0OyAmZ3Q7ICZn dDsgY29udGV4dCBpbiB1c2UuPGJyPg0KJmd0OyAmZ3Q7PGJyPg0KJmd0OyAmZ3Q7IE9rLCBnb3Qg aXQuIEhvdyB0byBjaGVjayB0aGlzIGFzIHdlbGw/PGJyPg0KJmd0Ozxicj4NCiZndDs8YnI+DQom Z3Q7IFRoZSBjb25zb2xlIGdldHMgYSBtZXNzYWdlIGxpa2U6PGJyPg0KJmd0Ozxicj4NCiZndDsg V0FSTklORzogL21udDogVFJJTSBmbGFnIG9uIGZzIGJ1dCBkaXNrIGRvZXMgbm90IHN1cHBvcnQg VFJJTTxicj4NCiZndDs8YnI+DQomZ3Q7IHdoZW4gYSBtb3VudCBpcyBhdHRlbXB0ZWQgKGF1dG9t YXRpYyBvciBtYW51YWxseSkgZm9yIGE8YnI+DQomZ3Q7IGZpbGUgc3lzdGVtIHdpdGggdGhlIHRy aW0gZmxhZyBlbmFibGVkIGJ1dCB0cmltIGRvZXMgbm90PGJyPg0KJmd0OyBlbmQgdXAgYWN0aXZl Ljxicj4NCiZndDs8YnI+DQomZ3Q7IFNvLCBmb3IgZXhhbXBsZTo8YnI+DQomZ3Q7PGJyPg0KJmd0 OyAjIGRtZXNnIC1hIHwgZ3JlcCBUUklNPGJyPg0KJmd0OyBXQVJOSU5HOiAvbW50OiBUUklNIGZs YWcgb24gZnMgYnV0IGRpc2sgZG9lcyBub3Qgc3VwcG9ydCBUUklNPGJyPg0KJmd0Ozxicj4NCiZn dDsgKFRoaXMgd2FzIGEgbWljcm9zZCBjYXJkIGluIGEgVVNCIHJlYWRlci93cml0ZXIgdGhhdCB3 YXMgbm90PGJyPg0KJmd0OyB1c2VkIGFzIHRoZSBib290IG1lZGlhOiBhIHNlcGFyYXRlIG1hbnVh bCBtb3VudCB3YXMgdXNlZC4pPGJyPg0KJmd0Ozxicj4NCiZndDsgVGhlIGZpbGUgc3lzdGVtJ3Mg VFJJTSBmbGFnIHN0YXR1cyBjYW4gYmUgY2hlY2tlZCB2aWEgdXNlIG9mPGJyPg0KJmd0OyAidHVu ZWZzIC1wIC4gLiAuIjo8YnI+DQomZ3Q7PGJyPg0KJmd0OyAjIHR1bmVmcyAtcCAvbW50PGJyPg0K Jmd0OyB0dW5lZnM6IFBPU0lYLjFlIEFDTHM6ICgtYSkgZGlzYWJsZWQ8YnI+DQomZ3Q7IHR1bmVm czogTkZTdjQgQUNMczogKC1OKSBkaXNhYmxlZDxicj4NCiZndDsgdHVuZWZzOiBNQUMgbXVsdGls YWJlbDogKC1sKSBkaXNhYmxlZDxicj4NCiZndDsgdHVuZWZzOiBzb2Z0IHVwZGF0ZXM6ICgtbikg ZW5hYmxlZDxicj4NCiZndDsgdHVuZWZzOiBzb2Z0IHVwZGF0ZSBqb3VybmFsaW5nOiAoLWopIGRp c2FibGVkPGJyPg0KJmd0OyB0dW5lZnM6IGdqb3VybmFsOiAoLUopIGRpc2FibGVkPGJyPg0KJmd0 OyB0dW5lZnM6IHRyaW06ICgtdCkgZW5hYmxlZDxicj4NCiZndDsgdHVuZWZzOiBtYXhpbXVtIGJs b2NrcyBwZXIgZmlsZSBpbiBhIGN5bGluZGVyIGdyb3VwOiAoLWUpIDQwOTY8YnI+DQomZ3Q7IHR1 bmVmczogYXZlcmFnZSBmaWxlIHNpemU6ICgtZikgMTYzODQ8YnI+DQomZ3Q7IHR1bmVmczogYXZl cmFnZSBudW1iZXIgb2YgZmlsZXMgaW4gYSBkaXJlY3Rvcnk6ICgtcykgNjQ8YnI+DQomZ3Q7IHR1 bmVmczogbWluaW11bSBwZXJjZW50YWdlIG9mIGZyZWUgc3BhY2U6ICgtbSkgOCU8YnI+DQomZ3Q7 IHR1bmVmczogc3BhY2UgdG8gaG9sZCBmb3IgbWV0YWRhdGEgYmxvY2tzOiAoLWspIDY0MDA8YnI+ DQomZ3Q7IHR1bmVmczogb3B0aW1pemF0aW9uIHByZWZlcmVuY2U6ICgtbykgdGltZTxicj4NCiZn dDsgdHVuZWZzOiB2b2x1bWUgbGFiZWw6ICgtTCk8YnI+DQomZ3Q7PGJyPg0KJmd0OyBJZiB0aGUg dHJpbSBmbGFnIGlzIGVuYWJsZWQgYnV0IHRoZSBtb3VudCBkb2VzIG5vdCBwcm9kdWNlIHRoZTxi cj4NCiZndDsgY29uc29sZSBtZXNzYWdlLCB0aGVuIFRSSU0gaXMgYWN0aXZlLjxicj4NCiZndDs8 YnI+DQo8YnI+DQpPaCwgdGhhdCdzIGFtYXppbmchIEkgdHJ5IGl0IHdpdGggbXkgU2FuRGlzayBV bHRyYSBtaWNyb1NEWEMgNjRHQiBhbmQgbW91bnQgaXQgaW4gYSBVU0IgcmVhZGVyIGFuZCBpdCBz aG93cyBlbmFibGVkIGluIHRoZSB0dW5lZnMgYW5kIGlzIGRldGVjdGVkIGluIHRoZSBkbWVzZywg dGhlIHNhbWUgYXMgeW91cnMuIFRoZXJlZm9yZSwgU2FuRGlzayBVbHRyYSBtaWNyb1NEIHN1cHBv cnRzIGl0ITxicj4NCjxicj4NCnJvb3RAc2QtZXh0cmVtZTp+ICMgL3NiaW4vbmV3ZnMgLVUgLXQg L2Rldi9kYTBzMmE8YnI+DQovZGV2L2RhMHMyYTogNTkwMDAuOU1CICgxMjA4MzM5MjAgc2VjdG9y cykgYmxvY2sgc2l6ZSAzMjc2OCwgZnJhZ21lbnQgc2l6ZSA0MDk2PGJyPg0KJm5ic3A7Jm5ic3A7 Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7dXNpbmcgOTUgY3lsaW5kZXIgZ3Jv dXBzIG9mIDYyNS4yMk1CLCAyMDAwNyBibGtzLCA4MDEyOCBpbm9kZXMuPGJyPg0KJm5ic3A7Jm5i c3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7d2l0aCBzb2Z0IHVwZGF0ZXM8 YnI+DQpzdXBlci1ibG9jayBiYWNrdXBzIChmb3IgZnNja19mZnMgLWIgIykgYXQ6PGJyPg0KJm5i c3A7MTkyLCAxMjgwNjQwLCAyNTYxMDg4LCAzODQxNTM2LCA1MTIxOTg0LCA2NDAyNDMyLCA3Njgy ODgwLCA4OTYzMzI4LCAxMDI0Mzc3NiwgMTE1MjQyMjQsIDEyODA0NjcyLCAxNDA4NTEyMCw8YnI+ DQombmJzcDsxNTM2NTU2OCwgMTY2NDYwMTYsIDE3OTI2NDY0LCAxOTIwNjkxMiwgMjA0ODczNjAs IDIxNzY3ODA4LCAyMzA0ODI1NiwgMjQzMjg3MDQsIDI1NjA5MTUyLCAyNjg4OTYwMCwgMjgxNzAw NDgsPGJyPg0KJm5ic3A7Mjk0NTA0OTYsIDMwNzMwOTQ0LCAzMjAxMTM5MiwgMzMyOTE4NDAsIDM0 NTcyMjg4LCAzNTg1MjczNiwgMzcxMzMxODQsIDM4NDEzNjMyLCAzOTY5NDA4MCwgNDA5NzQ1Mjgs IDQyMjU0OTc2LDxicj4NCiZuYnNwOzQzNTM1NDI0LCA0NDgxNTg3MiwgNDYwOTYzMjAsIDQ3Mzc2 NzY4LCA0ODY1NzIxNiwgNDk5Mzc2NjQsIDUxMjE4MTEyLCA1MjQ5ODU2MCwgNTM3NzkwMDgsIDU1 MDU5NDU2LCA1NjMzOTkwNCw8YnI+DQombmJzcDs1NzYyMDM1MiwgNTg5MDA4MDAsIDYwMTgxMjQ4 LCA2MTQ2MTY5NiwgNjI3NDIxNDQsIDY0MDIyNTkyLCA2NTMwMzA0MCwgNjY1ODM0ODgsIDY3ODYz OTM2LCA2OTE0NDM4NCwgNzA0MjQ4MzIsPGJyPg0KJm5ic3A7NzE3MDUyODAsIDcyOTg1NzI4LCA3 NDI2NjE3NiwgNzU1NDY2MjQsIDc2ODI3MDcyLCA3ODEwNzUyMCwgNzkzODc5NjgsIDgwNjY4NDE2 LCA4MTk0ODg2NCwgODMyMjkzMTIsIDg0NTA5NzYwLDxicj4NCiZuYnNwOzg1NzkwMjA4LCA4NzA3 MDY1NiwgODgzNTExMDQsIDg5NjMxNTUyLCA5MDkxMjAwMCwgOTIxOTI0NDgsIDkzNDcyODk2LCA5 NDc1MzM0NCwgOTYwMzM3OTIsIDk3MzE0MjQwLCA5ODU5NDY4OCw8YnI+DQombmJzcDs5OTg3NTEz NiwgMTAxMTU1NTg0LCAxMDI0MzYwMzIsIDEwMzcxNjQ4MCwgMTA0OTk2OTI4LCAxMDYyNzczNzYs IDEwNzU1NzgyNCwgMTA4ODM4MjcyLCAxMTAxMTg3MjAsIDExMTM5OTE2OCw8YnI+DQombmJzcDsx MTI2Nzk2MTYsIDExMzk2MDA2NCwgMTE1MjQwNTEyLCAxMTY1MjA5NjAsIDExNzgwMTQwOCwgMTE5 MDgxODU2LCAxMjAzNjIzMDQ8YnI+DQo8YnI+DQpyb290QHNkLWV4dHJlbWU6fiAjIGRtZXNnIHwg Z3JlcCBUUklNPGJyPg0KV0FSTklORzogL21udDogVFJJTSBmbGFnIG9uIGZzIGJ1dCBkaXNrIGRv ZXMgbm90IHN1cHBvcnQgVFJJTTxicj4NCjxicj4NCnJvb3RAc2QtZXh0cmVtZTp+ICMgdHVuZWZz IC1wIC9tbnQ8YnI+DQp0dW5lZnM6IFBPU0lYLjFlIEFDTHM6ICgtYSkgJm5ic3A7Jm5ic3A7Jm5i c3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7 Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5i c3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7 ZGlzYWJsZWQ8YnI+DQp0dW5lZnM6IE5GU3Y0IEFDTHM6ICgtTikgJm5ic3A7Jm5ic3A7Jm5ic3A7 Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5i c3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7 Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5i c3A7Jm5ic3A7Jm5ic3A7ZGlzYWJsZWQ8YnI+DQp0dW5lZnM6IE1BQyBtdWx0aWxhYmVsOiAoLWwp ICZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZu YnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNw OyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZu YnNwOyZuYnNwO2Rpc2FibGVkPGJyPg0KdHVuZWZzOiBzb2Z0IHVwZGF0ZXM6ICgtbikgJm5ic3A7 Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5i c3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7 Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5i c3A7Jm5ic3A7Jm5ic3A7ZW5hYmxlZDxicj4NCnR1bmVmczogc29mdCB1cGRhdGUgam91cm5hbGlu ZzogKC1qKSAmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsm bmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJz cDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDtkaXNhYmxlZDxicj4NCnR1bmVmczogZ2pvdXJuYWw6 ICgtSikgJm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5i c3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7 Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5i c3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7ZGlzYWJs ZWQ8YnI+DQp0dW5lZnM6IHRyaW06ICgtdCkgJm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7 Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5i c3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7 Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5i c3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7ZW5hYmxlZDxicj4NCnR1bmVm czogbWF4aW11bSBibG9ja3MgcGVyIGZpbGUgaW4gYSBjeWxpbmRlciBncm91cDogKC1lKSAmbmJz cDs0MDk2PGJyPg0KdHVuZWZzOiBhdmVyYWdlIGZpbGUgc2l6ZTogKC1mKSAmbmJzcDsmbmJzcDsm bmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJz cDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsm bmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsxNjM4NDxicj4NCnR1bmVmczogYXZl cmFnZSBudW1iZXIgb2YgZmlsZXMgaW4gYSBkaXJlY3Rvcnk6ICgtcykgJm5ic3A7Jm5ic3A7Jm5i c3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7NjQ8YnI+DQp0dW5lZnM6IG1pbmltdW0gcGVyY2VudGFnZSBv ZiBmcmVlIHNwYWNlOiAoLW0pICZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZu YnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOzglPGJyPg0KdHVuZWZzOiBzcGFjZSB0 byBob2xkIGZvciBtZXRhZGF0YSBibG9ja3M6ICgtaykgJm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7 Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7NjQwMDxicj4NCnR1bmVm czogb3B0aW1pemF0aW9uIHByZWZlcmVuY2U6ICgtbykgJm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7 Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5i c3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7dGltZTxicj4NCnR1 bmVmczogdm9sdW1lIGxhYmVsOiAoLUwpPGJyPg0KPGJyPg0KSSBwbGFuIHRvIGhhdmUgVFJJTSBm bGFnIGVuYWJsZWQgaW4gdGhlIHJvb3RmcyBwYXJ0aXRpb24gKC9kZXYvdWZzL3Jvb3Rmcykgb2Yg bXkgUmFzcGJlcnJ5IFBpLiBEbyB5b3UgdGhpbmsgb2YgYW55IGltcGxpY2F0aW9ucyB3aGVuIGVu YWJsZWQ/IEFzIHNob3duIGJlbG93LCBpdCBpcyBkaXNhYmxlZC48YnI+DQo8YnI+DQpyb290QHNk LWV4dHJlbWU6fiAjIHR1bmVmcyAtcCAvZGV2L3Vmcy9yb290ZnM8YnI+DQp0dW5lZnM6IFBPU0lY LjFlIEFDTHM6ICgtYSkgJm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7 Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5i c3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7 Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7ZGlzYWJsZWQ8YnI+DQp0dW5lZnM6IE5GU3Y0 IEFDTHM6ICgtTikgJm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5i c3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7 Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5i c3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7ZGlzYWJsZWQ8YnI+ DQp0dW5lZnM6IE1BQyBtdWx0aWxhYmVsOiAoLWwpICZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZu YnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNw OyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZu YnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwO2Rpc2FibGVkPGJyPg0KdHVu ZWZzOiBzb2Z0IHVwZGF0ZXM6ICgtbikgJm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5i c3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7 Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5i c3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7ZW5hYmxlZDxicj4N CnR1bmVmczogc29mdCB1cGRhdGUgam91cm5hbGluZzogKC1qKSAmbmJzcDsmbmJzcDsmbmJzcDsm bmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJz cDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDtk aXNhYmxlZDxicj4NCnR1bmVmczogZ2pvdXJuYWw6ICgtSikgJm5ic3A7Jm5ic3A7Jm5ic3A7Jm5i c3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7 Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5i c3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7 Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7ZGlzYWJsZWQ8YnI+DQp0dW5lZnM6IHRyaW06ICgtdCkg Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5i c3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7 Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5i c3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7 Jm5ic3A7Jm5ic3A7ZGlzYWJsZWQ8YnI+DQp0dW5lZnM6IG1heGltdW0gYmxvY2tzIHBlciBmaWxl IGluIGEgY3lsaW5kZXIgZ3JvdXA6ICgtZSkgJm5ic3A7NDA5Njxicj4NCnR1bmVmczogYXZlcmFn ZSBmaWxlIHNpemU6ICgtZikgJm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5i c3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7 Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5i c3A7Jm5ic3A7MTYzODQ8YnI+DQp0dW5lZnM6IGF2ZXJhZ2UgbnVtYmVyIG9mIGZpbGVzIGluIGEg ZGlyZWN0b3J5OiAoLXMpICZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOzY0PGJy Pg0KdHVuZWZzOiBtaW5pbXVtIHBlcmNlbnRhZ2Ugb2YgZnJlZSBzcGFjZTogKC1tKSAmbmJzcDsm bmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJz cDsmbmJzcDs4JTxicj4NCnR1bmVmczogc3BhY2UgdG8gaG9sZCBmb3IgbWV0YWRhdGEgYmxvY2tz OiAoLWspICZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZu YnNwOyZuYnNwOyZuYnNwOzY0MDA8YnI+DQp0dW5lZnM6IG9wdGltaXphdGlvbiBwcmVmZXJlbmNl OiAoLW8pICZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZu YnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNw OyZuYnNwOyZuYnNwOyZuYnNwO3RpbWU8YnI+DQp0dW5lZnM6IHZvbHVtZSBsYWJlbDogKC1MKSAm bmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJz cDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsm bmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJz cDsmbmJzcDsmbmJzcDsmbmJzcDtyb290ZnM8YnI+DQo8YnI+DQpCUiw8YnI+DQpvcmJpdDxicj4N CiZuYnNwOzxicj4NCjxicj4NCjxicj4NCiZuYnNwOzwvZGl2Pg0KDQo8aHI+PC9kaXY+DQo8L2Js b2NrcXVvdGU+DQo8YnI+DQpIaSw8YnI+DQo8YnI+DQpJJ20gc29ycnkgdG8gc3BvaWwgdGhlIGZ1 biBidXQgdGhpcyBtZXNzYWdlIHNheXMgdGhlIGRldmljZSBkb2VzICpub3QqIHN1cHBvcnQgdHJp bS48YnI+DQo8YnI+DQoicm9vdEBzZC1leHRyZW1lOn4gIyBkbWVzZyB8IGdyZXAgVFJJTTxicj4N CldBUk5JTkc6IC9tbnQ6IFRSSU0gZmxhZyBvbiBmcyBidXQgZGlzayBkb2VzIG5vdCBzdXBwb3J0 IFRSSU0iPGJyPg0KPGJyPg0KPGJyPg0KQnV0IHRvIGdpdmUgc29tZSBob3BlLiBBcyBNYXJrIHBv aW50ZWQgb3V0IGl0IGlzIG5vdCBvbmx5IGFib3V0IHRoZSBkaXNrLiBUaGUgY29udHJvbGxlciBh bHNvIG5lZWRzIHRvIHN1cHBvcnQgaXQuPGJyPg0KPGJyPg0KQXMgYSByZWFsIHdvcmxkIGV4YW1w bGU6PGJyPg0KPGJyPg0KSSBoYXZlIGFuIFJQSTMgd2l0aCBhbiBTU0QgKHN1cHBvcnRpbmcgdHJp bSkgY29ubmVjdGVkIHZpYSBVU0ItdG8tU0FUQSAobm90IHN1cHBvcnRpbmcgdHJpbSkuIFNvIHRo ZSB0b3RhbCBzeXN0ZW0gZG9lcyBub3Qgc3VwcG9ydCB0cmltLjxicj4NCiQgY2F0IC92YXIvcnVu L2RtZXNnLmJvb3QgfCBncmVwIC1FICJ1bWFzczB8ZGEwfFRSSU0iPGJyPg0KdW1hc3MwIG9uIHVo dWIxPGJyPg0KdW1hc3MwOiAmbHQ7R09ETyBVU0IzLjAgdG8gU0FUQSBhZGFwdGVyLCBjbGFzcyAw LzAsIHJldiAyLjEwLzEuMDAsIGFkZHIgNSZndDsgb24gdXNidXMxPGJyPg0KdW1hc3MwOiZuYnNw OyBTQ1NJIG92ZXIgQnVsay1Pbmx5OyBxdWlya3MgPSAweDAxMDA8YnI+DQp1bWFzczA6MDowOiBB dHRhY2hlZCB0byBzY2J1czA8YnI+DQpkYTAgYXQgdW1hc3Mtc2ltMCBidXMgMCBzY2J1czAgdGFy Z2V0IDAgbHVuIDA8YnI+DQpkYTA6ICZsdDtMSVRFT04gQyBWMy1ERTI1NiAwJmd0OyBGaXhlZCBE aXJlY3QgQWNjZXNzIFNQQy00IFNDU0kgZGV2aWNlPGJyPg0KZGEwOiBTZXJpYWwgTnVtYmVyIDIw MTUxMDAwMDBENjxicj4NCmRhMDogNDAuMDAwTUIvcyB0cmFuc2ZlcnM8YnI+DQpkYTA6IDI0NDE5 OE1CICg1MDAxMTgxOTIgNTEyIGJ5dGUgc2VjdG9ycyk8YnI+DQpkYTA6IHF1aXJrcz0weDImbHQ7 Tk9fNl9CWVRFJmd0Ozxicj4NCldBUk5JTkc6IC86IFRSSU0gZmxhZyBvbiBmcyBidXQgZGlzayBk b2VzIG5vdCBzdXBwb3J0IFRSSU08YnI+DQo8YnI+DQokIHN5c2N0bCBrZXJuLmNhbS5kYS4wIHwg Z3JlcCAtRSAidHJpbXxkZWxldGUiPGJyPg0Ka2Vybi5jYW0uZGEuMC50cmltX3RpY2tzOiAwPGJy Pg0Ka2Vybi5jYW0uZGEuMC50cmltX2dvYWw6IDA8YnI+DQprZXJuLmNhbS5kYS4wLnRyaW1fbGJh czogMDxicj4NCmtlcm4uY2FtLmRhLjAudHJpbV9yYW5nZXM6IDA8YnI+DQprZXJuLmNhbS5kYS4w LnRyaW1fY291bnQ6IDA8YnI+DQprZXJuLmNhbS5kYS4wLmRlbGV0ZV9tYXg6IDY1NTM2PGJyPg0K a2Vybi5jYW0uZGEuMC5kZWxldGVfbWV0aG9kOiBOT05FPGJyPg0KPGJyPg0KPGJyPg0KT24gYW4g UlBJNEIgSSBoYXZlIGFuIFNTRCAoc3VwcG9ydGluZyB0cmltKSBjb25uZWN0ZWQgdmlhIFVTQi10 by1TQVRBICh3aGljaCBkb2VzIHN1cHBvcnQgdHJpbSBhbHNvKS4gSGVyZSB0cmltIHdvcmtzIGZp bmUuIChOQjogdGhpcyB1c2VzIFpGUywgYnV0IHRoYXQgZG9lcyBub3QgbWF0dGVyKTxicj4NCiQg ZG1lc2cgfCBncmVwIC1FICJ1bWFzczB8ZGEwIjxicj4NCnVtYXNzMCBvbiB1aHViMDxicj4NCnVt YXNzMDogJmx0O1VTQiAzLjAgRGV2aWNlIFVTQiAzLjAgRGV2aWNlLCBjbGFzcyAwLzAsIHJldiAz LjAwLzMuMDEsIGFkZHIgMyZndDsgb24gdXNidXMwPGJyPg0KZGEwIGF0IHVtYXNzLXNpbTAgYnVz IDAgc2NidXMwIHRhcmdldCAwIGx1biAwPGJyPg0KZGEwOiAmbHQ7TWljcm9uIDEgMTAwIFNBVEEg MjU2R0IgMDMwMSZndDsgRml4ZWQgRGlyZWN0IEFjY2VzcyBTUEMtNCBTQ1NJIGRldmljZTxicj4N CmRhMDogU2VyaWFsIE51bWJlciAwMDAwMDAwMDRCQTg8YnI+DQpkYTA6IDQwMC4wMDBNQi9zIHRy YW5zZmVyczxicj4NCmRhMDogMjQ0MTk4TUIgKDUwMDExODE5MiA1MTIgYnl0ZSBzZWN0b3JzKTxi cj4NCmRhMDogcXVpcmtzPTB4MiZsdDtOT182X0JZVEUmZ3Q7PGJyPg0KPGJyPg0KJCBzeXNjdGwg a2Vybi5jYW0uZGEuMCB8IGdyZXAgLUUgInRyaW18ZGVsZXRlIjxicj4NCmtlcm4uY2FtLmRhLjAu dHJpbV90aWNrczogMDxicj4NCmtlcm4uY2FtLmRhLjAudHJpbV9nb2FsOiAwPGJyPg0Ka2Vybi5j YW0uZGEuMC50cmltX2xiYXM6IDU0Njg0MTcwNDxicj4NCmtlcm4uY2FtLmRhLjAudHJpbV9yYW5n ZXM6IDMxNzI4MDxicj4NCmtlcm4uY2FtLmRhLjAudHJpbV9jb3VudDogMzAxNDI0PGJyPg0Ka2Vy bi5jYW0uZGEuMC5kZWxldGVfbWF4OiA0Mjk0OTAxNzYwPGJyPg0Ka2Vybi5jYW0uZGEuMC5kZWxl dGVfbWV0aG9kOiBVTk1BUDxicj4NCjxicj4NCjxicj4NCkJUVzogZW5hYmxpbmcgdHJpbSBvbiB0 aGUgZnMgd2hpbGUgdGhlIGRldmljZSBkb2VzIG5vdCBzdXBwb3J0IGl0IGRvZXMgbm90IG1hdHRl ciBhbmQgZG9lcyBubyBoYXJtLiBVc2luZyB0aGUgc3lzY3RsIGFib3ZlIHlvdSBjYW4gc2VlIGlm IHRoZSBkZXZpY2UgYWN0dWFsbHkgdXNlcyB0cmltLjxicj4NCjxicj4NClJ1bm5pbmcgImdzdGF0 IC1kIiB3aGlsZSBkZWxldGluZyBhIGxvdCBvZiBmaWxlcyB3aWxsIGFsc28gc2hvdyBpZiB0cmlt L2RlbGV0ZSBhY3Rpb25zIGFyZSBzZW50IHRvIHRoZSBkaXNrLjxicj4NCjxicj4NClNvIHRoZSBj b21iaW5hdGlvbiBvZiB5b3VyIFVTQiByZWFkZXIgKyBtaWNyb1NEIGRvIG5vdCBzdXBwb3J0IFRS SU0sIGJ1dCB5b3UgY2FuIGp1c3QgdHJ5IHRvIHB1dCB0aGUgbWljcm9TRCBkaXJlY3RseSBpbiB0 aGUgUlBJNCBhbmQgc2VlIHdoYXQgdGhhdCBkb2VzLjxicj4NCjxicj4NClJlZ2FyZHMsPGJyPg0K Um9uYWxkLjxicj48L2Jsb2NrcXVvdGU+PC9kaXY+PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6IEFy aWFsLCBzYW5zLXNlcmlmOyBmb250LXNpemU6IDE0cHg7IGNvbG9yOiByZ2IoMCwgMCwgMCk7IGJh Y2tncm91bmQtY29sb3I6IHJnYigyNTUsIDI1NSwgMjU1KTsiIGNsYXNzPSJwcm90b25tYWlsX3F1 b3RlIj48YnI+PC9kaXY+PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6IEFyaWFsLCBzYW5zLXNlcmlm OyBmb250LXNpemU6IDE0cHg7IGNvbG9yOiByZ2IoMCwgMCwgMCk7IGJhY2tncm91bmQtY29sb3I6 IHJnYigyNTUsIDI1NSwgMjU1KTsiIGNsYXNzPSJwcm90b25tYWlsX3F1b3RlIj5UaGFua3MgZm9y IHRoZSBjYWxsb3V0ISBHb3Qgd2FybWVkLXVwIHdoZW4gSSBzYXcgdGhlIFRSSU0gaXMgZW5hYmxl ZCA6LSkgYnV0IG15IHVsdGltYXRlIGdvYWwgaXMgcmVhbGx5IHRoZSBtaWNyb1NEIGNhcmQgc2xv dCBvZiBteSBSYXNwYmVycnkgUGkgMyBvciA0IGFuZCBub3Qgd2l0aCB0aGUgVVNCIHJlYWRlci4g SSdsbCBjcmVhdGUgYW4gaW1hZ2UgdGhlbiBzZWUgaG93IGl0IGRvZXMuPC9kaXY+PGRpdiBzdHls ZT0iZm9udC1mYW1pbHk6IEFyaWFsLCBzYW5zLXNlcmlmOyBmb250LXNpemU6IDE0cHg7IGNvbG9y OiByZ2IoMCwgMCwgMCk7IGJhY2tncm91bmQtY29sb3I6IHJnYigyNTUsIDI1NSwgMjU1KTsiIGNs YXNzPSJwcm90b25tYWlsX3F1b3RlIj48YnI+PC9kaXY+PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6 IEFyaWFsLCBzYW5zLXNlcmlmOyBmb250LXNpemU6IDE0cHg7IGNvbG9yOiByZ2IoMCwgMCwgMCk7 IGJhY2tncm91bmQtY29sb3I6IHJnYigyNTUsIDI1NSwgMjU1KTsiIGNsYXNzPSJwcm90b25tYWls X3F1b3RlIj5CUiw8L2Rpdj48ZGl2IHN0eWxlPSJmb250LWZhbWlseTogQXJpYWwsIHNhbnMtc2Vy aWY7IGZvbnQtc2l6ZTogMTRweDsgY29sb3I6IHJnYigwLCAwLCAwKTsgYmFja2dyb3VuZC1jb2xv cjogcmdiKDI1NSwgMjU1LCAyNTUpOyIgY2xhc3M9InByb3Rvbm1haWxfcXVvdGUiPm9yYml0Jm5i c3A7IDxicj48L2Rpdj48L2Jsb2NrcXVvdGU+PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6IEFyaWFs LCBzYW5zLXNlcmlmOyBmb250LXNpemU6IDE0cHg7IGNvbG9yOiByZ2IoMCwgMCwgMCk7IGJhY2tn cm91bmQtY29sb3I6IHJnYigyNTUsIDI1NSwgMjU1KTsiIGNsYXNzPSJwcm90b25tYWlsX3F1b3Rl Ij48YnI+PC9kaXY+PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6IEFyaWFsLCBzYW5zLXNlcmlmOyBm b250LXNpemU6IDE0cHg7IGNvbG9yOiByZ2IoMCwgMCwgMCk7IGJhY2tncm91bmQtY29sb3I6IHJn YigyNTUsIDI1NSwgMjU1KTsiIGNsYXNzPSJwcm90b25tYWlsX3F1b3RlIj5Db25maXJtZWQsIHRo ZSBTYW5EaXNrIFVsdHJhIG1pY3JvU0QgZG9lcyBub3Qgc3VwcG9ydCBUUklNIHVzaW5nIHRoZSBt aWNyb1NEIHNsb3QgaW4gYm90aCBSYXNwYmVycnkgUGkgMyBhbmQgNC4gSW5zdGVhZCBvZiBidWls ZGluZyBhbiBpbWFnZSBmcm9tIHNjcmF0Y2gsIEkgZG93bmxvYWRlZCBhIEZyZWVCU0QgMTQuMCBp bWFnZSBhbmQgY3JlYXRlZCBhIG5ldyBwYXJ0aXRpb24gKDhHQiBpbiBzaXplKSB3aXRoIFVGUy9G RlMgd2l0aCBUUklNIGVuYWJsZWQgYW5kIG1vdW50IGl0LiBUaGUgcGFydGl0aW9uIGlzIDxzcGFu Pi9kZXYvbW1jc2QwczM8L3NwYW4+LiBUaGVyZSdzIG5vIGRlbGV0ZSBhY3Rpdml0eSBvYnNlcnZl ZCB3aXRoICJnc3RhdCAtZCIgd2hlbiBJIGRlbGV0ZWQgc29tZSBmaWxlcyBpbiBpdC48L2Rpdj48 ZGl2IHN0eWxlPSJmb250LWZhbWlseTogQXJpYWwsIHNhbnMtc2VyaWY7IGZvbnQtc2l6ZTogMTRw eDsgY29sb3I6IHJnYigwLCAwLCAwKTsgYmFja2dyb3VuZC1jb2xvcjogcmdiKDI1NSwgMjU1LCAy NTUpOyIgY2xhc3M9InByb3Rvbm1haWxfcXVvdGUiPjxicj48L2Rpdj48ZGl2IHN0eWxlPSJmb250 LWZhbWlseTogQXJpYWwsIHNhbnMtc2VyaWY7IGZvbnQtc2l6ZTogMTRweDsgY29sb3I6IHJnYigw LCAwLCAwKTsgYmFja2dyb3VuZC1jb2xvcjogcmdiKDI1NSwgMjU1LCAyNTUpOyIgY2xhc3M9InBy b3Rvbm1haWxfcXVvdGUiPjxkaXY+PHNwYW4+PHNwYW4+cm9vdEBzZC1leHRyZW1lOn4gIyAvc2Jp bi9ncGFydCBhZGQgLXMgOEcgLXQgZnJlZWJzZCBkYTA8L3NwYW4+PGRpdj48c3Bhbj5kYTBzMyBh ZGRlZDwvc3Bhbj48L2Rpdj48ZGl2PjxzcGFuPnJvb3RAc2QtZXh0cmVtZTp+ICMgZ3BhcnQgc2hv dzwvc3Bhbj48L2Rpdj48ZGl2PjxzcGFuPj0mZ3Q7ICZuYnNwOyAmbmJzcDsgJm5ic3A7IDYzICZu YnNwOzEyNDczNTQyNSAmbmJzcDttbWNzZDAgJm5ic3A7TUJSICZuYnNwOyg1OUcpPC9zcGFuPjwv ZGl2PjxkaXY+PHNwYW4+Jm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7ICZuYnNwOzYzICZuYnNw OyAmbmJzcDsgJm5ic3A7IDE5ODUgJm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7ICZuYnNwOy0g ZnJlZSAtICZuYnNwOyg5OTNLKTwvc3Bhbj48L2Rpdj48ZGl2PjxzcGFuPiZuYnNwOyAmbmJzcDsg Jm5ic3A7ICZuYnNwOzIwNDggJm5ic3A7ICZuYnNwOyAxMDI0MDAgJm5ic3A7ICZuYnNwOyAmbmJz cDsgMSAmbmJzcDtmYXQzMmxiYSAmbmJzcDtbYWN0aXZlXSAmbmJzcDsoNTBNKTwvc3Bhbj48L2Rp dj48ZGl2PjxzcGFuPiZuYnNwOyAmbmJzcDsgJm5ic3A7MTA0NDQ4ICZuYnNwOzEyNDYyNjk0NCAm bmJzcDsgJm5ic3A7ICZuYnNwOyAyICZuYnNwO2ZyZWVic2QgJm5ic3A7KDU5Ryk8L3NwYW4+PC9k aXY+PGRpdj48c3Bhbj4mbmJzcDsgMTI0NzMxMzkyICZuYnNwOyAmbmJzcDsgJm5ic3A7IDQwOTYg Jm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7ICZuYnNwOy0gZnJlZSAtICZuYnNwOygyLjBNKTwv c3Bhbj48L2Rpdj48ZGl2Pjxicj48L2Rpdj48ZGl2PjxzcGFuPj0mZ3Q7ICZuYnNwOyAmbmJzcDsg Jm5ic3A7ICZuYnNwOzAgJm5ic3A7MTI0NjI2OTQ0ICZuYnNwO21tY3NkMHMyICZuYnNwO0JTRCAm bmJzcDsoNTlHKTwvc3Bhbj48L2Rpdj48ZGl2PjxzcGFuPiZuYnNwOyAmbmJzcDsgJm5ic3A7ICZu YnNwOyAmbmJzcDsgMCAmbmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsxMjggJm5ic3A7ICZuYnNw OyAmbmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJzcDstIGZyZWUgLSAmbmJzcDsoNjRLKTwvc3Bhbj48 L2Rpdj48ZGl2PjxzcGFuPiZuYnNwOyAmbmJzcDsgJm5ic3A7ICZuYnNwOyAxMjggJm5ic3A7MTIw ODMzOTIwICZuYnNwOyAmbmJzcDsgJm5ic3A7ICZuYnNwOyAxICZuYnNwO2ZyZWVic2QtdWZzICZu YnNwOyg1OEcpPC9zcGFuPjwvZGl2PjxkaXY+PHNwYW4+Jm5ic3A7IDEyMDgzNDA0OCAmbmJzcDsg Jm5ic3A7Mzc5Mjg5NiAmbmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsgMiAmbmJzcDtmcmVlYnNk LXN3YXAgJm5ic3A7KDEuOEcpPC9zcGFuPjwvZGl2PjxkaXY+PGJyPjwvZGl2PjxkaXY+PHNwYW4+ PSZndDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsgNjMgJm5ic3A7MTI0NzM1NDI1ICZuYnNwO2RhMCAm bmJzcDtNQlIgJm5ic3A7KDU5Ryk8L3NwYW4+PC9kaXY+PGRpdj48c3Bhbj4mbmJzcDsgJm5ic3A7 ICZuYnNwOyAmbmJzcDsgJm5ic3A7NjMgJm5ic3A7ICZuYnNwOyAmbmJzcDsgMTk4NSAmbmJzcDsg Jm5ic3A7ICZuYnNwOyAtIGZyZWUgLSAmbmJzcDsoOTkzSyk8L3NwYW4+PC9kaXY+PGRpdj48c3Bh bj4mbmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsyMDQ4ICZuYnNwOyAmbmJzcDsgMTAyNDAwICZu YnNwOyAmbmJzcDsxICZuYnNwO2ZhdDMybGJhICZuYnNwO1thY3RpdmVdICZuYnNwOyg1ME0pPC9z cGFuPjwvZGl2PjxkaXY+PHNwYW4+Jm5ic3A7ICZuYnNwOyAmbmJzcDsxMDQ0NDggJm5ic3A7IDEw MzgxMzEyICZuYnNwOyAmbmJzcDsyICZuYnNwO2ZyZWVic2QgJm5ic3A7KDUuMEcpPC9zcGFuPjwv ZGl2PjxkaXY+PHNwYW4+Jm5ic3A7ICZuYnNwOzEwNDg1NzYwICZuYnNwOyAxNjc3NzIxNiAmbmJz cDsgJm5ic3A7MyAmbmJzcDtmcmVlYnNkICZuYnNwOyg4LjBHKTwvc3Bhbj48L2Rpdj48ZGl2Pjxz cGFuPiZuYnNwOyAmbmJzcDsyNzI2Mjk3NiAmbmJzcDsgOTc0NzI1MTIgJm5ic3A7ICZuYnNwOyAm bmJzcDsgLSBmcmVlIC0gJm5ic3A7KDQ2Ryk8L3NwYW4+PC9kaXY+PGRpdj48YnI+PC9kaXY+PGRp dj48c3Bhbj49Jmd0OyAmbmJzcDsgJm5ic3A7ICZuYnNwOyAwICZuYnNwOzEwMzgxMzEyICZuYnNw O2RhMHMyICZuYnNwO0JTRCAmbmJzcDsoNS4wRyk8L3NwYW4+PC9kaXY+PGRpdj48c3Bhbj4mbmJz cDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7MCAmbmJzcDsgJm5ic3A7ICZuYnNwOyAxMjgg Jm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7IC0gZnJlZSAtICZuYnNwOyg2NEspPC9zcGFuPjwv ZGl2PjxkaXY+PHNwYW4+Jm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7MTI4ICZuYnNwOzEwMzgx MTg0ICZuYnNwOyAmbmJzcDsgJm5ic3A7MSAmbmJzcDtmcmVlYnNkLXVmcyAmbmJzcDsoNC45Ryk8 L3NwYW4+PC9kaXY+PGRpdj48YnI+PC9kaXY+PGRpdj48c3Bhbj49Jmd0OyAmbmJzcDsgJm5ic3A7 ICZuYnNwOyA2MyAmbmJzcDsxMjQ3MzU0MjUgJm5ic3A7ZGlza2lkL0RJU0stMTIxMjIwMTYwMjA0 ICZuYnNwO01CUiAmbmJzcDsoNTlHKTwvc3Bhbj48L2Rpdj48ZGl2PjxzcGFuPiZuYnNwOyAmbmJz cDsgJm5ic3A7ICZuYnNwOyAmbmJzcDs2MyAmbmJzcDsgJm5ic3A7ICZuYnNwOyAxOTg1ICZuYnNw OyAmbmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7 ICZuYnNwOyAmbmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJzcDstIGZyZWUgLSAmbmJzcDsoOTkzSyk8 L3NwYW4+PC9kaXY+PGRpdj48c3Bhbj4mbmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsyMDQ4ICZu YnNwOyAmbmJzcDsgMTAyNDAwICZuYnNwOyAmbmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5i c3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7IDEgJm5ic3A7ZmF0 MzJsYmEgJm5ic3A7W2FjdGl2ZV0gJm5ic3A7KDUwTSk8L3NwYW4+PC9kaXY+PGRpdj48c3Bhbj4m bmJzcDsgJm5ic3A7ICZuYnNwOzEwNDQ0OCAmbmJzcDsgMTAzODEzMTIgJm5ic3A7ICZuYnNwOyAm bmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7ICZu YnNwOyAmbmJzcDsgMiAmbmJzcDtmcmVlYnNkICZuYnNwOyg1LjBHKTwvc3Bhbj48L2Rpdj48ZGl2 PjxzcGFuPiZuYnNwOyAmbmJzcDsxMDQ4NTc2MCAmbmJzcDsgMTY3NzcyMTYgJm5ic3A7ICZuYnNw OyAmbmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7 ICZuYnNwOyAmbmJzcDsgMyAmbmJzcDtmcmVlYnNkICZuYnNwOyg4LjBHKTwvc3Bhbj48L2Rpdj48 ZGl2PjxzcGFuPiZuYnNwOyAmbmJzcDsyNzI2Mjk3NiAmbmJzcDsgOTc0NzI1MTIgJm5ic3A7ICZu YnNwOyAmbmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5i c3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7ICZuYnNwOy0gZnJlZSAtICZuYnNwOyg0NkcpPC9zcGFu PjwvZGl2PjxkaXY+PGJyPjwvZGl2PjxkaXY+PHNwYW4+PSZndDsgJm5ic3A7ICZuYnNwOyAmbmJz cDsgMCAmbmJzcDsxMDM4MTMxMiAmbmJzcDtkaXNraWQvRElTSy0xMjEyMjAxNjAyMDRzMiAmbmJz cDtCU0QgJm5ic3A7KDUuMEcpPC9zcGFuPjwvZGl2PjxkaXY+PHNwYW4+Jm5ic3A7ICZuYnNwOyAm bmJzcDsgJm5ic3A7ICZuYnNwOzAgJm5ic3A7ICZuYnNwOyAmbmJzcDsgMTI4ICZuYnNwOyAmbmJz cDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7ICZuYnNw OyAmbmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7LSBmcmVlIC0gJm5ic3A7KDY0Syk8 L3NwYW4+PC9kaXY+PGRpdj48c3Bhbj4mbmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsxMjggJm5i c3A7MTAzODExODQgJm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5i c3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7IDEgJm5ic3A7ZnJl ZWJzZC11ZnMgJm5ic3A7KDQuOUcpPC9zcGFuPjwvZGl2PjxkaXY+PGJyPjwvZGl2PjxkaXY+PHNw YW4+cm9vdEBzZC1leHRyZW1lOn4gIyBscyAtbGFoIC9kZXYvZGEwPC9zcGFuPjwvZGl2PjxkaXY+ PHNwYW4+L2Rldi9kYTAgJm5ic3A7ICZuYnNwOyAvZGV2L2RhMHMxICZuYnNwOyAvZGV2L2RhMHMy ICZuYnNwOyAvZGV2L2RhMHMyYSAmbmJzcDsvZGV2L2RhMHMzPC9zcGFuPjwvZGl2PjxkaXY+PHNw YW4+PGJyPjwvc3Bhbj48L2Rpdj48ZGl2PjxzcGFuPnJvb3RAc2QtZXh0cmVtZTp+ICMgbmV3ZnMg LVUgLXQgL2Rldi9kYTBzMzwvc3Bhbj48L2Rpdj48ZGl2PjxzcGFuPi9kZXYvZGEwczM6IDgxOTIu ME1CICgxNjc3NzIxNiBzZWN0b3JzKSBibG9jayBzaXplIDMyNzY4LCBmcmFnbWVudCBzaXplIDQw OTY8L3NwYW4+PC9kaXY+PGRpdj48c3Bhbj4mbmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsgdXNp bmcgMTQgY3lsaW5kZXIgZ3JvdXBzIG9mIDYyNS4yMk1CLCAyMDAwNyBibGtzLCA4MDEyOCBpbm9k ZXMuPC9zcGFuPjwvZGl2PjxkaXY+PHNwYW4+Jm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7IHdp dGggc29mdCB1cGRhdGVzPC9zcGFuPjwvZGl2PjxkaXY+PHNwYW4+c3VwZXItYmxvY2sgYmFja3Vw cyAoZm9yIGZzY2tfZmZzIC1iICMpIGF0Ojwvc3Bhbj48L2Rpdj48ZGl2PjxzcGFuPiZuYnNwOzE5 MiwgMTI4MDY0MCwgMjU2MTA4OCwgMzg0MTUzNiwgNTEyMTk4NCwgNjQwMjQzMiwgNzY4Mjg4MCwg ODk2MzMyOCwgMTAyNDM3NzYsIDExNTI0MjI0LCAxMjgwNDY3MiwgMTQwODUxMjAsPC9zcGFuPjwv ZGl2PjxkaXY+PHNwYW4+Jm5ic3A7MTUzNjU1NjgsIDE2NjQ2MDE2PC9zcGFuPjwvZGl2PjxzcGFu Pjwvc3Bhbj48YnI+PC9zcGFuPjwvZGl2PjxkaXY+PHNwYW4+cm9vdEBzZC11bHRyYTp+ICMgZ3Bh cnQgc2hvdyAtbDwvc3Bhbj48ZGl2PjxzcGFuPj0mZ3Q7ICZuYnNwOyAmbmJzcDsgJm5ic3A7IDYz ICZuYnNwOzEyNDczNTQyNSAmbmJzcDttbWNzZDAgJm5ic3A7TUJSICZuYnNwOyg1OUcpPC9zcGFu PjwvZGl2PjxkaXY+PHNwYW4+Jm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7ICZuYnNwOzYzICZu YnNwOyAmbmJzcDsgJm5ic3A7IDE5ODUgJm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7ICZuYnNw Oy0gZnJlZSAtICZuYnNwOyg5OTNLKTwvc3Bhbj48L2Rpdj48ZGl2PjxzcGFuPiZuYnNwOyAmbmJz cDsgJm5ic3A7ICZuYnNwOzIwNDggJm5ic3A7ICZuYnNwOyAxMDI0MDAgJm5ic3A7ICZuYnNwOyAm bmJzcDsgMSAmbmJzcDsobnVsbCkgJm5ic3A7W2FjdGl2ZV0gJm5ic3A7KDUwTSk8L3NwYW4+PC9k aXY+PGRpdj48c3Bhbj4mbmJzcDsgJm5ic3A7ICZuYnNwOzEwNDQ0OCAmbmJzcDsgMTAzODEzMTIg Jm5ic3A7ICZuYnNwOyAmbmJzcDsgMiAmbmJzcDsobnVsbCkgJm5ic3A7KDUuMEcpPC9zcGFuPjwv ZGl2PjxkaXY+PHNwYW4+Jm5ic3A7ICZuYnNwOzEwNDg1NzYwICZuYnNwOyAxNjc3NzIxNiAmbmJz cDsgJm5ic3A7ICZuYnNwOyAzICZuYnNwOyhudWxsKSAmbmJzcDsoOC4wRyk8L3NwYW4+PC9kaXY+ PGRpdj48c3Bhbj4mbmJzcDsgJm5ic3A7MjcyNjI5NzYgJm5ic3A7IDk3NDcyNTEyICZuYnNwOyAm bmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJzcDstIGZyZWUgLSAmbmJzcDsoNDZHKTwvc3Bhbj48L2Rp dj48ZGl2Pjxicj48L2Rpdj48ZGl2PjxzcGFuPj0mZ3Q7ICZuYnNwOyAmbmJzcDsgJm5ic3A7IDAg Jm5ic3A7MTAzODEzMTIgJm5ic3A7bW1jc2QwczIgJm5ic3A7QlNEICZuYnNwOyg1LjBHKTwvc3Bh bj48L2Rpdj48ZGl2PjxzcGFuPiZuYnNwOyAmbmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJzcDswICZu YnNwOyAmbmJzcDsgJm5ic3A7IDEyOCAmbmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7 ICZuYnNwOy0gZnJlZSAtICZuYnNwOyg2NEspPC9zcGFuPjwvZGl2PjxkaXY+PHNwYW4+Jm5ic3A7 ICZuYnNwOyAmbmJzcDsgJm5ic3A7MTI4ICZuYnNwOzEwMzgxMTg0ICZuYnNwOyAmbmJzcDsgJm5i c3A7ICZuYnNwOyAxICZuYnNwOyhudWxsKSAmbmJzcDsoNC45Ryk8L3NwYW4+PC9kaXY+PGRpdj48 YnI+PC9kaXY+PGRpdj48c3Bhbj5yb290QHNkLXVsdHJhOn4gIyBtb3VudCAtbDwvc3Bhbj48L2Rp dj48ZGl2PjxzcGFuPi9kZXYvdWZzL3Jvb3RmcyBvbiAvICh1ZnMsIGxvY2FsLCBzb2Z0LXVwZGF0 ZXMpPC9zcGFuPjwvZGl2PjxkaXY+PHNwYW4+ZGV2ZnMgb24gL2RldiAoZGV2ZnMpPC9zcGFuPjwv ZGl2PjxkaXY+PHNwYW4+L2Rldi9tc2Rvc2ZzL0VGSSBvbiAvYm9vdC9lZmkgKG1zZG9zZnMsIGxv Y2FsLCBub2F0aW1lKTwvc3Bhbj48L2Rpdj48ZGl2PjxzcGFuPnRtcGZzIG9uIC90bXAgKHRtcGZz LCBsb2NhbCk8L3NwYW4+PC9kaXY+PGRpdj48c3Bhbj4vZGV2L21tY3NkMHMzIG9uIC9tbnQgKHVm cywgbG9jYWwsIHNvZnQtdXBkYXRlcyk8L3NwYW4+PC9kaXY+PGRpdj48c3Bhbj48YnI+PC9zcGFu PjwvZGl2PjxkaXY+PHNwYW4+PHNwYW4+cm9vdEBzZC11bHRyYTp+ICMgZ3N0YXQgLWQ8L3NwYW4+ PGRpdj48c3Bhbj5kVDogMS4wNjVzICZuYnNwO3c6IDEuMDAwczwvc3Bhbj48L2Rpdj48ZGl2Pjxz cGFuPiZuYnNwO0wocSkgJm5ic3A7b3BzL3MgJm5ic3A7ICZuYnNwO3IvcyAmbmJzcDsga0JwcyAm bmJzcDsgbXMvciAmbmJzcDsgJm5ic3A7dy9zICZuYnNwOyBrQnBzICZuYnNwOyBtcy93ICZuYnNw OyAmbmJzcDtkL3MgJm5ic3A7IGtCcHMgJm5ic3A7IG1zL2QgJm5ic3A7ICVidXN5IE5hbWU8L3Nw YW4+PC9kaXY+PGRpdj48c3Bhbj4mbmJzcDsgJm5ic3A7IDAgJm5ic3A7ICZuYnNwOyAmbmJzcDsw ICZuYnNwOyAmbmJzcDsgJm5ic3A7MCAmbmJzcDsgJm5ic3A7ICZuYnNwOzAgJm5ic3A7ICZuYnNw OzAuMCAmbmJzcDsgJm5ic3A7ICZuYnNwOzAgJm5ic3A7ICZuYnNwOyAmbmJzcDswICZuYnNwOyAm bmJzcDswLjAgJm5ic3A7ICZuYnNwOyAmbmJzcDswICZuYnNwOyAmbmJzcDsgJm5ic3A7MCAmbmJz cDsgJm5ic3A7MC4wICZuYnNwOyAmbmJzcDswLjB8IG1tY3NkMDwvc3Bhbj48L2Rpdj48ZGl2Pjxz cGFuPiZuYnNwOyAmbmJzcDsgMCAmbmJzcDsgJm5ic3A7ICZuYnNwOzAgJm5ic3A7ICZuYnNwOyAm bmJzcDswICZuYnNwOyAmbmJzcDsgJm5ic3A7MCAmbmJzcDsgJm5ic3A7MC4wICZuYnNwOyAmbmJz cDsgJm5ic3A7MCAmbmJzcDsgJm5ic3A7ICZuYnNwOzAgJm5ic3A7ICZuYnNwOzAuMCAmbmJzcDsg Jm5ic3A7ICZuYnNwOzAgJm5ic3A7ICZuYnNwOyAmbmJzcDswICZuYnNwOyAmbmJzcDswLjAgJm5i c3A7ICZuYnNwOzAuMHwgbW1jc2QwczE8L3NwYW4+PC9kaXY+PGRpdj48c3Bhbj4mbmJzcDsgJm5i c3A7IDAgJm5ic3A7ICZuYnNwOyAmbmJzcDswICZuYnNwOyAmbmJzcDsgJm5ic3A7MCAmbmJzcDsg Jm5ic3A7ICZuYnNwOzAgJm5ic3A7ICZuYnNwOzAuMCAmbmJzcDsgJm5ic3A7ICZuYnNwOzAgJm5i c3A7ICZuYnNwOyAmbmJzcDswICZuYnNwOyAmbmJzcDswLjAgJm5ic3A7ICZuYnNwOyAmbmJzcDsw ICZuYnNwOyAmbmJzcDsgJm5ic3A7MCAmbmJzcDsgJm5ic3A7MC4wICZuYnNwOyAmbmJzcDswLjB8 IG1tY3NkMHMyPC9zcGFuPjwvZGl2PjxkaXY+PHNwYW4+Jm5ic3A7ICZuYnNwOyAwICZuYnNwOyAm bmJzcDsgJm5ic3A7MCAmbmJzcDsgJm5ic3A7ICZuYnNwOzAgJm5ic3A7ICZuYnNwOyAmbmJzcDsw ICZuYnNwOyAmbmJzcDswLjAgJm5ic3A7ICZuYnNwOyAmbmJzcDswICZuYnNwOyAmbmJzcDsgJm5i c3A7MCAmbmJzcDsgJm5ic3A7MC4wICZuYnNwOyAmbmJzcDsgJm5ic3A7MCAmbmJzcDsgJm5ic3A7 ICZuYnNwOzAgJm5ic3A7ICZuYnNwOzAuMCAmbmJzcDsgJm5ic3A7MC4wfCBtbWNzZDBzMzwvc3Bh bj48L2Rpdj48ZGl2PjxzcGFuPiZuYnNwOyAmbmJzcDsgMCAmbmJzcDsgJm5ic3A7ICZuYnNwOzAg Jm5ic3A7ICZuYnNwOyAmbmJzcDswICZuYnNwOyAmbmJzcDsgJm5ic3A7MCAmbmJzcDsgJm5ic3A7 MC4wICZuYnNwOyAmbmJzcDsgJm5ic3A7MCAmbmJzcDsgJm5ic3A7ICZuYnNwOzAgJm5ic3A7ICZu YnNwOzAuMCAmbmJzcDsgJm5ic3A7ICZuYnNwOzAgJm5ic3A7ICZuYnNwOyAmbmJzcDswICZuYnNw OyAmbmJzcDswLjAgJm5ic3A7ICZuYnNwOzAuMHwgbXNkb3Nmcy9FRkk8L3NwYW4+PC9kaXY+PGRp dj48c3Bhbj4mbmJzcDsgJm5ic3A7IDAgJm5ic3A7ICZuYnNwOyAmbmJzcDswICZuYnNwOyAmbmJz cDsgJm5ic3A7MCAmbmJzcDsgJm5ic3A7ICZuYnNwOzAgJm5ic3A7ICZuYnNwOzAuMCAmbmJzcDsg Jm5ic3A7ICZuYnNwOzAgJm5ic3A7ICZuYnNwOyAmbmJzcDswICZuYnNwOyAmbmJzcDswLjAgJm5i c3A7ICZuYnNwOyAmbmJzcDswICZuYnNwOyAmbmJzcDsgJm5ic3A7MCAmbmJzcDsgJm5ic3A7MC4w ICZuYnNwOyAmbmJzcDswLjB8IG1tY3NkMHMyYTwvc3Bhbj48L2Rpdj48c3Bhbj48c3Bhbj4mbmJz cDsgJm5ic3A7IDAgJm5ic3A7ICZuYnNwOyAmbmJzcDswICZuYnNwOyAmbmJzcDsgJm5ic3A7MCAm bmJzcDsgJm5ic3A7ICZuYnNwOzAgJm5ic3A7ICZuYnNwOzAuMCAmbmJzcDsgJm5ic3A7ICZuYnNw OzAgJm5ic3A7ICZuYnNwOyAmbmJzcDswICZuYnNwOyAmbmJzcDswLjAgJm5ic3A7ICZuYnNwOyAm bmJzcDswICZuYnNwOyAmbmJzcDsgJm5ic3A7MCAmbmJzcDsgJm5ic3A7MC4wICZuYnNwOyAmbmJz cDswLjB8IHVmcy9yb290ZnM8L3NwYW4+PC9zcGFuPjxzcGFuPjwvc3Bhbj48L3NwYW4+PC9kaXY+ PC9kaXY+PGJyPjwvZGl2PjxkaXYgc3R5bGU9ImZvbnQtZmFtaWx5OiBBcmlhbCwgc2Fucy1zZXJp ZjsgZm9udC1zaXplOiAxNHB4OyBjb2xvcjogcmdiKDAsIDAsIDApOyBiYWNrZ3JvdW5kLWNvbG9y OiByZ2IoMjU1LCAyNTUsIDI1NSk7IiBjbGFzcz0icHJvdG9ubWFpbF9xdW90ZSI+PHNwYW4+cm9v dEBzZC11bHRyYTp+ICMgbW91bnQgLWwgLXY8L3NwYW4+PGRpdj48c3Bhbj4vZGV2L3Vmcy9yb290 ZnMgb24gLw0KICh1ZnMsIGxvY2FsLCBzb2Z0LXVwZGF0ZXMsIHdyaXRlczogc3luYyA4MzggYXN5 bmMgNDMxNiwgcmVhZHM6IHN5bmMgDQoyMjQxIGFzeW5jIDI2OTcsIGZzaWQgODlmYzRkNjUyYWJh MTdjZCwgdm5vZGVzOiBjb3VudCAxMjY0ICk8L3NwYW4+PC9kaXY+PGRpdj48c3Bhbj5kZXZmcyBv biAvZGV2IChkZXZmcywgZnNpZCAwMGZmMDA3MTcxMDAwMDAwLCB2bm9kZXM6IGNvdW50IDM0ICk8 L3NwYW4+PC9kaXY+PGRpdj48c3Bhbj4vZGV2L21zZG9zZnMvRUZJDQogb24gL2Jvb3QvZWZpICht c2Rvc2ZzLCBsb2NhbCwgbm9hdGltZSwgd3JpdGVzOiBzeW5jIDEgYXN5bmMgMCwgcmVhZHM6IA0K c3luYyA4IGFzeW5jIDAsIGZzaWQgNWIwMDAwMDAzMjAwMDAwMCwgdm5vZGVzOiBjb3VudCAxICk8 L3NwYW4+PC9kaXY+PGRpdj48c3Bhbj50bXBmcyBvbiAvdG1wICh0bXBmcywgbG9jYWwsIGZzaWQg MDFmZjAwODc4NzAwMDAwMCwgdm5vZGVzOiBjb3VudCA2ICk8L3NwYW4+PC9kaXY+PGRpdj48c3Bh bj4vZGV2L21tY3NkMHMzDQogb24gL21udCAodWZzLCBsb2NhbCwgc29mdC11cGRhdGVzLCB3cml0 ZXM6IHN5bmMgMiBhc3luYyA3ODY2LCByZWFkczogDQpzeW5jIDEyMTQ5MSBhc3luYyAwLCBmc2lk IDRlZTBjODY1MDYzMzA4OTAsIHZub2RlczogY291bnQgMyApPC9zcGFuPjwvZGl2PjxkaXY+PHNw YW4+PGJyPjwvc3Bhbj48L2Rpdj48ZGl2PjxzcGFuPjxzcGFuPnJvb3RAc2QtdWx0cmE6fiAjIHN5 c2N0bCAtYSB8IGdyZXAgdHJpbTwvc3Bhbj48ZGl2PjxzcGFuPiZsdDsxMTgmZ3Q7Q3JlYXRpbmcg YW5kL29yIHRyaW1taW5nIGxvZyBmaWxlcy48L3NwYW4+PC9kaXY+PGRpdj48c3Bhbj5rZXJuLmNh bS5uZGEubWF4X3RyaW06IDI1Njwvc3Bhbj48L2Rpdj48c3Bhbj52ZnMuZmZzLmRvdHJpbWNvbnM6 IDE8L3NwYW4+PGJyPjwvc3Bhbj48L2Rpdj48ZGl2PjxzcGFuPjxicj48L3NwYW4+PC9kaXY+PGRp dj48c3Bhbj5JJ2xsIHByb2NlZWQgd2l0aCBwbGFuIEIsIHRvIGJ1eSBpbmR1c3RyaWFsIGdyYWRl IG1pY3JvU0QgY2FyZCBoYXZpbmcgYSBnYXJiYWdlIGNvbGxlY3Rpb24gZmVhdHVyZSBpbiB0aGUg Y29udHJvbGxlciBpbnN0ZWFkIG9mIHRoaXMgY29uc3VtZXIgZ3JhZGUgY2FyZC4gRWl0aGVyIFRS SU0gd2lsbCB3b3JrIG9yIG5vdCB0aGVyZSdzIHN0aWxsIGdhcmJhZ2UgY29sbGVjdGlvbiBJIGNv dWxkIGJlbmVmaXQuIDxicj48L3NwYW4+PC9kaXY+PGRpdj48c3Bhbj48YnI+PC9zcGFuPjwvZGl2 PjxkaXY+PHNwYW4+QlIsPC9zcGFuPjwvZGl2PjxkaXY+PHNwYW4+b3JiaXQgPGJyPjwvc3Bhbj48 L2Rpdj4mbmJzcDsmbmJzcDsgPGJyPjwvZGl2PjxibG9ja3F1b3RlIGNsYXNzPSJwcm90b25tYWls X3F1b3RlIiB0eXBlPSJjaXRlIj48ZGl2IGNsYXNzPSJwcm90b25tYWlsX3F1b3RlIj4NCiAgICA8 L2Rpdj4NCiAgICAgICAgPC9ibG9ja3F1b3RlPjxicj4NCiAgICA8L2Rpdj4= --b1_hp6lbu7uCCqtZcnP3HWOX4Pb94jbjUYIGrM6EMDv3o-- From nobody Sat Feb 17 03:29:01 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TcDms4jhGz5BZ0q for ; Sat, 17 Feb 2024 03:29:17 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic305-21.consmr.mail.gq1.yahoo.com (sonic305-21.consmr.mail.gq1.yahoo.com [98.137.64.84]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4TcDms19Hyz4CTD for ; Sat, 17 Feb 2024 03:29:17 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1708140555; bh=rf11xTSAjJQpgTfboyp0FTYB38x8+YTkBQOTzTHsc4Q=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=HtlfcBjKgqOuohZUi6RG07ATCehBf6ZoZ3yXBO5OKWbxbCO2qT935HeJNcaYdTHmNT579NBgPqIWocJtZ+inQvbuN0VifipYJ17XFiA0UCqPnebuy3A/4AF5b21WJliJGeUZaPqWwOdJ1ZnLO4VnwFnrVQuU4AwL5iqmJnth2mvyqZIvBMu9IoltLHCMqG7i2zfFrPnCCC1LeojHCrmGW1sfqx8sgdm563RosF/NOvj1i8p1TBEyXBxwWyQONOgMgy3pgRGqjXLxXZfExZSy71Vl81GT/KbcHKB9vL/viHU/kZxsSxaoe3I7eJgsVAuKfSV58iv96tCipHklzxUJiw== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1708140555; bh=L97x82WBikIWuBLseQCabZ9rTl+gz6GcTZmDXrTmnDm=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=cXRvFXalgPP+tlKH6xzdk4BZTTMEUaUYDnH1yP2zpvOphvLvq9F4L6JcEc6jIzWknteLpKSYPtlEQL/9RU+/6qokMw2+r/WI+D2pRUol5mbuoat9aL8RSmKHzUY1Ng3rAWIK0voMJyzvdzpmR8s4Veuq9SoE4fQMT2An0P+1TzUYic3lqZQt8qOSqg4x3bKClSTy+vXEOcagNiI3ZLadoLXutdfbabqsL3+PApI4s36j20TEH53sFL3QXLD+h5fHddpPg5/wb9IBCFDst3+IG01CAVZ0S/cKqAqj70SVTOE1XoaVcSmTuqHrnRbkSLkCU6zh/ohOQcwncqoZKqKrWw== X-YMail-OSG: fh3JMToVM1ma2BTLLbtMHeFeaO3m8_eDPkGUTy18tMqG2HSI4kTZYcmSGXdcddj T6X9xLSwtz4LZneqqUsiJX.oVvpj2OoIkYN1zU7fXVH3w9qptPiSOsZrxc2y1ZIp1AZYk_58cVq5 i8FStdAiJJvV6lIynrbalPWzLt75u5Mlp3YYOpWrYknjQAjI1_Bjg_8MQeh0AYpY1tmpN4wbUFXd _sTxP8pe0PXQj_8h283FU_Ad_8vRMDCFBoH07UOT3pepHKFmSYni9ix1pX5RO_cbDQ91jHfXcPTK F0JmM_fUmr2PqNAiIfLWbnYKzsvLmx3.3VPgS8hrx4Piairc6.iynK5m_eQu91.4JkHczzK_iFDB 7fTxTa52r9MkchF5WsFw_N8LgSr9EyTHipYpj79T3Aum0z4gW5yGQan6YpdYyBZexpw5LJuNoxbV CHJvZ4XoMiOwJhdKoubIroUsKH9qAspARa9uXwMtqGQSmAHSQ_CXSWm6.xThSoaeknP8jSqoJ.4V HXeRaQQFxVycmlUt98sazeTPkOFhvT5kDgLN2ay1w1fDCe2wR7xxWZCfKKvJ_Hv.1cTINF8GmVwY mVU0KQKmQL_9XmWqqofQap51DRvh5o01XhMR7_8wHWP9Hdsy77lpS6dj0W7tQo.0FJ2NQfPV43_d 55lusP7yaxkmjBZvWUEDEGFrIV1CA9SoQAKBbXalhORgMyobH1jLnBKy.lIVT1ZbfHtWt8XkvrZM QbXP7Dpx8.39JnvjLtffpReB0j2Rywz02DLpijm_1eHYlPVTKygfNG5Xv5stx_ubyAbKveaRtEsV 8tx2BsyiBJDq9B0lFg.EctJIAz5TGJezc7c5rdQQviyEAs34_JN3XiLNpWgPO6Z2J_wA8Br1c5WB QH6hHpvWzwrCgRDvgOkkCj1VBZAQxjBIi62iOxVG5TiGyMWBBIFuQuYNzehpu9kEsRwKdU2NKvbq wH0Ru02WMHIhnf0FKJoEtZDo8_IyZ2UwfA3YDBImv9XK7AAIYVZKZ5XFaKw97l.D2YCTjFLqe.NX tZNNgT842zdEQmX_AblTRlq6ylx95nexPv1d6R2ZczhkNTXiGM8FBy6PciSHdFdILFun8eqkvqJ3 nWAbe2iM_L9gBbdpqSiUTv2koWvz6pdEJFkBxm5Dy60yLhH3EMIIlPedG6bVC8P5cSB6g1cKYuDc B0ZY.sWnAPn71rQ7by30CQp8uT.llxaTd0RJ_t_9FWwz6HIyhIwuqpPWL62zQ5KExp0_bsQFbX2B Zs0fACBj2..ne0ELJp5JUbiz7.cVpEYH92I34PhRgAZqfsw8VgvWudrf6czROGbcoOxZSxMtxo4x KAEe4Q66AF2MwR6U_O08ZRUD484.4RGWuUb_5M91DrlNLWzViTsMn2bx0cXYuWkeCODUCkEi6KeS RxJSvGbA1.MhPVAvZj85InKf4jmLGp.KN576cMD_JE6MPlN7_oNJ4VHYuiSjGgGgDNCNVxu5Jfd0 vhn3IVNevXciuRcJbcHw0DKOoZVUgLWVNEStqu942FpCXiMB3LlNRnbH5bPjbr.1z.XdJ07ZaZtP s0y.2zSsPVv2VPJ4fS2owpgWQCvFG_THpwhkEwIoHZvrm9JLPqUQXQ07QquFYeEO3qhywcA4pdqd O0I89zFqLNepp8Z_wplgYROGq6.DHXHGgmct5naMhqqx1xf1BtBh.wxysbqYZoIuk_nSgMqGaZb9 giFTChEaqmAr_xtGuDAAKpMpagLAejxu8G9SPptTaCXwlp0BshRLCe7v1GbZxZReYtIkj0c3fM5u zShPaV2ARum6LXNN63gJluZrsIZZDN4gU6fXXXF_cfZ.THh8yVYYYsiykSWB6q_i3MxuBFoESBDh Q8prfv.zOrByY6JiQU2uVa1wVW9VKqLoYB_gKXxgLkk5CMb9_mjAwOfwxLLKiGQl7RBapWlXWQsA ojSUI8PKic87GHacT..AFeJmWg81dV525K881USx_OjPj.IVnnUvGtSq0EN_P_Yz_yzuPQZSi0TI KykQr8HZoTZHUBZRJim8lj08qmOcMBzL9krTTeUMQXUCc1cGl6UFXoh0exQ1R0No4x3SY8aQDe55 lKVCjbOJoiM4r6N081.dMUrKgCly0z9ohWFJsPCYsag0a4xfpKxhAE1nVDiu4bdJ2z3TGrhqcri2 Aru5qoA.GR.2hh3bvt9xrYS_floyV4lpPWAR5Soxm8S3zZB7zLSuh9PsaAJ49dkXqBOpxQbASRM_ s2UOb1ifwMSRxYBLaFwsy9btKp009qU9qHN40A9huIM_EmZLm8YEEs19Lak8Umxy5E6fuy0Zx7i1 H X-Sonic-MF: X-Sonic-ID: 32dd325e-51d4-4e6f-8c20-4d2c693eb915 Received: from sonic.gate.mail.ne1.yahoo.com by sonic305.consmr.mail.gq1.yahoo.com with HTTP; Sat, 17 Feb 2024 03:29:15 +0000 Received: by hermes--production-gq1-5c57879fdf-wt62k (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 16c431e614640dbc0d9209c3a9dd5c09; Sat, 17 Feb 2024 03:29:12 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.400.31\)) Subject: Re: newfs TRIM flag device support From: Mark Millard In-Reply-To: Date: Fri, 16 Feb 2024 19:29:01 -0800 Cc: Ronald Klop , "freebsd-arm@freebsd.org" Content-Transfer-Encoding: quoted-printable Message-Id: <59693EE7-0DD6-4B41-BA64-0F3D69D3ECC9@yahoo.com> References: <6AD380D4-9B42-4511-9E02-94EB0005D278@yahoo.com> <8D8AA3F5-74A3-4AFE-A560-F52C7C55CF12@yahoo.com> <7okx4eDPse2uF1n3aX4W8RIOdbHXziUhqgvHxqqxpYN-7NsJfEGrfHbDxLgNS3PdOwZt7j9aOXp-0MHDjQAryL4t1Gd2QHUG1sFNmd5dY0k=@proton.me> <454941689.1180.1708079489598@localhost> To: Ordinary Bit X-Mailer: Apple Mail (2.3774.400.31) X-Spamd-Bar: ---- X-Rspamd-Queue-Id: 4TcDms19Hyz4CTD X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US] On Feb 16, 2024, at 18:40, Ordinary Bit wrote: >=20 > On Friday, 16 February 2024 at 18:54, Ordinary Bit = wrote: >>=20 >>=20 >> On Friday, 16 February 2024 at 18:31, Ronald Klop = wrote: >>>=20 >>>=20 >>>=20 >>> Van: Ordinary Bit >>> Datum: vrijdag, 16 februari 2024 10:18 >>> Aan: Mark Millard >>> CC: "freebsd-arm@freebsd.org" >>> Onderwerp: Re: newfs TRIM flag device support >>>=20 >>>=20 >>>=20 >>>=20 >>>=20 >>> Sent with Proton Mail secure email. >>>=20 >>> On Friday, 16 February 2024 at 14:02, Mark Millard = wrote: >>>=20 >>> > On Feb 15, 2024, at 20:08, Ordinary Bit ordinarybit@proton.me = wrote: >>> > >>> > > Hi! >>> > >>> > >>> > Hello. >>> > >>> > > On Friday, 16 February 2024 at 11:41, Mark Millard = marklmi@yahoo.com wrote: >>> > > >>> > > > [Only replying to lists I subscribe to.] >>> > > > >>> > > > On Feb 15, 2024, at 19:19, Ordinary Bit ordinarybit@proton.me = wrote: >>> > > > >>> > > > > I'm reading the newfs manual = https://man.freebsd.org/cgi/man.cgi?newfs(8) to be able to know about = the TRIM flag. In the manual under -t parameter, it mentioned about = "underlying device support", what exactly is this device? >>> > > > >>> > > > 2 contrasting examples: >>> > > > >>> > > > Example 0: Optane NVMe media (PCIe card or U.2, for example) >>> > > > >>> > > > Optane has no need of TRIM and, so, never supports TRIM. >>> > > > >>> > > > Example 1: microsd card media usage >>> > > > >>> > > > A microsd card in the normal type of microsd card slot on >>> > > > Small Board Computers (normally) supports TRIM. Take the >>> > > > same card and put it in a USB reader/writer and use it >>> > > > via USB on the same system: no TRIM is supported by >>> > > > FreeBSD over USB. >>> > > >>> > > So you mean to say that if I have a Rasperry Pi 3 or 4 now and = then have my FreeBSD installed in a microSD card (for example, SanDisk = Extreme card) with UFS/FFS filesytem in it with TRIM enabled parameter = then is it going to recognize it? How to verify? >>> > > >>> > > > FYI: >>> > > > >>> > > > When the file system has TRIM enabled, FreeBSD put out a >>> > > > notice if TRIM will not actually be used in the actual >>> > > > context in use. >>> > > >>> > > Ok, got it. How to check this as well? >>> > >>> > >>> > The console gets a message like: >>> > >>> > WARNING: /mnt: TRIM flag on fs but disk does not support TRIM >>> > >>> > when a mount is attempted (automatic or manually) for a >>> > file system with the trim flag enabled but trim does not >>> > end up active. >>> > >>> > So, for example: >>> > >>> > # dmesg -a | grep TRIM >>> > WARNING: /mnt: TRIM flag on fs but disk does not support TRIM >>> > >>> > (This was a microsd card in a USB reader/writer that was not >>> > used as the boot media: a separate manual mount was used.) >>> > >>> > The file system's TRIM flag status can be checked via use of >>> > "tunefs -p . . .": >>> > >>> > # tunefs -p /mnt >>> > tunefs: POSIX.1e ACLs: (-a) disabled >>> > tunefs: NFSv4 ACLs: (-N) disabled >>> > tunefs: MAC multilabel: (-l) disabled >>> > tunefs: soft updates: (-n) enabled >>> > tunefs: soft update journaling: (-j) disabled >>> > tunefs: gjournal: (-J) disabled >>> > tunefs: trim: (-t) enabled >>> > tunefs: maximum blocks per file in a cylinder group: (-e) 4096 >>> > tunefs: average file size: (-f) 16384 >>> > tunefs: average number of files in a directory: (-s) 64 >>> > tunefs: minimum percentage of free space: (-m) 8% >>> > tunefs: space to hold for metadata blocks: (-k) 6400 >>> > tunefs: optimization preference: (-o) time >>> > tunefs: volume label: (-L) >>> > >>> > If the trim flag is enabled but the mount does not produce the >>> > console message, then TRIM is active. >>> > >>>=20 >>> Oh, that's amazing! I try it with my SanDisk Ultra microSDXC 64GB = and mount it in a USB reader and it shows enabled in the tunefs and is = detected in the dmesg, the same as yours. Therefore, SanDisk Ultra = microSD supports it! >>>=20 >>> root@sd-extreme:~ # /sbin/newfs -U -t /dev/da0s2a >>> /dev/da0s2a: 59000.9MB (120833920 sectors) block size 32768, = fragment size 4096 >>> using 95 cylinder groups of 625.22MB, 20007 blks, 80128 = inodes. >>> with soft updates >>> super-block backups (for fsck_ffs -b #) at: >>> 192, 1280640, 2561088, 3841536, 5121984, 6402432, 7682880, 8963328, = 10243776, 11524224, 12804672, 14085120, >>> 15365568, 16646016, 17926464, 19206912, 20487360, 21767808, = 23048256, 24328704, 25609152, 26889600, 28170048, >>> 29450496, 30730944, 32011392, 33291840, 34572288, 35852736, = 37133184, 38413632, 39694080, 40974528, 42254976, >>> 43535424, 44815872, 46096320, 47376768, 48657216, 49937664, = 51218112, 52498560, 53779008, 55059456, 56339904, >>> 57620352, 58900800, 60181248, 61461696, 62742144, 64022592, = 65303040, 66583488, 67863936, 69144384, 70424832, >>> 71705280, 72985728, 74266176, 75546624, 76827072, 78107520, = 79387968, 80668416, 81948864, 83229312, 84509760, >>> 85790208, 87070656, 88351104, 89631552, 90912000, 92192448, = 93472896, 94753344, 96033792, 97314240, 98594688, >>> 99875136, 101155584, 102436032, 103716480, 104996928, 106277376, = 107557824, 108838272, 110118720, 111399168, >>> 112679616, 113960064, 115240512, 116520960, 117801408, 119081856, = 120362304 >>>=20 >>> root@sd-extreme:~ # dmesg | grep TRIM >>> WARNING: /mnt: TRIM flag on fs but disk does not support TRIM >>>=20 >>> root@sd-extreme:~ # tunefs -p /mnt >>> tunefs: POSIX.1e ACLs: (-a) disabled >>> tunefs: NFSv4 ACLs: (-N) disabled >>> tunefs: MAC multilabel: (-l) disabled >>> tunefs: soft updates: (-n) enabled >>> tunefs: soft update journaling: (-j) disabled >>> tunefs: gjournal: (-J) disabled >>> tunefs: trim: (-t) enabled >>> tunefs: maximum blocks per file in a cylinder group: (-e) 4096 >>> tunefs: average file size: (-f) 16384 >>> tunefs: average number of files in a directory: (-s) 64 >>> tunefs: minimum percentage of free space: (-m) 8% >>> tunefs: space to hold for metadata blocks: (-k) 6400 >>> tunefs: optimization preference: (-o) time >>> tunefs: volume label: (-L) >>>=20 >>> I plan to have TRIM flag enabled in the rootfs partition = (/dev/ufs/rootfs) of my Raspberry Pi. Do you think of any implications = when enabled? As shown below, it is disabled. >>>=20 >>> root@sd-extreme:~ # tunefs -p /dev/ufs/rootfs >>> tunefs: POSIX.1e ACLs: (-a) disabled >>> tunefs: NFSv4 ACLs: (-N) disabled >>> tunefs: MAC multilabel: (-l) disabled >>> tunefs: soft updates: (-n) enabled >>> tunefs: soft update journaling: (-j) disabled >>> tunefs: gjournal: (-J) disabled >>> tunefs: trim: (-t) disabled >>> tunefs: maximum blocks per file in a cylinder group: (-e) 4096 >>> tunefs: average file size: (-f) 16384 >>> tunefs: average number of files in a directory: (-s) 64 >>> tunefs: minimum percentage of free space: (-m) 8% >>> tunefs: space to hold for metadata blocks: (-k) 6400 >>> tunefs: optimization preference: (-o) time >>> tunefs: volume label: (-L) rootfs >>>=20 >>> BR, >>> orbit >>> =20 >>>=20 >>>=20 >>> =20 >>> Hi, >>>=20 >>> I'm sorry to spoil the fun but this message says the device does = *not* support trim. >>>=20 >>> "root@sd-extreme:~ # dmesg | grep TRIM >>> WARNING: /mnt: TRIM flag on fs but disk does not support TRIM" >>>=20 >>>=20 >>> But to give some hope. As Mark pointed out it is not only about the = disk. The controller also needs to support it. >>>=20 >>> As a real world example: >>>=20 >>> I have an RPI3 with an SSD (supporting trim) connected via = USB-to-SATA (not supporting trim). So the total system does not support = trim. >>> $ cat /var/run/dmesg.boot | grep -E "umass0|da0|TRIM" >>> umass0 on uhub1 >>> umass0: on usbus1 >>> umass0: SCSI over Bulk-Only; quirks =3D 0x0100 >>> umass0:0:0: Attached to scbus0 >>> da0 at umass-sim0 bus 0 scbus0 target 0 lun 0 >>> da0: Fixed Direct Access SPC-4 SCSI device >>> da0: Serial Number 2015100000D6 >>> da0: 40.000MB/s transfers >>> da0: 244198MB (500118192 512 byte sectors) >>> da0: quirks=3D0x2 >>> WARNING: /: TRIM flag on fs but disk does not support TRIM >>>=20 >>> $ sysctl kern.cam.da.0 | grep -E "trim|delete" >>> kern.cam.da.0.trim_ticks: 0 >>> kern.cam.da.0.trim_goal: 0 >>> kern.cam.da.0.trim_lbas: 0 >>> kern.cam.da.0.trim_ranges: 0 >>> kern.cam.da.0.trim_count: 0 >>> kern.cam.da.0.delete_max: 65536 >>> kern.cam.da.0.delete_method: NONE >>>=20 >>>=20 >>> On an RPI4B I have an SSD (supporting trim) connected via = USB-to-SATA (which does support trim also). Here trim works fine. (NB: = this uses ZFS, but that does not matter) >>> $ dmesg | grep -E "umass0|da0" >>> umass0 on uhub0 >>> umass0: on usbus0 >>> da0 at umass-sim0 bus 0 scbus0 target 0 lun 0 >>> da0: Fixed Direct Access SPC-4 SCSI = device >>> da0: Serial Number 000000004BA8 >>> da0: 400.000MB/s transfers >>> da0: 244198MB (500118192 512 byte sectors) >>> da0: quirks=3D0x2 >>>=20 >>> $ sysctl kern.cam.da.0 | grep -E "trim|delete" >>> kern.cam.da.0.trim_ticks: 0 >>> kern.cam.da.0.trim_goal: 0 >>> kern.cam.da.0.trim_lbas: 546841704 >>> kern.cam.da.0.trim_ranges: 317280 >>> kern.cam.da.0.trim_count: 301424 >>> kern.cam.da.0.delete_max: 4294901760 >>> kern.cam.da.0.delete_method: UNMAP >>>=20 >>>=20 >>> BTW: enabling trim on the fs while the device does not support it = does not matter and does no harm. Using the sysctl above you can see if = the device actually uses trim. >>>=20 >>> Running "gstat -d" while deleting a lot of files will also show if = trim/delete actions are sent to the disk. >>>=20 >>> So the combination of your USB reader + microSD do not support TRIM, = but you can just try to put the microSD directly in the RPI4 and see = what that does. >>>=20 >>> Regards, >>> Ronald. >>=20 >> Thanks for the callout! Got warmed-up when I saw the TRIM is enabled = :-) but my ultimate goal is really the microSD card slot of my Raspberry = Pi 3 or 4 and not with the USB reader. I'll create an image then see how = it does. >>=20 >> BR, >> orbit=20 >=20 > Confirmed, the SanDisk Ultra microSD does not support TRIM using the = microSD slot in both Raspberry Pi 3 and 4. Instead of building an image = from scratch, I downloaded a FreeBSD 14.0 image and created a new = partition (8GB in size) with UFS/FFS with TRIM enabled and mount it. The = partition is /dev/mmcsd0s3. There's no delete activity observed with = "gstat -d" when I deleted some files in it. You do not show the console output for the mount of /dev/mmcsd0s3 on /mnt. Did it show: WARNING: /mnt: TRIM flag on fs but disk does not support TRIM ? You do not show creating or deleting files. gstat does not give running totals. Your gstat output shown was only for a single 1.065s interval. Did you start the delete during that interval with enough time for the delete to happen during that interval? If not, the command output has no such implications. gstat has a command line option that allows specifying a longer interval. The deletes should be started and completed during the interval in order to see the counts for the deletes. I'll note that the SanDisk Ultra microSD cards that I use do TRIM when in the microsd card slots of the RPi4's I have access to. > root@sd-extreme:~ # /sbin/gpart add -s 8G -t freebsd da0 > da0s3 added /dev/da0 is not the microsd card slot, but a USB device. > root@sd-extreme:~ # gpart show The result would be easier to track with # gpart show -p That would show notation that could be used after /dev/ instead of index numbers. > =3D> 63 124735425 mmcsd0 MBR (59G) > 63 1985 - free - (993K) > 2048 102400 1 fat32lba [active] (50M) > 104448 124626944 2 freebsd (59G) > 124731392 4096 - free - (2.0M) >=20 > =3D> 0 124626944 mmcsd0s2 BSD (59G) > 0 128 - free - (64K) > 128 120833920 1 freebsd-ufs (58G) > 120834048 3792896 2 freebsd-swap (1.8G) There is no /dev/mmcsd0s3 above. > =3D> 63 124735425 da0 MBR (59G) > 63 1985 - free - (993K) > 2048 102400 1 fat32lba [active] (50M) > 104448 10381312 2 freebsd (5.0G) > 10485760 16777216 3 freebsd (8.0G) > 27262976 97472512 - free - (46G) There is a /dev/da0s3 already existing above. But it is not (yet) freebsd-ufs. > =3D> 0 10381312 da0s2 BSD (5.0G) > 0 128 - free - (64K) > 128 10381184 1 freebsd-ufs (4.9G) That freebsd-ufs is /dev/da0s2a . > =3D> 63 124735425 diskid/DISK-121220160204 MBR (59G) > 63 1985 - free - (993K) > 2048 102400 1 fat32lba [active] = (50M) > 104448 10381312 2 freebsd (5.0G) > 10485760 16777216 3 freebsd (8.0G) > 27262976 97472512 - free - (46G) >=20 > =3D> 0 10381312 diskid/DISK-121220160204s2 BSD (5.0G) > 0 128 - free - (64K) > 128 10381184 1 freebsd-ufs (4.9G) >=20 > root@sd-extreme:~ # ls -lah /dev/da0 > /dev/da0 /dev/da0s1 /dev/da0s2 /dev/da0s2a /dev/da0s3 >=20 > root@sd-extreme:~ # newfs -U -t /dev/da0s3 /dev/da0s3 is not in the microsd card slot, but on a USB device. > /dev/da0s3: 8192.0MB (16777216 sectors) block size 32768, fragment = size 4096 > using 14 cylinder groups of 625.22MB, 20007 blks, 80128 = inodes. > with soft updates > super-block backups (for fsck_ffs -b #) at: > 192, 1280640, 2561088, 3841536, 5121984, 6402432, 7682880, 8963328, = 10243776, 11524224, 12804672, 14085120, > 15365568, 16646016 >=20 > root@sd-ultra:~ # gpart show -l > =3D> 63 124735425 mmcsd0 MBR (59G) > 63 1985 - free - (993K) > 2048 102400 1 (null) [active] (50M) > 104448 10381312 2 (null) (5.0G) > 10485760 16777216 3 (null) (8.0G) > 27262976 97472512 - free - (46G) >=20 > =3D> 0 10381312 mmcsd0s2 BSD (5.0G) > 0 128 - free - (64K) > 128 10381184 1 (null) (4.9G) Looks like you moved the microsd card to the microsd card slot from the usb device? > root@sd-ultra:~ # mount -l > /dev/ufs/rootfs on / (ufs, local, soft-updates) > devfs on /dev (devfs) > /dev/msdosfs/EFI on /boot/efi (msdosfs, local, noatime) > tmpfs on /tmp (tmpfs, local) > /dev/mmcsd0s3 on /mnt (ufs, local, soft-updates) >=20 > root@sd-ultra:~ # gstat -d When did you create and delete the files? > dT: 1.065s w: 1.000s During that 1.065s interval? > L(q) ops/s r/s kBps ms/r w/s kBps ms/w d/s kBps = ms/d %busy Name > 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| mmcsd0 > 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| mmcsd0s1 > 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| mmcsd0s2 > 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| mmcsd0s3 > 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| msdosfs/EFI > 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| mmcsd0s2a > 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| ufs/rootfs >=20 > root@sd-ultra:~ # mount -l -v > /dev/ufs/rootfs on / (ufs, local, soft-updates, writes: sync 838 async = 4316, reads: sync 2241 async 2697, fsid 89fc4d652aba17cd, vnodes: count = 1264 ) > devfs on /dev (devfs, fsid 00ff007171000000, vnodes: count 34 ) > /dev/msdosfs/EFI on /boot/efi (msdosfs, local, noatime, writes: sync 1 = async 0, reads: sync 8 async 0, fsid 5b00000032000000, vnodes: count 1 ) > tmpfs on /tmp (tmpfs, local, fsid 01ff008787000000, vnodes: count 6 ) > /dev/mmcsd0s3 on /mnt (ufs, local, soft-updates, writes: sync 2 async = 7866, reads: sync 121491 async 0, fsid 4ee0c86506330890, vnodes: count 3 = ) >=20 > root@sd-ultra:~ # sysctl -a | grep trim # sysctl -a | grep -i trim ? (TRIM is sometimes in capitals.) # dmesg -a | grep -i trim ? > <118>Creating and/or trimming log files. > kern.cam.nda.max_trim: 256 > vfs.ffs.dotrimcons: 1 >=20 > I'll proceed with plan B, to buy industrial grade microSD card having = a garbage collection feature in the controller instead of this consumer = grade card. Either TRIM will work or not there's still garbage = collection I could benefit. It looks to me like the testing procedure may have been flawed. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Sat Feb 17 10:39:19 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TcQKR2SMvz5B1Ff for ; Sat, 17 Feb 2024 10:39:39 +0000 (UTC) (envelope-from ordinarybit@proton.me) Received: from mail-4319.protonmail.ch (mail-4319.protonmail.ch [185.70.43.19]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "protonmail.com", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TcQKQ4krNz42RW for ; Sat, 17 Feb 2024 10:39:38 +0000 (UTC) (envelope-from ordinarybit@proton.me) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=proton.me; s=protonmail; t=1708166374; x=1708425574; bh=DEIjm5Fv+ceuqiuEh6JqJC2htNZS7km3riokd66FURQ=; h=Date:To:From:Cc:Subject:Message-ID:In-Reply-To:References: Feedback-ID:From:To:Cc:Date:Subject:Reply-To:Feedback-ID: Message-ID:BIMI-Selector; b=S9tfhs+LILhYyswCVXXkd20mm95EtNMdpYVcJuQlzEmX805Bi9LE07aGLZj8SYDtW 9HkHO7DrLDOc0Fi7hYM7r+0d0Na6mqzXAOurL35w2eE/55xT8GAKk7W2ztUtRO09wX mIRV+3zQvuWioQRmwGlGaBQH4Fv58rAzKETQu6trozz8psd2Rgo/LwQdT5qvUdU3Jc 06A0Zd8IeN0mcUjG27dGjtMJmPAOolmyB5ItGXiQ/s2Jq4XGYhFj/uL9J7ZhexrOZc 4PfBOjhdAroxOhKmVI/21kY7P64yVlJkHGrnDo4Ecp0nJ8F+dw3tE4eAue8LPvYNVN T6qVL5KSkF2GA== Date: Sat, 17 Feb 2024 10:39:19 +0000 To: Mark Millard From: Ordinary Bit Cc: Ronald Klop , "freebsd-arm@freebsd.org" Subject: Re: newfs TRIM flag device support Message-ID: In-Reply-To: <59693EE7-0DD6-4B41-BA64-0F3D69D3ECC9@yahoo.com> References: <6AD380D4-9B42-4511-9E02-94EB0005D278@yahoo.com> <8D8AA3F5-74A3-4AFE-A560-F52C7C55CF12@yahoo.com> <7okx4eDPse2uF1n3aX4W8RIOdbHXziUhqgvHxqqxpYN-7NsJfEGrfHbDxLgNS3PdOwZt7j9aOXp-0MHDjQAryL4t1Gd2QHUG1sFNmd5dY0k=@proton.me> <454941689.1180.1708079489598@localhost> <59693EE7-0DD6-4B41-BA64-0F3D69D3ECC9@yahoo.com> Feedback-ID: 100108111:user:proton List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: ---- X-Rspamd-Queue-Id: 4TcQKQ4krNz42RW X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:62371, ipnet:185.70.43.0/24, country:CH] > >=20 > > Confirmed, the SanDisk Ultra microSD does not support TRIM using the mi= croSD slot in both Raspberry Pi 3 and 4. Instead of building an image from = scratch, I downloaded a FreeBSD 14.0 image and created a new partition (8GB= in size) with UFS/FFS with TRIM enabled and mount it. The partition is /de= v/mmcsd0s3. There's no delete activity observed with "gstat -d" when I dele= ted some files in it. >=20 >=20 > You do not show the console output for the mount of > /dev/mmcsd0s3 on /mnt. Did it show: >=20 > WARNING: /mnt: TRIM flag on fs but disk does not support TRIM >=20 > ? No, it didn't show up in the dmesg output after I mounted /dev/mmcsd03 on /= mnt. root@sd-ultra:~ # gpart show -p =3D> 63 124735425 mmcsd0 MBR (59G) 63 1985 - free - (993K) 2048 102400 mmcsd0s1 fat32lba [active] (50M) 104448 10381312 mmcsd0s2 freebsd (5.0G) 10485760 16777216 mmcsd0s3 freebsd (8.0G) 27262976 97472512 - free - (46G) =3D> 0 10381312 mmcsd0s2 BSD (5.0G) 0 128 - free - (64K) 128 10381184 mmcsd0s2a freebsd-ufs (4.9G) root@sd-ultra:~ # mount -lv /dev/ufs/rootfs on / (ufs, local, soft-updates, writes: sync 454 async 3046= , reads: sync 38935 async 3, fsid 89fc4d652aba17cd, vnodes: count 542 ) devfs on /dev (devfs, fsid 00ff007171000000, vnodes: count 34 ) /dev/msdosfs/EFI on /boot/efi (msdosfs, local, noatime, writes: sync 1 asyn= c 0, reads: sync 8 async 0, fsid 5c00000032000000, vnodes: count 1 ) tmpfs on /tmp (tmpfs, local, fsid 01ff008787000000, vnodes: count 6 ) root@sd-ultra:~ # root@sd-ultra:~ # mount /dev/mmcsd0 (pressing Tab key) /dev/mmcsd0 /dev/mmcsd0s1 /dev/mmcsd0s2 /dev/mmcsd0s2a /dev/mmcsd0= s3 root@sd-ultra:~ # mount /dev/mmcsd0s3 /mnt root@sd-ultra:~ # root@sd-ultra:~ # dmesg | grep TRIM root@sd-ultra:~ # dmesg | grep trim root@sd-ultra:~ # mount -lv /dev/ufs/rootfs on / (ufs, local, soft-updates, writes: sync 458 async 3065= , reads: sync 38935 async 3, fsid 89fc4d652aba17cd, vnodes: count 542 ) devfs on /dev (devfs, fsid 00ff007171000000, vnodes: count 34 ) /dev/msdosfs/EFI on /boot/efi (msdosfs, local, noatime, writes: sync 1 asyn= c 0, reads: sync 8 async 0, fsid 5c00000032000000, vnodes: count 1 ) tmpfs on /tmp (tmpfs, local, fsid 01ff008787000000, vnodes: count 6 ) /dev/mmcsd0s3 on /mnt (ufs, local, soft-updates, writes: sync 2 async 0, re= ads: sync 3 async 0, fsid 4ee0c86506330890, vnodes: count 2 ) Did I missed something here? but when the microSD card is inserted in my US= B reader, then it shows this up --> WARNING: /mnt: TRIM flag on fs but disk= does not support TRIM. =20 >=20 > You do not show creating or deleting files. gstat does not > give running totals. Your gstat output shown was only for a > single 1.065s interval. Did you start the delete during that > interval with enough time for the delete to happen during > that interval? If not, the command output has no such > implications. > While redoing the test with the same procedures, I do have a file with size= of 1.4G replicated as file01, file02 in the /mnt (/dev/mmcsd0s3). While re= plicating, gstat shows intervals of reading and writing. root@sd-ultra:/mnt # pwd /mnt root@sd-ultra:/mnt # cp file01 file02 root@sd-ultra:/mnt # gstat -d dT: 1.003s w: 1.000s L(q) ops/s r/s kBps ms/r w/s kBps ms/w d/s kBps ms/d= %busy Name 0 449 449 14358 2.1 0 0 0.0 0 0 0.0= 94.0| mmcsd0 0 0 0 0 0.0 0 0 0.0 0 0 0.0= 0.0| mmcsd0s1 0 0 0 0 0.0 0 0 0.0 0 0 0.0= 0.0| mmcsd0s2 0 449 449 14358 2.1 0 0 0.0 0 0 0.0= 94.5| mmcsd0s3 0 0 0 0 0.0 0 0 0.0 0 0 0.0= 0.0| msdosfs/EFI 0 0 0 0 0.0 0 0 0.0 0 0 0.0= 0.0| mmcsd0s2a 0 0 0 0 0.0 0 0 0.0 0 0 0.0= 0.0| ufs/rootfs root@sd-ultra:/mnt # gstat -d dT: 1.046s w: 1.000s L(q) ops/s r/s kBps ms/r w/s kBps ms/w d/s kBps ms/d= %busy Name 4 15 0 0 0.0 15 15665 229.3 0 0 0.0= 100.0| mmcsd0 0 0 0 0 0.0 0 0 0.0 0 0 0.0= 0.0| mmcsd0s1 0 0 0 0 0.0 0 0 0.0 0 0 0.0= 0.0| mmcsd0s2 4 16 0 0 0.0 16 16644 227.6 0 0 0.0= 105.8| mmcsd0s3 0 0 0 0 0.0 0 0 0.0 0 0 0.0= 0.0| msdosfs/EFI 0 0 0 0 0.0 0 0 0.0 0 0 0.0= 0.0| mmcsd0s2a 0 0 0 0 0.0 0 0 0.0 0 0 0.0= 0.0| ufs/rootfs root@sd-ultra:/mnt # gstat -d dT: 1.017s w: 1.000s L(q) ops/s r/s kBps ms/r w/s kBps ms/w d/s kBps ms/d= %busy Name 4 50 35 1133 2.1 15 14478 200.3 0 0 0.0= 99.7| mmcsd0 0 0 0 0 0.0 0 0 0.0 0 0 0.0= 0.0| mmcsd0s1 0 0 0 0 0.0 0 0 0.0 0 0 0.0= 0.0| mmcsd0s2 4 50 35 1133 2.1 15 14478 200.3 0 0 0.0= 99.8| mmcsd0s3 0 0 0 0 0.0 0 0 0.0 0 0 0.0= 0.0| msdosfs/EFI 0 0 0 0 0.0 0 0 0.0 0 0 0.0= 0.0| mmcsd0s2a 0 0 0 0 0.0 0 0 0.0 0 0 0.0= 0.0| ufs/rootfs root@sd-ultra:/mnt # ls -lah total 5684108 drwxr-xr-x 3 root wheel 512B Feb 11 13:04 . drwxr-xr-x 21 root wheel 512B Feb 11 14:13 .. drwxrwxr-x 2 root operator 512B Feb 11 14:57 .snap -rw-r--r-- 1 root wheel 1.4G Feb 11 17:20 file01 -rw-r--r-- 1 root wheel 1.4G Feb 11 12:56 file02 and then delete the files with rm root@sd-ultra:/mnt # rm * Executing delete is very fast and waited for a little bit more then suddenl= y I observed mmcsd0 and mmcsd0s3 showing deleting of data. This is the firs= t time I found. How about this one? It's a one time show and I never find a= nother stats after this interval. root@sd-ultra:/mnt # gstat -d dT: 1.028s w: 1.000s L(q) ops/s r/s kBps ms/r w/s kBps ms/w d/s kBps ms/d= %busy Name 22 422 14 436 71.8 0 0 0.0 409 1613422 38.= 6 96.7| mmcsd0 0 0 0 0 0.0 0 0 0.0 0 0 0.0= 0.0| mmcsd0s1 0 0 0 0 0.0 0 0 0.0 0 0 0.0= 0.0| mmcsd0s2 5 41 14 436 71.9 0 0 0.0 27 1637328 76.= 7 98.0| mmcsd0s3 0 0 0 0 0.0 0 0 0.0 0 0 0.0= 0.0| msdosfs/EFI 0 0 0 0 0.0 0 0 0.0 0 0 0.0= 0.0| mmcsd0s2a 0 0 0 0 0.0 0 0 0.0 0 0 0.0= 0.0| ufs/rootfs > gstat has a command line option that allows specifying a > longer interval. The deletes should be started and completed > during the interval in order to see the counts for the > deletes. >=20 > I'll note that the SanDisk Ultra microSD cards that I use > do TRIM when in the microsd card slots of the RPi4's I have > access to. > > > root@sd-extreme:~ # /sbin/gpart add -s 8G -t freebsd da0 > > da0s3 added >=20 > /dev/da0 is not the microsd card slot, but a USB device. >=20 > > root@sd-extreme:~ # gpart show >=20 >=20 >=20 > The result would be easier to track with >=20 > # gpart show -p >=20 > That would show notation that could be used after > /dev/ instead of index numbers. >=20 > > =3D> 63 124735425 mmcsd0 MBR (59G) > > 63 1985 - free - (993K) > > 2048 102400 1 fat32lba [active] (50M) > > 104448 124626944 2 freebsd (59G) > > 124731392 4096 - free - (2.0M) > >=20 > > =3D> 0 124626944 mmcsd0s2 BSD (59G) > > 0 128 - free - (64K) > > 128 120833920 1 freebsd-ufs (58G) > > 120834048 3792896 2 freebsd-swap (1.8G) >=20 >=20 > There is no /dev/mmcsd0s3 above. >=20 > > =3D> 63 124735425 da0 MBR (59G) > > 63 1985 - free - (993K) > > 2048 102400 1 fat32lba [active] (50M) > > 104448 10381312 2 freebsd (5.0G) > > 10485760 16777216 3 freebsd (8.0G) > > 27262976 97472512 - free - (46G) >=20 >=20 > There is a /dev/da0s3 already existing above. But > it is not (yet) freebsd-ufs. >=20 > > =3D> 0 10381312 da0s2 BSD (5.0G) > > 0 128 - free - (64K) > > 128 10381184 1 freebsd-ufs (4.9G) >=20 >=20 > That freebsd-ufs is /dev/da0s2a . >=20 > > =3D> 63 124735425 diskid/DISK-121220160204 MBR (59G) > > 63 1985 - free - (993K) > > 2048 102400 1 fat32lba [active] (50M) > > 104448 10381312 2 freebsd (5.0G) > > 10485760 16777216 3 freebsd (8.0G) > > 27262976 97472512 - free - (46G) > >=20 > > =3D> 0 10381312 diskid/DISK-121220160204s2 BSD (5.0G) > > 0 128 - free - (64K) > > 128 10381184 1 freebsd-ufs (4.9G) > >=20 > > root@sd-extreme:~ # ls -lah /dev/da0 > > /dev/da0 /dev/da0s1 /dev/da0s2 /dev/da0s2a /dev/da0s3 > >=20 > > root@sd-extreme:~ # newfs -U -t /dev/da0s3 >=20 >=20 > /dev/da0s3 is not in the microsd card slot, but on a USB device. >=20 > > /dev/da0s3: 8192.0MB (16777216 sectors) block size 32768, fragment size= 4096 > > using 14 cylinder groups of 625.22MB, 20007 blks, 80128 inodes. > > with soft updates > > super-block backups (for fsck_ffs -b #) at: > > 192, 1280640, 2561088, 3841536, 5121984, 6402432, 7682880, 8963328, 102= 43776, 11524224, 12804672, 14085120, > > 15365568, 16646016 > >=20 > > root@sd-ultra:~ # gpart show -l > > =3D> 63 124735425 mmcsd0 MBR (59G) > > 63 1985 - free - (993K) > > 2048 102400 1 (null) [active] (50M) > > 104448 10381312 2 (null) (5.0G) > > 10485760 16777216 3 (null) (8.0G) > > 27262976 97472512 - free - (46G) > >=20 > > =3D> 0 10381312 mmcsd0s2 BSD (5.0G) > > 0 128 - free - (64K) > > 128 10381184 1 (null) (4.9G) >=20 >=20 > Looks like you moved the microsd card to the microsd card > slot from the usb device? > Sorry for the confusion, I dd'ed the FreeBSD 14 image from my @sd-extreme (= a Rasberry Pi 3 host) where my SanDisk microSD is inserted in a USB reader = and then after the installer had been created I also created the new partit= ion which is the /dev/da0s3 having UFS/FFS plus the TRIM flag enabled. I al= so mounted the rootfs /dev/da0s2a to modify the /etc/rc.conf and remove the= firstboot file to intervene auto-resize of the default rootfs FS size. Fro= m here, I moved the microsSD card from @sd-extreme and insert it into my Ra= spberry Pi 4 microSD slot with hostname @sd-ultra. From here, partitions ch= anged from /dev/da0* to /dev/mmcsd0*.=20 > > root@sd-ultra:~ # mount -l > > /dev/ufs/rootfs on / (ufs, local, soft-updates) > > devfs on /dev (devfs) > > /dev/msdosfs/EFI on /boot/efi (msdosfs, local, noatime) > > tmpfs on /tmp (tmpfs, local) > > /dev/mmcsd0s3 on /mnt (ufs, local, soft-updates) > >=20 > > root@sd-ultra:~ # gstat -d >=20 >=20 > When did you create and delete the files? >=20 > > dT: 1.065s w: 1.000s >=20 >=20 > During that 1.065s interval? >=20 > > L(q) ops/s r/s kBps ms/r w/s kBps ms/w d/s kBps ms/d %busy Name > > 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| mmcsd0 > > 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| mmcsd0s1 > > 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| mmcsd0s2 > > 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| mmcsd0s3 > > 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| msdosfs/EFI > > 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| mmcsd0s2a > > 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| ufs/rootfs > >=20 > > root@sd-ultra:~ # mount -l -v > > /dev/ufs/rootfs on / (ufs, local, soft-updates, writes: sync 838 async = 4316, reads: sync 2241 async 2697, fsid 89fc4d652aba17cd, vnodes: count 126= 4 ) > > devfs on /dev (devfs, fsid 00ff007171000000, vnodes: count 34 ) > > /dev/msdosfs/EFI on /boot/efi (msdosfs, local, noatime, writes: sync 1 = async 0, reads: sync 8 async 0, fsid 5b00000032000000, vnodes: count 1 ) > > tmpfs on /tmp (tmpfs, local, fsid 01ff008787000000, vnodes: count 6 ) > > /dev/mmcsd0s3 on /mnt (ufs, local, soft-updates, writes: sync 2 async 7= 866, reads: sync 121491 async 0, fsid 4ee0c86506330890, vnodes: count 3 ) > >=20 > > root@sd-ultra:~ # sysctl -a | grep trim >=20 >=20 > # sysctl -a | grep -i trim >=20 > ? (TRIM is sometimes in capitals.) >=20 > # dmesg -a | grep -i trim >=20 > ? No output results. root@sd-ultra:~ # dmesg | grep trim root@sd-ultra:~ # dmesg | grep TRIM >=20 > > <118>Creating and/or trimming log files. > > kern.cam.nda.max_trim: 256 > > vfs.ffs.dotrimcons: 1 > >=20 > > I'll proceed with plan B, to buy industrial grade microSD card having a= garbage collection feature in the controller instead of this consumer grad= e card. Either TRIM will work or not there's still garbage collection I cou= ld benefit. >=20 >=20 > It looks to me like the testing procedure may have been > flawed. > Maybe but upon re-testing I already found this gstat deleting activity, thi= s is after I executed rm to the 2 files as I stated above. root@sd-ultra:/mnt # gstat -d dT: 1.028s w: 1.000s L(q) ops/s r/s kBps ms/r w/s kBps ms/w d/s kBps ms/d= %busy Name 22 422 14 436 71.8 0 0 0.0 409 1613422 38.= 6 96.7| mmcsd0 0 0 0 0 0.0 0 0 0.0 0 0 0.0= 0.0| mmcsd0s1 0 0 0 0 0.0 0 0 0.0 0 0 0.0= 0.0| mmcsd0s2 5 41 14 436 71.9 0 0 0.0 27 1637328 76.= 7 98.0| mmcsd0s3 0 0 0 0 0.0 0 0 0.0 0 0 0.0= 0.0| msdosfs/EFI 0 0 0 0 0.0 0 0 0.0 0 0 0.0= 0.0| mmcsd0s2a 0 0 0 0 0.0 0 0 0.0 0 0 0.0= 0.0| ufs/rootfs If still not the expected outcome, do you have procedures how did TRIM happ= ened in your simulation? BR, orbit From nobody Sat Feb 17 15:20:52 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TcXZB5Xmnz5BH26 for ; Sat, 17 Feb 2024 15:21:06 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic312-23.consmr.mail.gq1.yahoo.com (sonic312-23.consmr.mail.gq1.yahoo.com [98.137.69.204]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4TcXZB3CGCz4dN3 for ; Sat, 17 Feb 2024 15:21:06 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1708183264; bh=Y77NbXQiFHmWZF4T6Eyk5HUrDWMqePrfxp/FJaLcPbw=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=mHjMBYECXj12vNi6RPaCOkbgLJZq+wfiB1AJHI8+QwEp/plrYcvbxavfAkhiRTn3f/WPt2QZk732HhNRXBnpQtF5lfdXyuEeught16B95baL/a/gkqVwhNE5TtYP8YDL7/U0Dfiy81v7cdZkTVMxVP7eGaK7MxxXwbOg7xvXPhFf4tua53eEXhetyetVQGADiXA0vicEDLJUb4iiByA5slXIUl+fgvMGA2I90Hzk8/uTwQy2gm/dTNxOg07KUZ9PNPpjmEgYRSOvfsY+r5NU84FAxaiZGtwceJ/mjAEpHjFBHpOv82AyvPVA4/ckGEMysxhDyk6Stokfcyo3gneEJw== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1708183264; bh=gv1xdD7Y0r4qvYEghINV9ExvV4xj08rp/xmFKTu48Ku=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=CTq+divVFvTPENL/Irp0xZTT7XOPULhoO7310+LFswLmpwbnYqryWnBS/9uvwrWbqrWeBnyVvVi5qoK8inGj7v6ROrQtcEWFwjXKVmtElHEZVQ7Y289S4qr1CZXYslKqc3oHYbX8g9GqxFmHZTZrASzCfrmkuPGcdP3k5CU+hom7ZB27se3bFZlXuRKt+bUpkkHWH3QR6esJd4a9e2aU9vkeKN0QQGhdUT5oO+iGXZUlTG7rh5XsS7WVb7GWWWSyNu/JgkR8jpdEsGpauTiTvEp7fqObj6ITn6Bi+rv5QoJy8oatVYsFtCSevjnCjbSpkA1krXQmSzPzKO5v5VBJ6A== X-YMail-OSG: 7HY9oqsVM1nis0iLEAnuMVGas.wRZ0tkHtfHfcjyL5NIegLO.dpObCfb_ogxcag M7vtcYXOSSHFw3uG.Kd8p0Jh07m9KsXFZ_2EqX9AeaviJhKCyezCMT3.rz9JHkCn7IJUT4RGXLEK qiAa0u2bTTyl7m5jEuAoY56N1xAvEHODVgLq.iV.UO46oyqpmbj6WRCasuFuczDaO0lAdBtiFs53 5qsNvz6WAwotlX4yN7_1HAoKkK8C9uIc2_XNf3RYnG.Pd3xiSm6cJBdZE0xGH1Gzd5.NUECrM4h9 wpVIoGgEu1nPH1M93gGMlkO82opxQhn_Z8Ktk_7sHwiAx_RlYahwSqoIMPZco3Sgr2Dpu0LwqDvh rLqcZ.Y0n109VnVjoAHFoJf_8sPCNS2hIZFpoJoRDbZs_3Cm_j82eEMwrS3zD_Dwr5PgQUVyqLcT f3EoYGrn5K5NDzQVlu9MkuFjnQNIHScIdzGLpexXQ8XZFExUj2dPNg41CTJIwJvSDTADMXD568i. 7NRiQuIAHSdQ14uGzbFDQ8oCy8X_0kSuOHSkam.LwlPYNTkL6rA_O0UlPjyu5dJTALtdGH2pGrIE LhBXuFEm7ZXxWkchoyQHhHHG6_JeeX72v4sml5NrSH8AGtOElsIYGWzXSDBt2wPBN5QZhdkEzmiJ Kh1AVzIOSnb7J1chPhpUlAec9LMI89A3LVYvR1IsZ6pMVd.wzV6I_fBIUvT5kdAFpxGLqyLmKhs0 91B5UbVzaQQ4hNNDlvmLufFWAvZg_aWjjNZIGbR47JIIXzz.PTU1dJAzEqzlcQt6ChKkoyY0ke6w Kjw2if5d.tWjzJy86mxFyQnxaw5usjxWZJCH42h3bnmdLA2RHsZVDZLeFhVPgJztwxjvxhtMIS7R UyJ0QHNY5jJcGEDfb1VYOiAx6sGdkv0KdQuV8FEKZyAnJnFCWmXm_XNlrJ0NM_mKeurE7J4I5XOY OZvP3Jh7eyWVX3NzwQbDUvIKcLjSv.j6uszjL9JrwObmqNPEnc5.WrY2HSAMZeDmoYXV4OmEyHLM atK.iPNax.JhaET5tqcoqLPccDys0UHgQfJMzv7w03hS9auKLOAFoJ3DkpkEN5iJSAoJJQRLyflA mN6GGnK7idYwBVhjw4CXpqos1Ij3nF7UPOqtTZVTeL1NOrlVzqSktsLYGgEnQJtbvVALcFMIoHH0 HNQNxjoow3t_zqmkodVINAKQI_tFnozVz50lR1UbZEQRs.UQkOGQg0e9xcCcryNC1lANK.T6mscL _jpGmeAh6a7hvMdWQKXfvJRqFVPMLZEh1krpvCXdDPMerq4MoTBocaBt8CsG2CAo2RLWTL11RVTo 8GkEM8g2tI9y2dr5QrgVSCJC00k2dMbtRUvWO0ngbv2KzVkdrpWdUk7sKpfCzAh5TEmLPyKYFC_q QeewrEFMX2VD5QXjyt2SUD_EPhuseS1NNdQrOHBzeEMQwJPcIwXj9ZNwYkP1h0d9tOVBKYZH9qsB 68Y4NLMLU2s2.TgtHdUCm7avwsJS.48XROW6K1BSs8wczKK94xctKVQXFrTIa.hUKmJo9bR5wDGP seAtzelN18pl1E6vr92duat6cOWDxqG5f0UNyuGdcUVluHbqRLnzhTigsT_EdJqDTUjIeLbaQJay .LQktGXrAC6kdoWDeJ4yQP0cA8y2h9Xmga1ECM2CvvUnC4Idd8HpB0giwTOycg8OtVDY3GGuuOCJ DcM6hWFNkcXPtZV2WPC0UKV9lreM5f_ZkvnWLns9Q_6CN_jwIpQ4z0jdbJitu4KmSDyVXbVE2CdW OiWDeP_sCyjn4OisERPoX4Q2GcQq5K4VJi.Y514vJp.3whiqqF3b.u1VPumfJyAofz2dJf7W9HRD myVM5gA7pgPkVoDFWqchnyNkNWAHvPb0T6v3cZY.Avhb8En3XWZcyNU3ozaTTV4AV96dObmUaA5Q xiaQhHUmAotCbFBCEc0jCFlYlLI59MKPz7eD2Bkr8khTu5AOSut3hLqbhgFlsN9Dbm6GW_576uCu 1oZcu99yjTWJQRfvSsng92RsCx04KtW4pPoRu7BKWhcv9N4_gcPeNzCctteLo9bnumpxI5IZUnHA CK8TWliAJzDHZ5MzHIipxFff2duNHiIR.7CNKdcZ7HFD0vRGXjFkn7jKLqjqGnCvfZBDUT_6YiUN FhTLyax.rf3l.uEMBM97.ZzIO3unbALaR6hASnkh3FLGU8Cm4bbu6rHBhmH4k7LgdLzUGrvusJbd YCBwuM97_FcF9yM4gi6z._.ptCaiedHrzaW.Hw28BgX.CYEVyxl52rWPR8_paAHWEk.qFA2vunYp J X-Sonic-MF: X-Sonic-ID: 5511fe43-bec6-4df2-8c61-b43991710440 Received: from sonic.gate.mail.ne1.yahoo.com by sonic312.consmr.mail.gq1.yahoo.com with HTTP; Sat, 17 Feb 2024 15:21:04 +0000 Received: by hermes--production-gq1-5c57879fdf-qprqq (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 3376f75d834121a6e4d4f9505cc3e8f2; Sat, 17 Feb 2024 15:21:03 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.400.31\)) Subject: Re: newfs TRIM flag device support From: Mark Millard In-Reply-To: Date: Sat, 17 Feb 2024 07:20:52 -0800 Cc: Ronald Klop , "freebsd-arm@freebsd.org" Content-Transfer-Encoding: quoted-printable Message-Id: <1B0C2E27-6776-40A1-A8BF-29094FF8FC03@yahoo.com> References: <6AD380D4-9B42-4511-9E02-94EB0005D278@yahoo.com> <8D8AA3F5-74A3-4AFE-A560-F52C7C55CF12@yahoo.com> <7okx4eDPse2uF1n3aX4W8RIOdbHXziUhqgvHxqqxpYN-7NsJfEGrfHbDxLgNS3PdOwZt7j9aOXp-0MHDjQAryL4t1Gd2QHUG1sFNmd5dY0k=@proton.me> <454941689.1180.1708079489598@localhost> <59693EE7-0DD6-4B41-BA64-0F3D69D3ECC9@yahoo.com> To: Ordinary Bit X-Mailer: Apple Mail (2.3774.400.31) X-Spamd-Bar: ---- X-Rspamd-Queue-Id: 4TcXZB3CGCz4dN3 X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US] On Feb 17, 2024, at 02:39, Ordinary Bit wrote: >=20 >>>=20 >>> Confirmed, the SanDisk Ultra microSD does not support TRIM using the = microSD slot in both Raspberry Pi 3 and 4. Instead of building an image = from scratch, I downloaded a FreeBSD 14.0 image and created a new = partition (8GB in size) with UFS/FFS with TRIM enabled and mount it. The = partition is /dev/mmcsd0s3. There's no delete activity observed with = "gstat -d" when I deleted some files in it. My notes were bad by not saying how to force FreeBSD to actually do the file system updates (sync use). But you got the evidence that you were looking for anyway: you have now seen evidence of the TRIM activity when the microsd card is in the built-in slot on a RPi* . Below, you showed the SanDisk Ultra microSD doing TRIM activity. >> You do not show the console output for the mount of >> /dev/mmcsd0s3 on /mnt. Did it show: >>=20 >> WARNING: /mnt: TRIM flag on fs but disk does not support TRIM >>=20 >> ? >=20 > No, it didn't show up in the dmesg output after I mounted /dev/mmcsd03 = on /mnt. >=20 > root@sd-ultra:~ # gpart show -p > =3D> 63 124735425 mmcsd0 MBR (59G) > 63 1985 - free - (993K) > 2048 102400 mmcsd0s1 fat32lba [active] (50M) > 104448 10381312 mmcsd0s2 freebsd (5.0G) > 10485760 16777216 mmcsd0s3 freebsd (8.0G) An oddity just above is that mmcsd0s3 is not set to the type freebsd-ufs . But the oddity does not mess up the testing you were doing. > 27262976 97472512 - free - (46G) >=20 > =3D> 0 10381312 mmcsd0s2 BSD (5.0G) > 0 128 - free - (64K) > 128 10381184 mmcsd0s2a freebsd-ufs (4.9G) >=20 > . . . > root@sd-ultra:~ # mount /dev/mmcsd0s3 /mnt > root@sd-ultra:~ # > root@sd-ultra:~ # dmesg | grep TRIM > root@sd-ultra:~ # dmesg | grep trim The above is evidence that TRIM is supported for the combination in use. > . . . >=20 > Did I missed something here? but when the microSD card is inserted in = my USB reader, then it shows this up --> WARNING: /mnt: TRIM flag on fs = but disk does not support TRIM. =20 I already wrote that the combination FreeBSD+USB-reader/writer normally does not support TRIM for any microsd card. This is normal. >> . . . > . . . > root@sd-ultra:/mnt # cp file01 file02 I should have noted that the actual file I/O can continue to happen after the shell gets to the next prompt. Using a larger enough file can be a good technique of allowing gstat to be run after a cp or rm command but to see activity in the later gstat. > . . . >=20 > root@sd-ultra:/mnt # ls -lah > total 5684108 > drwxr-xr-x 3 root wheel 512B Feb 11 13:04 . > drwxr-xr-x 21 root wheel 512B Feb 11 14:13 .. > drwxrwxr-x 2 root operator 512B Feb 11 14:57 .snap > -rw-r--r-- 1 root wheel 1.4G Feb 11 17:20 file01 > -rw-r--r-- 1 root wheel 1.4G Feb 11 12:56 file02 >=20 > and then delete the files with rm >=20 > root@sd-ultra:/mnt # rm * >=20 > Executing delete is very fast and waited for a little bit more then = suddenly I observed mmcsd0 and mmcsd0s3 showing deleting of data. This = is the first time I found. How about this one? It's a one time show and = I never find another stats after this interval. >=20 > root@sd-ultra:/mnt # gstat -d > dT: 1.028s w: 1.000s > L(q) ops/s r/s kBps ms/r w/s kBps ms/w d/s kBps = ms/d %busy Name > 22 422 14 436 71.8 0 0 0.0 409 1613422 = 38.6 96.7| mmcsd0 > 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| mmcsd0s1 > 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| mmcsd0s2 > 5 41 14 436 71.9 0 0 0.0 27 1637328 = 76.7 98.0| mmcsd0s3 > 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| msdosfs/EFI > 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| mmcsd0s2a > 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| ufs/rootfs The above gives evidence of specific TRIM activity happening. >>> . . . >>=20 >>=20 >> # sysctl -a | grep -i trim >>=20 >> ? (TRIM is sometimes in capitals.) >>=20 >> # dmesg -a | grep -i trim >>=20 >> ? >=20 > No output results. >=20 > root@sd-ultra:~ # dmesg | grep trim > root@sd-ultra:~ # dmesg | grep TRIM The above afer the mount is evidence that TRIM is supported for the combination in use. >> . . . >>=20 >> It looks to me like the testing procedure may have been >> flawed. >>=20 >=20 > Maybe but upon re-testing I already found this gstat deleting = activity, this is after I executed rm to the 2 files as I stated above. >=20 > root@sd-ultra:/mnt # gstat -d > dT: 1.028s w: 1.000s > L(q) ops/s r/s kBps ms/r w/s kBps ms/w d/s kBps = ms/d %busy Name > 22 422 14 436 71.8 0 0 0.0 409 1613422 = 38.6 96.7| mmcsd0 > 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| mmcsd0s1 > 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| mmcsd0s2 > 5 41 14 436 71.9 0 0 0.0 27 1637328 = 76.7 98.0| mmcsd0s3 > 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| msdosfs/EFI > 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| mmcsd0s2a > 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| ufs/rootfs >=20 > If still not the expected outcome, do you have procedures how did TRIM = happened in your simulation? It is as expected. You got a good sequence of well timed test steps this time. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Sat Feb 17 15:42:35 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TcY3C1ckzz5BKng for ; Sat, 17 Feb 2024 15:42:47 +0000 (UTC) (envelope-from ordinarybit@proton.me) Received: from mail-4325.protonmail.ch (mail-4325.protonmail.ch [185.70.43.25]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "protonmail.com", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TcY3B5lNTz4jJp for ; Sat, 17 Feb 2024 15:42:46 +0000 (UTC) (envelope-from ordinarybit@proton.me) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=proton.me; s=protonmail; t=1708184564; x=1708443764; bh=7URUQCL3dkz+tXEQgGJcFDmwcfIXx15MqqQfOZTPGW4=; h=Date:To:From:Cc:Subject:Message-ID:In-Reply-To:References: Feedback-ID:From:To:Cc:Date:Subject:Reply-To:Feedback-ID: Message-ID:BIMI-Selector; b=CPpmQFYCQ6r1lkk4hN8ul7EvyuJHu4T0jtQjzq8hYOWLvAftGL4HYYh6D0ONUucye qqiluwoVC68TGtLWRbtwTeNlVTKypI0uzJ7rUwYelwNya05AYXltFtZW+SXEqloGlA +ncaz+8mIF2XbIElYOtX6kIgHaDjiZNkLgNTBNtnydLHkHt9Ce4R5kwFKOcjyDoqYE araAPa8HEnkQYi/+DMvsmA/YDVmYP3zD554aKSSoLaTdhn6YZOD1o0Nd7M6obpizw3 wd68AxXlR8tOZLOQmT9oQKuL8dfHkiOLGRZKaJsPukVGisWi3xoX8pn1ZSPV1GWOCO fqoZZy3JilYXQ== Date: Sat, 17 Feb 2024 15:42:35 +0000 To: Ordinary Bit From: Ordinary Bit Cc: Mark Millard , Ronald Klop , "freebsd-arm@freebsd.org" Subject: Re: newfs TRIM flag device support Message-ID: In-Reply-To: References: <6AD380D4-9B42-4511-9E02-94EB0005D278@yahoo.com> <8D8AA3F5-74A3-4AFE-A560-F52C7C55CF12@yahoo.com> <7okx4eDPse2uF1n3aX4W8RIOdbHXziUhqgvHxqqxpYN-7NsJfEGrfHbDxLgNS3PdOwZt7j9aOXp-0MHDjQAryL4t1Gd2QHUG1sFNmd5dY0k=@proton.me> <454941689.1180.1708079489598@localhost> <59693EE7-0DD6-4B41-BA64-0F3D69D3ECC9@yahoo.com> Feedback-ID: 100108111:user:proton List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: ---- X-Rspamd-Queue-Id: 4TcY3B5lNTz4jJp X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:62371, ipnet:185.70.43.0/24, country:CH] On Saturday, 17 February 2024 at 18:39, Ordinary Bit wrote: > > > Confirmed, the SanDisk Ultra microSD does not support TRIM using the = microSD slot in both Raspberry Pi 3 and 4. Instead of building an image fro= m scratch, I downloaded a FreeBSD 14.0 image and created a new partition (8= GB in size) with UFS/FFS with TRIM enabled and mount it. The partition is /= dev/mmcsd0s3. There's no delete activity observed with "gstat -d" when I de= leted some files in it. > >=20 > > You do not show the console output for the mount of > > /dev/mmcsd0s3 on /mnt. Did it show: > >=20 > > WARNING: /mnt: TRIM flag on fs but disk does not support TRIM > >=20 > > ? >=20 >=20 > No, it didn't show up in the dmesg output after I mounted /dev/mmcsd03 on= /mnt. >=20 > root@sd-ultra:~ # gpart show -p > =3D> 63 124735425 mmcsd0 MBR (59G) >=20 > 63 1985 - free - (993K) > 2048 102400 mmcsd0s1 fat32lba [active] (50M) > 104448 10381312 mmcsd0s2 freebsd (5.0G) > 10485760 16777216 mmcsd0s3 freebsd (8.0G) > 27262976 97472512 - free - (46G) >=20 > =3D> 0 10381312 mmcsd0s2 BSD (5.0G) >=20 > 0 128 - free - (64K) > 128 10381184 mmcsd0s2a freebsd-ufs (4.9G) >=20 > root@sd-ultra:~ # mount -lv > /dev/ufs/rootfs on / (ufs, local, soft-updates, writes: sync 454 async 30= 46, reads: sync 38935 async 3, fsid 89fc4d652aba17cd, vnodes: count 542 ) > devfs on /dev (devfs, fsid 00ff007171000000, vnodes: count 34 ) > /dev/msdosfs/EFI on /boot/efi (msdosfs, local, noatime, writes: sync 1 as= ync 0, reads: sync 8 async 0, fsid 5c00000032000000, vnodes: count 1 ) > tmpfs on /tmp (tmpfs, local, fsid 01ff008787000000, vnodes: count 6 ) > root@sd-ultra:~ # > root@sd-ultra:~ # mount /dev/mmcsd0 (pressing Tab key) > /dev/mmcsd0 /dev/mmcsd0s1 /dev/mmcsd0s2 /dev/mmcsd0s2a /dev/mmcsd0s3 > root@sd-ultra:~ # mount /dev/mmcsd0s3 /mnt > root@sd-ultra:~ # > root@sd-ultra:~ # dmesg | grep TRIM > root@sd-ultra:~ # dmesg | grep trim >=20 > root@sd-ultra:~ # mount -lv > /dev/ufs/rootfs on / (ufs, local, soft-updates, writes: sync 458 async 30= 65, reads: sync 38935 async 3, fsid 89fc4d652aba17cd, vnodes: count 542 ) > devfs on /dev (devfs, fsid 00ff007171000000, vnodes: count 34 ) > /dev/msdosfs/EFI on /boot/efi (msdosfs, local, noatime, writes: sync 1 as= ync 0, reads: sync 8 async 0, fsid 5c00000032000000, vnodes: count 1 ) > tmpfs on /tmp (tmpfs, local, fsid 01ff008787000000, vnodes: count 6 ) > /dev/mmcsd0s3 on /mnt (ufs, local, soft-updates, writes: sync 2 async 0, = reads: sync 3 async 0, fsid 4ee0c86506330890, vnodes: count 2 ) >=20 > Did I missed something here? but when the microSD card is inserted in my = USB reader, then it shows this up --> WARNING: /mnt: TRIM flag on fs but di= sk does not support TRIM. >=20 > > You do not show creating or deleting files. gstat does not > > give running totals. Your gstat output shown was only for a > > single 1.065s interval. Did you start the delete during that > > interval with enough time for the delete to happen during > > that interval? If not, the command output has no such > > implications. >=20 >=20 > While redoing the test with the same procedures, I do have a file with si= ze of 1.4G replicated as file01, file02 in the /mnt (/dev/mmcsd0s3). While = replicating, gstat shows intervals of reading and writing. >=20 > root@sd-ultra:/mnt # pwd > /mnt >=20 > root@sd-ultra:/mnt # cp file01 file02 >=20 > root@sd-ultra:/mnt # gstat -d > dT: 1.003s w: 1.000s > L(q) ops/s r/s kBps ms/r w/s kBps ms/w d/s kBps ms/d %busy Name > 0 449 449 14358 2.1 0 0 0.0 0 0 0.0 94.0| mmcsd0 > 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| mmcsd0s1 > 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| mmcsd0s2 > 0 449 449 14358 2.1 0 0 0.0 0 0 0.0 94.5| mmcsd0s3 > 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| msdosfs/EFI > 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| mmcsd0s2a > 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| ufs/rootfs >=20 > root@sd-ultra:/mnt # gstat -d > dT: 1.046s w: 1.000s > L(q) ops/s r/s kBps ms/r w/s kBps ms/w d/s kBps ms/d %busy Name > 4 15 0 0 0.0 15 15665 229.3 0 0 0.0 100.0| mmcsd0 > 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| mmcsd0s1 > 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| mmcsd0s2 > 4 16 0 0 0.0 16 16644 227.6 0 0 0.0 105.8| mmcsd0s3 > 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| msdosfs/EFI > 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| mmcsd0s2a > 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| ufs/rootfs >=20 > root@sd-ultra:/mnt # gstat -d > dT: 1.017s w: 1.000s > L(q) ops/s r/s kBps ms/r w/s kBps ms/w d/s kBps ms/d %busy Name > 4 50 35 1133 2.1 15 14478 200.3 0 0 0.0 99.7| mmcsd0 > 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| mmcsd0s1 > 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| mmcsd0s2 > 4 50 35 1133 2.1 15 14478 200.3 0 0 0.0 99.8| mmcsd0s3 > 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| msdosfs/EFI > 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| mmcsd0s2a > 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| ufs/rootfs >=20 > root@sd-ultra:/mnt # ls -lah > total 5684108 > drwxr-xr-x 3 root wheel 512B Feb 11 13:04 . > drwxr-xr-x 21 root wheel 512B Feb 11 14:13 .. > drwxrwxr-x 2 root operator 512B Feb 11 14:57 .snap > -rw-r--r-- 1 root wheel 1.4G Feb 11 17:20 file01 > -rw-r--r-- 1 root wheel 1.4G Feb 11 12:56 file02 >=20 > and then delete the files with rm >=20 > root@sd-ultra:/mnt # rm * >=20 > Executing delete is very fast and waited for a little bit more then sudde= nly I observed mmcsd0 and mmcsd0s3 showing deleting of data. This is the fi= rst time I found. How about this one? It's a one time show and I never find= another stats after this interval. >=20 > root@sd-ultra:/mnt # gstat -d > dT: 1.028s w: 1.000s > L(q) ops/s r/s kBps ms/r w/s kBps ms/w d/s kBps ms/d %busy Name > 22 422 14 436 71.8 0 0 0.0 409 1613422 38.6 96.7| mmcsd0 > 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| mmcsd0s1 > 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| mmcsd0s2 > 5 41 14 436 71.9 0 0 0.0 27 1637328 76.7 98.0| mmcsd0s3 > 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| msdosfs/EFI > 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| mmcsd0s2a > 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| ufs/rootfs >=20 > > gstat has a command line option that allows specifying a > > longer interval. The deletes should be started and completed > > during the interval in order to see the counts for the > > deletes. > >=20 > > I'll note that the SanDisk Ultra microSD cards that I use > > do TRIM when in the microsd card slots of the RPi4's I have > > access to. > >=20 > > > root@sd-extreme:~ # /sbin/gpart add -s 8G -t freebsd da0 > > > da0s3 added > >=20 > > /dev/da0 is not the microsd card slot, but a USB device. > >=20 > > > root@sd-extreme:~ # gpart show > >=20 > > The result would be easier to track with > >=20 > > # gpart show -p > >=20 > > That would show notation that could be used after > > /dev/ instead of index numbers. > >=20 > > > =3D> 63 124735425 mmcsd0 MBR (59G) > > > 63 1985 - free - (993K) > > > 2048 102400 1 fat32lba [active] (50M) > > > 104448 124626944 2 freebsd (59G) > > > 124731392 4096 - free - (2.0M) > > >=20 > > > =3D> 0 124626944 mmcsd0s2 BSD (59G) > > > 0 128 - free - (64K) > > > 128 120833920 1 freebsd-ufs (58G) > > > 120834048 3792896 2 freebsd-swap (1.8G) > >=20 > > There is no /dev/mmcsd0s3 above. > >=20 > > > =3D> 63 124735425 da0 MBR (59G) > > > 63 1985 - free - (993K) > > > 2048 102400 1 fat32lba [active] (50M) > > > 104448 10381312 2 freebsd (5.0G) > > > 10485760 16777216 3 freebsd (8.0G) > > > 27262976 97472512 - free - (46G) > >=20 > > There is a /dev/da0s3 already existing above. But > > it is not (yet) freebsd-ufs. > >=20 > > > =3D> 0 10381312 da0s2 BSD (5.0G) > > > 0 128 - free - (64K) > > > 128 10381184 1 freebsd-ufs (4.9G) > >=20 > > That freebsd-ufs is /dev/da0s2a . > >=20 > > > =3D> 63 124735425 diskid/DISK-121220160204 MBR (59G) > > > 63 1985 - free - (993K) > > > 2048 102400 1 fat32lba [active] (50M) > > > 104448 10381312 2 freebsd (5.0G) > > > 10485760 16777216 3 freebsd (8.0G) > > > 27262976 97472512 - free - (46G) > > >=20 > > > =3D> 0 10381312 diskid/DISK-121220160204s2 BSD (5.0G) > > > 0 128 - free - (64K) > > > 128 10381184 1 freebsd-ufs (4.9G) > > >=20 > > > root@sd-extreme:~ # ls -lah /dev/da0 > > > /dev/da0 /dev/da0s1 /dev/da0s2 /dev/da0s2a /dev/da0s3 > > >=20 > > > root@sd-extreme:~ # newfs -U -t /dev/da0s3 > >=20 > > /dev/da0s3 is not in the microsd card slot, but on a USB device. > >=20 > > > /dev/da0s3: 8192.0MB (16777216 sectors) block size 32768, fragment si= ze 4096 > > > using 14 cylinder groups of 625.22MB, 20007 blks, 80128 inodes. > > > with soft updates > > > super-block backups (for fsck_ffs -b #) at: > > > 192, 1280640, 2561088, 3841536, 5121984, 6402432, 7682880, 8963328, 1= 0243776, 11524224, 12804672, 14085120, > > > 15365568, 16646016 > > >=20 > > > root@sd-ultra:~ # gpart show -l > > > =3D> 63 124735425 mmcsd0 MBR (59G) > > > 63 1985 - free - (993K) > > > 2048 102400 1 (null) [active] (50M) > > > 104448 10381312 2 (null) (5.0G) > > > 10485760 16777216 3 (null) (8.0G) > > > 27262976 97472512 - free - (46G) > > >=20 > > > =3D> 0 10381312 mmcsd0s2 BSD (5.0G) > > > 0 128 - free - (64K) > > > 128 10381184 1 (null) (4.9G) > >=20 > > Looks like you moved the microsd card to the microsd card > > slot from the usb device? >=20 >=20 > Sorry for the confusion, I dd'ed the FreeBSD 14 image from my @sd-extreme= (a Rasberry Pi 3 host) where my SanDisk microSD is inserted in a USB reade= r and then after the installer had been created I also created the new part= ition which is the /dev/da0s3 having UFS/FFS plus the TRIM flag enabled. I = also mounted the rootfs /dev/da0s2a to modify the /etc/rc.conf and remove t= he firstboot file to intervene auto-resize of the default rootfs FS size. F= rom here, I moved the microsSD card from @sd-extreme and insert it into my = Raspberry Pi 4 microSD slot with hostname @sd-ultra. From here, partitions = changed from /dev/da0* to /dev/mmcsd0*. >=20 > > > root@sd-ultra:~ # mount -l > > > /dev/ufs/rootfs on / (ufs, local, soft-updates) > > > devfs on /dev (devfs) > > > /dev/msdosfs/EFI on /boot/efi (msdosfs, local, noatime) > > > tmpfs on /tmp (tmpfs, local) > > > /dev/mmcsd0s3 on /mnt (ufs, local, soft-updates) > > >=20 > > > root@sd-ultra:~ # gstat -d > >=20 > > When did you create and delete the files? > >=20 > > > dT: 1.065s w: 1.000s > >=20 > > During that 1.065s interval? > >=20 > > > L(q) ops/s r/s kBps ms/r w/s kBps ms/w d/s kBps ms/d %busy Name > > > 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| mmcsd0 > > > 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| mmcsd0s1 > > > 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| mmcsd0s2 > > > 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| mmcsd0s3 > > > 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| msdosfs/EFI > > > 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| mmcsd0s2a > > > 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| ufs/rootfs > > >=20 > > > root@sd-ultra:~ # mount -l -v > > > /dev/ufs/rootfs on / (ufs, local, soft-updates, writes: sync 838 asyn= c 4316, reads: sync 2241 async 2697, fsid 89fc4d652aba17cd, vnodes: count 1= 264 ) > > > devfs on /dev (devfs, fsid 00ff007171000000, vnodes: count 34 ) > > > /dev/msdosfs/EFI on /boot/efi (msdosfs, local, noatime, writes: sync = 1 async 0, reads: sync 8 async 0, fsid 5b00000032000000, vnodes: count 1 ) > > > tmpfs on /tmp (tmpfs, local, fsid 01ff008787000000, vnodes: count 6 ) > > > /dev/mmcsd0s3 on /mnt (ufs, local, soft-updates, writes: sync 2 async= 7866, reads: sync 121491 async 0, fsid 4ee0c86506330890, vnodes: count 3 ) > > >=20 > > > root@sd-ultra:~ # sysctl -a | grep trim > >=20 > > # sysctl -a | grep -i trim > >=20 > > ? (TRIM is sometimes in capitals.) > >=20 > > # dmesg -a | grep -i trim > >=20 > > ? >=20 >=20 > No output results. >=20 > root@sd-ultra:~ # dmesg | grep trim > root@sd-ultra:~ # dmesg | grep TRIM >=20 > > > <118>Creating and/or trimming log files. > > > kern.cam.nda.max_trim: 256 > > > vfs.ffs.dotrimcons: 1 > > >=20 > > > I'll proceed with plan B, to buy industrial grade microSD card having= a garbage collection feature in the controller instead of this consumer gr= ade card. Either TRIM will work or not there's still garbage collection I c= ould benefit. > >=20 > > It looks to me like the testing procedure may have been > > flawed. >=20 >=20 > Maybe but upon re-testing I already found this gstat deleting activity, t= his is after I executed rm to the 2 files as I stated above. >=20 > root@sd-ultra:/mnt # gstat -d > dT: 1.028s w: 1.000s > L(q) ops/s r/s kBps ms/r w/s kBps ms/w d/s kBps ms/d %busy Name > 22 422 14 436 71.8 0 0 0.0 409 1613422 38.6 96.7| mmcsd0 > 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| mmcsd0s1 > 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| mmcsd0s2 > 5 41 14 436 71.9 0 0 0.0 27 1637328 76.7 98.0| mmcsd0s3 > 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| msdosfs/EFI > 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| mmcsd0s2a > 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| ufs/rootfs >=20 > If still not the expected outcome, do you have procedures how did TRIM ha= ppened in your simulation? >=20 > BR, > orbit By redoing what I've performed in Raspberry Pi 4B with the same SanDisk mic= roSD card inserted in Raspberry Pi 3B, the delete activity were also observ= ed in gstat. root@sd-ultra:~ # mount /dev/mmcsd0s3 /mnt root@sd-ultra:~ # mount -lv /dev/ufs/rootfs on / (ufs, local, soft-updates, writes: sync 12 async 185, = reads: sync 900 async 3, fsid 89fc4d652aba17cd, vnodes: count 504 ) devfs on /dev (devfs, fsid 00ff007171000000, vnodes: count 37 ) /dev/msdosfs/EFI on /boot/efi (msdosfs, local, noatime, writes: sync 1 asyn= c 0, reads: sync 8 async 0, fsid 5c00000032000000, vnodes: count 1 ) tmpfs on /tmp (tmpfs, local, fsid 01ff008787000000, vnodes: count 6 ) /dev/mmcsd0s3 on /mnt (ufs, local, soft-updates, writes: sync 2 async 21, r= eads: sync 65 async 0, fsid 4ee0c86506330890, vnodes: count 3 ) root@sd-ultra:~ # cd /mnt root@sd-ultra:/mnt # ls -lah total 5684108 drwxr-xr-x 3 root wheel 512B Feb 11 20:41 . drwxr-xr-x 21 root wheel 512B Feb 11 20:55 .. drwxrwxr-x 2 root operator 512B Feb 11 14:57 .snap -rw-r--r-- 1 root wheel 1.4G Feb 11 20:35 file1 -rw-r--r-- 1 root wheel 1.4G Feb 11 20:38 file2 root@sd-ultra:/mnt # rm * root@sd-ultra:~ # gstat -d dT: 1.041s w: 1.000s L(q) ops/s r/s kBps ms/r w/s kBps ms/w d/s kBps ms/d= %busy Name 23 586 20 645 44.6 0 0 0.0 566 2229919 27.= 0 90.2| mmcsd0 0 0 0 0 0.0 0 0 0.0 0 0 0.0= 0.0| mmcsd0s1 0 0 0 0 0.0 0 0 0.0 0 0 0.0= 0.0| mmcsd0s2 3 59 20 645 44.6 0 0 0.0 38 2225985 49.= 6 90.1| mmcsd0s3 0 0 0 0 0.0 0 0 0.0 0 0 0.0= 0.0| msdosfs/EFI 0 0 0 0 0.0 0 0 0.0 0 0 0.0= 0.0| mmcsd0s2a 0 0 0 0 0.0 0 0 0.0 0 0 0.0= 0.0| ufs/rootfs So, it's just a matter of waiting until it occurs. BR, orbit From nobody Sat Feb 17 16:21:14 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TcYvn04cDz5BR2V for ; Sat, 17 Feb 2024 16:21:25 +0000 (UTC) (envelope-from ordinarybit@proton.me) Received: from mail-4324.protonmail.ch (mail-4324.protonmail.ch [185.70.43.24]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "protonmail.com", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TcYvm4KS8z4mRS for ; Sat, 17 Feb 2024 16:21:24 +0000 (UTC) (envelope-from ordinarybit@proton.me) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=proton.me; s=vo4slgzljvhubgtgwclqig63ly.protonmail; t=1708186881; x=1708446081; bh=qb4s+J/9oUk5mWbIwbjaVhlkBz9XBkdp1gieDIJ35Yg=; h=Date:To:From:Cc:Subject:Message-ID:In-Reply-To:References: Feedback-ID:From:To:Cc:Date:Subject:Reply-To:Feedback-ID: Message-ID:BIMI-Selector; b=NVc1v9T3pHF9VY9ktoTxNU/bSdz/0d8cG+vE2I4jEceQVyjSalm58T74Eumd5hkhD f32M/q4RiMXZw5E78MmtyKCQo8RWRc8e4chSnlD4ZhipqI9rn5BpgbU6yZTfEdH2QB r7imhhPrl+MvwtH0mAUDedVjYZW6pa+LabXQezWBc1MP1CXXIHuRlnw+puKvB3bN+l 2KhxBtf4qyNmRPbWe+6RVcKd+ZablkmRCkQVU17mJmExsPAHZIO07lkuXDePK6tE4h 43OC7bOBa5V0fOqekR7NcnLXqRo3QeNIi6AuuOb6on7DmqyLR9G7OwpFS8wrzScPMr xl56DZ919mkEA== Date: Sat, 17 Feb 2024 16:21:14 +0000 To: Mark Millard From: Ordinary Bit Cc: Ronald Klop , "freebsd-arm@freebsd.org" Subject: Re: newfs TRIM flag device support Message-ID: In-Reply-To: <1B0C2E27-6776-40A1-A8BF-29094FF8FC03@yahoo.com> References: <8D8AA3F5-74A3-4AFE-A560-F52C7C55CF12@yahoo.com> <7okx4eDPse2uF1n3aX4W8RIOdbHXziUhqgvHxqqxpYN-7NsJfEGrfHbDxLgNS3PdOwZt7j9aOXp-0MHDjQAryL4t1Gd2QHUG1sFNmd5dY0k=@proton.me> <454941689.1180.1708079489598@localhost> <59693EE7-0DD6-4B41-BA64-0F3D69D3ECC9@yahoo.com> <1B0C2E27-6776-40A1-A8BF-29094FF8FC03@yahoo.com> Feedback-ID: 100108111:user:proton List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: ---- X-Rspamd-Queue-Id: 4TcYvm4KS8z4mRS X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:62371, ipnet:185.70.43.0/24, country:CH] On Saturday, 17 February 2024 at 23:20, Mark Millard wr= ote: > On Feb 17, 2024, at 02:39, Ordinary Bit ordinarybit@proton.me wrote: >=20 > > > > Confirmed, the SanDisk Ultra microSD does not support TRIM using th= e microSD slot in both Raspberry Pi 3 and 4. Instead of building an image f= rom scratch, I downloaded a FreeBSD 14.0 image and created a new partition = (8GB in size) with UFS/FFS with TRIM enabled and mount it. The partition is= /dev/mmcsd0s3. There's no delete activity observed with "gstat -d" when I = deleted some files in it. >=20 >=20 > My notes were bad by not saying how to force FreeBSD to > actually do the file system updates (sync use). But you > got the evidence that you were looking for anyway: you > have now seen evidence of the TRIM activity when the > microsd card is in the built-in slot on a RPi* . >=20 > Below, you showed the SanDisk Ultra microSD doing > TRIM activity. > Yeah, thanks for the help Mark! My latest testing were also showing Raspber= ry Pi 3B performed TRIM activity. So, both Raspberry Pi 3B and 4B can do TR= IM with SanDisk Ultra microSD using their built-in slots. =20 > > > You do not show the console output for the mount of > > > /dev/mmcsd0s3 on /mnt. Did it show: > > >=20 > > > WARNING: /mnt: TRIM flag on fs but disk does not support TRIM > > >=20 > > > ? > >=20 > > No, it didn't show up in the dmesg output after I mounted /dev/mmcsd03 = on /mnt. > >=20 > > root@sd-ultra:~ # gpart show -p > > =3D> 63 124735425 mmcsd0 MBR (59G) > > 63 1985 - free - (993K) > > 2048 102400 mmcsd0s1 fat32lba [active] (50M) > > 104448 10381312 mmcsd0s2 freebsd (5.0G) > > 10485760 16777216 mmcsd0s3 freebsd (8.0G) >=20 >=20 > An oddity just above is that mmcsd0s3 is not set to the > type freebsd-ufs . But the oddity does not mess up the > testing you were doing. > Let me test this one with freebsd-ufs type because you're right it's only f= reebsd. =20 > > 27262976 97472512 - free - (46G) > >=20 > > =3D> 0 10381312 mmcsd0s2 BSD (5.0G) > > 0 128 - free - (64K) > > 128 10381184 mmcsd0s2a freebsd-ufs (4.9G) > >=20 > > . . . > > root@sd-ultra:~ # mount /dev/mmcsd0s3 /mnt > > root@sd-ultra:~ # > > root@sd-ultra:~ # dmesg | grep TRIM > > root@sd-ultra:~ # dmesg | grep trim >=20 >=20 > The above is evidence that TRIM is supported for the > combination in use. > > > . . . > >=20 > > Did I missed something here? but when the microSD card is inserted in m= y USB reader, then it shows this up --> WARNING: /mnt: TRIM flag on fs but = disk does not support TRIM. >=20 >=20 > I already wrote that the combination FreeBSD+USB-reader/writer > normally does not support TRIM for any microsd card. This is > normal. >=20 > > > . . . > > > . . . > > > root@sd-ultra:/mnt # cp file01 file02 >=20 >=20 > I should have noted that the actual file I/O can continue > to happen after the shell gets to the next prompt. Using > a larger enough file can be a good technique of allowing > gstat to be run after a cp or rm command but to see > activity in the later gstat. > Got it, I can do test with this scenario too. =20 > > . . . > >=20 > > root@sd-ultra:/mnt # ls -lah > > total 5684108 > > drwxr-xr-x 3 root wheel 512B Feb 11 13:04 . > > drwxr-xr-x 21 root wheel 512B Feb 11 14:13 .. > > drwxrwxr-x 2 root operator 512B Feb 11 14:57 .snap > > -rw-r--r-- 1 root wheel 1.4G Feb 11 17:20 file01 > > -rw-r--r-- 1 root wheel 1.4G Feb 11 12:56 file02 > >=20 > > and then delete the files with rm > >=20 > > root@sd-ultra:/mnt # rm * > >=20 > > Executing delete is very fast and waited for a little bit more then sud= denly I observed mmcsd0 and mmcsd0s3 showing deleting of data. This is the = first time I found. How about this one? It's a one time show and I never fi= nd another stats after this interval. > >=20 > > root@sd-ultra:/mnt # gstat -d > > dT: 1.028s w: 1.000s > > L(q) ops/s r/s kBps ms/r w/s kBps ms/w d/s kBps ms/d %busy Name > > 22 422 14 436 71.8 0 0 0.0 409 1613422 38.6 96.7| mmcsd0 > > 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| mmcsd0s1 > > 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| mmcsd0s2 > > 5 41 14 436 71.9 0 0 0.0 27 1637328 76.7 98.0| mmcsd0s3 > > 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| msdosfs/EFI > > 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| mmcsd0s2a > > 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| ufs/rootfs >=20 >=20 > The above gives evidence of specific TRIM activity happening. > Alright! =20 > > > > . . . > > >=20 > > > # sysctl -a | grep -i trim > > >=20 > > > ? (TRIM is sometimes in capitals.) > > >=20 > > > # dmesg -a | grep -i trim > > >=20 > > > ? > >=20 > > No output results. > >=20 > > root@sd-ultra:~ # dmesg | grep trim > > root@sd-ultra:~ # dmesg | grep TRIM >=20 >=20 > The above afer the mount is evidence that TRIM is supported > for the combination in use. >=20 > > > . . . > > >=20 > > > It looks to me like the testing procedure may have been > > > flawed. > >=20 > > Maybe but upon re-testing I already found this gstat deleting activity,= this is after I executed rm to the 2 files as I stated above. > >=20 > > root@sd-ultra:/mnt # gstat -d > > dT: 1.028s w: 1.000s > > L(q) ops/s r/s kBps ms/r w/s kBps ms/w d/s kBps ms/d %busy Name > > 22 422 14 436 71.8 0 0 0.0 409 1613422 38.6 96.7| mmcsd0 > > 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| mmcsd0s1 > > 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| mmcsd0s2 > > 5 41 14 436 71.9 0 0 0.0 27 1637328 76.7 98.0| mmcsd0s3 > > 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| msdosfs/EFI > > 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| mmcsd0s2a > > 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| ufs/rootfs > >=20 > > If still not the expected outcome, do you have procedures how did TRIM = happened in your simulation? >=20 >=20 > It is as expected. You got a good sequence of well timed test steps > this time. >=20 Apologize for the confusion of my first test with poor documentation provid= ed but at last it's proven. Thanks once again! BR, orbit From nobody Sat Feb 17 16:44:19 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TcZQM6SgVz5BTY7 for ; Sat, 17 Feb 2024 16:44:27 +0000 (UTC) (envelope-from mike@karels.net) Received: from mail2.karels.net (mail2.karels.net [3.19.118.201]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "freebsd", Issuer "freebsd" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TcZQM4NMrz4q1x for ; Sat, 17 Feb 2024 16:44:27 +0000 (UTC) (envelope-from mike@karels.net) Authentication-Results: mx1.freebsd.org; none Received: from mail2.karels.net (localhost [IPv6:0:0:0:0:0:0:0:1]) by mail2.karels.net (8.17.1/8.17.1) with ESMTP id 41HGiK6q020439; Sat, 17 Feb 2024 10:44:20 -0600 (CST) (envelope-from mike@karels.net) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=karels.net; s=mail2; t=1708188261; bh=LBRgPI1ZylS/xjT4UII+zLHAZZPHMKbPdrfXP1o0XzI=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=ExnAUCIxJcEgkVSJChJCmimiikaRVLARmg6vrBFx7Zz3uoP/EA94un7jtA7wdSpCx 1ICFkNYaVmq+g0l45RmHYoziYVRjjVouqzBYcl6m+PXPJPYhpA5/msmpgaRzYpAoEY MlMqHSbsJI47Hw9+JxvAvicFtoYbP79e7S5oiwxcRQO0Jp99rrqjDATVhDkr+EaXPi qRNFv77xWrU5HfTK1QG/JCYsSxYwMb1c3ojkvi7OsiVQ6cD+SqjX9pDObnhjOI/Ow/ P3GuAv/vGP29RAqwsy8LLooDKnPSkJvQUkXGGgw22D7KIy9e5AMR7MxGOt1eSoQBTt 1ylcPCDzTxm5Q== Received: from [10.0.2.130] ([73.62.165.147]) by mail2.karels.net with ESMTPSA id 4ucFI2Ti0GXVTwAAs/W3XQ (envelope-from ); Sat, 17 Feb 2024 10:44:20 -0600 From: Mike Karels To: Ordinary Bit Cc: Mark Millard , Ronald Klop , freebsd-arm@freebsd.org Subject: Re: newfs TRIM flag device support Date: Sat, 17 Feb 2024 10:44:19 -0600 X-Mailer: MailMate (1.14r6015) Message-ID: In-Reply-To: References: <8D8AA3F5-74A3-4AFE-A560-F52C7C55CF12@yahoo.com> <7okx4eDPse2uF1n3aX4W8RIOdbHXziUhqgvHxqqxpYN-7NsJfEGrfHbDxLgNS3PdOwZt7j9aOXp-0MHDjQAryL4t1Gd2QHUG1sFNmd5dY0k=@proton.me> <454941689.1180.1708079489598@localhost> <59693EE7-0DD6-4B41-BA64-0F3D69D3ECC9@yahoo.com> <1B0C2E27-6776-40A1-A8BF-29094FF8FC03@yahoo.com> List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 Content-Type: text/plain Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: ---- X-Rspamd-Queue-Id: 4TcZQM4NMrz4q1x X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:16509, ipnet:3.16.0.0/14, country:US] On 17 Feb 2024, at 10:21, Ordinary Bit wrote: > On Saturday, 17 February 2024 at 23:20, Mark Millard wrote: > >> On Feb 17, 2024, at 02:39, Ordinary Bit ordinarybit@proton.me wrote: >> >>>>> Confirmed, the SanDisk Ultra microSD does not support TRIM using th= e microSD slot in both Raspberry Pi 3 and 4. Instead of building an image= from scratch, I downloaded a FreeBSD 14.0 image and created a new partit= ion (8GB in size) with UFS/FFS with TRIM enabled and mount it. The partit= ion is /dev/mmcsd0s3. There's no delete activity observed with "gstat -d"= when I deleted some files in it. >> >> >> My notes were bad by not saying how to force FreeBSD to >> actually do the file system updates (sync use). But you >> got the evidence that you were looking for anyway: you >> have now seen evidence of the TRIM activity when the >> microsd card is in the built-in slot on a RPi* . >> >> Below, you showed the SanDisk Ultra microSD doing >> TRIM activity. >> > > Yeah, thanks for the help Mark! My latest testing were also showing Ras= pberry Pi 3B performed TRIM activity. So, both Raspberry Pi 3B and 4B can= do TRIM with SanDisk Ultra microSD using their built-in slots. > >>>> You do not show the console output for the mount of >>>> /dev/mmcsd0s3 on /mnt. Did it show: >>>> >>>> WARNING: /mnt: TRIM flag on fs but disk does not support TRIM >>>> >>>> ? >>> >>> No, it didn't show up in the dmesg output after I mounted /dev/mmcsd0= 3 on /mnt. >>> >>> root@sd-ultra:~ # gpart show -p >>> =3D> 63 124735425 mmcsd0 MBR (59G) >>> 63 1985 - free - (993K) >>> 2048 102400 mmcsd0s1 fat32lba [active] (50M) >>> 104448 10381312 mmcsd0s2 freebsd (5.0G) >>> 10485760 16777216 mmcsd0s3 freebsd (8.0G) >> >> >> An oddity just above is that mmcsd0s3 is not set to the >> type freebsd-ufs . But the oddity does not mess up the >> testing you were doing. >> > > Let me test this one with freebsd-ufs type because you're right it's on= ly freebsd. freebsd-ufs is a GPT type; this is MBR. The ufs filesystem is presumably= mmcsd0s3a. If the filesystem gets mounted, the partitioning is OK. Mike >>> 27262976 97472512 - free - (46G) >>> >>> =3D> 0 10381312 mmcsd0s2 BSD (5.0G) >>> 0 128 - free - (64K) >>> 128 10381184 mmcsd0s2a freebsd-ufs (4.9G) >>> >>> . . . >>> root@sd-ultra:~ # mount /dev/mmcsd0s3 /mnt >>> root@sd-ultra:~ # >>> root@sd-ultra:~ # dmesg | grep TRIM >>> root@sd-ultra:~ # dmesg | grep trim >> >> >> The above is evidence that TRIM is supported for the >> combination in use. >> >>> . . . >>> >>> Did I missed something here? but when the microSD card is inserted in= my USB reader, then it shows this up --> WARNING: /mnt: TRIM flag on fs = but disk does not support TRIM. >> >> >> I already wrote that the combination FreeBSD+USB-reader/writer >> normally does not support TRIM for any microsd card. This is >> normal. >> >>>> . . . >>>> . . . >>>> root@sd-ultra:/mnt # cp file01 file02 >> >> >> I should have noted that the actual file I/O can continue >> to happen after the shell gets to the next prompt. Using >> a larger enough file can be a good technique of allowing >> gstat to be run after a cp or rm command but to see >> activity in the later gstat. >> > > Got it, I can do test with this scenario too. > >>> . . . >>> >>> root@sd-ultra:/mnt # ls -lah >>> total 5684108 >>> drwxr-xr-x 3 root wheel 512B Feb 11 13:04 . >>> drwxr-xr-x 21 root wheel 512B Feb 11 14:13 .. >>> drwxrwxr-x 2 root operator 512B Feb 11 14:57 .snap >>> -rw-r--r-- 1 root wheel 1.4G Feb 11 17:20 file01 >>> -rw-r--r-- 1 root wheel 1.4G Feb 11 12:56 file02 >>> >>> and then delete the files with rm >>> >>> root@sd-ultra:/mnt # rm * >>> >>> Executing delete is very fast and waited for a little bit more then s= uddenly I observed mmcsd0 and mmcsd0s3 showing deleting of data. This is = the first time I found. How about this one? It's a one time show and I ne= ver find another stats after this interval. >>> >>> root@sd-ultra:/mnt # gstat -d >>> dT: 1.028s w: 1.000s >>> L(q) ops/s r/s kBps ms/r w/s kBps ms/w d/s kBps ms/d %busy Name >>> 22 422 14 436 71.8 0 0 0.0 409 1613422 38.6 96.7| mmcsd0 >>> 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| mmcsd0s1 >>> 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| mmcsd0s2 >>> 5 41 14 436 71.9 0 0 0.0 27 1637328 76.7 98.0| mmcsd0s3 >>> 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| msdosfs/EFI >>> 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| mmcsd0s2a >>> 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| ufs/rootfs >> >> >> The above gives evidence of specific TRIM activity happening. >> > > Alright! > >>>>> . . . >>>> >>>> # sysctl -a | grep -i trim >>>> >>>> ? (TRIM is sometimes in capitals.) >>>> >>>> # dmesg -a | grep -i trim >>>> >>>> ? >>> >>> No output results. >>> >>> root@sd-ultra:~ # dmesg | grep trim >>> root@sd-ultra:~ # dmesg | grep TRIM >> >> >> The above afer the mount is evidence that TRIM is supported >> for the combination in use. >> >>>> . . . >>>> >>>> It looks to me like the testing procedure may have been >>>> flawed. >>> >>> Maybe but upon re-testing I already found this gstat deleting activit= y, this is after I executed rm to the 2 files as I stated above. >>> >>> root@sd-ultra:/mnt # gstat -d >>> dT: 1.028s w: 1.000s >>> L(q) ops/s r/s kBps ms/r w/s kBps ms/w d/s kBps ms/d %busy Name >>> 22 422 14 436 71.8 0 0 0.0 409 1613422 38.6 96.7| mmcsd0 >>> 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| mmcsd0s1 >>> 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| mmcsd0s2 >>> 5 41 14 436 71.9 0 0 0.0 27 1637328 76.7 98.0| mmcsd0s3 >>> 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| msdosfs/EFI >>> 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| mmcsd0s2a >>> 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| ufs/rootfs >>> >>> If still not the expected outcome, do you have procedures how did TRI= M happened in your simulation? >> >> >> It is as expected. You got a good sequence of well timed test steps >> this time. >> > > Apologize for the confusion of my first test with poor documentation pr= ovided but at last it's proven. Thanks once again! > > BR, > orbit From nobody Sat Feb 17 19:04:06 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TcdWp4NZzz53ty4 for ; Sat, 17 Feb 2024 19:04:22 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic307-55.consmr.mail.gq1.yahoo.com (sonic307-55.consmr.mail.gq1.yahoo.com [98.137.64.31]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4TcdWp1y96z4CTx for ; Sat, 17 Feb 2024 19:04:22 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1708196659; bh=Fnynw+mjwj+mCCKauZBLa8DJhIWrDH897cRMYzUA0F4=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=AKiBSPOe3/FSbgT2tvcbKuLaKB3meqCVGWuFD2wO6zOMg0DZlYT+AWclrxoC8qP33oKJq3qJMV9xUsCIsvLN1kqpqaAyxjmUDZ1fZkDb27f65RZOTql//7G9eJSEgr3/4zFUCaXbWFGHeQXgByd6BxOd2eNTQCvFvHwuu+7PIs0/ZubnI6egYyQ18lhdmh4vGAvCl6HX2FFMNO3AHD+Z9PZwd5yiiJXtwWT3JoHZw1T2YQwmcP96qR/Gnzl1n1i2BDLMOq0KFTQtc77FhfjPQFQ7dOaQTRE6R4fwumFV+sLwjMTe5Suaxs32WqeobYEPVuQs1jsaz518tZhb5zg4pg== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1708196659; bh=xOA4Oa7yIMBAMjjBXF/9klRK4PRkHMzL77yyL8ZUIY5=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=tRbuvjek3BMvV07sclJV1UzDodvRIielqMvteI0HqZMvEkRiNTYbpbmAw1DHWWzTwH32ycmNU7KdrfcdczJBmAJB55tUUVyRAJV3D8BLvZiX3NVNym2vAii3Q/MTulPKAdCwfhZt8G8Am1uDrmxggKADUmaIoRpInS5a1fkxFBZX5yLzr4EoXpN13qzMUfy0YkBCjsKftqGABFG5B6U02qSwdhZ9l7FTpVdCYZa3laRSdl58CHUO6C0fXC5C24blU1Cu+LDZP1vnptHB5VsCpRQilEHa6xefjNNHLpkDlg+za0k0ldWuIIc7qJLsjwPI7LZYh+gsP0EEsEGkUTDGVg== X-YMail-OSG: 5XxA9tEVM1kLar0PNVfFLsTpX.q4iE054_CdYgQadmUecSokvfsBS6C3jM.ssTR nXEQvqi2pR.8vIRzC_PS0G1FCzRh6eo7o7MTW3cjhUGzIOFeP.Z3uvivs8I0KxnwumvLVhX_COXc ltBqhhqmWAF2gIyKw9LSoPkDD4FxHg3ylRyb5O7FyJmYUsrTw2A5.JPRs2SbalYdIbvjvMrOkDqk KPzyTQF2KkFzrKM2PoFo5DyLczb51sfP8gC8Vpt74TGBpUqSkcHvd4oVUIE692ooReSRTs0UlAOS ShxSF2346R6oXBmNutsgX5gg3UDpLLI2cxGk62tePUL2tZNakbguoZjRYtSz2.x1_8OPJMce97jS ONIFgad0Bp.z_uefk06h26iFWzn.O791eOE2ZdXgHsDFvnUL0GUDeDbs6Bl.I6ikqhrKGQw2kwR8 NOGoR0KlQltnAtEK5DJAKhm_B9ec66xqbeb.qOUXJheg9qWE3tsXguTuO7xd2.7EpeVNQWbIZB4H MSgUgPH6O57zYXCCKbTx.SqXkalyPIRN5Og0RWdlVDp.XsnFVj.cPikCkRanOg8vv5fK8ti_H8Ew k5fxEd68peURTGZOTN5HLOhYhqYYkLx9qOzqwCue2mwdVDfK4IlvgieteI.qqPxPsObyxhgPb7rQ xJB.TzMBwLO.6HpizTVlC7.FOgDCOyI2c0s192HLo0dSNytO5yocnUFT7KCn_qhZSM6oRKT6CFsL 5lSfsFMC8YyhnSMZZR.XfB3jHrMAMAX07zuWhKCLw2eDTgRkvigxEaNpo0EDjAK4Ek8lYhBxVbAL lSx3aKIPx_A92mLb6kkr0RfYdx9EbK.2UJbbK_MDuSfRtuFBrLQL6eeJHL4UEmTi4OMNNfsKIkuE tStBZcGr4jcErtkk68Tkx_4NFD2IoVLi_QeNuhS7U74POxN3dx6c2tEyvK7rOr7jeYpq3y9.aDpI 0F3GA6X5PPh5WQAJ6z.jNJo7nvt2It3ODbQYpGI4tNa76QNLd51FhIoZyAI5Rt6o6cXeLKTJPDxH lVP22Akoa5KfE7pNhWl3epci6fRreQ3EbHoxIz9z5i6CvBfxcrkSVSskpJcziaaFGtII_UoQYk6b skqoHKuMudI2RNqRbaTeg1Lb99X6cSrXcD7yv4BbQJMClnloazybJxco7kXQhh7IVRvFiD_iEbjj Hrklz2yTXh4XQ6GfD.TkK7IKx9DAAntYDgY_UFxhCr.G58ql3VRgQiogGOyXO4O1NBi3juIinp._ acRkCbM91EmewI3FK8zLt0vq6tmNH4K0MZ35HrgqUFJmkVNjx6jyAHb5bYt6z4kWxbaffCUuO1Um jRqoO547LOugpG2UJS31nClcgNmwFmlCXGc6bP2_z_SjlxsdUQ88.8veCmFvVzI2IXJ2iJfBRShY H4NGTnwWb2XiQHhYtlA0zYzHgkv.7fQBRdvGo79VfB8hTyc2_DN_JFPbYzsjzyTB_BpOOO2Kbbjs T0.fMtyCdURV57Zc1Pyzf1jSTaNz1qUUuzmv150ggE9HrB.u5K3u2qCLznVwb4JsYt7WES2.nu6U Cig0e391wQLFc7ypPyAGfWcDfTW7HjB.VZ4asxgg.brWaPyq9BR0ulx5mCAfn5V9dmwQXKip6.4_ 5Xrf_nk01zWqHA_KaUF_kkwt3eA_Tj1wUgNlBIwe8tsIHS_6XA_keV.U0VFi7ZamuqXgObzQszCi d.erGYLFvrlGBww0xFpFrx_Pi0iATqtcWRo10yYjTJ_X1M5AyObMnUCb.npbu6yRbLnL4pdfF2XN RdadJDr0ByXCUmJ1xI6yGV_s9dR4G0qZ10Snj_S7UD9UK9Wj2oMGFHWMQlAw.1FfOpu4QdnmRsuK .VPtm6f3V43wYsQmWDVK_SEyMhPF2jeFm0XiRT54lb8WgAe8PytaxJAGIxz48Ai0HDvPTn5IFHth rx8pAEkzVUOgxHzpbKpoqKzj0FyLUIiDKm_5YKfioKA5ZncZxGJeK7PSLYNNkgV79XG7BNq07ZuX U1xsR7hgl0X2.fCxSGsEOGJAETpAF9myl_gCxl6aUqZK_c6XLRjEje.9UHfsQJql0uZ_raXGnsWZ 4NROaWpJDgHuWz5jyr.1QysOCXD7PfO9Q2X.lxTCzUdw0sxlUKmWFdIMlNuYV5XvUtbzJQSPraaz 10S3n2L.4nQe.BcpaeOczmA.x.W0mWobcEqPBr8i5ooFGz6fs0bftxNrFqrkEcnJd9vBlZtWhNs9 .LxX2LfwU9SPPHxubKYZjlWvsBPS5fWSmWtEdHjNTviI7Gicp2c_X7_vr643Bgxnw8vgttxx8L9Z AyHsT9Pg- X-Sonic-MF: X-Sonic-ID: 10f2a22d-01c1-4e77-8ede-8ce03fa4ee73 Received: from sonic.gate.mail.ne1.yahoo.com by sonic307.consmr.mail.gq1.yahoo.com with HTTP; Sat, 17 Feb 2024 19:04:19 +0000 Received: by hermes--production-gq1-5c57879fdf-vxz7c (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID a87c17a9e9b7da56116b22839a43c27b; Sat, 17 Feb 2024 19:04:17 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.400.31\)) Subject: Re: newfs TRIM flag device support From: Mark Millard In-Reply-To: Date: Sat, 17 Feb 2024 11:04:06 -0800 Cc: Ronald Klop , freebsd-arm Content-Transfer-Encoding: quoted-printable Message-Id: <9F657059-E39A-4E2A-8AF5-CA491BBC7ABB@yahoo.com> References: <8D8AA3F5-74A3-4AFE-A560-F52C7C55CF12@yahoo.com> <7okx4eDPse2uF1n3aX4W8RIOdbHXziUhqgvHxqqxpYN-7NsJfEGrfHbDxLgNS3PdOwZt7j9aOXp-0MHDjQAryL4t1Gd2QHUG1sFNmd5dY0k=@proton.me> <454941689.1180.1708079489598@localhost> <59693EE7-0DD6-4B41-BA64-0F3D69D3ECC9@yahoo.com> <1B0C2E27-6776-40A1-A8BF-29094FF8FC03@yahoo.com> To: Mike Karels , Ordinary Bit X-Mailer: Apple Mail (2.3774.400.31) X-Spamd-Bar: ---- X-Rspamd-Queue-Id: 4TcdWp1y96z4CTx X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US] On Feb 17, 2024, at 08:44, Mike Karels wrote: > On 17 Feb 2024, at 10:21, Ordinary Bit wrote: >=20 >> On Saturday, 17 February 2024 at 23:20, Mark Millard = wrote: >>=20 >>> On Feb 17, 2024, at 02:39, Ordinary Bit ordinarybit@proton.me wrote: >>> . . .=20 >>>>=20 >>>> root@sd-ultra:~ # gpart show -p >>>> =3D> 63 124735425 mmcsd0 MBR (59G) >>>> 63 1985 - free - (993K) >>>> 2048 102400 mmcsd0s1 fat32lba [active] (50M) >>>> 104448 10381312 mmcsd0s2 freebsd (5.0G) >>>> 10485760 16777216 mmcsd0s3 freebsd (8.0G) >>>=20 >>>=20 >>> An oddity just above is that mmcsd0s3 is not set to the >>> type freebsd-ufs . But the oddity does not mess up the >>> testing you were doing. >>>=20 >>=20 >> Let me test this one with freebsd-ufs type because you're right it's = only freebsd. >=20 > freebsd-ufs is a GPT type; this is MBR. The ufs filesystem is = presumably mmcsd0s3a. > If the filesystem gets mounted, the partitioning is OK. >=20 > Mike >=20 >>>> 27262976 97472512 - free - (46G) >>>>=20 >>>> =3D> 0 10381312 mmcsd0s2 BSD (5.0G) >>>> 0 128 - free - (64K) >>>> 128 10381184 mmcsd0s2a freebsd-ufs (4.9G) >>>> . . . >=20 My splitting of the gpart output at the line I was interested in may have left some confusion. So I'll quote the original full output. I was not very explicit about the details of the oddities involved before but will be so below. QUOTE root@sd-ultra:~ # gpart show -p =3D> 63 124735425 mmcsd0 MBR (59G) 63 1985 - free - (993K) 2048 102400 mmcsd0s1 fat32lba [active] (50M) 104448 10381312 mmcsd0s2 freebsd (5.0G) 10485760 16777216 mmcsd0s3 freebsd (8.0G) 27262976 97472512 - free - (46G) =3D> 0 10381312 mmcsd0s2 BSD (5.0G) 0 128 - free - (64K) 128 10381184 mmcsd0s2a freebsd-ufs (4.9G) END QUOTE First note that mmcsd0s2a says freebsd-ufs (but is inside a BSD). That is what I consider normal for MBR partitions that contain UFS file systems. Next note that there was no "mmcsd0s3 BSD" shown. Nor was there a "mmcsd0s3a freebsd-ufs" shown. "mmcsd0s3 freebsd" shows no evidence of any MBR/BSD substructure. (But it was mountable as a UFS file system after the newfs that was done on mmcsd0s3 .) None of this messed up the testing that was being done. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Sat Feb 17 23:10:05 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TckzT0R3Vz5B1JX for ; Sat, 17 Feb 2024 23:10:13 +0000 (UTC) (envelope-from madis555@hot.ee) Received: from SMTPOUT09.DKA.mailcore.net (smtpout09.dka.mailcore.net [194.19.134.9]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtpout09.dka.mailcore.net", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TckzR5clpz5830 for ; Sat, 17 Feb 2024 23:10:11 +0000 (UTC) (envelope-from madis555@hot.ee) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=online.ee header.s=mailcore header.b=QK8P+c7q; spf=pass (mx1.freebsd.org: domain of madis555@hot.ee designates 194.19.134.9 as permitted sender) smtp.mailfrom=madis555@hot.ee; dmarc=pass (policy=reject) header.from=hot.ee Received: from SMTP.DKA.mailcore.net (unknown [10.1.0.53]) by SMTPOUT01.DKA.mailcore.net (Postfix) with ESMTP id 1E05AE00EE; Sun, 18 Feb 2024 00:10:08 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=online.ee; s=mailcore; t=1708211408; bh=mzXDxfwZoI3aoA6nxHA74nQuCBVe7A4kdDyzpuaK2kk=; h=Date:From:To:Subject:In-Reply-To:References:From; b=QK8P+c7qNZbiWO7bdGceXhwFy+xwkBEVq6SHNX1iU8eGW91qEAEk3gvMD89OI0PBs 9EC2gEnWfO6yGR/LlPQChXoXi3wHvTkO0rgTNLM8CtXGD+smpJkSDhrDsgrgLTjugm TgXspFvmBD35TcVfJpc4OKk2KQwSJUPOAG/4jLj0pnBAXXwDelkaxe7v3cwAiAtWla GMu7ecI9WaXTeEKw7z5JqUQcCRZB+hjoBE3VLIYUxsqyMisYR96wfL7L2PGjSZ2JVc 87PODp8oOhPRfsrH/+b+kYygk3ZfRG8oZdz/Cfz1eaviumARJq1vKy2uXosq0eHhAC La8uyao4lY+8w== Received: from [127.0.0.1] (65-43-196-88.dyn.estpak.ee [88.196.43.65]) by SMTP.DKA.mailcore.net (Postfix) with ESMTPSA id E563F40146; Sun, 18 Feb 2024 00:10:07 +0100 (CET) Date: Sun, 18 Feb 2024 01:10:05 +0200 From: Sulev-Madis Silber To: freebsd-arm@freebsd.org Subject: Re: newfs TRIM flag device support User-Agent: K-9 Mail for Android In-Reply-To: <9F657059-E39A-4E2A-8AF5-CA491BBC7ABB@yahoo.com> References: <7okx4eDPse2uF1n3aX4W8RIOdbHXziUhqgvHxqqxpYN-7NsJfEGrfHbDxLgNS3PdOwZt7j9aOXp-0MHDjQAryL4t1Gd2QHUG1sFNmd5dY0k=@proton.me> <454941689.1180.1708079489598@localhost> <59693EE7-0DD6-4B41-BA64-0F3D69D3ECC9@yahoo.com> <1B0C2E27-6776-40A1-A8BF-29094FF8FC03@yahoo.com> <9F657059-E39A-4E2A-8AF5-CA491BBC7ABB@yahoo.com> Message-ID: <984C8EF0-6A74-4A59-906D-1E27F2F16BEA@hot.ee> List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 4TckzR5clpz5830 X-Spamd-Bar: --- X-Spamd-Result: default: False [-4.00 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.996]; DMARC_POLICY_ALLOW(-0.50)[hot.ee,reject]; R_SPF_ALLOW(-0.20)[+ip4:194.19.134.0/25]; R_DKIM_ALLOW(-0.20)[online.ee:s=mailcore]; MIME_GOOD(-0.10)[text/plain]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:3308, ipnet:194.19.128.0/18, country:SE]; FREEMAIL_FROM(0.00)[hot.ee]; ARC_NA(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; MLMMJ_DEST(0.00)[freebsd-arm@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; FREEMAIL_ENVFROM(0.00)[hot.ee]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; TO_DN_NONE(0.00)[]; RCVD_TLS_LAST(0.00)[]; DKIM_TRACE(0.00)[online.ee:+] i wonder if there is *any* usb reader that *does* support trim or could be = used for any other low level sd operations, eg for sysutils/mmc-utils or i should just use sbc as "usb-ethernet-sd-reader-ip-bridge" for that? From nobody Sun Feb 18 11:09:28 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Td2xb2vldz53sQY for ; Sun, 18 Feb 2024 11:09:39 +0000 (UTC) (envelope-from ordinarybit@proton.me) Received: from mail-4319.protonmail.ch (mail-4319.protonmail.ch [185.70.43.19]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "protonmail.com", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Td2xZ44MRz4LPF for ; Sun, 18 Feb 2024 11:09:38 +0000 (UTC) (envelope-from ordinarybit@proton.me) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=proton.me; s=protonmail; t=1708254574; x=1708513774; bh=guT1IVVSwfgxNC3MacQTwbC8Md6j8D/Rxn0KqbGWYB8=; h=Date:To:From:Cc:Subject:Message-ID:In-Reply-To:References: Feedback-ID:From:To:Cc:Date:Subject:Reply-To:Feedback-ID: Message-ID:BIMI-Selector; b=KSoF7UU7ORDV+kWET6jH9UOho1JrECTsVz30C2/pr1ozCaonj3fk93uYCpvqPDzpg h9z92rLXAgpegSossodYcwQlNWe99xP8qAQDLGpPcwZepBkc4fDu9/cHvEDu3p8YRE iq7WMKVo8+CA9H5CDumoX6S4oxPB/N9OYrzW40IEFlivNu/yU7zVJtNXavH/U5uxRo CDmRvD0UPVhbgVBmrc918O+DZkJPez1FtdG8rs+qffzWDJBRC/sCR7f675AHzxI/7a y8mYVqHDF0ebR9Y5NisLmblhynOAemeKsMfAa8AhKgIRtBQTY9WmyAg5DsMMMRRWBI 1c9rn+24uXexg== Date: Sun, 18 Feb 2024 11:09:28 +0000 To: Mark Millard From: Ordinary Bit Cc: Mike Karels , Ronald Klop , freebsd-arm Subject: Re: newfs TRIM flag device support Message-ID: In-Reply-To: <9F657059-E39A-4E2A-8AF5-CA491BBC7ABB@yahoo.com> References: <59693EE7-0DD6-4B41-BA64-0F3D69D3ECC9@yahoo.com> <1B0C2E27-6776-40A1-A8BF-29094FF8FC03@yahoo.com> <9F657059-E39A-4E2A-8AF5-CA491BBC7ABB@yahoo.com> Feedback-ID: 100108111:user:proton List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: ---- X-Rspamd-Queue-Id: 4Td2xZ44MRz4LPF X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:62371, ipnet:185.70.43.0/24, country:CH] On Sunday, 18 February 2024 at 03:04, Mark Millard wrot= e: > On Feb 17, 2024, at 08:44, Mike Karels mike@karels.net wrote: >=20 > > On 17 Feb 2024, at 10:21, Ordinary Bit wrote: > >=20 > > > On Saturday, 17 February 2024 at 23:20, Mark Millard marklmi@yahoo.co= m wrote: > > >=20 > > > > On Feb 17, 2024, at 02:39, Ordinary Bit ordinarybit@proton.me wrote= : > > > > . . . > > > >=20 > > > > > root@sd-ultra:~ # gpart show -p > > > > > =3D> 63 124735425 mmcsd0 MBR (59G) > > > > > 63 1985 - free - (993K) > > > > > 2048 102400 mmcsd0s1 fat32lba [active] (50M) > > > > > 104448 10381312 mmcsd0s2 freebsd (5.0G) > > > > > 10485760 16777216 mmcsd0s3 freebsd (8.0G) > > > >=20 > > > > An oddity just above is that mmcsd0s3 is not set to the > > > > type freebsd-ufs . But the oddity does not mess up the > > > > testing you were doing. > > >=20 > > > Let me test this one with freebsd-ufs type because you're right it's = only freebsd. > >=20 > > freebsd-ufs is a GPT type; this is MBR. The ufs filesystem is presumabl= y mmcsd0s3a. > > If the filesystem gets mounted, the partitioning is OK. > >=20 > > Mike > >=20 > > > > > 27262976 97472512 - free - (46G) > > > > >=20 > > > > > =3D> 0 10381312 mmcsd0s2 BSD (5.0G) > > > > > 0 128 - free - (64K) > > > > > 128 10381184 mmcsd0s2a freebsd-ufs (4.9G) > > > > > . . . >=20 >=20 > My splitting of the gpart output at the line I was interested in > may have left some confusion. So I'll quote the original full > output. I was not very explicit about the details of the oddities > involved before but will be so below. >=20 > QUOTE > root@sd-ultra:~ # gpart show -p > =3D> 63 124735425 mmcsd0 MBR (59G) >=20 > 63 1985 - free - (993K) > 2048 102400 mmcsd0s1 fat32lba [active] (50M) > 104448 10381312 mmcsd0s2 freebsd (5.0G) > 10485760 16777216 mmcsd0s3 freebsd (8.0G) > 27262976 97472512 - free - (46G) >=20 > =3D> 0 10381312 mmcsd0s2 BSD (5.0G) >=20 > 0 128 - free - (64K) > 128 10381184 mmcsd0s2a freebsd-ufs (4.9G) > END QUOTE >=20 > First note that mmcsd0s2a says freebsd-ufs (but is > inside a BSD). That is what I consider normal for MBR > partitions that contain UFS file systems. >=20 > Next note that there was no "mmcsd0s3 BSD" shown. > Nor was there a "mmcsd0s3a freebsd-ufs" shown. > "mmcsd0s3 freebsd" shows no evidence of any MBR/BSD > substructure. (But it was mountable as a UFS file > system after the newfs that was done on mmcsd0s3 .) >=20 > None of this messed up the testing that was being > done. >=20 Basically what I did after creating the mmcsd0s3 partition, I format it rig= ht away with newfs. So I re-created the setup to clear things out. This tim= e having mmcsd0s3 partition and mmcsd0s3a in freebsd-ufs type. You can see = below sequences of procedures. I deleted the existing mmcsd0s3 partition fi= rst and then create a new one in the same 8GB size. root@sd-ultra:~ # /sbin/gpart delete -i 3 mmcsd0 mmcsd0s3 deleted root@sd-ultra:~ # /sbin/gpart show -p =3D> 63 124735425 mmcsd0 MBR (59G) 63 1985 - free - (993K) 2048 102400 mmcsd0s1 fat32lba [active] (50M) 104448 10381312 mmcsd0s2 freebsd (5.0G) 10485760 114249728 - free - (54G) =3D> 0 10381312 mmcsd0s2 BSD (5.0G) 0 128 - free - (64K) 128 10381184 mmcsd0s2a freebsd-ufs (4.9G) root@sd-ultra:~ # /sbin/gpart add -t freebsd -s 8G mmcsd0 mmcsd0s3 added root@sd-ultra:~ # /sbin/gpart show -p =3D> 63 124735425 mmcsd0 MBR (59G) 63 1985 - free - (993K) 2048 102400 mmcsd0s1 fat32lba [active] (50M) 104448 10381312 mmcsd0s2 freebsd (5.0G) 10485760 16777216 mmcsd0s3 freebsd (8.0G) 27262976 97472512 - free - (46G) =3D> 0 10381312 mmcsd0s2 BSD (5.0G) 0 128 - free - (64K) 128 10381184 mmcsd0s2a freebsd-ufs (4.9G) =3D> 0 16777216 mmcsd0s3 BSD (8.0G) 0 16777216 mmcsd0s3a freebsd-ufs (8.0G) =3D> 0 16777216 ufsid/65c95062ec444331 BSD (8.0G) 0 16777216 ufsid/65c95062ec444331a freebsd-ufs (8.0G) root@sd-ultra:~ # root@sd-ultra:~ # /sbin/newfs -U -t /dev/mmcsd0s3a /dev/mmcsd0s3a: 8192.0MB (16777216 sectors) block size 32768, fragment size= 4096 using 14 cylinder groups of 625.22MB, 20007 blks, 80128 inodes. with soft updates super-block backups (for fsck_ffs -b #) at: 192, 1280640, 2561088, 3841536, 5121984, 6402432, 7682880, 8963328, 102437= 76, 11524224, 12804672, 14085120, 15365568, 16646016 root@sd-ultra:~ # root@sd-ultra:~ # /sbin/gpart show -p =3D> 63 124735425 mmcsd0 MBR (59G) 63 1985 - free - (993K) 2048 102400 mmcsd0s1 fat32lba [active] (50M) 104448 10381312 mmcsd0s2 freebsd (5.0G) 10485760 16777216 mmcsd0s3 freebsd (8.0G) 27262976 97472512 - free - (46G) =3D> 0 10381312 mmcsd0s2 BSD (5.0G) 0 128 - free - (64K) 128 10381184 mmcsd0s2a freebsd-ufs (4.9G) =3D> 0 16777216 mmcsd0s3 BSD (8.0G) 0 16777216 mmcsd0s3a freebsd-ufs (8.0G) =3D> 0 16777216 ufsid/65c93f4a5cb247cf BSD (8.0G) 0 16777216 ufsid/65c93f4a5cb247cfa freebsd-ufs (8.0G) root@sd-ultra:~ # root@sd-ultra:~ # mount /dev/mmcsd0s3a /mnt root@sd-ultra:~ # root@sd-ultra:~ # /sbin/gpart show -p =3D> 63 124735425 mmcsd0 MBR (59G) 63 1985 - free - (993K) 2048 102400 mmcsd0s1 fat32lba [active] (50M) 104448 10381312 mmcsd0s2 freebsd (5.0G) 10485760 16777216 mmcsd0s3 freebsd (8.0G) 27262976 97472512 - free - (46G) =3D> 0 10381312 mmcsd0s2 BSD (5.0G) 0 128 - free - (64K) 128 10381184 mmcsd0s2a freebsd-ufs (4.9G) =3D> 0 16777216 mmcsd0s3 BSD (8.0G) 0 16777216 mmcsd0s3a freebsd-ufs (8.0G) root@sd-ultra:~ # cp /home/freebsd/file01 /mnt/file01 root@sd-ultra:~ # cd /mnt root@sd-ultra:/mnt # root@sd-ultra:/mnt # ls -lah total 1421036 drwxr-xr-x 3 root wheel 512B Feb 11 21:44 . drwxr-xr-x 21 root wheel 512B Feb 11 20:04 .. drwxrwxr-x 2 root operator 512B Feb 11 21:42 .snap -rw-r--r-- 1 root wheel 1.4G Feb 11 21:47 file01 root@sd-ultra:/mnt # root@sd-ultra:/mnt # cp file01 file02 root@sd-ultra:/mnt # ls -lah total 2842060 drwxr-xr-x 3 root wheel 512B Feb 11 22:01 . drwxr-xr-x 21 root wheel 512B Feb 11 20:04 .. drwxrwxr-x 2 root operator 512B Feb 11 21:42 .snap -rw-r--r-- 1 root wheel 1.4G Feb 11 21:47 file01 -rw-r--r-- 1 root wheel 1.4G Feb 11 22:03 file02 root@sd-ultra:/mnt # root@sd-ultra:/mnt # rm * root@sd-ultra:/mnt # root@sd-ultra:/mnt # gstat -d dT: 1.003s w: 1.000s L(q) ops/s r/s kBps ms/r w/s kBps ms/w d/s kBps ms/d= %busy Name 15 327 15 479 31.0 1 32 7.6 311 1242254 30.= 9 53.1| mmcsd0 0 0 0 0 0.0 0 0 0.0 0 0 0.0= 0.0| mmcsd0s1 0 0 0 0 0.0 0 0 0.0 0 0 0.0= 0.0| mmcsd0s2 5 33 15 479 31.0 1 32 7.6 17 1176886 53.= 3 50.9| mmcsd0s3 0 0 0 0 0.0 0 0 0.0 0 0 0.0= 0.0| msdosfs/EFI 0 0 0 0 0.0 0 0 0.0 0 0 0.0= 0.0| mmcsd0s2a 5 33 15 479 31.0 1 32 7.6 17 1176886 53.= 3 50.9| mmcsd0s3a 0 0 0 0 0.0 0 0 0.0 0 0 0.0= 0.0| ufs/rootfs The TRIM activity were observed the same as previous but having the mmcsd0s= 3a along with mmcsd0s3 and mmcsd0. BR, orbit From nobody Sun Feb 18 11:46:38 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Td3mV0VD9z53yXc for ; Sun, 18 Feb 2024 11:46:50 +0000 (UTC) (envelope-from ordinarybit@proton.me) Received: from mail-40132.protonmail.ch (mail-40132.protonmail.ch [185.70.40.132]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "protonmail.com", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Td3mT52DLz4V9f for ; Sun, 18 Feb 2024 11:46:49 +0000 (UTC) (envelope-from ordinarybit@proton.me) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=proton.me; s=protonmail; t=1708256807; x=1708516007; bh=K5I5zYU9ZsEhzuXLMBuKSq2bXj0t41vPi2n9/dRShjQ=; h=Date:To:From:Cc:Subject:Message-ID:In-Reply-To:References: Feedback-ID:From:To:Cc:Date:Subject:Reply-To:Feedback-ID: Message-ID:BIMI-Selector; b=GBTuDv7yPhLUr8iSKvA7pYZRSToAOaD1LdRg/Dr9f+rTn9mE5PmQWC7BwzU4SaNdH VJC5oDuQBoSia20nQ4e6eymg3j4gkswPRMWMa/R8kYLjxFtwzSPNHATTGsWVoPzgRO Bg8jNQoEBNSUKIWLibMEfK95a/KI3pw/7ZALNUdTL/UT6zvy7eM2F6k8pUi+cP1xNs 0vydTIu17jmc4mZZTDuomebZmFZVb9fEEe2qWZNfKyLBoqWU/GuORySklQO7hLvn9l EHiqArI5RR0g/hA2i8EPkgjKv3mYn6+sDRMZQoOLYAgg735OJMNRBC0jWRTBwcx4SY G/Kglk0vAULNg== Date: Sun, 18 Feb 2024 11:46:38 +0000 To: Ordinary Bit From: Ordinary Bit Cc: Mark Millard , Mike Karels , Ronald Klop , freebsd-arm Subject: Re: newfs TRIM flag device support Message-ID: In-Reply-To: References: <59693EE7-0DD6-4B41-BA64-0F3D69D3ECC9@yahoo.com> <1B0C2E27-6776-40A1-A8BF-29094FF8FC03@yahoo.com> <9F657059-E39A-4E2A-8AF5-CA491BBC7ABB@yahoo.com> Feedback-ID: 100108111:user:proton List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: ---- X-Rspamd-Queue-Id: 4Td3mT52DLz4V9f X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:62371, ipnet:185.70.40.0/24, country:CH] Sent with Proton Mail secure email. On Sunday, 18 February 2024 at 19:09, Ordinary Bit = wrote: > On Sunday, 18 February 2024 at 03:04, Mark Millard marklmi@yahoo.com wrot= e: >=20 > > On Feb 17, 2024, at 08:44, Mike Karels mike@karels.net wrote: > >=20 > > > On 17 Feb 2024, at 10:21, Ordinary Bit wrote: > > >=20 > > > > On Saturday, 17 February 2024 at 23:20, Mark Millard marklmi@yahoo.= com wrote: > > > >=20 > > > > > On Feb 17, 2024, at 02:39, Ordinary Bit ordinarybit@proton.me wro= te: > > > > > . . . > > > > >=20 > > > > > > root@sd-ultra:~ # gpart show -p > > > > > > =3D> 63 124735425 mmcsd0 MBR (59G) > > > > > > 63 1985 - free - (993K) > > > > > > 2048 102400 mmcsd0s1 fat32lba [active] (50M) > > > > > > 104448 10381312 mmcsd0s2 freebsd (5.0G) > > > > > > 10485760 16777216 mmcsd0s3 freebsd (8.0G) > > > > >=20 > > > > > An oddity just above is that mmcsd0s3 is not set to the > > > > > type freebsd-ufs . But the oddity does not mess up the > > > > > testing you were doing. > > > >=20 > > > > Let me test this one with freebsd-ufs type because you're right it'= s only freebsd. > > >=20 > > > freebsd-ufs is a GPT type; this is MBR. The ufs filesystem is presuma= bly mmcsd0s3a. > > > If the filesystem gets mounted, the partitioning is OK. > > >=20 > > > Mike > > >=20 > > > > > > 27262976 97472512 - free - (46G) > > > > > >=20 > > > > > > =3D> 0 10381312 mmcsd0s2 BSD (5.0G) > > > > > > 0 128 - free - (64K) > > > > > > 128 10381184 mmcsd0s2a freebsd-ufs (4.9G) > > > > > > . . . > >=20 > > My splitting of the gpart output at the line I was interested in > > may have left some confusion. So I'll quote the original full > > output. I was not very explicit about the details of the oddities > > involved before but will be so below. > >=20 > > QUOTE > > root@sd-ultra:~ # gpart show -p > > =3D> 63 124735425 mmcsd0 MBR (59G) > >=20 > > 63 1985 - free - (993K) > > 2048 102400 mmcsd0s1 fat32lba [active] (50M) > > 104448 10381312 mmcsd0s2 freebsd (5.0G) > > 10485760 16777216 mmcsd0s3 freebsd (8.0G) > > 27262976 97472512 - free - (46G) > >=20 > > =3D> 0 10381312 mmcsd0s2 BSD (5.0G) > >=20 > > 0 128 - free - (64K) > > 128 10381184 mmcsd0s2a freebsd-ufs (4.9G) > > END QUOTE > >=20 > > First note that mmcsd0s2a says freebsd-ufs (but is > > inside a BSD). That is what I consider normal for MBR > > partitions that contain UFS file systems. > >=20 > > Next note that there was no "mmcsd0s3 BSD" shown. > > Nor was there a "mmcsd0s3a freebsd-ufs" shown. > > "mmcsd0s3 freebsd" shows no evidence of any MBR/BSD > > substructure. (But it was mountable as a UFS file > > system after the newfs that was done on mmcsd0s3 .) > >=20 > > None of this messed up the testing that was being > > done. >=20 >=20 > Basically what I did after creating the mmcsd0s3 partition, I format it r= ight away with newfs. So I re-created the setup to clear things out. This t= ime having mmcsd0s3 partition and mmcsd0s3a in freebsd-ufs type. You can se= e below sequences of procedures. I deleted the existing mmcsd0s3 partition = first and then create a new one in the same 8GB size. >=20 > root@sd-ultra:~ # /sbin/gpart delete -i 3 mmcsd0 > mmcsd0s3 deleted > root@sd-ultra:~ # /sbin/gpart show -p > =3D> 63 124735425 mmcsd0 MBR (59G) >=20 > 63 1985 - free - (993K) > 2048 102400 mmcsd0s1 fat32lba [active] (50M) > 104448 10381312 mmcsd0s2 freebsd (5.0G) > 10485760 114249728 - free - (54G) >=20 > =3D> 0 10381312 mmcsd0s2 BSD (5.0G) >=20 > 0 128 - free - (64K) > 128 10381184 mmcsd0s2a freebsd-ufs (4.9G) >=20 > root@sd-ultra:~ # /sbin/gpart add -t freebsd -s 8G mmcsd0 > mmcsd0s3 added > root@sd-ultra:~ # /sbin/gpart show -p > =3D> 63 124735425 mmcsd0 MBR (59G) >=20 > 63 1985 - free - (993K) > 2048 102400 mmcsd0s1 fat32lba [active] (50M) > 104448 10381312 mmcsd0s2 freebsd (5.0G) > 10485760 16777216 mmcsd0s3 freebsd (8.0G) > 27262976 97472512 - free - (46G) >=20 > =3D> 0 10381312 mmcsd0s2 BSD (5.0G) >=20 > 0 128 - free - (64K) > 128 10381184 mmcsd0s2a freebsd-ufs (4.9G) >=20 > =3D> 0 16777216 mmcsd0s3 BSD (8.0G) >=20 > 0 16777216 mmcsd0s3a freebsd-ufs (8.0G) >=20 > =3D> 0 16777216 ufsid/65c95062ec444331 BSD (8.0G) >=20 > 0 16777216 ufsid/65c95062ec444331a freebsd-ufs (8.0G) > root@sd-ultra:~ # > root@sd-ultra:~ # /sbin/newfs -U -t /dev/mmcsd0s3a > /dev/mmcsd0s3a: 8192.0MB (16777216 sectors) block size 32768, fragment si= ze 4096 > using 14 cylinder groups of 625.22MB, 20007 blks, 80128 inodes. > with soft updates > super-block backups (for fsck_ffs -b #) at: > 192, 1280640, 2561088, 3841536, 5121984, 6402432, 7682880, 8963328, 10243= 776, 11524224, 12804672, 14085120, > 15365568, 16646016 > root@sd-ultra:~ # > root@sd-ultra:~ # /sbin/gpart show -p > =3D> 63 124735425 mmcsd0 MBR (59G) >=20 > 63 1985 - free - (993K) > 2048 102400 mmcsd0s1 fat32lba [active] (50M) > 104448 10381312 mmcsd0s2 freebsd (5.0G) > 10485760 16777216 mmcsd0s3 freebsd (8.0G) > 27262976 97472512 - free - (46G) >=20 > =3D> 0 10381312 mmcsd0s2 BSD (5.0G) >=20 > 0 128 - free - (64K) > 128 10381184 mmcsd0s2a freebsd-ufs (4.9G) >=20 > =3D> 0 16777216 mmcsd0s3 BSD (8.0G) >=20 > 0 16777216 mmcsd0s3a freebsd-ufs (8.0G) >=20 > =3D> 0 16777216 ufsid/65c93f4a5cb247cf BSD (8.0G) >=20 > 0 16777216 ufsid/65c93f4a5cb247cfa freebsd-ufs (8.0G) > root@sd-ultra:~ # > root@sd-ultra:~ # mount /dev/mmcsd0s3a /mnt > root@sd-ultra:~ # > root@sd-ultra:~ # /sbin/gpart show -p > =3D> 63 124735425 mmcsd0 MBR (59G) >=20 > 63 1985 - free - (993K) > 2048 102400 mmcsd0s1 fat32lba [active] (50M) > 104448 10381312 mmcsd0s2 freebsd (5.0G) > 10485760 16777216 mmcsd0s3 freebsd (8.0G) > 27262976 97472512 - free - (46G) >=20 > =3D> 0 10381312 mmcsd0s2 BSD (5.0G) >=20 > 0 128 - free - (64K) > 128 10381184 mmcsd0s2a freebsd-ufs (4.9G) >=20 > =3D> 0 16777216 mmcsd0s3 BSD (8.0G) >=20 > 0 16777216 mmcsd0s3a freebsd-ufs (8.0G) >=20 > root@sd-ultra:~ # cp /home/freebsd/file01 /mnt/file01 > root@sd-ultra:~ # cd /mnt > root@sd-ultra:/mnt # > root@sd-ultra:/mnt # ls -lah > total 1421036 > drwxr-xr-x 3 root wheel 512B Feb 11 21:44 . > drwxr-xr-x 21 root wheel 512B Feb 11 20:04 .. > drwxrwxr-x 2 root operator 512B Feb 11 21:42 .snap > -rw-r--r-- 1 root wheel 1.4G Feb 11 21:47 file01 > root@sd-ultra:/mnt # > root@sd-ultra:/mnt # cp file01 file02 > root@sd-ultra:/mnt # ls -lah > total 2842060 > drwxr-xr-x 3 root wheel 512B Feb 11 22:01 . > drwxr-xr-x 21 root wheel 512B Feb 11 20:04 .. > drwxrwxr-x 2 root operator 512B Feb 11 21:42 .snap > -rw-r--r-- 1 root wheel 1.4G Feb 11 21:47 file01 > -rw-r--r-- 1 root wheel 1.4G Feb 11 22:03 file02 > root@sd-ultra:/mnt # > root@sd-ultra:/mnt # rm * > root@sd-ultra:/mnt # > root@sd-ultra:/mnt # gstat -d > dT: 1.003s w: 1.000s > L(q) ops/s r/s kBps ms/r w/s kBps ms/w d/s kBps ms/d %busy Name > 15 327 15 479 31.0 1 32 7.6 311 1242254 30.9 53.1| mmcsd0 > 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| mmcsd0s1 > 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| mmcsd0s2 > 5 33 15 479 31.0 1 32 7.6 17 1176886 53.3 50.9| mmcsd0s3 > 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| msdosfs/EFI > 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| mmcsd0s2a > 5 33 15 479 31.0 1 32 7.6 17 1176886 53.3 50.9| mmcsd0s3a > 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| ufs/rootfs >=20 > The TRIM activity were observed the same as previous but having the mmcsd= 0s3a along with mmcsd0s3 and mmcsd0. >=20 > BR, > orbit Oh I see, with tunefs it's also possible to enable/disable TRIM flag. I jus= t tried out the rootfs partition (/dev/da0s2a) enabled and it did.=20 root@sd-ultra:~ # /sbin/tunefs -t enable /dev/da0s2a tunefs: issue TRIM to the disk set Thanks for this great tool! BR, orbit =20 > From nobody Sun Feb 18 21:00:01 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TdJ2n4FFlz5Bkbn for ; Sun, 18 Feb 2024 21:00:01 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TdJ2n2b2tz4sw8 for ; Sun, 18 Feb 2024 21:00:01 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1708290001; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=ceboJltszDUiStn/c4HLdsnTgQuzplljx9CHVsQLsVM=; b=jLIuFk6p627Eam5bZeGQBY4GH6rn4oaxX+YZNn/vCIFU5WQMjc7RwalTBSLTJZe1X5/MPX MTl1qaEqPQ2Bhu8x5bnQgN5BjAXHEed7nexFa3XJ+5Pq1J8euPeUINawZGF1xt0xQwWxjy rsf95STFktfDIlCPYdtJNzjWoETVKcB0H+xbckcZT1cORssmzBwTzujwedwjm5fwDw5wkT 35Nh9AEO67mNQ6eVrw7k5+blk3GzZ9SNJcFA4KVzUVcmx7yLp8B8CggJ4mii/7qsdz5U/S HguX9VRkkYrzwFxn7jGValbYS6ha+zK/+Btrrb2mCAUWnF9LdIiPYRBzPuCgpg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1708290001; a=rsa-sha256; cv=none; b=qOaJAZ+BP0sP3LPyX9RslDLxErU9uzvEPfTf3nOUqgaEaLFifDUfLps2fDeZYUmZrYU2X4 r5KWvDj4cvBlJGAfswYFqaLve59WZjBBvCnhdaW5oKfQPQU7pXhoPEXoyHhB9OFWpX5xdH kL6FI0v1YBd1KvxwnxMmVK51RAYdUIxxJO52p39k2xKpuQzs1gpdQgVMcogGZCqK5ucqy4 ZrG1wrNgIXGJ2xUN6xc1/sarqz33GF1098u+K2EfTaqU2imKhZYTWSEpUS3wuqoROKSUos 5UqgrvKKvOEVSZrrIiBF4tGCgu4ezEzwl8NWNzlcAc2mKQ0AzGld/blc4ARy6Q== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4TdJ2n1dc4z15tN for ; Sun, 18 Feb 2024 21:00:01 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 41IL01x1097017 for ; Sun, 18 Feb 2024 21:00:01 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Received: (from bugzilla@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 41IL01Lc097012 for freebsd-arm@FreeBSD.org; Sun, 18 Feb 2024 21:00:01 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <202402182100.41IL01Lc097012@kenobi.freebsd.org> X-Authentication-Warning: kenobi.freebsd.org: bugzilla set sender to bugzilla-noreply@FreeBSD.org using -f From: bugzilla-noreply@FreeBSD.org To: freebsd-arm@FreeBSD.org Subject: Problem reports for freebsd-arm@FreeBSD.org that need special attention Date: Sun, 18 Feb 2024 21:00:01 +0000 List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="17082900011.5051393.96481" Content-Transfer-Encoding: 7bit --17082900011.5051393.96481 Date: Sun, 18 Feb 2024 21:00:01 +0000 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ------------+-----------+--------------------------------------------------- Open | 257670 | mpr(4): SAS3008 PCI-Express Fusion-MPT SAS-3: Fat Open | 264574 | sdhci(4): Support ACPI attachment in BCM2835_sdhc 2 problems total for which you should take action. --17082900011.5051393.96481 Date: Sun, 18 Feb 2024 21:00:01 +0000 MIME-Version: 1.0 Content-Type: text/html; charset="UTF-8" 
The following is a listing of current problems submitted by FreeBSD users,
which need special attention. These represent problem reports covering
all versions including experimental development code and obsolete releases.

Status      |    Bug Id | Description
------------+-----------+---------------------------------------------------
Open        |    257670 | mpr(4): SAS3008 PCI-Express Fusion-MPT SAS-3: Fat
Open        |    264574 | sdhci(4): Support ACPI attachment in BCM2835_sdhc

2 problems total for which you should take action.
--17082900011.5051393.96481--