Date: Thu, 18 Jan 2024 14:49:36 -0700 From: Warner Losh <imp@bsdimp.com> To: John Baldwin <jhb@freebsd.org> Cc: Warner Losh <imp@freebsd.org>, src-committers <src-committers@freebsd.org>, dev-commits-src-all@freebsd.org, dev-commits-src-main@freebsd.org Subject: Re: git: 773c13c686e4 - main - kldxref: skip .pkgsave files Message-ID: <CANCZdfreOGPMfts6JON2bHm05kwEOCyBth1G_YFh6EwBt6t5ag@mail.gmail.com> In-Reply-To: <CANCZdfo-oZYeicbuZBYhVuj8DFFJDMw-A-CzX_dz7Da=SV_7SQ@mail.gmail.com> References: <202302251737.31PHb2R8072300@gitrepo.freebsd.org> <e30bb1e4-11c7-4829-a7b6-21bfd590e558@FreeBSD.org> <CANCZdfp_Vz-K24%2BkBy1DZhMKoAjJ%2B-Z1m_9wB6DZ24=QTopBiA@mail.gmail.com> <a0746fe6-62b6-4a0a-99ad-d8a0b2e01d09@FreeBSD.org> <CANCZdfozE%2BB721S800eSKQ8eaUgdCC4BcyYXOf4BLEfbgL-34A@mail.gmail.com> <d6562b38-4770-4ae6-903d-dc00d557ea26@FreeBSD.org> <CANCZdfpkEmPrTfU7bbw%2BauWuvvu=DuJf5LbepBzbZxM0_SGbGg@mail.gmail.com> <a2bc3fcd-1d17-4dca-aaeb-28e61cd41949@FreeBSD.org> <CANCZdfo-oZYeicbuZBYhVuj8DFFJDMw-A-CzX_dz7Da=SV_7SQ@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--000000000000a01678060f3f577c Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Thu, Jan 18, 2024 at 2:47=E2=80=AFPM Warner Losh <imp@bsdimp.com> wrote: > Doh! This is what I have hanging around > > diff --git a/usr.sbin/kldxref/kldxref.c b/usr.sbin/kldxref/kldxref.c > index 1694f069564b..25bfab7129e1 100644 > --- a/usr.sbin/kldxref/kldxref.c > +++ b/usr.sbin/kldxref/kldxref.c > @@ -842,10 +842,10 @@ main(int argc, char *argv[]) > continue; > /* > * Skip files that generate errors like .debug, .symbol > and .pkgsave > - * by generally skipping all files with 2 dots. > + * by generally skipping all files not ending with ".ko". > */ > - dot =3D strchr(p->fts_name, '.'); > - if (dot && strchr(dot + 1, '.') !=3D NULL) > + dot =3D strrchr(p->fts_name, '.'); > + if (dot =3D=3D NULL || strcmp(dot, ".ko") !=3D 0) > continue; > read_kld(p->fts_path, p->fts_name); > } > > See anything obviously wrong with it before I make it into a phab? > > Sorry for the delay... > https://reviews.freebsd.org/D43507 Warner > Warner > > On Thu, Jan 18, 2024 at 2:35=E2=80=AFPM John Baldwin <jhb@freebsd.org> wr= ote: > >> On 12/6/23 5:35 PM, Warner Losh wrote: >> > On Wed, Dec 6, 2023, 2:53 PM John Baldwin <jhb@freebsd.org> wrote: >> > >> >> On 12/6/23 1:41 PM, Warner Losh wrote: >> >>> Hey John, >> >>> >> >>> On Wed, Dec 6, 2023 at 2:13=E2=80=AFPM John Baldwin <jhb@freebsd.org= > wrote: >> >>> >> >>>> On 12/6/23 1:02 PM, Warner Losh wrote: >> >>>>> On Wed, Dec 6, 2023, 1:04 PM John Baldwin <jhb@freebsd.org> wrote: >> >>>>> >> >>>>>> On 2/25/23 9:37 AM, Warner Losh wrote: >> >>>>>>> The branch main has been updated by imp: >> >>>>>>> >> >>>>>>> URL: >> >>>>>> >> >>>> >> >> >> https://cgit.FreeBSD.org/src/commit/?id=3D773c13c686e4b6ae9dbbc150b342b8= 2c3f47d73a >> >>>>>>> >> >>>>>>> commit 773c13c686e4b6ae9dbbc150b342b82c3f47d73a >> >>>>>>> Author: Mina Gali=C4=87 <freebsd@igalic.co> >> >>>>>>> AuthorDate: 2023-02-25 17:31:58 +0000 >> >>>>>>> Commit: Warner Losh <imp@FreeBSD.org> >> >>>>>>> CommitDate: 2023-02-25 17:35:43 +0000 >> >>>>>>> >> >>>>>>> kldxref: skip .pkgsave files >> >>>>>>> >> >>>>>>> This should help people transitioning from traditional >> setups >> >> to >> >>>>>> pkgbase >> >>>>>>> experience a lot less friction. >> >>>>>>> >> >>>>>>> We do this by skipping all files containing two dots. >> >>>>>>> >> >>>>>>> Reviewed by: imp >> >>>>>>> Pull Request: >> https://github.com/freebsd/freebsd-src/pull/661 >> >>>>>>> Differential Revision: https://reviews.freebsd.org/D2795= 9 >> >>>>>> >> >>>>>> This restriction is too broad and omits all of the modern wifi >> >> firmware >> >>>>>> klds from linker.hints, e.g. >> >>>>>> >> >>>>>> /boot/kernel/iwlwifi-3160-17.ucode.ko >> >>>>>> /boot/kernel/iwlwifi-3168-29.ucode.ko >> >>>>>> /boot/kernel/iwlwifi-7260-17.ucode.ko >> >>>>>> /boot/kernel/iwlwifi-7265-17.ucode.ko >> >>>>>> /boot/kernel/iwlwifi-7265D-29.ucode.ko >> >>>>>> /boot/kernel/iwlwifi-8000C-36.ucode.ko >> >>>>>> /boot/kernel/iwlwifi-8265-36.ucode.ko >> >>>>>> /boot/kernel/iwlwifi-9000-pu-b0-jf-b0-46.ucode.ko >> >>>>>> /boot/kernel/iwlwifi-9260-th-b0-jf-b0-46.ucode.ko >> >>>>>> /boot/kernel/iwlwifi-Qu-b0-hr-b0-77.ucode.ko >> >>>>>> /boot/kernel/iwlwifi-Qu-b0-jf-b0-77.ucode.ko >> >>>>>> /boot/kernel/iwlwifi-Qu-c0-hr-b0-77.ucode.ko >> >>>>>> /boot/kernel/iwlwifi-Qu-c0-jf-b0-77.ucode.ko >> >>>>>> /boot/kernel/iwlwifi-QuZ-a0-hr-b0-77.ucode.ko >> >>>>>> /boot/kernel/iwlwifi-QuZ-a0-jf-b0-77.ucode.ko >> >>>>>> /boot/kernel/iwlwifi-cc-a0-77.ucode.ko >> >>>>>> /boot/kernel/iwlwifi-so-a0-gf-a0-83.ucode.ko >> >>>>>> /boot/kernel/iwlwifi-so-a0-gf-a0.pnvm.ko >> >>>>>> /boot/kernel/iwlwifi-so-a0-gf4-a0-83.ucode.ko >> >>>>>> /boot/kernel/iwlwifi-so-a0-gf4-a0.pnvm.ko >> >>>>>> /boot/kernel/iwlwifi-so-a0-hr-b0-81.ucode.ko >> >>>>>> /boot/kernel/iwlwifi-so-a0-jf-b0-77.ucode.ko >> >>>>>> /boot/kernel/iwlwifi-ty-a0-gf-a0-83.ucode.ko >> >>>>>> /boot/kernel/iwlwifi-ty-a0-gf-a0.pnvm.ko >> >>>>>> /boot/kernel/rtw8723d_fw.bin.ko >> >>>>>> /boot/kernel/rtw8821c_fw.bin.ko >> >>>>>> /boot/kernel/rtw8822b_fw.bin.ko >> >>>>>> /boot/kernel/rtw8822c_fw.bin.ko >> >>>>>> /boot/kernel/rtw8822c_wow_fw.bin.ko >> >>>>>> >> >>>>>> all match this pattern and are skipped. >> >>>>>> >> >>>>>> I'm busy rewriting a bunch of kldxref to be a cross tool using >> libelf, >> >>>>>> but I think here you want to probably revert this and just add >> pkgsave >> >>>>>> to the list of "known bad" suffixes. >> >>>>>> >> >>>>> >> >>>>> Sure. Any reason to not just require .ko? Or do we have to index t= he >> >>>> kernel >> >>>>> too? >> >>>> >> >>>> We do index the kernel as well, yes. However, we could probably ge= t >> by >> >>>> with "kernel" and ends in ".ko" as a valid set of files. This woul= d >> >> also >> >>>> avoid bogusly warning about linker.hints not being a valid ELF file >> on >> >>>> re-runs if you use -v. >> >>>> >> >>> >> >>> Yea, that sounds good. I'll code it up and add you to the review. >> >>> >> >>> But why does it matter for these? Firmware is usually loaded by >> filename >> >>> and need not be elf... or are these wrapped in elf sections... >> >>> >> >>> I haven't noticed it breaking my linuxkpi wifi driver that have >> >> autoloaded >> >>> firmware... >> >> >> >> Hmm, afaik firmwares are loaded by "module name" where a firmware .ko >> >> contains >> >> one or more of the firmware modules. We happen today to generally on= ly >> >> store one module in a single .ko (and with the same name), and in tha= t >> case >> >> kern_linker.c may fallback to just trying to load "foo".ko if it >> doesn't >> >> find >> >> an entry in linker.hints, but if that is why it is working that is >> >> certainly >> >> by happy accident. >> >> >> >> I only found this by comparing klxref output btw on a stale i386 VM >> between >> >> the native kldxref in the VM (before this change) and my cross-arch >> version >> >> of kldxref. >> >> >> > >> > Ok. That all makes sense. I'll update my working tree tomorrow with th= e >> > revert and the replacement. Since it "works" today, I'll push the reve= rt >> > and the fix at the same time unless the review takes too long. >> >> Ping, do you still have this fix in your pending tree? I noticed it is >> still there when doing MFC's of the libelf kldxref changes today. >> >> -- >> John Baldwin >> >> --000000000000a01678060f3f577c Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div dir=3D"ltr"><br></div><br><div class=3D"gmail_quote">= <div dir=3D"ltr" class=3D"gmail_attr">On Thu, Jan 18, 2024 at 2:47=E2=80=AF= PM Warner Losh <<a href=3D"mailto:imp@bsdimp.com">imp@bsdimp.com</a>>= wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px = 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir= =3D"ltr">Doh! This is what I have hanging around<div><br></div><div>diff --= git a/usr.sbin/kldxref/kldxref.c b/usr.sbin/kldxref/kldxref.c<br>index 1694= f069564b..25bfab7129e1 100644<br>--- a/usr.sbin/kldxref/kldxref.c<br>+++ b/= usr.sbin/kldxref/kldxref.c<br>@@ -842,10 +842,10 @@ main(int argc, char *ar= gv[])<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 continue;<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 /*<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0* Skip files that generate errors like .debug, .symbol and .pkgsave<= br>- =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0* by generally skipping all f= iles with 2 dots.<br>+ =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0* by generally ski= pping all files not ending with ".ko".<br>=C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0*/<br>- =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 dot =3D strchr(p->fts_name, '.');<br>- =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 if (dot && strchr(dot + 1, '.') !=3D N= ULL)<br>+ =C2=A0 =C2=A0 =C2=A0 =C2=A0 dot =3D strrchr(p->fts_name, '= .');<br>+ =C2=A0 =C2=A0 =C2=A0 =C2=A0 if (dot =3D=3D NULL || strcmp(dot= , ".ko") !=3D 0)<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 continue;<br>=C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 read_kld(p->fts_path, p->fts_name)= ;<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 }<br></div><div><br></div><div>See anythin= g obviously wrong with it before I make it into a phab?</div><div><br></div= ><div>Sorry for the delay...</div></div></blockquote><div><br></div><div><b= r></div><div><a href=3D"https://reviews.freebsd.org/D43507">https://reviews= .freebsd.org/D43507</a></div><div><br></div><div>Warner=C2=A0</div><blockqu= ote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px= solid rgb(204,204,204);padding-left:1ex"><div dir=3D"ltr"><div>Warner</div= ></div><br><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr"= >On Thu, Jan 18, 2024 at 2:35=E2=80=AFPM John Baldwin <<a href=3D"mailto= :jhb@freebsd.org" target=3D"_blank">jhb@freebsd.org</a>> wrote:<br></div= ><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border= -left:1px solid rgb(204,204,204);padding-left:1ex">On 12/6/23 5:35 PM, Warn= er Losh wrote:<br> > On Wed, Dec 6, 2023, 2:53 PM John Baldwin <<a href=3D"mailto:jhb@fr= eebsd.org" target=3D"_blank">jhb@freebsd.org</a>> wrote:<br> > <br> >> On 12/6/23 1:41 PM, Warner Losh wrote:<br> >>> Hey John,<br> >>><br> >>> On Wed, Dec 6, 2023 at 2:13=E2=80=AFPM John Baldwin <<a hre= f=3D"mailto:jhb@freebsd.org" target=3D"_blank">jhb@freebsd.org</a>> wrot= e:<br> >>><br> >>>> On 12/6/23 1:02 PM, Warner Losh wrote:<br> >>>>> On Wed, Dec 6, 2023, 1:04 PM John Baldwin <<a href= =3D"mailto:jhb@freebsd.org" target=3D"_blank">jhb@freebsd.org</a>> wrote= :<br> >>>>><br> >>>>>> On 2/25/23 9:37 AM, Warner Losh wrote:<br> >>>>>>> The branch main has been updated by imp:<br> >>>>>>><br> >>>>>>> URL:<br> >>>>>><br> >>>><br> >> <a href=3D"https://cgit.FreeBSD.org/src/commit/?id=3D773c13c686e4b= 6ae9dbbc150b342b82c3f47d73a" rel=3D"noreferrer" target=3D"_blank">https://c= git.FreeBSD.org/src/commit/?id=3D773c13c686e4b6ae9dbbc150b342b82c3f47d73a</= a><br> >>>>>>><br> >>>>>>> commit 773c13c686e4b6ae9dbbc150b342b82c3f47d73= a<br> >>>>>>> Author:=C2=A0 =C2=A0 =C2=A0Mina Gali=C4=87 <= ;<a href=3D"mailto:freebsd@igalic.co" target=3D"_blank">freebsd@igalic.co</= a>><br> >>>>>>> AuthorDate: 2023-02-25 17:31:58 +0000<br> >>>>>>> Commit:=C2=A0 =C2=A0 =C2=A0Warner Losh <imp= @FreeBSD.org><br> >>>>>>> CommitDate: 2023-02-25 17:35:43 +0000<br> >>>>>>><br> >>>>>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0kldxref: skip= .pkgsave files<br> >>>>>>><br> >>>>>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0This should h= elp people transitioning from traditional setups<br> >> to<br> >>>>>> pkgbase<br> >>>>>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0experience a = lot less friction.<br> >>>>>>><br> >>>>>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0We do this by= skipping all files containing two dots.<br> >>>>>>><br> >>>>>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Reviewed by: = imp<br> >>>>>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Pull Request:= <a href=3D"https://github.com/freebsd/freebsd-src/pull/661" rel=3D"norefer= rer" target=3D"_blank">https://github.com/freebsd/freebsd-src/pull/661</a><= br> >>>>>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Differential = Revision: <a href=3D"https://reviews.freebsd.org/D27959" rel=3D"noreferrer"= target=3D"_blank">https://reviews.freebsd.org/D27959</a><br>; >>>>>><br> >>>>>> This restriction is too broad and omits all of the= modern wifi<br> >> firmware<br> >>>>>> klds from linker.hints, e.g.<br> >>>>>><br> >>>>>> /boot/kernel/iwlwifi-3160-17.ucode.ko<br> >>>>>> /boot/kernel/iwlwifi-3168-29.ucode.ko<br> >>>>>> /boot/kernel/iwlwifi-7260-17.ucode.ko<br> >>>>>> /boot/kernel/iwlwifi-7265-17.ucode.ko<br> >>>>>> /boot/kernel/iwlwifi-7265D-29.ucode.ko<br> >>>>>> /boot/kernel/iwlwifi-8000C-36.ucode.ko<br> >>>>>> /boot/kernel/iwlwifi-8265-36.ucode.ko<br> >>>>>> /boot/kernel/iwlwifi-9000-pu-b0-jf-b0-46.ucode.ko<= br> >>>>>> /boot/kernel/iwlwifi-9260-th-b0-jf-b0-46.ucode.ko<= br> >>>>>> /boot/kernel/iwlwifi-Qu-b0-hr-b0-77.ucode.ko<br> >>>>>> /boot/kernel/iwlwifi-Qu-b0-jf-b0-77.ucode.ko<br> >>>>>> /boot/kernel/iwlwifi-Qu-c0-hr-b0-77.ucode.ko<br> >>>>>> /boot/kernel/iwlwifi-Qu-c0-jf-b0-77.ucode.ko<br> >>>>>> /boot/kernel/iwlwifi-QuZ-a0-hr-b0-77.ucode.ko<br> >>>>>> /boot/kernel/iwlwifi-QuZ-a0-jf-b0-77.ucode.ko<br> >>>>>> /boot/kernel/iwlwifi-cc-a0-77.ucode.ko<br> >>>>>> /boot/kernel/iwlwifi-so-a0-gf-a0-83.ucode.ko<br> >>>>>> /boot/kernel/iwlwifi-so-a0-gf-a0.pnvm.ko<br> >>>>>> /boot/kernel/iwlwifi-so-a0-gf4-a0-83.ucode.ko<br> >>>>>> /boot/kernel/iwlwifi-so-a0-gf4-a0.pnvm.ko<br> >>>>>> /boot/kernel/iwlwifi-so-a0-hr-b0-81.ucode.ko<br> >>>>>> /boot/kernel/iwlwifi-so-a0-jf-b0-77.ucode.ko<br> >>>>>> /boot/kernel/iwlwifi-ty-a0-gf-a0-83.ucode.ko<br> >>>>>> /boot/kernel/iwlwifi-ty-a0-gf-a0.pnvm.ko<br> >>>>>> /boot/kernel/rtw8723d_fw.bin.ko<br> >>>>>> /boot/kernel/rtw8821c_fw.bin.ko<br> >>>>>> /boot/kernel/rtw8822b_fw.bin.ko<br> >>>>>> /boot/kernel/rtw8822c_fw.bin.ko<br> >>>>>> /boot/kernel/rtw8822c_wow_fw.bin.ko<br> >>>>>><br> >>>>>> all match this pattern and are skipped.<br> >>>>>><br> >>>>>> I'm busy rewriting a bunch of kldxref to be a = cross tool using libelf,<br> >>>>>> but I think here you want to probably revert this = and just add pkgsave<br> >>>>>> to the list of "known bad" suffixes.<br> >>>>>><br> >>>>><br> >>>>> Sure. Any reason to not just require .ko? Or do we hav= e to index the<br> >>>> kernel<br> >>>>> too?<br> >>>><br> >>>> We do index the kernel as well, yes.=C2=A0 However, we cou= ld probably get by<br> >>>> with "kernel" and ends in ".ko" as a v= alid set of files.=C2=A0 This would<br> >> also<br> >>>> avoid bogusly warning about linker.hints not being a valid= ELF file on<br> >>>> re-runs if you use -v.<br> >>>><br> >>><br> >>> Yea, that sounds good. I'll code it up and add you to the = review.<br> >>><br> >>> But why does it matter for these? Firmware is usually loaded b= y filename<br> >>> and need not be elf... or are these wrapped in elf sections...= <br> >>><br> >>> I haven't noticed it breaking my linuxkpi wifi driver that= have<br> >> autoloaded<br> >>> firmware...<br> >><br> >> Hmm, afaik firmwares are loaded by "module name" where a= firmware .ko<br> >> contains<br> >> one or more of the firmware modules.=C2=A0 We happen today to gene= rally only<br> >> store one module in a single .ko (and with the same name), and in = that case<br> >> kern_linker.c may fallback to just trying to load "foo".= ko if it doesn't<br> >> find<br> >> an entry in linker.hints, but if that is why it is working that is= <br> >> certainly<br> >> by happy accident.<br> >><br> >> I only found this by comparing klxref output btw on a stale i386 V= M between<br> >> the native kldxref in the VM (before this change) and my cross-arc= h version<br> >> of kldxref.<br> >><br> > <br> > Ok. That all makes sense. I'll update my working tree tomorrow wit= h the<br> > revert and the replacement. Since it "works" today, I'll= push the revert<br> > and the fix at the same time unless the review takes too long.<br> <br> Ping, do you still have this fix in your pending tree?=C2=A0 I noticed it i= s<br> still there when doing MFC's of the libelf kldxref changes today.<br> <br> -- <br> John Baldwin<br> <br> </blockquote></div> </blockquote></div></div> --000000000000a01678060f3f577c--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CANCZdfreOGPMfts6JON2bHm05kwEOCyBth1G_YFh6EwBt6t5ag>