Date: Fri, 31 Dec 2021 09:40:12 -0700 From: Warner Losh <imp@bsdimp.com> To: Kyle Evans <kevans@freebsd.org> Cc: Konstantin Belousov <kostikbel@gmail.com>, Stefan Esser <se@freebsd.org>, Antoine Brodin <antoine@freebsd.org>, src-committers <src-committers@freebsd.org>, "<dev-commits-src-all@freebsd.org>" <dev-commits-src-all@freebsd.org>, dev-commits-src-main@freebsd.org, FreeBSD Ports Management Team <portmgr@freebsd.org> Subject: Re: git: e2650af157bc - main - Make CPU_SET macros compliant with other implementations Message-ID: <CANCZdfrnVESrXkf%2B%2BFqmrO55RQQMV-D8psPXL60WcEJL1HzaMQ@mail.gmail.com> In-Reply-To: <CACNAnaEWGMSM5beWZSfg_vsYwLL56PNzN-59Oq_7Bbvkzro10A@mail.gmail.com> References: <202112301154.1BUBsR1q017491@gitrepo.freebsd.org> <CAALwa8m3u3xrO3N0j8um57qGTVnMEQwx1gP2YxJbzE5%2BLhbsWA@mail.gmail.com> <d1553b68-23dd-128e-6ac0-6c3c1f66c7cd@freebsd.org> <CACNAnaEbjRj9yGWtBMByVOET8FFjJQxQEiFuPbrya2CMePf8BQ@mail.gmail.com> <Yc8tgfSVhKZelwxg@kib.kiev.ua> <CACNAnaEWGMSM5beWZSfg_vsYwLL56PNzN-59Oq_7Bbvkzro10A@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--000000000000cfd2d605d473d36d Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Fri, Dec 31, 2021 at 9:31 AM Kyle Evans <kevans@freebsd.org> wrote: > On Fri, Dec 31, 2021 at 10:19 AM Konstantin Belousov > <kostikbel@gmail.com> wrote: > > > > On Fri, Dec 31, 2021 at 09:37:16AM -0600, Kyle Evans wrote: > > > On Fri, Dec 31, 2021 at 4:22 AM Stefan Esser <se@freebsd.org> wrote: > > > > > > > > Am 31.12.21 um 09:01 schrieb Antoine Brodin: > > > > > On Thu, Dec 30, 2021 at 11:54 AM Stefan E=C3=9Fer <se@freebsd.org= > > wrote: > > > > >> > > > > >> The branch main has been updated by se: > > > > >> > > > > >> URL: > https://cgit.FreeBSD.org/src/commit/?id=3De2650af157bc7489deaf2c9054995f0= f88a6e5da > > > > >> > > > > >> commit e2650af157bc7489deaf2c9054995f0f88a6e5da > > > > >> Author: Stefan E=C3=9Fer <se@FreeBSD.org> > > > > >> AuthorDate: 2021-12-30 11:20:32 +0000 > > > > >> Commit: Stefan E=C3=9Fer <se@FreeBSD.org> > > > > >> CommitDate: 2021-12-30 11:20:32 +0000 > > > > >> > > > > [...] > > > > >> Ports that have added -D_WITH_CPU_SET_T to build on -CURRENT > do > > > > >> no longer require that option. > > > > >> > > > > >> The FreeBSD version has been bumped to 1400046 to reflect th= is > > > > >> incompatible change. > > > > >> > > > > >> Reviewed by: kib > > > > >> MFC after: 2 weeks > > > > >> Relnotes: yes > > > > >> Differential Revision: https://reviews.freebsd.org/D33451 > > > > > > > > > > Hi, > > > > > > > > > > This breaks a lot of ports, like lang/python38. > > > > > Could these kinds of changes on public headers be tested with an > > > > > exp-run, and reverted in the mean-time? > > > > > > > > I'm sorry for the breakage. The commit had the goal to lessen > > > > port build problems caused by the misled assumptions that the > > > > port was being built on a GLIBC based system. > > > > > > > > > > Given that we've now iterated on this a couple of times, this likely > > > should have all been backed out and exp-run'd *way* sooner. > > Exp-runs are great when there is a path forward from fixing the breakag= e. > > In the case where you have some random ports broken, and no ports > maintainers > > responses for explicit queries, it is basically a deadlock. > > > > I definitely cannot go over some random but large set of ports fixing > them, > > while maintainers are silent. > > > > We do not know in advance that maintainers are silent, because we do > not know in advance which ports are even affected. We can't outright > claim there's a problem without knowing the scope. Nevertheless, yes, > you're right- this is even a conversation we've had recently re: > toolchain breakage, and I don't recall what the outcome of that > conversation was. > > > In fact, with this set of changes, I initially provided some tools in > base > > that were intended to ease the ports life, but still required some > (minimal) > > involvement from the maintainers, like passing -DWITH_CPU_SET_T to C/C+= + > > compiler. I asked more than once if these tools are desirable helper o= r > > not, with no avail. > > > > And that was indeed helpful, thanks! > > > So my only route forward was to leave the state in the minimal damaging > > mode as I see it from bug reports, and wait for maintainers to do > > _something_. I am very grateful that Stefan took the torch and started > > massaging the CPU_XXX ugliness into more compatibility with glibc. This > > again happens in the same silence mode from maintainers, so if we want = a > > progress in this area, it have to go this way. > > > > None of this is really applicable to this specific commit, though. The > breakage identified this time was that a couple more definitions were > expected, which does lean towards iteration on the patch rather than > yet requiring the maintainer or ports committer aide. > > I suspect the answer for other scenarios is that we run the exp-run > and shoot out a solicitation for help to -ports@ to cast a broader > net. Ports committers already get stuck with the fallout from this > stuff when maintainers don't notice or step up to the plate or even > when the breakage is just bad enough. I imagine you'd have no problem > catching some folks willing to help be proactive rather than reactive. > > > > > > > > In the case of the Python language ports, one additional macro > > > > was required and has been added in commit cb65d4432aed11. > > > > > > > > Since the official package builders have not been upgraded to > > > > a -CURRENT with this change, they are not affected. But I'll > > > > watch the failed build logs on beefy18. > > Right. > > > > > > > > This is a mindset that we all take, but we really need to work toward= s > > > improving. Once we're watching fallout logs on the official builders, > > > we've already lost. This is the kind of thing that helps promote the > > > idea that -CURRENT isn't stable enough for production uses: we start > > > accepting that we can be a little more lenient on identifying > > > ports-breaking changes because it's -CURRENT and we lose a fraction o= f > > > the ports tree because we've only sniped off individual ports as they > > > come up. > > > > > > portmgr@ is able and willing to run exp-runs for changes like this, w= e > > > really need to take advantage of that to avoid this kind of follow-up= . > > There, you are blocking src changes by putting unreasonable requirement= s > > on src committers to fix ports breakage. I am willing to work together > > with ports maintainers, but I am not willing to handle things in silenc= e > > and neglect of other' (my) work. > > > > > > I have similar experience with ino64 FWIW, but I was too naive at that > time > > and indeed tried to fix all ports breakage, including digging into > rust/ghc > > builds. I learned since, I will not do that again. > > > > I don't want src committers to fix all of the breakage, I want them to > at least do the bare minimum to identify when a change is going to > break a lot of stuff and figure out the magnitude of that breakage. > Sometimes we have to work with ports people on it, sometimes we just > need to iterate on our own changes further. We can't punt on the > latter because we haven't perfected the former yet. > I have a bunch of endian.h changes. I've been iterating for months, though most of that is my fault. After the initial exp-run, I've been able to use poudriere to identify whether or not my fixes actually fix things, or if it's time to try to get things in upstream. I'm down to 4 ports I need to fix and upstream because they do stupid things that accidentally worked in my case. When I'm ready, I've found portmgr quite responsive to my requests and in general the ports maintainers I've contacted for this and other changes responsive enough for my needs. I have the changes in a branch for when I can get back to them, and a list of currently broken ports I can run poudriere on when I have time to pick it back up. Even though there's only a few that were broken by my last iteration, they are important enough that I'm holding off pushing those int= o the tree... I strongly agree with Kyle that we need to move away from the mindset of futility. If there's maintainers that are unresponsive, we have timeouts. If we need a lot of changes to ports, portmgr can grant blanket approval. There's no urgency to get these changes in, to be honest. Breaking python and who knows what else really isn't acceptable on the basis that people had a bad experience in the past. That's how things stay dysfunctional and how we never get any better. Especially for a change that's motivation is to make us more compatible and for there to be less work as a result... Warner --000000000000cfd2d605d473d36d Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div dir=3D"ltr"><br></div><br><div class=3D"gmail_quote">= <div dir=3D"ltr" class=3D"gmail_attr">On Fri, Dec 31, 2021 at 9:31 AM Kyle = Evans <<a href=3D"mailto:kevans@freebsd.org">kevans@freebsd.org</a>> = wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0= px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On Fri, D= ec 31, 2021 at 10:19 AM Konstantin Belousov<br> <<a href=3D"mailto:kostikbel@gmail.com" target=3D"_blank">kostikbel@gmai= l.com</a>> wrote:<br> ><br> > On Fri, Dec 31, 2021 at 09:37:16AM -0600, Kyle Evans wrote:<br> > > On Fri, Dec 31, 2021 at 4:22 AM Stefan Esser <<a href=3D"mailt= o:se@freebsd.org" target=3D"_blank">se@freebsd.org</a>> wrote:<br> > > ><br> > > > Am 31.12.21 um 09:01 schrieb Antoine Brodin:<br> > > > > On Thu, Dec 30, 2021 at 11:54 AM Stefan E=C3=9Fer <<= a href=3D"mailto:se@freebsd.org" target=3D"_blank">se@freebsd.org</a>> w= rote:<br> > > > >><br> > > > >> The branch main has been updated by se:<br> > > > >><br> > > > >> URL: <a href=3D"https://cgit.FreeBSD.org/src/commit= /?id=3De2650af157bc7489deaf2c9054995f0f88a6e5da" rel=3D"noreferrer" target= =3D"_blank">https://cgit.FreeBSD.org/src/commit/?id=3De2650af157bc7489deaf2= c9054995f0f88a6e5da</a><br> > > > >><br> > > > >> commit e2650af157bc7489deaf2c9054995f0f88a6e5da<br> > > > >> Author:=C2=A0 =C2=A0 =C2=A0Stefan E=C3=9Fer <se@= FreeBSD.org><br> > > > >> AuthorDate: 2021-12-30 11:20:32 +0000<br> > > > >> Commit:=C2=A0 =C2=A0 =C2=A0Stefan E=C3=9Fer <se@= FreeBSD.org><br> > > > >> CommitDate: 2021-12-30 11:20:32 +0000<br> > > > >><br> > > > [...]<br> > > > >>=C2=A0 =C2=A0 =C2=A0Ports that have added -D_WITH_CP= U_SET_T to build on -CURRENT do<br> > > > >>=C2=A0 =C2=A0 =C2=A0no longer require that option.<b= r> > > > >><br> > > > >>=C2=A0 =C2=A0 =C2=A0The FreeBSD version has been bum= ped to 1400046 to reflect this<br> > > > >>=C2=A0 =C2=A0 =C2=A0incompatible change.<br> > > > >><br> > > > >>=C2=A0 =C2=A0 =C2=A0Reviewed by:=C2=A0 =C2=A0 kib<br= > > > > >>=C2=A0 =C2=A0 =C2=A0MFC after:=C2=A0 =C2=A0 =C2=A0 2= weeks<br> > > > >>=C2=A0 =C2=A0 =C2=A0Relnotes:=C2=A0 =C2=A0 =C2=A0 = =C2=A0yes<br> > > > >>=C2=A0 =C2=A0 =C2=A0Differential Revision:=C2=A0 <a = href=3D"https://reviews.freebsd.org/D33451" rel=3D"noreferrer" target=3D"_b= lank">https://reviews.freebsd.org/D33451</a><br>; > > > ><br> > > > > Hi,<br> > > > ><br> > > > > This breaks a lot of ports,=C2=A0 like=C2=A0 lang/pytho= n38.<br> > > > > Could these kinds of changes on public headers be teste= d with an<br> > > > > exp-run,=C2=A0 and reverted in the mean-time?<br> > > ><br> > > > I'm sorry for the breakage. The commit had the goal to l= essen<br> > > > port build problems caused by the misled assumptions that th= e<br> > > > port was being built on a GLIBC based system.<br> > > ><br> > ><br> > > Given that we've now iterated on this a couple of times, this= likely<br> > > should have all been backed out and exp-run'd *way* sooner.<b= r> > Exp-runs are great when there is a path forward from fixing the breaka= ge.<br> > In the case where you have some random ports broken, and no ports main= tainers<br> > responses for explicit queries, it is basically a deadlock.<br> ><br> > I definitely cannot go over some random but large set of ports fixing = them,<br> > while maintainers are silent.<br> ><br> <br> We do not know in advance that maintainers are silent, because we do<br> not know in advance which ports are even affected. We can't outright<br= > claim there's a problem without knowing the scope. Nevertheless, yes,<b= r> you're right- this is even a conversation we've had recently re:<br= > toolchain breakage, and I don't recall what the outcome of that<br> conversation was.<br> <br> > In fact, with this set of changes, I initially provided some tools in = base<br> > that were intended to ease the ports life, but still required some (mi= nimal)<br> > involvement from the maintainers, like passing -DWITH_CPU_SET_T to C/C= ++<br> > compiler.=C2=A0 I asked more than once if these tools are desirable he= lper or<br> > not, with no avail.<br> ><br> <br> And that was indeed helpful, thanks!<br> <br> > So my only route forward was to leave the state in the minimal damagin= g<br> > mode as I see it from bug reports, and wait for maintainers to do<br> > _something_. I am very grateful that Stefan took the torch and started= <br> > massaging the CPU_XXX ugliness into more compatibility with glibc. Thi= s<br> > again happens in the same silence mode from maintainers, so if we want= a<br> > progress in this area, it have to go this way.<br> ><br> <br> None of this is really applicable to this specific commit, though. The<br> breakage identified this time was that a couple more definitions were<br> expected, which does lean towards iteration on the patch rather than<br> yet requiring the maintainer or ports committer aide.<br> <br> I suspect the answer for other scenarios is that we run the exp-run<br> and shoot out a solicitation for help to -ports@ to cast a broader<br> net. Ports committers already get stuck with the fallout from this<br> stuff when maintainers don't notice or step up to the plate or even<br> when the breakage is just bad enough. I imagine you'd have no problem<b= r> catching some folks willing to help be proactive rather than reactive.<br> <br> > ><br> > > > In the case of the Python language ports, one additional mac= ro<br> > > > was required and has been added in commit cb65d4432aed11.<br= > > > ><br> > > > Since the official package builders have not been upgraded t= o<br> > > > a -CURRENT with this change, they are not affected. But I= 9;ll<br> > > > watch the failed build logs on beefy18.<br> > Right.<br> ><br> > ><br> > > This is a mindset that we all take, but we really need to work to= wards<br> > > improving. Once we're watching fallout logs on the official b= uilders,<br> > > we've already lost. This is the kind of thing that helps prom= ote the<br> > > idea that -CURRENT isn't stable enough for production uses: w= e start<br> > > accepting that we can be a little more lenient on identifying<br> > > ports-breaking changes because it's -CURRENT and we lose a fr= action of<br> > > the ports tree because we've only sniped off individual ports= as they<br> > > come up.<br> > ><br> > > portmgr@ is able and willing to run exp-runs for changes like thi= s, we<br> > > really need to take advantage of that to avoid this kind of follo= w-up.<br> > There, you are blocking src changes by putting unreasonable requiremen= ts<br> > on src committers to fix ports breakage.=C2=A0 I am willing to work to= gether<br> > with ports maintainers, but I am not willing to handle things in silen= ce<br> > and neglect of other' (my) work.<br> ><br> ><br> > I have similar experience with ino64 FWIW, but I was too naive at that= time<br> > and indeed tried to fix all ports breakage, including digging into rus= t/ghc<br> > builds.=C2=A0 I learned since, I will not do that again.<br> ><br> <br> I don't want src committers to fix all of the breakage, I want them to<= br> at least do the bare minimum to identify when a change is going to<br> break a lot of stuff and figure out the magnitude of that breakage.<br> Sometimes we have to work with ports people on it, sometimes we just<br> need to iterate on our own changes further. We can't punt on the<br> latter because we haven't perfected the former yet.<br></blockquote><di= v><br></div><div>I have a bunch of endian.h changes. I've been iteratin= g for months, though</div><div>most of that is my fault. After the initial = exp-run, I've been able to use poudriere</div><div>to identify whether = or not my fixes actually fix things, or if it's time to try to</div><di= v>get things in upstream. I'm down to 4 ports I need to fix and upstrea= m because</div><div>they do stupid things that accidentally worked in my ca= se. When I'm ready,</div><div>I've found portmgr quite responsive t= o my requests and in general the ports</div><div>maintainers I've conta= cted for this and other changes responsive enough</div><div>for my needs. I= have the changes in a branch for when I can get back to them,</div><div>an= d a list of currently broken ports I can run poudriere on when I have time = to</div><div>pick it back up. Even though there's only a few that were = broken by my last</div><div>iteration, they are important enough that I'= ;m holding off pushing those into</div><div>the tree...</div><div><br></div= ><div>I strongly agree with Kyle that we need to move away from the mindset= of</div><div>futility. If there's maintainers that are unresponsive, w= e have timeouts. If we</div><div>need a lot of changes to ports, portmgr ca= n grant blanket approval. There's</div><div>no urgency to get these cha= nges in, to be honest. Breaking python and who</div><div>knows what else re= ally isn't acceptable on the basis that people had a bad</div><div>expe= rience in the past. That's how things stay dysfunctional and how we</di= v><div>never get any better. Especially for a change that's motivation = is to make us</div><div>more compatible and for there to be less work as a = result...</div><div><br></div><div>Warner<br></div><div><br></div></div></d= iv> --000000000000cfd2d605d473d36d--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CANCZdfrnVESrXkf%2B%2BFqmrO55RQQMV-D8psPXL60WcEJL1HzaMQ>