Date: Thu, 18 Apr 2019 09:26:47 -0700 From: Cy Schubert <Cy.Schubert@cschubert.com> To: Warner Losh <imp@bsdimp.com> Cc: "Rodney W. Grimes" <rgrimes@freebsd.org>, "Rodney W. Grimes" <freebsd@gndrsh.dnsmgr.net>, Kyle Evans <kevans@freebsd.org>, Cy Schubert <cy@freebsd.org>, src-committers <src-committers@freebsd.org>, svn-src-all <svn-src-all@freebsd.org>, svn-src-head <svn-src-head@freebsd.org> Subject: Re: svn commit: r346341 - head/tools/build Message-ID: <AA719490-5A6F-4218-90B8-FB7FA724C94E@cschubert.com> In-Reply-To: <CANCZdfrkON8J_6ZHt1UKa95G0=JLwZ8KoebspZzF0%2BeQ71BY4A@mail.gmail.com> References: <201904181422.x3IEMrux005930@gndrsh.dnsmgr.net> <3095E422-7865-4EA5-BF13-6A48CB542AEE@cschubert.com> <CANCZdfrkON8J_6ZHt1UKa95G0=JLwZ8KoebspZzF0%2BeQ71BY4A@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On April 18, 2019 8:11:49 AM PDT, Warner Losh <imp@bsdimp=2Ecom> wrote: >On Thu, Apr 18, 2019 at 8:44 AM Cy Schubert <Cy=2ESchubert@cschubert=2Eco= m> >wrote: > >> On April 18, 2019 7:22:53 AM PDT, "Rodney W=2E Grimes" < >> freebsd@gndrsh=2Ednsmgr=2Enet> wrote: >> >> On Thu, Apr 18, 2019 at 8:46 AM Rodney W=2E Grimes >> >> <freebsd@gndrsh=2Ednsmgr=2Enet> wrote: >> >> > >> >> > > In message <201904180107=2Ex3I17QDc002945@gndrsh=2Ednsmgr=2Enet>= , >> >"Rodney W=2E >> >> > > Grimes" >> >> > > writes: >> >> > > > > Author: cy >> >> > > > > Date: Thu Apr 18 01:02:00 2019 >> >> > > > > New Revision: 346341 >> >> > > > > URL: https://svnweb=2Efreebsd=2Eorg/changeset/base/346341 >> >> > > > > >> >> > > > > Log: >> >> > > > > As an interim measure until a more permanent solution is >> >implemented >> >> > > > > workaround the following error: >> >> > > > > >> >> > > > > /usr/src/contrib/elftoolchain/strings/strings=2Ec:198:55: >> >error: use of >> >> > > > > undeclared identifier >> >> > > > > 'FA_OPEN' fa =3D fileargs_init(argc, argv, O_RDONLY, 0, >> >&rights, FA_OPEN); >> >> > > > > >> >> > > > > Reported by: O=2E Hartmann <ohartmann@walstatt=2Eorg> >> >> > > > > Reported by: Michael Butler ><imb@protected-networks=2Enet> >> >> > > > > Reported by: gjb@ & cy@ (implicit) >> >> > > > > Reviewed by: emaste@ >> >> > > > > Noted by: rgrimes@ >> >> > > > > >> >> > > > > Modified: >> >> > > > > head/tools/build/Makefile >> >> > > > > >> >> > > > > Modified: head/tools/build/Makefile >> >> > > > > >> >> >>=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D >> >> > > > =3D=3D=3D >> >> > > > > --- head/tools/build/Makefile Thu Apr 18 00:38:54 2019 >> > (r34634 >> >> > > > 0) >> >> > > > > +++ head/tools/build/Makefile Thu Apr 18 01:02:00 2019 >> > (r34634 >> >> > > > 1) >> >> > > > > @@ -59,9 +59,7 @@ INCS+=3D capsicum_helpers=2Eh >> >> > > > > INCS+=3D libcasper=2Eh >> >> > > > > =2Eendif >> >> > > > > >> >> > > > > -=2Eif !exists(/usr/include/casper/cap_fileargs=2Eh) >> >> > > > > CASPERINC+=3D >> >${SRCTOP}/lib/libcasper/services/cap_fileargs/cap_filea >> >> > > > rgs=2Eh >> >> > > > > -=2Eendif >> >> > > > >> >> > > > As a further note, we should probably hunt for any thing >> >> > > > that is explicity looking at /usr/include/=2E=2E=2E in a Makef= ile, >> >> > > > as that is minimally missing a ${DESTDIR} argument=2E >> >> > > > >> >> > > > The above may of actually worked if it had been written: >> >> > > > =2Eif !exists(${DESTDIR}/usr/include/casper/cap_fileargs=2Eh) >> >> > > > someone may wish to test that=2E >> >> > > > >> >> > > > Also a pathname rooted at / without ${DESTDIR} is almost >> >certainly a mistake=2E >> >> > > >> >> > > This is a better solution=2E I tested this in a tree with a >> >duplicated >> >> > > environment: Problem solved=2E Before this is committed it >should >> >be >> >> > > tested on one of the universe machines=2E >> >> > >> >> > From what Ed just said this would also be wrong, >> >> > as well as CASPERINC+=3D above being wrong, if this >> >> > is being built for the host we should not be using >> >> > any headers from ${SRCTOP} at all=2E >> >> > >> >> > if capfileargs=2Eh does not exist on the host that functionality >> >> > must not be compiled into the buildtool as the host does not >> >> > have this feature and attempting to use it from SRCTOP is wrong=2E >> >> > >> >> >> >> Keep in mind that this is bootstrap; it's being built for the host >> >> system, but it will link against a version of libcasper that's >been >> >> built in an earlier stage with the proper featureset=2E >> > >> >Ok, flip flop again, if infact this is linked against a >> >library that implements the stuff from cap_fileargs=2Eh then >> >infact the ${DESTDIR} addition so that the build peaks into >> >the cross build tree is correct, or what ever the equivelent >> >to DESTDIR is for that ? BUILDDIR? The point is it should >> >be picking this header up from the object tree, NOT from >> >the running system=2E >> >> Yes, this was my conclusion when working on kerberos and ntp=2E This is >also >> true of libraries, else one would need to installworld and >buildworld >> again to get a properly built library/binary=2E >> >> IIRC ngie@ fixed a number of these across the tree a couple of years >ago=2E > > >OK=2E There's a number of different issues going on=2E As the original >author >of libegacy (which is what we're seeing fail), let me address the >design >generically and comment on different things that have come up in this >thread=2E > >Since this is going into the libegacy that we're using to build the >system, >the check for file is bogus, for reasons I'll discuss below=2E When we >add >new includes to the system, it is appropriate to do it this way=2E And >when >this file was added to the system, the check was correct=2E > >First off, DESTDIR is absolutely not correct since this is to build the >legacy library and legacy includes which augment the host's sources on >legacay system, hence the name=2E It's never the correct thing to use=2E > >The problem that we have here is not that the file is missing (which is >why >it was added the way it was a long time ago), but rather missing >functionality in a file that's been around for a long time=2E > >So, since it is that class of problem, the canonical way we've dealt >with >it in the past is to do something like create a file foo=2Eh that looks >something like > >#include_next <foo=2Eh> > >#ifndev SOMETHING_NEW >#define SOMETHING_NEW 0 /* Or other values that are fail-safe on the >old >system */ >#endif > >and that's the change that needs to be made here=2E Sometimes, more >extensive >things need to be done when the old library can't work at all with the >new >code=2E In those cases, the pattern is for foo=2Eh to include #define >problem_fn my_problem_fn and then write a my_problem_fn that wraps >problem_fn in a way that works on the old system=2E Kinda a poor man's >symbol >versioning, in a way (note: we can't use symbol version for this since >we're building new binaries)=2E > >The "stop gap" gets things compiling, and maybe OK for the moment, >unless >the SOMETHING_NEW variable that's used (in this case FA_OPEN) causes >the >old library to do the wrong thing=2E I've not done the deep dive to see >if >this is the case or not=2E > >So, does that make sense? > >Warner This solves one problem but what about the cases when a new krb5, ntp, or = amd is imported but fails to build because it is using the old headers in /= usr/include and linking against old libraries on the running system? These examples BTW have been fixed=2E My concern is there could be other e= xamples in contrib, yet to be discovered, that might also have the same iss= ues=2E=20 --=20 Pardon the typos and autocorrect, small keyboard in use=2E Cheers, Cy Schubert <Cy=2ESchubert@cschubert=2Ecom> FreeBSD UNIX: <cy@FreeBSD=2Eorg> Web: http://www=2EFreeBSD=2Eorg The need of the many outweighs the greed of the few=2E From owner-svn-src-head@freebsd.org Thu Apr 18 16:59:06 2019 Return-Path: <owner-svn-src-head@freebsd.org> Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F1A3815766A6; Thu, 18 Apr 2019 16:59:05 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 92DB26EDC2; Thu, 18 Apr 2019 16:59:05 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from John-Baldwins-MacBook-Pro-3.local (ralph.baldwin.cx [66.234.199.215]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: jhb) by smtp.freebsd.org (Postfix) with ESMTPSA id 692359074; Thu, 18 Apr 2019 16:59:04 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Subject: Re: svn commit: r346341 - head/tools/build To: Cy Schubert <Cy.Schubert@cschubert.com>, Warner Losh <imp@bsdimp.com> Cc: "Rodney W. Grimes" <rgrimes@freebsd.org>, "Rodney W. Grimes" <freebsd@gndrsh.dnsmgr.net>, Kyle Evans <kevans@freebsd.org>, Cy Schubert <cy@freebsd.org>, src-committers <src-committers@freebsd.org>, svn-src-all <svn-src-all@freebsd.org>, svn-src-head <svn-src-head@freebsd.org> References: <201904181422.x3IEMrux005930@gndrsh.dnsmgr.net> <3095E422-7865-4EA5-BF13-6A48CB542AEE@cschubert.com> <CANCZdfrkON8J_6ZHt1UKa95G0=JLwZ8KoebspZzF0+eQ71BY4A@mail.gmail.com> <AA719490-5A6F-4218-90B8-FB7FA724C94E@cschubert.com> From: John Baldwin <jhb@FreeBSD.org> Openpgp: preference=signencrypt Autocrypt: addr=jhb@FreeBSD.org; keydata= mQGiBETQ+XcRBADMFybiq69u+fJRy/0wzqTNS8jFfWaBTs5/OfcV7wWezVmf9sgwn8TW0Dk0 c9MBl0pz+H01dA2ZSGZ5fXlmFIsee1WEzqeJzpiwd/pejPgSzXB9ijbLHZ2/E0jhGBcVy5Yo /Tw5+U/+laeYKu2xb0XPvM0zMNls1ah5OnP9a6Ql6wCgupaoMySb7DXm2LHD1Z9jTsHcAQMD /1jzh2BoHriy/Q2s4KzzjVp/mQO5DSm2z14BvbQRcXU48oAosHA1u3Wrov6LfPY+0U1tG47X 1BGfnQH+rNAaH0livoSBQ0IPI/8WfIW7ub4qV6HYwWKVqkDkqwcpmGNDbz3gfaDht6nsie5Z pcuCcul4M9CW7Md6zzyvktjnbz61BADGDCopfZC4of0Z3Ka0u8Wik6UJOuqShBt1WcFS8ya1 oB4rc4tXfSHyMF63aPUBMxHR5DXeH+EO2edoSwViDMqWk1jTnYza51rbGY+pebLQOVOxAY7k do5Ordl3wklBPMVEPWoZ61SdbcjhHVwaC5zfiskcxj5wwXd2E9qYlBqRg7QeSm9obiBCYWxk d2luIDxqaGJARnJlZUJTRC5vcmc+iGAEExECACAFAkTQ+awCGwMGCwkIBwMCBBUCCAMEFgID AQIeAQIXgAAKCRBy3lIGd+N/BI6RAJ9S97fvbME+3hxzE3JUyUZ6vTewDACdE1stFuSfqMvM jomvZdYxIYyTUpC5Ag0ERND5ghAIAPwsO0B7BL+bz8sLlLoQktGxXwXQfS5cInvL17Dsgnr3 1AKa94j9EnXQyPEj7u0d+LmEe6CGEGDh1OcGFTMVrof2ZzkSy4+FkZwMKJpTiqeaShMh+Goj XlwIMDxyADYvBIg3eN5YdFKaPQpfgSqhT+7El7w+wSZZD8pPQuLAnie5iz9C8iKy4/cMSOrH YUK/tO+Nhw8Jjlw94Ik0T80iEhI2t+XBVjwdfjbq3HrJ0ehqdBwukyeJRYKmbn298KOFQVHO EVbHA4rF/37jzaMadK43FgJ0SAhPPF5l4l89z5oPu0b/+5e2inA3b8J3iGZxywjM+Csq1tqz hltEc7Q+E08AAwUIAL+15XH8bPbjNJdVyg2CMl10JNW2wWg2Q6qdljeaRqeR6zFus7EZTwtX sNzs5bP8y51PSUDJbeiy2RNCNKWFMndM22TZnk3GNG45nQd4OwYK0RZVrikalmJY5Q6m7Z16 4yrZgIXFdKj2t8F+x613/SJW1lIr9/bDp4U9tw0V1g3l2dFtD3p3ZrQ3hpoDtoK70ioIAjjH aIXIAcm3FGZFXy503DOA0KaTWwvOVdYCFLm3zWuSOmrX/GsEc7ovasOWwjPn878qVjbUKWwx Q4QkF4OhUV9zPtf9tDSAZ3x7QSwoKbCoRCZ/xbyTUPyQ1VvNy/mYrBcYlzHodsaqUDjHuW+I SQQYEQIACQUCRND5ggIbDAAKCRBy3lIGd+N/BCO8AJ9j1dWVQWxw/YdTbEyrRKOY8YZNwwCf afMAg8QvmOWnHx3wl8WslCaXaE8= Message-ID: <5ba297cb-e97b-2ac7-58ee-971545581f08@FreeBSD.org> Date: Thu, 18 Apr 2019 09:59:02 -0700 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:60.0) Gecko/20100101 Thunderbird/60.6.1 MIME-Version: 1.0 In-Reply-To: <AA719490-5A6F-4218-90B8-FB7FA724C94E@cschubert.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 92DB26EDC2 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.98 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[]; NEURAL_HAM_SHORT(-0.98)[-0.980,0] X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: SVN commit messages for the src tree for head/-current <svn-src-head.freebsd.org> List-Unsubscribe: <https://lists.freebsd.org/mailman/options/svn-src-head>, <mailto:svn-src-head-request@freebsd.org?subject=unsubscribe> List-Archive: <http://lists.freebsd.org/pipermail/svn-src-head/>; List-Post: <mailto:svn-src-head@freebsd.org> List-Help: <mailto:svn-src-head-request@freebsd.org?subject=help> List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/svn-src-head>, <mailto:svn-src-head-request@freebsd.org?subject=subscribe> X-List-Received-Date: Thu, 18 Apr 2019 16:59:06 -0000 On 4/18/19 9:26 AM, Cy Schubert wrote: > On April 18, 2019 8:11:49 AM PDT, Warner Losh <imp@bsdimp.com> wrote: >> On Thu, Apr 18, 2019 at 8:44 AM Cy Schubert <Cy.Schubert@cschubert.com> >> wrote: >> >>> On April 18, 2019 7:22:53 AM PDT, "Rodney W. Grimes" < >>> freebsd@gndrsh.dnsmgr.net> wrote: >>>>> On Thu, Apr 18, 2019 at 8:46 AM Rodney W. Grimes >>>>> <freebsd@gndrsh.dnsmgr.net> wrote: >>>>>> >>>>>>> In message <201904180107.x3I17QDc002945@gndrsh.dnsmgr.net>, >>>> "Rodney W. >>>>>>> Grimes" >>>>>>> writes: >>>>>>>>> Author: cy >>>>>>>>> Date: Thu Apr 18 01:02:00 2019 >>>>>>>>> New Revision: 346341 >>>>>>>>> URL: https://svnweb.freebsd.org/changeset/base/346341 >>>>>>>>> >>>>>>>>> Log: >>>>>>>>> As an interim measure until a more permanent solution is >>>> implemented >>>>>>>>> workaround the following error: >>>>>>>>> >>>>>>>>> /usr/src/contrib/elftoolchain/strings/strings.c:198:55: >>>> error: use of >>>>>>>>> undeclared identifier >>>>>>>>> 'FA_OPEN' fa = fileargs_init(argc, argv, O_RDONLY, 0, >>>> &rights, FA_OPEN); >>>>>>>>> >>>>>>>>> Reported by: O. Hartmann <ohartmann@walstatt.org> >>>>>>>>> Reported by: Michael Butler >> <imb@protected-networks.net> >>>>>>>>> Reported by: gjb@ & cy@ (implicit) >>>>>>>>> Reviewed by: emaste@ >>>>>>>>> Noted by: rgrimes@ >>>>>>>>> >>>>>>>>> Modified: >>>>>>>>> head/tools/build/Makefile >>>>>>>>> >>>>>>>>> Modified: head/tools/build/Makefile >>>>>>>>> >>> >>> >>> =========================================================================== >>>>>>>> === >>>>>>>>> --- head/tools/build/Makefile Thu Apr 18 00:38:54 2019 >>>> (r34634 >>>>>>>> 0) >>>>>>>>> +++ head/tools/build/Makefile Thu Apr 18 01:02:00 2019 >>>> (r34634 >>>>>>>> 1) >>>>>>>>> @@ -59,9 +59,7 @@ INCS+= capsicum_helpers.h >>>>>>>>> INCS+= libcasper.h >>>>>>>>> .endif >>>>>>>>> >>>>>>>>> -.if !exists(/usr/include/casper/cap_fileargs.h) >>>>>>>>> CASPERINC+= >>>> ${SRCTOP}/lib/libcasper/services/cap_fileargs/cap_filea >>>>>>>> rgs.h >>>>>>>>> -.endif >>>>>>>> >>>>>>>> As a further note, we should probably hunt for any thing >>>>>>>> that is explicity looking at /usr/include/... in a Makefile, >>>>>>>> as that is minimally missing a ${DESTDIR} argument. >>>>>>>> >>>>>>>> The above may of actually worked if it had been written: >>>>>>>> .if !exists(${DESTDIR}/usr/include/casper/cap_fileargs.h) >>>>>>>> someone may wish to test that. >>>>>>>> >>>>>>>> Also a pathname rooted at / without ${DESTDIR} is almost >>>> certainly a mistake. >>>>>>> >>>>>>> This is a better solution. I tested this in a tree with a >>>> duplicated >>>>>>> environment: Problem solved. Before this is committed it >> should >>>> be >>>>>>> tested on one of the universe machines. >>>>>> >>>>>> From what Ed just said this would also be wrong, >>>>>> as well as CASPERINC+= above being wrong, if this >>>>>> is being built for the host we should not be using >>>>>> any headers from ${SRCTOP} at all. >>>>>> >>>>>> if capfileargs.h does not exist on the host that functionality >>>>>> must not be compiled into the buildtool as the host does not >>>>>> have this feature and attempting to use it from SRCTOP is wrong. >>>>>> >>>>> >>>>> Keep in mind that this is bootstrap; it's being built for the host >>>>> system, but it will link against a version of libcasper that's >> been >>>>> built in an earlier stage with the proper featureset. >>>> >>>> Ok, flip flop again, if infact this is linked against a >>>> library that implements the stuff from cap_fileargs.h then >>>> infact the ${DESTDIR} addition so that the build peaks into >>>> the cross build tree is correct, or what ever the equivelent >>>> to DESTDIR is for that ? BUILDDIR? The point is it should >>>> be picking this header up from the object tree, NOT from >>>> the running system. >>> >>> Yes, this was my conclusion when working on kerberos and ntp. This is >> also >>> true of libraries, else one would need to installworld and >> buildworld >>> again to get a properly built library/binary. >>> >>> IIRC ngie@ fixed a number of these across the tree a couple of years >> ago. >> >> >> OK. There's a number of different issues going on. As the original >> author >> of libegacy (which is what we're seeing fail), let me address the >> design >> generically and comment on different things that have come up in this >> thread. >> >> Since this is going into the libegacy that we're using to build the >> system, >> the check for file is bogus, for reasons I'll discuss below. When we >> add >> new includes to the system, it is appropriate to do it this way. And >> when >> this file was added to the system, the check was correct. >> >> First off, DESTDIR is absolutely not correct since this is to build the >> legacy library and legacy includes which augment the host's sources on >> legacay system, hence the name. It's never the correct thing to use. >> >> The problem that we have here is not that the file is missing (which is >> why >> it was added the way it was a long time ago), but rather missing >> functionality in a file that's been around for a long time. >> >> So, since it is that class of problem, the canonical way we've dealt >> with >> it in the past is to do something like create a file foo.h that looks >> something like >> >> #include_next <foo.h> >> >> #ifndev SOMETHING_NEW >> #define SOMETHING_NEW 0 /* Or other values that are fail-safe on the >> old >> system */ >> #endif >> >> and that's the change that needs to be made here. Sometimes, more >> extensive >> things need to be done when the old library can't work at all with the >> new >> code. In those cases, the pattern is for foo.h to include #define >> problem_fn my_problem_fn and then write a my_problem_fn that wraps >> problem_fn in a way that works on the old system. Kinda a poor man's >> symbol >> versioning, in a way (note: we can't use symbol version for this since >> we're building new binaries). >> >> The "stop gap" gets things compiling, and maybe OK for the moment, >> unless >> the SOMETHING_NEW variable that's used (in this case FA_OPEN) causes >> the >> old library to do the wrong thing. I've not done the deep dive to see >> if >> this is the case or not. >> >> So, does that make sense? >> >> Warner > > This solves one problem but what about the cases when a new krb5, ntp, or amd is imported but fails to build because it is using the old headers in /usr/include and linking against old libraries on the running system? > > These examples BTW have been fixed. My concern is there could be other examples in contrib, yet to be discovered, that might also have the same issues. We don't build those as bootstrap tools. This step is a special thing only needed to build tools that run on the _host_ that are used when compiling the world. All of the rest of world, etc. is compiled against the SYSROOT. This step is how you build the tools you need to generate the SYSROOT. strings is one of these special tools, so it requires extra care. -- John Baldwin
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?AA719490-5A6F-4218-90B8-FB7FA724C94E>