Date: Fri, 26 Jan 2024 11:28:19 +0100 From: Alexander Leidinger <Alexander@Leidinger.net> To: Gleb Popov <arrowd@freebsd.org> Cc: Stefan Esser <se@freebsd.org>, freebsd-ports <ports@freebsd.org>, portmgr <portmgr@freebsd.org>, FreeBSD Core Team <core@freebsd.org> Subject: Re: This is going to break port building without poudriere! Message-ID: <4b1f2470bf476f0f9e8f8b689c585c43@Leidinger.net> In-Reply-To: <CALH631ntQ8VzqhDmyxpcXwpZU0jsALgi_74qzLpNDSBLtGNXRA@mail.gmail.com> References: <CAB88xy-8hAknWJDRBjbJo2%2Bw878ZMosKcvQbpKVzwq%2BH7%2Bzuyg@mail.gmail.com> <cd0c0cb0-6035-45b4-b3e8-d99115e6c013@FreeBSD.org> <CAB88xy8gTC4UJK0fOiHnVCFf0AGtLoHfHdOAF29zChQ8=5SV6w@mail.gmail.com> <d6a7c9725edd734aca842d6ce85b0be2@Leidinger.net> <CALH631ntQ8VzqhDmyxpcXwpZU0jsALgi_74qzLpNDSBLtGNXRA@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --=_cf3966ffa22ae96a960c44f1231e7418 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=UTF-8; format=flowed Am 2024-01-26 09:41, schrieb Gleb Popov: > On Fri, Jan 26, 2024 at 11:01 AM Alexander Leidinger > <Alexander@leidinger.net> wrote: >> >> Am 2024-01-25 20:57, schrieb Luca Pizzamiglio: >> >> >> I think Stefans expectations about such a feature are different from >> the understand of the implementor of the feature in our tree. Somewhat >> a clash of an idealistic view and reality. >> >> To me (and I assume to Stefan too) it looks like slave ports will go >> away and subpackages will be used instead. A slave port may only >> compile a subset, and a subpackages aware port will compile everything >> (simplified view, not true where slave ports exclude a feature instead >> of excluding a file). From this point of view, a port can not depend >> on a subpackage in the sense of a port can not depend only on a subset >> of the php extensions included in the main php build (if/when it is >> converted to subpackages). As such a build from ports will have all >> php extensions included in the php port installed, whereas a pkp based >> install can limit the amount of installed php extensions to what is >> required. At least this is what I understand based upon what I have >> read about subpackages. As the documentation is not ready and I >> haven't looked at the code, this understanding may off course be >> wrong. >> >> Is my understanding correct that subpackages aware ports are supposed >> to replace master/slave ports? In the sense of slave ports will be >> deleted once a port is converted to a subpackages aware port (except >> where slave ports exclude features from binaries like git-lite... yes, >> git-lite is implemented as a flavour and as such we cover this in a >> different way and such slave ports if they still exist can maybe be >> converted to use flavours)? I assume yes to both questions. With this >> assumption: >> >> With master/slave ports this is possible. And replacing master/slave >> ports with a subpackages aware port will remove this possibility. > > Subpackages are not a master/slave ports replacement, nor vice versa. > Both you and Stephan seem to mix a lot of concepts which causes all > that confusion. Did I misunderstand that Luca wants to convert master/slave ports like my php case into subpages aware ports to cut down on package build times? So if there is a port which supports subpackages but has no slave port it is a bug? How do we express that a slave port is a subset of another port so that poudriere doesn't build too much (also take the parallel build into account) and only builds the subpackages aware master port? What you wrote in the provided URL, in particular this about master/slave ports: ---snip--- - With flavors and subpackages at our disposal, the master/slave ports should only be considered as a last resort. ---snip--- Based upon this I would assume stuff which can be expressed as subpackages aware ports shall not have slave ports. Can you please try to explain in different words if my understanding is not correct? Please also see the rest of the mail first for suggestions with specific examples. > The idea behind master/slave ports **templating**. The master Makefile > serves as a template and the slave Makefile overrides some variables > and/or targets. This is a very general technique which allows for > implementing many things. Including subpackages, by the way! Look at > how devel/appstream and devel/appstream-qt are implemented. lang/php82 is not a good example? Why not? > The idea behind subpackages is producing multiple packages from a > single build (a single work/ directory). This idea is simple and > concise in its shell. It can be applied right away without literally The complaint from Stefan is not about the idea of subpackages. It is about the implemntation of them. So it is not about the simple and concise idea (which I get), what we are talking about is a specific implemntation of this concept. To make it explicit: I do not complain at all about anything. I try to understand the implementation we got (without looking at the implementation and only looking at what written here in this thread). Obviously what was written here so far was not clear enough to me to understand it. I have an idea what this is able to do, and an expectation how it is supposed to work. Clearly Stefan things there are some drawbacks to the implementation with his idea what such a feature shall be able to do. And it seems his ideas about how something like that is supposed to work is not far away from what I expect from something like that. And what you wrote, matches with some parts of my expectations what such a feature is supposed to do. What I try to get at here is to get rid of some misunderstanding between people. To get all into the same picture. May it by that Stefan and I "see" how it works and understand, that there is no issue, or may it be that you and Luca "see" what the problem is Stefan and I see (note, all my installations of stuff which we have in our ports collection happens via packages build by my own poudriere instance, installing a ports directly is a rare situation for me, but the concern that "make install" and "pkg install" will not produce the same result while thinking it shall produce the same result in all cases is a real one). > degrading anything else, but for now it requires much caution (again, > see the revert commit for my attempt to subpackagize devel/appstream). > > Finally, ports are mostly declarative build recipes, a pile of > variables and some command invocations that **describe** something. > Subpackages allows us to refine the description of a given software > product by saying "From the resulting build artifact we can pick out > this and that into their own package". Nothing is broken in Ports. The > breakage you're talking about is the breakage in **interpretation**. Yes, I interpreted what Luco wrote as replacing some master/slave ports with subpackages aware ports. Removing the slave ports and having only one port which produces everything and generates several packages out of this one port. I interpreted all this out of what he wrote about reducing the build time on the build machines. If this is a misinterpretion, then we should understand this as a suggestion for the upcoming documentation for the subpackages feature to: - give clear advise to create slave ports for all the subpackages, - what needs to be done in the slave ports to denote the parent port which shall be build on the package build cluster instead of this slave port to cut down on package build times - make it clear the dependencies shall be expressed to slave ports and not to the subpages aware port If I misunderstood this again, please provide a mock-up of an example. In my opinion, lang/php82 would be a good candiate for a subpackages aware port, and a mock-up example (portname, dependencies) with a fictive webapplication would be a good example to show how the ports and packages are used/created/handled/... I base my "good candidate for a subpackages aware port" on what you wrote in the link you provided, in particular on this: ---snip--- - If you have master/slave ports which follow the same build procedure but then remove some files from the `STAGEDIR` to make packages different - they are begging for being subpackaged. ---snip--- For php it doesn't remove files, but generates a few files based upon the same build procedure. If you think that lang/php82 is not a good candidate for subpackages, please explain this rationale and take it as a request to make it clear in the upcoming documentation (e.g. with lang/phpXX as an example) why this shall not be handled with subpackages and what shall be used for such cases. > Dependencies describe relations between the **port** we're building > and **packages** it depends on. It might be confusing at first, one > may argue "but I do see ports origin in the *_DEPENDS lines!". Yes, > but origins are there for convenience only. It is the part that > precedes ":" which matters - it declares what the port is requiring. > Where to get it is actually an orthogonal question and we already have > two options for that - a package repository and compiling a port (and > this is where the origin after ":" comes into play). Note that in the > resulting package all dependencies do not contain origins - exactly > because they are an additional info provided for convenience. > >> What is broken is that you _have_ to install such dependencies via >> packages in this case if you want to have the minimal install. > > There you're talking about a one specific interpretation, a naive one. > It build down to just "make"ing the Makefile. This approach still If I go into mail/roundcube and run "make install", this is exacly what happens for all the dependencies right now. Can you please describe what happens in ports if we assume that lang/php82 is subpackages aware? Without looking at the implementation of subpackages, and based upon what you wrote in the link provided below, my expectation would be that there are no slave ports for lang/php82 and all subpackages-modules, even those which mail/roundcube doesn't depend upon, are installed by this. And my expectation of a pkg install of mail/roundcube would be that only those subpackages-modules are installed, which the mail/roundcube port depends upon. > works, but there is a fundamental problem with building on host (aka > "not in poudriere"). Building on host requires you to install even > BUILD_DEPENDS (and transitively!). Are you OK with installing > BUILD_DEPENDS for a given port? Then subpackages doesn't even install A build depends is usually not exposed at runtime in the webinterface of e.g. the mail/roundcube port. If lang/php82 is subpackages aware and the dependency of mail/roundcube doesn't point to a slave port in www/php82-session but to a subpackage of lang/php82, a "make install" in mail/roundcube would install in my interpretation of the subpackages featue all the php extensions the lang/php82 port would provide by default and not only the dependencies of mail/roundcube. All those extensions would be enalbe by default for POLA and convenience reasons. All those extensions are then exposed in some way in the webinterface of mail/roundcube via the php interpreter. This is a higher security risk than an installed build depends of autoconf, gmake or such. > anything, they only require building more and only under certain > circumstances (when the subpackage is not hidden behind an option). > > With all this said, what's exactly broken for your case? You're > building/installing too much? But you were installing much more by > BUILD_DEPENDS, so it is hardly an argument. Heck, when using > build-on-host to install a simple Haskell program, you need to install > a 1Gb Haskell compiler package as its BUILD_DEPEND! See above. > P.S. I made a little writeup on Ports features, which tries to explain > what subpackages really are. You might find it useful: > http://arrowd.name/ports_writeup I have read it and it matches with what I expected as a concept. > P.P.S. It took a while to properly trim quotes from your message, > because your mail software did not mark Stefan's message as quotes. Ooops, sorry. The HTML mode of mail/roundcube which I use to answer has some strange behavior in some cases. This response was now done in plain text mode and I probably will switch to plain text mode for cases where roundcube comes up in html mode. Bye, Alexander. -- http://www.Leidinger.net Alexander@Leidinger.net: PGP 0x8F31830F9F2772BF http://www.FreeBSD.org netchild@FreeBSD.org : PGP 0x8F31830F9F2772BF --=_cf3966ffa22ae96a960c44f1231e7418 Content-Type: application/pgp-signature; name=signature.asc Content-Disposition: attachment; filename=signature.asc; size=833 Content-Description: OpenPGP digital signature -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEER9UlYXp1PSd08nWXEg2wmwP42IYFAmWziVMACgkQEg2wmwP4 2IZO3w/+IluPI9ja7y73wkWLdgkC7SpCGvC5z5VsjhiYkUxYzMt5MxHG9UwZ0YTY M4zM4R8DDr2Frwh6gna+cTCzaY12P3CCoOnng0V9BGApXaq/bZiqlwu9Vrii0QKQ Pd/R0SkFCGb0hTCvuMLGiaVinYaauwYQe5bhriMpVXNnLsNHBxTWFyuOZ714Ddae wm/UrM1mebo+ZodJamWXzWitZUZLrbL6ONPIcxuXl75NNykfjtGG5ah1UzuZ6eJ6 t1xbFuSlWAww9ADl6AVZcKU0ezPaqHfOPdb5RERbitebkhdB0bPm6PkcYrMR+eT8 JXXulJcjCzVG4x5QO1lLkgP2uKJGcNBZjzyP+gYvvfqzdtn7bC6NGiiUjRKX+x8Q Nbw+0OP91Y/TU0flH4S4nyQ2O3HgqsGm7vu7XoqitcbZ8z5wuHRVSUf+h/lgVxBU Z3yNps6l6sEN5Vz6rJ96NIFR9GVn0UBmDfW1tlmfpu/1zPr76hCylbyhTEWwgWzI uWFNNHkuJRqv07LatPM+PV9nQE0uhULkJOEJyeLhIgViSCkxN9VxrmpGw048Nxd4 8eror/JypZlrvdt6ukFS/n5llgoMXUqmjy5RoPWEtgr49avgXFGVjiFsrWyEITw+ JOkUEs4V6gYIntYSf7h54HauOcawp5qtk4nsjGFvEY1QWvWd61U= =Xth6 -----END PGP SIGNATURE----- --=_cf3966ffa22ae96a960c44f1231e7418--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4b1f2470bf476f0f9e8f8b689c585c43>