Date: Mon, 22 Jul 2019 10:55:49 -0700 From: Theron Tarigo <theron@freebsd.org> To: soc-status@freebsd.org Subject: GSoC: Separation of Ports Build Process from Local Installation Message-ID: <0016ceb4-b36e-8fa8-6597-8256541a96a8@freebsd.org>
next in thread | raw e-mail | index | archive | help
Hello all, This project aims to decouple the build process of the ports framework from the local installation. To work around the assumptions made by ports about dependency file locations, a userspace tool to remap processes' filesystem namespaces is under development: freebsd-user-namespace, provided as port devel/userns. This past week, I have made the following progress: - Eliminated the need for any special setup to be able to use PORTS_SEPARATED_BUILD. The mode may be enabled by this one variable, at the command line or in /etc/make.conf. - Created a lightweight framework for testing large numbers of ports under the separated build mode. It does the following: - Handles FLAVORS. - Saves success/failure/skipped status. - Skips ports which have failed dependencies. - Builds each port in its own pristine environment. (This allows concurrent building of unrelated ports) - Saves success/failure/skipped status. - Skips ports which have failed dependencies. - Builds each port in its own pristine environment. (This allows parallel (concurrent) port build to work) - Attempted build of 2000 ports, selected at random, and their dependencies. - Identified (and applied temporary fixes for) some ports causing highest number of skipped dependents. - Retried the skipped ports. - Developed a plan of a general fix for the most common cause of failure: custom target definitions in port Makefiles running build tools (also from ports) without the userns wrapper. Latest summary of port building success: - 5613 ports tried - 2958 (53%) succeeded - 177 (3%) failed - 2478 (44%) skipped due to failed dependencies - Success rate: 94% What I am working on next: - Implement the planned solution for running build tools inside userns wrapper. - Compare the packages built by separated-build method to either official packages or ports built by original FreeBSD method. Any differences should be attributable to built times and hostnames. - Debug the ports which fail by segfaut or buserror; this implicates userns library. - Provide more thorough documentation of PORTS_SEPARATED_BUILD, in particular its usage and its limitations, so that users and devs interested in this project can more easily test it for themselves. Project goals and status are kept at https://wiki.freebsd.org/SummerOfCode2019Projects/PortsSeparatedBuild . Source of the userspace filesystem namespace tool is shared at https://github.com/therontarigo/freebsd-user-namespace . Changes to ports framework are shared at https://github.com/freebsd/freebsd-ports/compare/master...therontarigo:separated. Theron Tarigo
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?0016ceb4-b36e-8fa8-6597-8256541a96a8>