Date: Fri, 17 Apr 2015 09:08:46 -0400 From: John Baldwin <jhb@freebsd.org> To: freebsd-arch@freebsd.org Cc: Bryan Drewery <bdrewery@freebsd.org> Subject: Re: [RFC] SA/EN ABI/Lib flagging for package rebuilding Message-ID: <3952433.yB8VVSoaLQ@ralph.baldwin.cx> In-Reply-To: <5526B8E0.1090403@FreeBSD.org> References: <5526B8E0.1090403@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thursday, April 09, 2015 12:37:36 PM Bryan Drewery wrote: > Currently for any SA/EN we force rebuild all packages for > pkg.freebsd.org. This is a very time-consuming process at 24 hours for > each set and 10 (non-head) sets to build. > > We really only need to rebuild packages if ABI/KBI is changed (which I > would think would happen almost never) or if a library is updated. The > library being updated needs a rebuild in case of packages linking > against the library statically. There's no quick way to determine what > package may be infected by a static linkage to a library so we just > rebuild them all. > > There are 2 somewhat conflicting goals here. We need head packages to be > usable on all ABI/KBI changes. The simplest here was decided to just > rebuilding head packages always. This is a compromise I am willing to > accept. I had wanted to use __FreeBSD_version for this in the past but > it tends to be missed or bumped gratuitously. > > > My idea to cover both cases so that packages will only rebuild if the > public API or libraries change in the jail or anything else which > affects the resulting binaries. > > I will take a checksum of the following and if anything differs then do > a rebuild: > > /lib /usr/lib /libexec (.so included due to symbol versioning and > detecting added/removed libraries which can change packages) > /usr/include (all public API for the system including kernel API for > modules) > /usr/libdata > > That is all I am aware of for the public API/KPI/ABI/KBI that is > relevant for packages. > > I also realize I need to include some of the binutils files as they may > change how binaries are produced. This is all of 'ar as ld nm objcopy > objdump ranlib readelf strip' in /usr/bin/. Also all of GCC/Clang > binaries in /usr/bin: 'cc clang clang++ clang-cpp clang-tblgen cpp > c++filt CC c++ g++ gcc gnu-ar gnu-ranlib'. > > I don't want to consider all of /bin /usr/bin /sbin and /usr/sbin as it > catches SA that do not really need a rebuild, such as > FreeBSD-SA-15:07.ntp which modified /usr/sbin/ntpd. > > By having a whitelist of files from *bin* I risk not rebuilding in a > rare case where /bin/sh or /bin/cp (random example) had a vulnerability > fixed in them that could change resulting binaries. I consider that to > be very unlikely though. Perhaps for head packages it is more relevant > but I still consider it unlikely other changes will require rebuilding > packages. > > If I forgot other critical pieces from *bin* please list them. > > An alternative would be for there to be a reliable flag in head and > SA/EN noting whether binaries need to be rebuilt. There had been a > technical limitation to bumping __FreeBSD_version for SA as it would > modify all binaries. I think that was fixed. Portmgr fought hard to > always have __FreeBSD_version bumped but that's a lost cause. As I > mentioned I don't trust this flag in head either. I also want __FreeBSD_version always bumped for SA/EN, but secteam@ actively refuses to do this, even after being explicitly told to do so by core@. :( Regardless, I think your approach is more robust than hoping developers remember to bump the version for any API/ABI change. Looking at your proposed list I can only think of a few other binaries that matter, in particular code-generators like lex, yacc, and rpcgen. I don't claim that that list is exhaustive, but I do think it is a class of binaries that were not in your list. -- John Baldwin
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3952433.yB8VVSoaLQ>