Date: Wed, 4 Oct 2017 17:27:11 -0700 (PDT) From: Don Lewis <truckman@FreeBSD.org> To: sgk@troutmask.apl.washington.edu Cc: linimon@lonesome.com, list1@gjunka.com, freebsd-ports@freebsd.org Subject: Re: portmaster, portupgrade, etc Message-ID: <201710050027.v950RBFT047711@gw.catspoiler.org> In-Reply-To: <20171004232819.GA86102@troutmask.apl.washington.edu>
next in thread | previous in thread | raw e-mail | index | archive | help
On 4 Oct, Steve Kargl wrote: > On Wed, Oct 04, 2017 at 05:29:14PM -0500, Mark Linimon wrote: >> Please understand that I'm not trying to be obstinate, >> I'm trying to understand. > > Me too. > >> Background: years ago I managed the cluster of i386 blades >> that we used in package building. 933MHz and 512MB IIRC. >> So I am familiar with constraint problems. > > The system in question is my last i686 laptop, which I > use for libm development and testing. Once I cannot use > that laptop (whether hardware failure or inability to > update the installed ports), I'll stop worrying about a > functional libm on 32-bit hardware. As an aside, this sort of thing could be done in an i386 VM or maybe an i386 jail on amd64 hardware. >> On Wed, Oct 04, 2017 at 02:22:25PM -0700, Steve Kargl wrote: >> > Can't find info on whether jails can be avoided. >> >> I have not checked the code but IIRC, no. I thought jails >> had low memory overhead, though. > > That's good, but memory overhead isn't the problem with a > jail. It's the diskspace used to duplicate everything > already available in /bin:/usr/bin:/usr/local/bin:... and > storage to hold the packages as things get built. > >> > If you only have 1 Gb of memory and 5-10 GB diskspace, >> > then using poudriere with zfs and jails is a nonstarter. >> >> For point of comparison, with those constraints, I do not >> understand how modern llvms can build at all. >> >> What happens if you use the manual approach on this same >> system? e.g. >> >> cd /usr/ports/devel/llvm40 >> mkdir -p /usr/ports/packages >> make && make package >> pkg install /usr/ports/packages/llvm<whatever>.txz > > I normally do > > portmaster -Byd devel/llvm40 > > and wait a day for the build to complete. I have > /usr/ports/distfiles symlinked to a USB2 /mnt/distfiles > (40 MB/s max throughput). I may also need to set > DISABLE_MAKE_JOBS="YES", but can't remember offhand. > >> >> Do you still run out of resources? >> > > No. I have 4 GB of swap space, which is well used during the > build. The system is actually quite usable as portmaster > runs. I also build libreoffice and octave on the system. That must be fun ... At some point you'll run into the problem that even one copy of the ports that you want to install and just one copy of all their dependencies will no longer fit and then you have to start tossing out stuff that you would really like to install and use. I'm pretty sure that I no longer have enough room to build all the stuff on my laptop that I currently have installed (openoffice warns that it needs 11GB of free space). In that case, if you move the package building off to another machine, you don't even have to install many of the build dependencies on the target machine. For instance, I have a list of 214 ports that I want to install on my desktop machine. When I feed that into poudriere, it builds something on the order of 1900 ports. When I do the pkg install on my desktop machine with the same list, only 1431 packages get installed. >> In that case, there's not much that can be done. The >> compilers, the office suites, and certain math packages >> are huge beasts. However you try to build them won't >> matter. >> >> I would think having a copy of the llvm workfiles in a jail >> is going to be equivalent to having them in /tmp? >> >> I must be missing something. > > portmaster satisfies its dependencies from already installed > ports in /usr/local. There isn't a clean room full of pkg'd > dependencies sitting in some jail while llvm40 builds. The llvm40 problem should be helped a lot by subpackages. If you install a package that was built by llvm40 and uses its shared libraries, hopefully pkg will install a subpackage that contains the llvm40 .so files and not all of the compiler bits. You would only get those if you manually installed llvm40 so that you could use the compiler bits. In that case, poudriere would only cause two copies of the .so files to be installed and the compiler bits would only be installed on the machine (in the build jail) when poudriere was using a port that needed the llvm40 compiler. The same will be true of other things that are installed because of LIB_DEPENDS. Things like the include files, docs, and static libraries won't get automatically installed just because a port is a dependency of something else. They would only need to be manually installed if you want to build something manually outside of the ports framework.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201710050027.v950RBFT047711>