Date: Wed, 7 May 2014 10:57:49 -0700 From: Warner Losh <imp@bsdimp.com> To: Benjamin Kaduk <kaduk@MIT.EDU> Cc: svn-doc-head@freebsd.org, svn-doc-all@freebsd.org, doc-committers@freebsd.org, Warner Losh <imp@freebsd.org> Subject: Re: svn commit: r44734 - head/en_US.ISO8859-1/books/handbook/cutting-edge Message-ID: <4D3E993B-0C34-40AB-B0B3-7D1C900B862A@bsdimp.com> In-Reply-To: <alpine.GSO.1.10.1405062252260.9713@multics.mit.edu> References: <201405011850.s41Iojk5091857@svn.freebsd.org> <alpine.GSO.1.10.1405062252260.9713@multics.mit.edu>
next in thread | previous in thread | raw e-mail | index | archive | help
[-- Attachment #1 --] On May 6, 2014, at 9:09 PM, Benjamin Kaduk <kaduk@MIT.EDU> wrote: > Warner, can you please double-check the facts about make variables? > I am not confident I have the new world order correct. > > On Thu, 1 May 2014, Dru Lavigne wrote: > >> >> Modified: head/en_US.ISO8859-1/books/handbook/cutting-edge/chapter.xml >> ============================================================================== >> --- head/en_US.ISO8859-1/books/handbook/cutting-edge/chapter.xml Thu May 1 18:12:22 2014 (r44733) >> +++ head/en_US.ISO8859-1/books/handbook/cutting-edge/chapter.xml Thu May 1 18:50:44 2014 (r44734) >> @@ -1396,6 +1396,25 @@ before running "/usr/sbin/freebsd-update >> sequence described in >> the following procedure.</para> >> >> + <note> >> + <para>It is a good idea to save the output from running >> + <command>make</command> to a file. If something goes wrong, a copy of >> + the error message can be posted to one of the &os; mailing >> + lists.</para> >> + >> + <para>The easiest way to do this is to use <command>script</command> with a >> + parameter that specifies the name of the file to save all >> + output to. Do not save the output to >> + <filename>/tmp</filename> as this directory may be cleared at >> + next reboot. A better place to save the file is >> + <filename>/var/tmp</filename>. Run this command immediately before rebuilding >> + the world, and then type <userinput>exit</userinput> when the >> + process has finished:</para> >> + >> + <screen>&prompt.root; <userinput>script <replaceable>/var/tmp/mw.out</replaceable></userinput> >> +Script started, output file is /var/tmp/mw.out</screen> >> + </note> >> + > > (I mentioned on IRC that this could be "script /var/tmp/mw.out make buildworld", but only on BSD boxes -- Linux's script(1) has a different syntax. This is more portable as-is, so I don't think we should change it.) make buildworld |& tee foo.log is what I use, but that’s csh/tcsh syntax and only catches one command. when I have to do multiples, I’ll add -a to the tee command. Sometimes I use emacs’ shell to get the same effect. >> @@ -1597,140 +1620,70 @@ before running "/usr/sbin/freebsd-update >> behavior of the <filename>Makefile</filename> is controlled by >> - these variables. These are the same variables as are set in >> - <filename>/etc/make.conf</filename>, and this provides >> - another way of setting them. For example:</para> >> + variables. These can either be set in >> + <filename>/etc/make.conf</filename> or they can be specified > > I think we must also mention /etc/src.conf here, in the file entitled "make world”. src.conf is used for all builds in /usr/src (no matter where /usr/src is located). If you are building things like ls from hand, you’ll need to also define MAKESYSPATH to be/have “…/share/mk” in it. >> + when using <command>make</command>. For example, this >> + variable specifies that profiled libraries >> + should not be built:</para> >> >> <screen>&prompt.root; <userinput>make -DNO_PROFILE <replaceable>target</replaceable></userinput></screen> > > Using the NO_FOO form of variables is highly deprecated at this point, at least in HEAD. (I think we're trying to deprecate it on the other branches, too.) While reviewing some of Warner's commits to look at the current state of things, I saw a comment in src.opts.mk that "Old instances [of NO_FOO] should be removed since they were just to bridge the gap between FreeBSD 4 and FreeBSD 5." The normal way to disable the building of profiled libraries would be to set WITHOUT_PROFILE in /etc/src.conf; I think that -DWITHOUT_PROFILE on the command line should also work, but would like confirmation from Warner. In 11 and forward, his is correct. Set WITHOUT_PROFILE=yes somewhere and all profiling is disabled. In some rare cases you may need to set MK_PROFILE=no. This replaces the NO_FOO syntax that was supposed to be a stop-gap measure between 4 and 5, but the gap never stopped. NO_FOO is limited to CTF, DEBUG_FILES, INSTALLLIB, MAN and profile, and so was irregular with the rest of things. A few more NO items may be added to the list in my next round of changes, but that’s likely a ways away. WITHOUT_PROFILE should work in 10 and earlier too, but that should be tested. I don’t plan on MFCing the NO_foo deprecation warning, and in fact realize that it is premature in head. >> - <para>is another way of specifying that profiled libraries >> - should not be built, and corresponds with the</para> >> + <para>It corresponds with this setting in >> + <filename>/etc/make.conf</filename>:</para> >> >> <programlisting>NO_PROFILE= true # Avoid compiling profiled libraries</programlisting> > > (Per the above, this is WITHOUT_PROFILE=true in /etc/src.conf. Maybe we should note that it doesn't matter if it's set to true or false of arglebarglepants; the fact that it's set is the only thing that matters. The inverse of WITHOUT_PROFILE is setting WITH_PROFILE explicitly.) Yes. >> - <para>line in <filename>/etc/make.conf</filename>.</para> >> - > [...] >> - <para>On a multi-CPU machine using an SMP configured kernel, try >> - values between 6 and 10 and see how they speed things >> + <screen>&prompt.root; <userinput>make -j4 buildworld</userinput></screen> >> + >> + <para>On a multi-<acronym>CPU</acronym> machine, try >> + values between <literal>6</literal> and <literal>10</literal> to see how they speed things > > -j10 is suboptimal on a 32-core machine, surely. The last advice I remember seeing on the mailing lists was that the limit was about 1.5 times the number of cores. Somewhere between 1.5x and 2.0x is where we saturate. 10 used to be a very optimistic number, in that the typical parallelism rarely got above 5 jobs running at any time. Now, however, running around #CPUs jobs tends to approach optimal, although some small but worthwhile gains can be had at 1.5 to 2.0 times the number of CPUs, especially if hyper threading is disabled. However, this massive parallelism is relatively recent, and problems exist starting in the -j 15-20 range and getting really bad in the 30 and up range. I’ll be hacking a bit in this area for some time to come, so maybe it would be good to do a review when I’m done? Warner > -Ben > >> up.</para> >> >> <indexterm> [-- Attachment #2 --] -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJTanQeAAoJEGwc0Sh9sBEAgQoQAKNQfLWnfCYdMhi2rTYsUl3A 0ZJN8H0P+VlMYBDB5Kgs+SykFo4VL3urA11Fpjsb3Lqscq48JKTvjhhndrlOHWD1 IFnhNDcAWQell5WA4tVfjjHn7j4zSdKsWdaV+e3B3Ipy/udrX+HhtqwAz/9LLP52 fEzYVYNbNjitc6NCKX8AW+DOKzya3pN6JBVWI6o1LbPaoZr5fMROxNpdqy7r0L3T GXcwbcEC6Oe+aYP0q8uud4mQSY/29WoT3DDzZBjmybfDF1H0VWjfH/dbMTfrygwi 0+wdje+CBbbEXxvhdDhvHU/AmJXTFAOgkHmySdH8SOf6TmEAhb379NWhnB9kqOIh ye3oAGsveVK0ePjPgMtliEXPkTRFPKLrKkBS7exNjABevco1pVy/Xc18wbI8d65h D2kHWNvIMGhqQ2O94Ts0Qsa3xE1G7kOq0E0Xwpt5IsM07i+VwPBBnF5K4Tmz7ImJ ITKUMhzEWpP1jt02Wn4FDUHXOFT3pRSmN/MrvgE6ayKL2tGHw6HoxSuOj8KFjRXB JWXejCCenk4arl9KQ80FWQfXEE90Ta2W6MGuRZjgYVEDwTe4LeJCcPkik5fuRanO iWlSrMbufvV6PLb6MD391QCJ5cvYAsLoAvpLrWk79xDI44xjzOF3XBzRbONWj0pk v55FogfRUg7u0uS2Bm2F =6SQ+ -----END PGP SIGNATURE-----
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4D3E993B-0C34-40AB-B0B3-7D1C900B862A>