Date: Fri, 31 Oct 1997 00:47:41 +1030 From: Mike Smith <mike@smith.net.au> To: current@freebsd.org Subject: Staying -current with FreeBSD (rev 2) Message-ID: <199710301417.AAA01090@word.smith.net.au>
next in thread | raw e-mail | index | archive | help
This is a multipart MIME message.
--==_Exmh_-13658490450
Content-Type: text/plain; charset=us-ascii
Thanks to everyone for the suggestions; sorry for the lag in getting
this version out. Again, any comments, expansions, extra text, etc.
would be appreciated. Please circulate, photocopy, fax, xerox, mimeo,
steno, staple, tack, glue, or otherwise forward this anywhere you think
it may either help or receive commentary.
Thanks.
Mike
--==_Exmh_-13658490450
Content-Type: text/plain; name="sc"; charset=us-ascii
Content-Description: sc
Content-Disposition: attachment; filename="sc"
Staying -current with FreeBSD r2
======================================================================
Maintained by Mike Smith <msmith@freebsd.org>
FreeBSD-current represents the bleeding edge of FreeBSD development.
It's where all the new features, performance improvements and bugfixes
are unleashed on the user community. If you are interested in
developing for FreeBSD, or just want the heady thrill of getting all
the latest stuff first, -current is what you want.
FreeBSD-current changes daily, and like any new product it goes through
patches of instability and unreliability. This is part and parcel of
using -current; if you don't think you can deal with it, now is a good
time to opt out. A -release or -stable version will still give you
excellent service, and with greatly improved dependability.
However, there are a lot of people that do run FreeBSD-current, and
many use nothing else. If you are serious about contributing to
ongoing FreeBSD development, you may need to run -current too.
In order to help new -current users, and old ones with bad habits, this
is a guide to getting along with FreeBSD-current.
If you're having problems, check here first because if you don't, you
may find your (quite valid) complaint ignored simply because there's
not enough information for anyone to do anything about it, and you just
might find the information you need to get you going again.
Notes
-----
If you're going to be running FreeBSD-current, it's assumed that
you are reasonably familiar with FreeBSD. No attempt will be made
here to explain basic operating procedure or commands; if this sounds
too complicated then it probably is.
Command examples here should work with any shell. Where there are
potential differences, an example of each form will be given.
Preparation
-----------
If you are not subscribed to the FreeBSD-current mailing list, you
should do this first. Running -current and not reading the mailing
list is like driving at night with duct tape over your eyes and
jellybeans in your ears; ie. not a good idea.
Another mailing list worth subscribing to is the cvs-all list. Every
change made to the FreeBSD source tree is accompanied by a mail
message sent to this list. These messages contain the name(s) of the
file(s) affected, the name of the person making the change, and a
description of the change itself. Reading this list can provide
you with valuable insight into the directions that -current is taking,
as well as advance warning of possible problems or fixes. There are
other cvs-* lists which may be more suitable, depending on your field
of interest.
Read the -current mailing list for a few days before starting with
FreeBSD-current; it's possible that you might have picked a really bad
time to start. If there are problems, people will be discussing them;
wait for a resolution to be announced, or for other people to indicate
that the problem has been solved. If you're not sure after a few days,
post a short message asking whether people think that it's OK to go
ahead.
As you become more familiar with the atmosphere and the people on
-current, you will learn to judge what constitutes "really bad" and
what is merely "interesting". Remember that while FreeBSD-current is a
development environment, the desired result is indeed a stable platform
and if something is broken people will be working to fix it.
Starting
--------
Once you're subscribed to the -current mailing list, there are a couple
of different ways to get FreeBSD-current onto your system. Which is
best for you will depend on your circumstances.
- FTP: You can install a FreeBSD-current snapshot via FTP just like
a release version. FreeBSD-current snapshots are normally built daily
and can be downloaded from ftp://current.freebsd.org/. When
installing you can either start from scratch, or upgrade an existing
system.
- CDROM: Walnut Creek CDROM occasionally release a FreeBSD-current
snapshot CDROM, and other vendors may too. These are generally a
bargain, as they come with a copy of the CVS repository which can
save you a lot of download time. You can install from a snapshot
CDROM in exactly the same fashion as you would a release version.
Be careful though; a snapshot more than a few months old may be just
as tough to upgrade from as a previous release.
- Bootstrap: If you have the FreeBSD CVS repository on your system
you can checkout a copy of the -current source tree and bootstrap
a previous release system. You can also get the -current source
using CVSup or CTM (see below). Bootstrapping may take a little
perseverance, but that's good practice for later. Note that
bootstrapping from anything older than the most recent release is
something best left to the truly adventurous.
If you have problems bootstrapping and really can't get -current
going any other way then you should post a concise message to the
FreeBSD-current mailing list clearly detailing what you are trying
to do, the release you are bootstrapping from and any error
messages that you receive during the process.
Keeping -current
----------------
Once you have a -current system up and running, congratulations, it's
obsolete! FreeBSD-current changes daily, and part of running a
-current system is keeping up to date.
There are two parts to keeping your system -current; getting the
source, and building it.
You can obtain the source in two forms; either the raw source tree, or
the CVS repository. The former requires around 200MB of disk space,
the latter perhaps twice times that; expect these values to grow with
time. The raw source tree is all that's required to build the source,
however the CVS repository is useful if you are interested in tracking or
becoming involved with development. In particular, the CVS repository
can be invaluable if you are trying to locate a new problem.
There are two distribution strategies in common use for keeping your
source up to date, named after the tools used to perform the updating:
CVSup and CTM.
CVSup
'''''
CVSup replaces an older tool, 'sup' (the Source Update Protocol), and
is the preferred method for tracking the -current source tree. It
requires an active network connection and provides guaranteed-accuracy
updates of your CVS repository or checked-out source tree. CVSup can
also repair a damaged or modified repository or tree.
If you have a -stable or -current system up and running, sample control
files for CVSup are located in /usr/share/examples/cvsup. You can get
CVSup itself from the FreeBSD ports collection.
CTM
'''
CTM can be interpreted as 'CVS Thru eMail'; it provides updates for
your CVS repository or source tree via email messages. CTM is ideal if
your network connection is indirect, expensive or otherwise cannot
support CVSup.
Documentation for the CTM programs is included with the system, start
with 'man ctm', and check the handbook for more details.
Checking out with CVS
'''''''''''''''''''''
If you are fetching the CVS repository there is one important step
required before you can build anything. This is known as "checking
out" the source; literally you will generate a complete copy of the
-current source tree from the information in the repository.
The source tree can be anywhere, but the intermediate files
generated during the build will be placed under /usr/obj. If the
filesystem containing /usr doesn't have ~150MB of free space, you will
need to create a directory elsewhere and use a symbolic link.
To check out the source tree, go to where the 'src' directory is to be
located and set the CVSROOT environment variable to point to the location
of your CVS repository. Using a csh-like shell:
# setenv CVSROOT <path-to-repository>
or with an sh-like shell:
# export CVSROOT=<path-to-repository>
Then issue the command:
# cvs checkout src
This will create a directory 'src' and populate it with the source
tree. When the repository is updated by CVSup or CTM you can propagate
the changes to this source tree by making the src directory your
current directory and saying:
# cvs update -Pd
If you have been making changes to your source tree, CVS will try to
merge these changes with any updates it makes. You should watch the
output from the update command and note any merges; these should be
checked before you go any further to make sure they won't cause any
problems. Identifying these merges is easier if you make CVS run
quieter, eg.
# cvs -q update -Pd
If in doubt, remove the file(s) in question and rerun the update,
which will replace them with fresh copies.
Building the World
''''''''''''''''''
Once you have your source tree checked out, make sure the 'src' tree is
reachable as /usr/src, become root and change to the /usr/src directory.
Now is the time to select your build options. If you've built the
FreeBSD tree before, you will be familiar with these. Options are set
via environment variables, by editing /etc/make.conf, or by passing
them on the commandline when you build the tree. We will use the last
method here.
To set an option when invoking 'make', specify its name as an argument to
the '-D' option, ie. to set the option VARIABLE, you would add
'-DVARIABLE' to the 'make' commandline.
The common variables and their uses are:
NOCLEAN Do not clean out the old build tree. This can save
on build time, but may introduce subtle problems.
NOSECURE Do not build the export-restricted modules. If you
haven't downloaded the source for these, then you
won't want to try to build them. This is a good
option to use if you are not in the USA.
NOGAMES Don't rebuild the games. If you don't have or want
them installed, set this.
NOSHARE Don't rebuild /usr/share. This can save some time,
and if /usr/share (containing manpages, data files and
examples) hasn't changed much it won't cause you any
trouble.
NOPROFILE Don't build the profiling libraries. If you're not
doing any development involving profiling, this can
save a considerable amount of time and space. If you
don't know what profiling libraries are, you don't
want them.
Once your options are set, start the build. It's generally a good idea
to keep a record of the build, in case there are problems and you want
to look over it. There are two good ways to do this. The first uses
the 'script' command, which arranges a transcript of a session. You
would say:
# script
Script started, output file is typescript
# make world
When the build has finished, use 'exit' to close the script.
The second method runs the build in the background, and watches it with
the 'tail' command. The advantage of this method is that you can let
the build run without having to watch it all the time; you can come
back to check on it at your leisure. To do this, you would say:
# make world >& makelog &
# tail -f makelog
You can hit ^C at any time to exit the 'tail' command, and then reissue
it whenever you want to check on the build.
A world build transcript generated like this will generally be around
the 7 megabyte mark; you should make sure you have space for it.
If you are unsure about whether the build will work, and would prefer
to make sure it builds properly before allowing it to install over
your running system, you can use the 'buildworld' rather than 'world'
target. This will compile everything, but not install it. Once you
are happy that the build has completed OK, you can invoke the
'installworld' target, which will install the built world.
If your 'buildworld' is failing on something that you think is trivial
you can add the '-k' option to the make command. This will cause make
to continue after an error, rather than stopping the build. If you
have built with '-k' you should examine the build log very carefully
before proceeding to install, as it is possible to make an enormous
mess like this.
Building the Kernel
'''''''''''''''''''
Changes in -current often require coordinated rebuilding of both the
'world' (userland) and the kernel. Normally you should build a new
kernel after you have built the world, but before you reboot. There
are a couple of important points to bear in mind regarding the kernel:
- The 'config' program is closely tied to the kernel. It is normally
built as part of userland, but it can also be built manually with
# cd /usr/src/usr.sbin/config
# make depend all install clean clendepend
Using an outdated 'config' program can cause strange errors when
configuring or building a kernel, althought 'config' does its best
to determine whether it is out of date and complain accordingly.
- The kernel and LKMs are closely related. Mismatches can cause fatal
kernel errors or strange behaviour. LKMs are also built as part of
userland, but can be regenerated and reinstalled with
# cd /usr/src/lkm
# make depend all install clean cleandepend
Note that both of these examples assume that your source tree is under
/usr/src; if you've put it elsewhere, adjust them accordingly.
Changes to -current may require matching changes in your kernel
configuration file. These are normally reflected in changes to the
LINT or GENERIC files, and/or described in messages posted to the
mailing list.
Once the kernel has been built and installed, reboot the system. It is
generally a good idea to do this at this point, in order to ensure that
the running kernel and userland programs correspond.
When Things go Wrong
--------------------
Problems with FreeBSD-current can generally be divided into three
groups:
- It won't build.
- It won't boot.
- Something funny happens when...
It won't Build
''''''''''''''
If it won't build, and the reason isn't immediately obvious, there are
several steps you should take. Always remember to go through these in
order; skipping them may miss the problem. In some cases, the only
remedy is to wait for someone to fix the problem before you can go on.
- Completely remove the object tree and start again. You can do this
with:
# chflags -R noschg /usr/obj/
# rm -rf /usr/obj/*
Normally the 'world' target will do this for you, unless you have
NOCLEAN or one of the related non-cleaning options set.
The 'chflags' command removes the system-immutable flag which is set
on some of the intermediate files generated during the build process.
This flag renders files unchangable (and undeletable) as a security
and safety measure. Its use on intermediate files can be considered
a 'feature'.
- If you have a local copy of the CVS repository, completely remove
the source tree, check out a new copy and try building it again.
Make sure you make a backup of your kernel configuration files
before doing this.
- Update your local sources, either by waiting for the next CTM mail,
or re-running CVSup. If you are using a local CVS repository don't
forget to update the checked-out soruce tree with the 'cvs update'
command described above. It is possible that your last update came
in the middle of a compound change to the source tree, and that
updating is all it will take to correct the problem.
- Wait a day or two, and watch the mailing list for messages about your
problem. If the problem is widespread (ie. not a result of local
circumstances) then other people will have encountered it, and a fix
is likely to be discussed rapidly.
While you are waiting, try investigating the problem yourself. The
build process isn't magic, and the error messages are emitted when
it fails can often provide you with helpful clues.
If you think you have found a problem, try fixing it yourself. If
you are using a local CVS repository or CVSup and you make a mess of
a file, just delete it and update again; the file will be magically
replaced. You need to be a bit more careful if you are fetching the
source tree directly with CTM.
- If you cannot obtain a clean build after a couple of days of
watching the list and updating your source tree, you should post a
message to the -current list describing your problem. It is
extremely important that you provide enough information to diagnose
your problem; without this information your post is likely to be
ignored because it represents a great deal of work.
At the very least, your post should include:
x The last update date of your source tree. If you are using CVSup,
the time at which you last ran CVSup, and the server you updated
from. If you are using CTM, the number of the last CTM delta that
you have applied.
x An excerpt from the the end of the build transcript above. Try to
include the last few lines of commands before the "error 1" lines,
as these will generally contain information needed to locate the
problem.
x A list of all the build options that were enabled.
You may be asked for additional information, or to try various things;
be prepared to become involved in solving your problem, and look to
learn from the experience. In time, you may be able to apply the
solutions you have discovered to someone else's problem, and in turn
help them out.
It Won't Boot
'''''''''''''
Occasionally you will encounter a newly-built kernel that won't boot.
"Won't boot" may mean that it produces a panic message, or a fatal trap
while starting up, or the system may freeze up, or reboot spontaenously.
There can be a number of reasons for these failures; the two most
common are:
- You have stale LKMs.
- A new change to the kernel is broken.
You should always check for an LKM problem before suspecting that the
kernel is broken. It is easy to check whether you have such a problem:
- Boot single-user (specify '-s' at the boot: prompt)
- Mount the root filesystem read/write:
# mount /
- Move the LKM directory out of the way:
# mv /lkm /lkm.bad
- Continue to multi-user:
# exit
If this fixes your problem, then you should remove all of the old LKMs,
and install a new set. This can be achieved with:
# mv /lkm.bad /lkm
# rm -rf /lkm/*
# cd /usr/src/lkm
# make depend all install clean cleandepend
Note that both this technique and the one above for config may make
temporary files in the source tree; this is dirty but expedient.
If the kernel doesn't get to the single-user stage, or if it still
fails, then it is likely that the kernel is broken. You should boot
the backup kernel, which is a copy of the previous kernel which was
made when you installed the new one. This is achieved by entering
the name of the backup kernel at the 'boot:' prompt; it's normally
called 'kernel.old'. If you are experimenting with new kernels,
you will want to make another backup, as everytime you install a
kernel the backup is overwritten with the current kernel; two bad
kernels in a row would leave you unable to boot.
You can copy the backup kernel over the newer kernel with the commands:
# chflags noschg /kernel
# cp /kernel.old /kerne
# chflags schg /kernel
Once you have your system up and running again in this fashion, you
should again keep an eye on the FreeBSD-current mailing list. It's
possible that your problem will already have been noticed, and a fix
may have been made already. If your problem persists after several
days and several rebuilds, you can participate in tracking it down as
follows.
It is important to provide a lot of information about kernel problems,
as they can be very subtle. You should build a kernel with the debugger
enabled, ie. with
options DDB
in the configuration file. Ideally, the kernel should be configured
with the '-g' flag to config, and installed in this form. Be aware
that this will result in a kernel that may be up to 10 megabytes in
size, as it includes copious debugging information.
When the system panics, or encounters a trap, it will print a message
giving details of the panic or trap, and leave the system in the
debugger at the 'ddb>' prompt. At this point, you should make a note
of all of this information. It is important that you do this, as it
identifies the basic nature of the problem, its location and the state
of the system when the problem occurred.
You should then issue the 'trace' command to ddb, and note all of its
output. This provides more vital information as to the state of the
system.
Armed with this information, you are ready to post to the
FreeBSD-current mailing list. You should include in your message:
- The trap/panic message and 'trace' output.
- The last update date/CTM number from which the kernel was built.
- Your hardware configuration details.
- The kernel configuration file from which the kernel was generated.
- The activity which causes the problem (booting, heavy load, etc.)
If your problem is due to a fault which is triggered by your system
configuration or load profile you may be asked to try making changes to
the source in order to locate a suitable fix. These changes are often
provided in the form of patches, which are discussed below.
There is a lot more that you can do with a debugging-enabled kernel;
see the online handbook for more details on this.
Something Funny happens When...
'''''''''''''''''''''''''''''''
You may notice that the new kernel or userland doesn't behave 'quite
right'. A command may behave differently, or the system may not
respond as it did previously under some circumstances. In the worst
case, you may find that the kernel may panic or encounter a fatal trap.
Under these circumstances, it is important to characterise the nature
of the fault. This means that you should attempt to establish, by
trial and error, exactly what it is that is causing the problem to
manifest. It may be a combination of arguments to a command; a
particular sort of system load, or any of many other things.
If the system is panicing or stopping on a trap, build and install a
kernel with DDB and preferably debugging enabled as above. Otherwise,
gather as much information about the problem and any identified causes
for it, and watch the FreeBSD-current mailing list. If the problem
isn't discussed after a couple of days, post your findings. As always,
provide as much information as you believe may be relevant, and be
prepared to become involved in finding the solution.
How Often?
----------
Now you are running -current, how often should you update?
This will depend on your personal schedule, the load and demand on your
system, and the general health of -current. In general once a week is
a good number if you are keen to keep up to date. Once a month is good
if you want to avoid falling too far behind. Another alternative is to
wait for a few days after a new major feature of interest is announced,
to make sure it is stable and worthwhile.
Wrapup
------
FreeBSD-current can be a rewarding and useful environment. Users of
-current provide vital feedback to the developer community, who in turn
depend on this feedback as a check on their work. -current users must
be prepared to work with developers in the pursuit of a stable,
reliable, high-performance system, and in turn developers must work
with users towards the same goals.
Using -current can be a lot more work than a release or -stable
version, but the rewards:
- Improved performance.
- New features and fixes, sooner.
- A sense of directly contributing to the ongoing development
of FreeBSD.
can be well worth the effort.
Patching it Up (sidebar)
--------------
When you are chasing a problem with your -current system, or if you
want to try out something so new that it hasn't been added yet, you may
encounter 'patch' or 'diff' files. Often someone will send you one
saying "here, try this".
These files contain information which allows you to make automated
changes to your source tree. They are fed as input to the 'patch'
program, usually as:
# patch -p < <patchfile>
It is important to be in the correct directory when applying a patch.
If you can't establish this from the details at the top of the patch
file, be sure to ask the author. You can check that a patch will apply
by supplying the '-C' option to patch; if any parts are rejected, the
patch will not apply cleanly.
Note that if you apply a patch to your source tree, CTM will no longer
be able to update the file(s) patched. The patch program makes a
backup of patched files with the suffix '.orig'; after you have tested
a patch you may wish to move these files back to their original names.
If you are using a CVS repository, CVS will try to merge your patched
changes with new changes to the repository. This can occasionally
result in files that cannot be used to build from, as they contain
merge comments. The easiest way to deal with such a file is to delete
it and then say:
# cvs update -Pd <filename>
If you are using CVSup to update your source tree directly, it will
replace any patched files with the current version when you update.
--==_Exmh_-13658490450--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199710301417.AAA01090>