Date: Wed, 17 Mar 2004 10:58:34 -0700 From: Scott Long <scottl@freebsd.org> To: FreeBSD Current List <current@freebsd.org>, freebsd-stable@freebsd.org Subject: January-February 2004 FreeBSD Status Report Message-ID: <405891CA.6000300@freebsd.org>
next in thread | raw e-mail | index | archive | help
January-February 2004 Status Report
Introduction:
2004 started with another exciting two months for the project. FreeBSD
5.2 was released in early January and then quickly followed in
February with the 5.2.1 bug-fix release. Looking forward, we are
expecting a late-April release date for FreeBSD 4.10, and mid-summer
date for FreeBSD 5.3. And don't forget to support the FreeBSD vendors
and developers by buying a copy of the latest CD or DVD sets.
Thanks,
Scott Long
* Bluetooth stack for FreeBSD (Netgraph implementation)
* Automatic sizing of TCP send buffers
* Compile FreeBSD with Intels C compiler (icc)
* Disk and device I/O
* FreeBSD GNOME Project Report
* FreeBSD Package Grid
* FreeBSD ports monitoring system
* FreeBSD/arm Status Report
* FreeBSD/ia64
* FreeSBIE
* kgi4BSD
* libarchive/bsdtar
* Move ARP out of routing table
* NanoBSD
* Network interface naming changes
* Network Stack Locking
* Porting OpenBSD's pf
* PowerPC Port
* SGI XFS port for FreeBSD
* Testbed for testing and qualification of TCP performance
* The FreeBSD Dutch Documentation Project.
* The FreeBSD Simplified Chinese Project
* Verify source reachability option for ipfw2
* vinum + GEOM
* Weekly cvs-src summaries
Bluetooth stack for FreeBSD (Netgraph implementation)
Contact: Maksim Yevmenkin <m_evmenkin@yahoo.com>
Not much to report. Bluetooth Service Discovery Procotol daemon sdpd
was integrated with existing Bluetooth utilities. From now on users
should not use GNU sdpd (Linux BlueZ port).
Bluetooth HID profile implementation is almost complete. Thanks to
Matt Peterson < matt at peterson dot org > for giving me Bluetooth
keyboard and mouse for development.
_________________________________________________________________
Automatic sizing of TCP send buffers
Contact: Andre Oppermann <andre@FreeBSD.org>
The current TCP send and receive buffers are static and set to a
conservative value to preserve kernel memory. This is sub-optimal for
connections with a high bandwidth*delay product because the size of
the TCP send buffer determines how big the send window can get. For
high bandwidth trans-continental links this seriously limits the
maximum transfer speed per TCP connection. For example a 170ms RTT and
a 32kB send buffer limit the speed to approximately 1.5Mbit per second
even thought you might have a 10Mbit pipe.
This project makes the TCP send buffer to automatically adapt to the
optimal buffer size for maximal link usage. In the case above this
would be a buffer of approximately 220kB. The main challenge is to
have a stable and reliable measurement of the link parameters and
manage the kernel memory properly and in a fair way. We don't want to
have a few connections to monopolize all available socket buffer space
and many edge cases have to be considered. The first implementation
will be tuned conservatively but even that will provide significantly
better performance than the static buffers currently. Work on this
project is already in progress.
_________________________________________________________________
Compile FreeBSD with Intels C compiler (icc)
URL: http://www.Leidinger.net/FreeBSD/
Contact: Alexander Leidinger <netchild@FreeBSD.org>
If nothing bad happened, the icc patches got committed around the date
of the deadline for submissions of this report. Please search the
archives of -current and/or cvs-all for more information.
The next steps in this project are to
* fix the kernel to also run without problems when compiled with icc
v8
* fix the kernel if some problems surface after more people give it
a try
* get some ports to compile with icc
_________________________________________________________________
Disk and device I/O
Contact: Poul-Henning Kamp <phk@FreeBSD.org>
In the overall area of disk and device I/O, a significant milestone
was reached with the implementation of proper reference counting on
dev_t. We are now able to properly allocate and free dev_t. Cloning
device drivers also had the job made easier for them with the addition
of the unit number management routines.
It is not quite decided which will be the next step in the quest for a
truly SMPng I/O subsystem, but a leading candidate is to implement the
device-access vnode bypass to get more concurrency in the system:
Instead of taking the tour through the vnodes for each i/o operation
on a device we will go directly from the file descriptor layer to
DEVFS/SPECFS. In addition to Giant-less disk I/O, this should enable
us to pull the entire tty subsystem and the PTY driver out from under
Giant and we expect that to improve the "snappiness" of the system
measurably.
_________________________________________________________________
FreeBSD GNOME Project Report
URL: http://www.FreeBSD.org/gnome/
Contact: FreeBSD GNOME Team <gnome@FreeBSD.org>
It has been a year since our last status report, but we haven't slowed
down. Since the last report, Alexander Nedotsukov (bland) and Pav
Lucistnik (pav) have joined the FreeBSD GNOME team. GNOME 2.4 was
released back in September 2003, followed by 2.4.1 and 2.4.2. We are
actively working on getting GNOME 2.6.0 out the door at the end of
March. GNOME 2.6 Beta releases can be obtained via the project URL
above.
To help make GNOME 2.6.0 our best release to date, we have created a
script to automate the upgrade from GNOME 2.4. We also have a new
GNOME package build server that builds and serves i386 packages for
all supported FreeBSD releases. We plan on having the GNOME 2.6.0
packages available the moment 2.6.0 hits the ports tree.
Included in the release of GNOME 2.6 is GTK+ 2.4, the next installment
in the GTK+ 2 series. Because GTK+ 2 has become very stable over the
past few years, the FreeBSD GNOME Team is pushing for GTK+ 2 support
to be included by default in all applications that support it. This
has already been done with Mozilla, Firefox, and Thunderbird. A
complete GNOME Desktop and application environment can already be
built using only GTK+ 2. The ultimate goal is to phase GTK+ 1 out of
the ports tree.
_________________________________________________________________
FreeBSD Package Grid
Contact: Kris Kennaway <kris@FreeBSD.org>
Distributed package builds are currently done using a set of
home-grown shell scripts for managing, scheduling and dispatching of
package builds on the client machines. This has been sufficient for
our needs in the past, but has a number of significant shortcomings
that limit future growth. I am rewriting the package build scripts to
work on top of Sun GridEngine (ports/sysutils/sge), as a client
application of a "FreeBSD package grid". Some of the design goals for
the new system are:
* Better robustness against machine failure, and more efficient
scheduling of build jobs
* Support for remote build machines, to make better use of machine
resources and clusters that are not on the same LAN as the build
master
* Ability for other committers to submit port build jobs to the
system, for testing of changes, new ports, etc.
_________________________________________________________________
FreeBSD ports monitoring system
URL: http://portsmon.firepipe.net/index.html
Contact: Mark Linimon <linimon_at_lonesome_dot_com>
Thanks to the loan of a box by Will Andrews, the system has been moved
into production. The previous installation at lonesome.com now refers
you to the new system. As part of the installation, a preliminary FAQ
was added.
The database is updated once per hour.
New reports available include ones about ports marked DEPRECATED,
since that function has now been incorporated into bsd.port.mk. (The
author hopes that this will allow the port deprecation process to be
much more visible to the general FreeBSD user community.) In addition,
a report for ports marked FORBIDDEN was added (the code was
essentially the same).
The next topic of interest is to try to identify ports which are slave
ports because the status of these ports is not currently being updated
automatically. This problem also affects FreshPorts. PR ports/63683 is
an attempt to address this problem. Also, preliminary work has been
done on creating some graphs and charts for various statistics, and in
creating a tool to browse port dependencies for the entire ports tree.
Some general observations about the trends in ports PRs can be made:
* In the past 6 months, the amount of time to get ports PRs
committed has dropped dramatically. (This is especially true of
PRs for new ports.)
* The queue of PRs for existing ports that are unmaintained has
similarly been trimmed. Both of these two items are due in large
part to a few very active committers (how do they ever get their
"real" work done?) Thanks, guys, you know who you are.
* There is still a fairly high number of PRs (~400/~750) which apply
to existing ports, and have been assigned to a FreeBSD committer.
This represents around 370 individual ports. We seem to have a
much harder time getting these numbers to go down; basically, we
just hold our own most weeks. This is somewhat disappointing.
* The number of ports marked BROKEN has jumped dramatically,
currently standing at over 250 (for i386-current). This represents
less a sudden problem as it does Kris' effort to bring existing
brokenness to people's attention -- thus, a much larger percentage
of ports with build errors are now labeled as BROKEN.
* Approximately two-thirds of the port build errors are still due to
compilation problems, primarily from the gcc3.3 import. Another
10% fail to install correctly. The reasons for the others are more
varied.
_________________________________________________________________
FreeBSD/arm Status Report
Contact: Olivier Houchard <cognet@FreeBSD.org>
Development goes reasonably fast, right now it boots single user. It
is still very simics-centric, and it deserves a huge cleanup and a few
bug fixes, but there's already a decent amount of code to work with,
mostly taken from NetBSD. I now plan to work on real hardware support
(as soon as I can get some), to get the missing userland bits (mainly
rtld and the pthread libs) so that I can build a full world.
_________________________________________________________________
FreeBSD/ia64
URL: http://www.FreeBSD.org/platforms/ia64/index.html
Contact: Marcel Moolenaar <marcel@FreeBSD.org>
Work on the PMAP overhaul has been put into gear. A lot of issues will
be addressed, including support for sparse physical memory and of
course SMP. Performance will be addressed to the extend possible, but
functionality has priority. The redesign will lay the foundation for
NUMA support where possible. An example of this is limiting TLB
shootdowns to processors that actually have or had TLBs belonging to
the PMAP loaded. Of course, without NUMA hardware the implementation
of NUMA support is quite limited.
_________________________________________________________________
FreeSBIE
URL: http://www.freesbie.org
URL: mailto:freesbie@gufi.org
URL: http://www.freesbie.org/?section=mirror-en
Contact: FreeSBIE Staff <staff@FreeSBIE.org>
The FreeSBIE Project aims to develop a set of scripts that allow
anyone to create their own FreeBSD Bootable Cdrom, with their own set
of installed packages. The Project releases an ISO builded with
FreeSBIE scripts, to show what they can do. On Sunday 29 February
2004, FreeSBIE 1.0 was released and it had a great success, as there
were post on Slashdot.org, OSnews, DaemonNews and BSDForums. Thanks to
the huge amount of feedback they got, FreeSBIE Developers are now
developing new features such as support for archs different from i386.
Website redesign is on the way too.
_________________________________________________________________
kgi4BSD
URL: http://www.FreeBSD.org/~nsouch/kgi4BSD
Contact: Nicholas Souchu <nsouch@FreeBSD.org>
Move to Perforce is done. I spent some time on building a common
compilation tree with Linux: until now drivers were build in a FreeBSD
makefile tree, not compatible with Linux.
The next priorities are ANSI support and keymaps in the KGC Kernel
Graphic Console system.
_________________________________________________________________
libarchive/bsdtar
URL: http://people.freebsd.org/~kientzle/
Contact: Tim Kientzle <kientzle@FreeBSD.org>
libarchive, with complete documentation, has been committed to
-CURRENT. bsdtar should follow soon. For a few months, gtar and bsdtar
will both be available in the base system. Once bsdtar is in the tree,
I hope to resume work on libpkg and my pkg_add rewrite.
Note that bsdtar is not an exact replacement for gtar: it does some
things better (reads/writes standard formats, archive ACLs and file
flags, detects format and compression automatically), some things
worse (does not handle multi-volume archives or sparse files) and a
few things just different (writes POSIX-format archives by default,
not GNU-format). The command lines are sufficiently similar that most
users should have no problems with the transition. However, people who
rely on peculiar options or capabilities of gtar may have to look to
ports.
_________________________________________________________________
Move ARP out of routing table
Contact: Andre Oppermann <andre@FreeBSD.org>
The ARP IP address to MAC address mapping does not belong into the
routing table (FIB) as it is currently done. This will move it to its
own hash based structure which will be instantiated per each 802.1
broadcast domain. With this change it is possible to have more than
one interface in the same IP subnet and layer 2 broadcast domain. The
ARP handling and the routing table will be quite a bit simplified
afterwards. As an additional benefit full MAC address based accosting
will be provided. Work on this project is already in progress.
_________________________________________________________________
NanoBSD
Contact: Poul-Henning Kamp <phk@FreeBSD.org>
NanoBSD, src/tools/tools/nanobsd, is a tool for stuffing FreeBSD onto
small disk media (like CompactFlash) for embedded applications. The
disk image is built with three partitions, two for software images and
one for configuration files. Having two software partitions means that
new software can be uploaded to the non-active partition while running
off the active partition.
The first really public version has been committed and many
suggestions and offers of patches have started pouring in.
_________________________________________________________________
Network interface naming changes
Contact: Brooks Davis <brooks@FreeBSD.org>
The first actual feature related to the if_xname conversion was
committed in early February. Network interfaces can now be renamed
with "ifconfig <if> name <newname>".
Work is slowly progressing on a new network interface cloning API to
enable interesting cloners like auto-configurating vlans. This work is
taking place in the perforce repository under:
//depot/user/brooks/xname/...
_________________________________________________________________
Network Stack Locking
Contact: Sam Leffler <sam@FreeBSD.org>
Contact: Robert Watson <rwatson@FreeBSD.org>
This project is aimed at converting the FreeBSD network stack from
running under the single Giant kernel lock to permitting it to run in
a fully parallel manner on multiple CPUs (i.e., a fully threaded
network stack). This will improve performance/latency through
reentrancy and preemption on single-processor machines, and also on
multi-processor machines by permitting real parallelism in the
processing of network traffic. As of FreeBSD 5.2, it was possible to
run low level network functions, as well as the IP filtering and
forwarding plane, without the Giant lock, as well as "process to
completion" in the interrupt handler.
Work continues to improve the maturity and completeness of the locking
(and performance) of the network stack for 5.3. The network stack
locking development branch has been updated cothe latest CVS HEAD,
tracking a variety of FreeBSD changes, including tracking and driving
changes in the interface and device cloning APIs, push-down and fixes
to locking in the Berkeley Packet Filter, consistency improvements in
allocation flags for network objects, diagnosis of excessive
acquisition of Giant in various system callouts and timeouts, removal
of Giant from several system callouts, "const"-ification of a number
of global variables in the network stack (IPv4, IPv6, elsewhere) as
part of ananalysis of locking requirements, fine-grain locking of a
number of pseudo-interfaces (disc, loopback, faith, stf, gif, tap,
tun), IP encapsulation and tunneling, initial review and locking of
parts of PPP and SLIP, experimentation with PCB assertions on IPv6,
additional socket locking assertions, graphing of the FreeBSD sockets
layer to support locking analysis, merging of theMT_TAG to m_tag
conversion to improve the ability to queue packets, moving of the
debug.mpsafenet tunable to controlling Giant over the forwarding plane
to Giant over the entire stack("dual-mode" to support non-MPSAFE
protocols), adaption of existing network lock assertions to also
assert Giant when running non-MPSAFE, analysis of high cost of
select() locking, improved locking and synchronization annotations,
TCP callouts run MPSAFE, logtimeout() runs MPSAFE, uma_timeout() runs
MPSAFE, callout sampling instrumentation, loadav() runs MPSAFE,
AppleTalk locking begun: AARP locked down and DDP analysis, rawcb list
locked, locking analysis of mrouter and IP ID code, IGMP locked, IPv6
analysis begun, IPX/SPX analysis begun, PPP timeouts converted to
callouts, Netgraph analysis begun. Many of these changes have not yet
been merged to the main FreeBSDtree, but this is a work in progress.
In related work on Pipe IPC (not quite network stack locking),
substantial time was invested in diagnosing an increase in the cost of
pipe allocation since FreeBSD 4.x, as well as coalescing the several
allocations needed to create a pipe, as well as moving to slab
allocation so as to amortize the cost of pipe initialization. Future
work here will include caching the VM structures supporting pipe
buffers.
Recent contributors include Robert Watson, Sam Leffler, MaxLaier,
Maurycy Pawlowski-Wieronski, Brooks Davis, and many others who are
omitted here only by accident.
_________________________________________________________________
Porting OpenBSD's pf
URL: http://pf4freebsd.love2party.net/
URL: http://www.benzedrine.cx/pf.html
URL: http://openbsd.org/faq/pf/index.html
URL: http://www.rofug.ro/projects/freebsd-altq/
Contact: Max Laier <max@love2party.net>
Contact: Pyun YongHyeon <yongari@kt-is.co.kr>
The sources were imported from OpenBSD 3.4R and patched with diffs
obtained from the port. Since March the 8th it is linked to the build
and install. There is some more work to be done in order make pf a
home inside the tree, but the biggest hunk of work was lifted during
the past two month.
OpenBSD 3.5 is scheduled for early May, so we might see an update
before 5.3R. Work towards integration of the - often requested - ALTQ
framework is in progress also, though it is not yet clear how well it
goes along with the ongoing work towards a giant free net stack.
_________________________________________________________________
PowerPC Port
Contact: Peter Grehan <grehan@FreeBSD.org>
After a slow time at the end of last year due to a disk crash, the
project is moving along rapidly. The loader is fully functional with
Forth support. Syscons has been integrated. New Powerbook models are
supported. Work is starting on a G5 port.
There's still lots to do, so as usual volunteers are most welcome.
_________________________________________________________________
SGI XFS port for FreeBSD
Contact: Alexander Kabaev <kan@FreeBSD.org>
Contact: Russell Cattelan <cattelan@thebarn.com>
Not much has changed since last report was submitted. The read-onle
access XFS volumes is quite stable now. The work is underway to
rewrite xfs_buf layer to minimize local changes intrusiveness. Initial
attempt to make XFS code to compile and run on amd64 is in progress
too.
We really need a care-taker for our userland tools.
_________________________________________________________________
Testbed for testing and qualification of TCP performance
Contact: Andre Oppermann <andre@FreeBSD.org>
The TCP performance test and qualification testbed is an automated
environment that simulates various common and uncommon end-to-end
network and link characteristics such as delay, bandwidth limitations,
congestion, packet drops, packet corruption and out of order arrival.
The testbed automatically steps through all link types and tests
various TCP optimizations and parameter adjustments. In the end all
data is graphically arranged and compared against standard behaviour
and each other to judge the positive or negative effects of the
modifications. Work on this project has just started and is based on
FreeBSDs dummynet.
_________________________________________________________________
The FreeBSD Dutch Documentation Project.
Contact: Remko Lodder <remko@elvandar.org>
The Dutch Documentation Project is a ongoing project in translating
the handbook and other documentation to the dutch language. Currently
there is 1 active person (me) translating the documentation. I am
currently working on the handbook/basics section. But i can use some
more hands, please drop me an email if you wish to help out so that
the dutch translation will speed up and be ready in some time. Contact
remko@elvandar.org for information.
_________________________________________________________________
The FreeBSD Simplified Chinese Project
URL: http://www.freebsd.org.cn
URL: http://www.freebsd.org.cn/snap/zh_CN/
URL: http://www.freebsd.org.cn/snap/doc/zh_CN.GB2312/books/handbook/
Contact: Dong LI <ld@FreeBSD.org.cn>
Contact: Xin LI <delphij@frontfree.net>
The project is a joint effort of volunteers, which focus in the
internationalization and localization of the FreeBSD Operating System
and applications running on FreeBSD. All of the work resulted in this
project will be contributed back to the FreeBSD project.
Thanks to many volunteers' help, by this time of writing, we have
finished more than 60% of the translation of the FreeBSD Handbook. We
plan to submit a preliminary translation of the FreeBSD website as
well as the FreeBSD Handbook when most part of them were finished,
which is expected to happen in a couple of months. The snapshot of the
documentation translation effort could be accessed through the URL
listed above.
The project also supported individual efforts on porting applications
(especially software that supports Simplified and/or Traditional
Chinese) to FreeBSD. We are also doing some research on making FreeBSD
kernel and base system more i18n-aware.
_________________________________________________________________
Verify source reachability option for ipfw2
URL: http://www.nrg4u.com/freebsd/ipfw_versrcreach.diff
Contact: Andre Oppermann <andre@FreeBSD.org>
The verify source reachability option for ipfw2 checks if the source
IP address of a packet entering the machine is reachable at all. Thus
if we can't send a packet back because we don't have a route back we
don't have to forward it because two way communication isn't possible
anyway. It is more than likely that such a packet is spoofed. This
option is almost the same as what is known on Cisco IOS as "ip verify
unicast source reachable-via [any|ifn]". Using this option only makes
sense when you don't have a default route which naturally always
matches. So this is useful for machines acting as routers with a
default-free view of the entire Internet as common when running a BGP
daemon (Zebra/Quagga or OpenBSD bgpd).
One useful way of enabling it globally on a router looks like this:
ipfw add xxxx deny ip from any to any not versrcreach or for an
individual interface only: ipfw add xxxx deny ip from any to any not
versrcreach recv fxp0
_________________________________________________________________
vinum + GEOM
URL: http://mailbox.univie.ac.at/~le/geom_vinum.tar.gz
Contact: Lukas Ertl <le@FreeBSD.org>
The "geomification" of vinum has made some progress. I now have all
basic setups working (concatenated plexes, striped plexes, RAID5
plexes, and RAID1), but I still have to implement correct error
handling and status change handling.
Still missing is a userland tool, so currently you still have to use
"old-style" vinum to configure your setup.
_________________________________________________________________
Weekly cvs-src summaries
URL: http://excel.xl0.org/FreeBSD/
URL: http://mocart.pinco.pl/FreeBSD/
Contact: Mark Johnston <mark@xl0.org>
I have been producing weekly summaries of commits and the surrounding
discussions as reported on the cvs-src mailing list. These summaries
are posted to -current on Sunday evenings and archived on the Web. The
reception has been overwhelmingly good. As of the end of February,
Polish translations are being produced by Lukasz Dudek and Szymon
Roczniak; they are also planning to translate the older summaries.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?405891CA.6000300>