From owner-svn-src-user@FreeBSD.ORG Sun Dec 6 21:42:25 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D78FF10656A3; Sun, 6 Dec 2009 21:42:25 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id AD7578FC0A; Sun, 6 Dec 2009 21:42:25 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nB6LgP2p032446; Sun, 6 Dec 2009 21:42:25 GMT (envelope-from dougb@svn.freebsd.org) Received: (from dougb@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nB6LgPSI032444; Sun, 6 Dec 2009 21:42:25 GMT (envelope-from dougb@svn.freebsd.org) Message-Id: <200912062142.nB6LgPSI032444@svn.freebsd.org> From: Doug Barton Date: Sun, 6 Dec 2009 21:42:25 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r200187 - user/dougb/portmaster X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Dec 2009 21:42:25 -0000 Author: dougb Date: Sun Dec 6 21:42:25 2009 New Revision: 200187 URL: http://svn.freebsd.org/changeset/base/200187 Log: Rather than repeat all the tests for when NOT to run read_distinfos() each time, depend on the tests for creating $DI_FILES then test for its existence. In fetch_packages() don't run all the code for finding out packages, etc. unless $ppd is actually unset. Modified: user/dougb/portmaster/portmaster Modified: user/dougb/portmaster/portmaster ============================================================================== --- user/dougb/portmaster/portmaster Sun Dec 6 21:07:47 2009 (r200186) +++ user/dougb/portmaster/portmaster Sun Dec 6 21:42:25 2009 (r200187) @@ -2019,10 +2019,8 @@ all_config () { CONFIG_SEEN_LIST="${CONFIG_SEEN_LIST}${origin}:" done } - if [ -z "$DONT_SCRUB_DISTFILES" -a -z "$FETCH_ONLY" \ - -a -z "$PM_PACKAGES" ]; then - (read_distinfos)& - fi + [ -n "$DI_FILES" ] && (read_distinfos)& + ports_by_category echo "===>>> Starting check of installed ports for available updates" @@ -2236,10 +2234,8 @@ check_state || { safe_exit 1; } # Do not start this in the background until we are sure we are going to build -if [ "$$" -eq "$PM_PARENT_PID" -a -z "$SHOW_WORK" -a -z "$PM_PACKAGES" \ - -a -z "$DONT_SCRUB_DISTFILES" -a -z "$FETCH_ONLY" ]; then - echo ''; (read_distinfos)& -fi +[ "$$" -eq "$PM_PARENT_PID" -a -n "$DI_FILES" ] && { + echo ''; (read_distinfos)& } # Do these things first time through, with or without 'make config' if [ -z "$PM_BUILDING" -a -z "$SHOW_WORK" -a -z "$NO_ACTION" ]; then @@ -2415,16 +2411,18 @@ if [ -n "$PM_PACKAGES" -o "$PM_PACKAGES_ fetch_package () { local do_fetch - # Duplicated from pkg_init() - [ -z "$packages" ] && + if [ -z "$ppd" ]; then + # Duplicated from pkg_init() + [ -z "$packages" ] && packages=`pm_make -f/usr/share/mk/bsd.port.mk -V PACKAGES` - [ -n "$packages" ] || fail 'The value of PACKAGES cannot be empty' - - [ -z "$ppd" ] && { ppd=$packages/portmaster-download; export ppd; } + [ -n "$packages" ] || + fail 'The value of PACKAGES cannot be empty' + ppd=$packages/portmaster-download + export ppd + fi if [ ! -d "$ppd" ]; then - [ -n "$PM_SU_VERBOSE" ] && - echo "===>>> Creating $ppd" + [ -n "$PM_SU_VERBOSE" ] && echo "===>>> Creating $ppd" pm_mkdir_s $ppd fi From owner-svn-src-user@FreeBSD.ORG Sun Dec 6 23:28:49 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 90763106574E; Sun, 6 Dec 2009 23:28:49 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 5EC888FC08; Sun, 6 Dec 2009 23:28:49 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nB6NSn0G034951; Sun, 6 Dec 2009 23:28:49 GMT (envelope-from dougb@svn.freebsd.org) Received: (from dougb@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nB6NSnlV034948; Sun, 6 Dec 2009 23:28:49 GMT (envelope-from dougb@svn.freebsd.org) Message-Id: <200912062328.nB6NSnlV034948@svn.freebsd.org> From: Doug Barton Date: Sun, 6 Dec 2009 23:28:49 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r200192 - user/dougb/portmaster X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Dec 2009 23:28:49 -0000 Author: dougb Date: Sun Dec 6 23:28:49 2009 New Revision: 200192 URL: http://svn.freebsd.org/changeset/base/200192 Log: Don't say that we're buildin a port when we're using packages Don't rely on -v to print the message about checking the package repo since that can take a long time Modified: user/dougb/portmaster/portmaster user/dougb/portmaster/portmaster.8 Modified: user/dougb/portmaster/portmaster ============================================================================== --- user/dougb/portmaster/portmaster Sun Dec 6 23:27:24 2009 (r200191) +++ user/dougb/portmaster/portmaster Sun Dec 6 23:28:49 2009 (r200192) @@ -1829,6 +1829,16 @@ create_master_rb_list () { [ -n "$MASTER_RB_LIST" ] && export MASTER_RB_LIST=" $MASTER_RB_LIST" } +pca () { + if [ "$PM_PACKAGES" = only ]; then + echo install + elif [ -n "$PM_PACKAGES" ]; then + echo 'build and/or install' + else + echo build + fi +} + multiport () { # Global PM_MULTI_PORTS=':' ; PM_MULTI_BUILT=':' @@ -1893,7 +1903,7 @@ multiport () { check_fetch_only unset CONFIG_SEEN_LIST CONFIG_ONLY echo '' - echo "===>>> Starting build for multiple ports <<<===" + echo "===>>> Starting `pca` for multiple ports <<<===" echo '' if [ -n "$PM_BUILD_ONLY_LIST" ]; then @@ -2050,7 +2060,7 @@ all_config () { unset CONFIG_SEEN_LIST CONFIG_ONLY echo '' - echo "===>>> Starting build for ports that need updating <<<===" + echo "===>>> Starting `pca` for ports that need updating <<<===" echo '' fi @@ -2366,7 +2376,7 @@ if [ -n "$CONFIG_ONLY" ]; then check_fetch_only unset CONFIG_SEEN_LIST CONFIG_ONLY - echo "===>>> Starting build for $portdir <<<===" + echo "===>>> Starting `pca` for $portdir <<<===" echo '' fi @@ -2481,7 +2491,6 @@ fetch_package () { sitepath="${sitepath%/}/${portdir%/*}/" - [ -n "$PM_VERBOSE" ] && echo "===>>> Checking package repository for latest available version" if [ -n "$LOCAL_PACKAGEDIR" ]; then Modified: user/dougb/portmaster/portmaster.8 ============================================================================== --- user/dougb/portmaster/portmaster.8 Sun Dec 6 23:27:24 2009 (r200191) +++ user/dougb/portmaster/portmaster.8 Sun Dec 6 23:28:49 2009 (r200192) @@ -24,7 +24,7 @@ .\" .\" $FreeBSD$ .\" -.Dd September 15, 2009 +.Dd December 1, 2009 .Dt PORTMASTER 8 .Os .Sh NAME @@ -33,6 +33,11 @@ .Sh SYNOPSIS Common Flags: .Op Fl -force-config CGHgntvw [B|b] [f|i] [D|d] +.Op Fl [--packages|--packages-only] [-P|-PP] | [--packages-build] +.Op Fl -packages-if-newer +.Op Fl -delete-build-only +.Op Fl -always-fetch +.Op Fl -local-packagedir= .Op Fl m Ar arguments for make .Op Fl x Ar glob pattern to exclude from building .Nm @@ -70,6 +75,23 @@ and/or multiple globs from /var/db/pkg .Op Common Flags .Fl a .Nm +.Fl -delete-build-only +.Nm +.Op Fl -packages-if-newer +.Op Fl -always-fetch +.Op Fl -local-packagedir= +.Fl P|--packages +.Nm +.Op Fl -packages-if-newer +.Op Fl -always-fetch +.Op Fl -local-packagedir= +.Fl PP|--packages-only +.Nm +.Op Fl -packages-if-newer +.Op Fl -always-fetch +.Op Fl -local-packagedir= +.Fl -packages-build +.Nm .Fl [l|L] .Nm .Op Fl b [D|d] @@ -109,8 +131,8 @@ Rather it uses the existing ports infras including what is located in .Pa /var/db/pkg . The focus of this tool is to keep the dependency -tracking information for your ports up to date, -which allows you to update a specific port without +tracking information for your ports up to date. +This allows you to update a specific port without having to update all of the ports .Qq above it. @@ -136,6 +158,9 @@ option. It will then start building all ports that need updating. While recursing through dependencies, +if you are not using any of the +.Fl -packages* +options, a 'make checksum' process will be launched in the background to either verify that the correct distfiles are available, @@ -146,7 +171,7 @@ with ^C, an attempt will be made to kill the child processes started for this purpose. .Pp If the recursion through the ports for 'make -config' does not find a port that needs updating, +config' does not find a port that needs updating the dependency check step will be skipped prior to building the port specified on the command line. In addition to this optimization, @@ -176,6 +201,17 @@ and if you have installed an alternate v that is required by the port you are building, it will be used in place of the default dependency. .Pp +There are a number of +.Fl -packages* +options available to save the time that would normally +be spent building the port(s). +Users interested in a reasonable balance between speed of +installation and maximum performance should consider the +.Fl -packages-build +option, perhaps combined with the +.Fl -delete-build-only +option. +.Pp If there is no .Fl B option specified when updating an existing port, @@ -234,17 +270,25 @@ any .Pa pkg-message files that were installed, and a summary of the work performed will be displayed. +If the +.Fl -delete-build-only +option is in use, those packages that were installed during +the current run of +.Nm +AND were only ever listed as build dependencies during this +run will be deleted. .Pp If something goes wrong during the process (e.g., a port build fails, a port is marked BROKEN) .Nm will report any work done successfully as described above, then exit. +.Pp The question is often asked, .Dq Why is it not possible to proceed with the ports that do not have errors? The answer is that (unfortunately) .Nm -is not omniscient, and therefore cannot guess what resolution the +is not omniscient, and cannot guess what resolution the user would like to have for this problem. Manual intervention is therefore required. Assuming that the failure occurred after the config phase has ended @@ -256,7 +300,7 @@ option to the command line to skip the c The options are as follows: .Bl -tag -width F1 .It Fl -force-config -run 'make config' for all ports (must be the first option) +run 'make config' for all ports .It Fl B prevents creation of the backup package for the installed port .It Fl C @@ -307,13 +351,10 @@ If a port is not already installed the e be run against the directory name from .Pa /usr/ports . .It Fl p Ar port directory in /usr/ports -specify the full path to a port directory. -This option is generally not necessary and may be -removed in future versions. +This option has been deprecated. .It Fl -show-work show what dependent ports are, and are not installed (implies .Fl t ) . -This flag must come first on the command line. .It Fl o Ar replace the installed port with a port from a different origin .It [-R] Fl r Ar name/glob of port in /var/db/pkg @@ -326,6 +367,23 @@ or options to skip ports updated on a previous run. .It Fl a check all ports, update as necessary +.It Fl -delete-build-only +delete ports that are build-only dependencies after a successful run, +only if installed this run +.It Fl P|--packages +use packages, but build port if not available +.It Fl PP|--packages-only +fail if no package is available +.It Fl -packages-build +use packages for all build dependencies +.It Fl -packages-if-newer +use package if newer than installed even if the package is not +the latest according to the ports tree +.It Fl -always-fetch +fetch package even if it already exists locally +.It Fl -local-packagedir= +where local packages can be found, +will fall back to fetching if no local version exists .It Fl l list all installed ports by category .It Fl L From owner-svn-src-user@FreeBSD.ORG Mon Dec 7 00:15:39 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C76D8106566B; Mon, 7 Dec 2009 00:15:39 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 81BFA8FC18; Mon, 7 Dec 2009 00:15:39 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nB70Fdtl036023; Mon, 7 Dec 2009 00:15:39 GMT (envelope-from dougb@svn.freebsd.org) Received: (from dougb@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nB70FdhS036021; Mon, 7 Dec 2009 00:15:39 GMT (envelope-from dougb@svn.freebsd.org) Message-Id: <200912070015.nB70FdhS036021@svn.freebsd.org> From: Doug Barton Date: Mon, 7 Dec 2009 00:15:39 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r200197 - user/dougb/portmaster X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Dec 2009 00:15:39 -0000 Author: dougb Date: Mon Dec 7 00:15:39 2009 New Revision: 200197 URL: http://svn.freebsd.org/changeset/base/200197 Log: Minor whitespace twiddle Modified: user/dougb/portmaster/portmaster Modified: user/dougb/portmaster/portmaster ============================================================================== --- user/dougb/portmaster/portmaster Sun Dec 6 23:56:54 2009 (r200196) +++ user/dougb/portmaster/portmaster Mon Dec 7 00:15:39 2009 (r200197) @@ -2825,6 +2825,7 @@ if [ -z "$use_package" ]; then elif grep -q DEPORIGIN $pdb/$new_port/+CONTENTS; then echo "===>>> Updating dependencies for $new_port to match installed versions" update_contents $pdb/$new_port/+CONTENTS + echo '' fi check_dependency_files $portdir $new_port From owner-svn-src-user@FreeBSD.ORG Mon Dec 7 00:29:11 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 113EB106566B; Mon, 7 Dec 2009 00:29:11 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 012468FC12; Mon, 7 Dec 2009 00:29:11 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nB70TAhE036382; Mon, 7 Dec 2009 00:29:10 GMT (envelope-from dougb@svn.freebsd.org) Received: (from dougb@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nB70TA3R036380; Mon, 7 Dec 2009 00:29:10 GMT (envelope-from dougb@svn.freebsd.org) Message-Id: <200912070029.nB70TA3R036380@svn.freebsd.org> From: Doug Barton Date: Mon, 7 Dec 2009 00:29:10 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r200199 - user/dougb/portmaster X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Dec 2009 00:29:11 -0000 Author: dougb Date: Mon Dec 7 00:29:10 2009 New Revision: 200199 URL: http://svn.freebsd.org/changeset/base/200199 Log: If --local-packagedir and -v notify if no local package exists Modified: user/dougb/portmaster/portmaster Modified: user/dougb/portmaster/portmaster ============================================================================== --- user/dougb/portmaster/portmaster Mon Dec 7 00:22:10 2009 (r200198) +++ user/dougb/portmaster/portmaster Mon Dec 7 00:29:10 2009 (r200199) @@ -2499,6 +2499,9 @@ fetch_package () { local_package=${LOCAL_PACKAGEDIR}/Latest/${s}.tbz latest_pv=`readlink ${LOCAL_PACKAGEDIR}/Latest/${s}.tbz` latest_pv=${latest_pv##*/} + else + [ -n "$PM_VERBOSE" ] && + echo "===>>> No local package for $new_port exists, attempting fetch" fi fi From owner-svn-src-user@FreeBSD.ORG Mon Dec 7 01:46:40 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 32C6F1065672; Mon, 7 Dec 2009 01:46:40 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 17E158FC15; Mon, 7 Dec 2009 01:46:40 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nB71kdsJ038074; Mon, 7 Dec 2009 01:46:39 GMT (envelope-from dougb@svn.freebsd.org) Received: (from dougb@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nB71kdfa038072; Mon, 7 Dec 2009 01:46:39 GMT (envelope-from dougb@svn.freebsd.org) Message-Id: <200912070146.nB71kdfa038072@svn.freebsd.org> From: Doug Barton Date: Mon, 7 Dec 2009 01:46:39 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r200201 - user/dougb/portmaster X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Dec 2009 01:46:40 -0000 Author: dougb Date: Mon Dec 7 01:46:39 2009 New Revision: 200201 URL: http://svn.freebsd.org/changeset/base/200201 Log: The last commit to this file was unintentional. :) In this update: 1. Fix (sort of) the markup to avoid spurious -'es with the --packages* options. 2. Remove a trailing space 3. Detail that -PP has to be isolated on the command line 4. Give more detail about what --local-packagedir should point to 5. Describe how we use the --packages*-related environment vars 6. In the example .portmasterrc section: a. Add a value to the verbose example b. Add examples for the --packages* options 7. Add a Pp before the port_log description to match the others 8. Add some examples for the --packages* options 9. Add the Xr or pkg_add The previous update included the following: 1. Document the --packages* and other new options 2. Remove the SYNOPSIS for -p, and indicate that it's deprecated 3. Try to improve the older text in some places, mostly remove commas and shorten some sentences. 4. Add text to describe how the --packages* options interact with the older code. 5. --force-config and --show-work no longer has to be the first option listed 6. Describe how -x works when there is no installed version 7. Detail that +IGNOREME files only work with installed ports Modified: user/dougb/portmaster/portmaster.8 Modified: user/dougb/portmaster/portmaster.8 ============================================================================== --- user/dougb/portmaster/portmaster.8 Mon Dec 7 00:54:02 2009 (r200200) +++ user/dougb/portmaster/portmaster.8 Mon Dec 7 01:46:39 2009 (r200201) @@ -33,7 +33,7 @@ .Sh SYNOPSIS Common Flags: .Op Fl -force-config CGHgntvw [B|b] [f|i] [D|d] -.Op Fl [--packages|--packages-only] [-P|-PP] | [--packages-build] +.Op [--packages|--packages-only] [-P|-PP] | [--packages-build] .Op Fl -packages-if-newer .Op Fl -delete-build-only .Op Fl -always-fetch @@ -206,7 +206,7 @@ There are a number of options available to save the time that would normally be spent building the port(s). Users interested in a reasonable balance between speed of -installation and maximum performance should consider the +installation and maximum performance should consider the .Fl -packages-build option, perhaps combined with the .Fl -delete-build-only @@ -253,7 +253,7 @@ the runtime dependencies will be checked they are up to date. If the .Fl g -option is used, a package will be created for the new +option is used a package will be created for the new (or newly installed) version. .Pp When installing a port or using the @@ -373,7 +373,13 @@ only if installed this run .It Fl P|--packages use packages, but build port if not available .It Fl PP|--packages-only -fail if no package is available +fail if no package is available. +The +.Fl PP +option must stand alone on the command line. +In other words, you cannot do +.Fl PPav +(for example). .It Fl -packages-build use packages for all build dependencies .It Fl -packages-if-newer @@ -383,7 +389,22 @@ the latest according to the ports tree fetch package even if it already exists locally .It Fl -local-packagedir= where local packages can be found, -will fall back to fetching if no local version exists +will fall back to fetching if no local version exists. +This option should point to the full path of a directory structure +created in the same way that 'make package' (or the +.Nm +.Fl g +option) creates it. +I.e., the package files are contained in +.Pa /All , +there are LATEST_LINK symlinks in the +.Pa /Latest +directory, and symlinks to the packages in +.Pa /All +in the category subdirectories, such as +.Pa /devel , +.Pa /ports-mgmt , +etc. .It Fl l list all installed ports by category .It Fl L @@ -455,6 +476,22 @@ so that you can create both a backup pac a package of the newly installed port even if they have the same version. .Pp +When using the +.Fl -packages* +options the package files will be downloaded to +.Pa ${PACKAGES}/portmaster-download . +.Nm +will respect the +.Ev PACKAGESITE +and +.Ev PACKAGEROOT +(by default http://ftp.freebsd.org) variables. +.Nm +attempts to use both of these variables in the same +way that +.Xr pkg_add 1 +does. +.Pp The .Ev UPGRADE_TOOL variable is set to @@ -531,10 +568,34 @@ along with their related options. # RECURSE_THOROUGH=topt # # Be verbose (-v) -# PM_VERBOSE= +# PM_VERBOSE=vopt # # Save copies of old shared libraries (recommended) (-w) # SAVE_SHARED=wopt +# +# Install a package if available (-P or --packages) +# PM_PACKAGES=first +# +# Only install packages (-PP or --packages-only) +# PM_PACKAGES=only +# +# Install packages for build-only dependencies (--packages-build) +# PM_PACKAGES_BUILD=pmp_build +# PM_BUILD_ONLY_LIST=pm_bol +# +# Delete build-only dependencies when finished (--delete-build-only) +# PM_DEL_BUILD_ONLY=pm_dbo +# PM_BUILD_ONLY_LIST=pm_bol +# +# Use packages if they are newer than installed (--packages-newer) +# PM_PACKAGES=newer +# PM_PACKAGES_NEWER=pmp_newer +# +# Always fetch new package files (--always-fetch) +# PM_ALWAYS_FETCH=pm_always_fetch +# +# Specify a local package repository (--local-packagedir) +# LOCAL_PACKAGEDIR= .Ed .Pp .It Pa /var/db/pkg/*/+IGNOREME @@ -572,6 +633,7 @@ run which includes the .Fl R option that a port has already been rebuilt, so it can be safely ignored if it is up to date. +.Pp .It Pa /tmp/port_log-* If the .Fl H @@ -662,9 +724,19 @@ Update one port: .Dl "portmaster fooport" .Dl "portmaster foo/fooport" .Pp +Use a package if available: +.Dl "portmaster --packages fooport-1.23" +.Pp Update multiple ports: .Dl "portmaster fooport-1.23 barport baz/blahport" .Pp +Build a port locally but use packages for build dependencies, +then delete the build dependencies when finished: +.Dl "portmaster --packages-build --delete-build-only fooport-1.23" +.Pp +Update a system using only packages that are available locally: +.Dl "portmaster -PP --local-packagedir= -a" +.Pp Update all ports that need updating: .Dl "portmaster -a" .Pp @@ -716,6 +788,7 @@ to avoid rebuilding ports already rebuil However the first method (delete everything and reinstall) is preferred. .Sh SEE ALSO .Xr make 1 , +.Xr pkg_add 1 , .Xr pkg_delete 1 , .Xr su 1 , .Xr ports 7 , From owner-svn-src-user@FreeBSD.ORG Mon Dec 7 06:27:47 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7732C1065676; Mon, 7 Dec 2009 06:27:47 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 663398FC12; Mon, 7 Dec 2009 06:27:47 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nB76Rl5R043875; Mon, 7 Dec 2009 06:27:47 GMT (envelope-from dougb@svn.freebsd.org) Received: (from dougb@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nB76Rla5043873; Mon, 7 Dec 2009 06:27:47 GMT (envelope-from dougb@svn.freebsd.org) Message-Id: <200912070627.nB76Rla5043873@svn.freebsd.org> From: Doug Barton Date: Mon, 7 Dec 2009 06:27:47 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r200206 - user/dougb/portmaster X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Dec 2009 06:27:47 -0000 Author: dougb Date: Mon Dec 7 06:27:47 2009 New Revision: 200206 URL: http://svn.freebsd.org/changeset/base/200206 Log: If we get to the end of the search for a package and latest_pv is still not set, don't attempt to modify it. Modified: user/dougb/portmaster/portmaster Modified: user/dougb/portmaster/portmaster ============================================================================== --- user/dougb/portmaster/portmaster Mon Dec 7 06:25:19 2009 (r200205) +++ user/dougb/portmaster/portmaster Mon Dec 7 06:27:47 2009 (r200206) @@ -2528,12 +2528,13 @@ fetch_package () { echo " Check the pkg_add(1) man page for information" echo " on setting the PACKAGESITE environment variable" fail 'No package archive found' + else + latest_pv=${latest_pv#*href=\"} + latest_pv=${latest_pv%%\.tbz*} + case "$latest_pv" in + *%2[cC]*) latest_pv=`echo $latest_pv | sed s#%2[cC]#,#` ;; + esac fi - latest_pv=${latest_pv#*href=\"} - latest_pv=${latest_pv%%\.tbz*} - case "$latest_pv" in - *%2[cC]*) latest_pv=`echo $latest_pv | sed s#%2[cC]#,#` ;; - esac notnewer () { echo '' From owner-svn-src-user@FreeBSD.ORG Tue Dec 8 01:29:53 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5347E106566C; Tue, 8 Dec 2009 01:29:53 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 42A0B8FC13; Tue, 8 Dec 2009 01:29:53 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nB81TrFJ071727; Tue, 8 Dec 2009 01:29:53 GMT (envelope-from dougb@svn.freebsd.org) Received: (from dougb@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nB81TrwV071725; Tue, 8 Dec 2009 01:29:53 GMT (envelope-from dougb@svn.freebsd.org) Message-Id: <200912080129.nB81TrwV071725@svn.freebsd.org> From: Doug Barton Date: Tue, 8 Dec 2009 01:29:53 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r200244 - user/dougb/portmaster X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Dec 2009 01:29:53 -0000 Author: dougb Date: Tue Dec 8 01:29:52 2009 New Revision: 200244 URL: http://svn.freebsd.org/changeset/base/200244 Log: 1. Make the output of --version more clear to everyone who isn't me :) 2. If DISTDIR is its own zfs filesystem them the code to delete empty DIST_SUBDIRs can cause problems, so special case it in its own function. 3. Don't unset the packages_init function till we're actually done using it 4. If you are using -g plus one of the --packages* options then creation of the new package will fail since there is nothing for 'make package' to work with. So update backup_package() to a more generic form called pm_pkg_create and use that for both making backup packages and making a new local package from an installed package. 5. In order to facilitate 4, update the dependency information in an installed package's +CONTENTS file before creating the new package. 6. Remove some (spurious?) white space Modified: user/dougb/portmaster/portmaster Modified: user/dougb/portmaster/portmaster ============================================================================== --- user/dougb/portmaster/portmaster Tue Dec 8 01:07:44 2009 (r200243) +++ user/dougb/portmaster/portmaster Tue Dec 8 01:29:52 2009 (r200244) @@ -50,7 +50,7 @@ version () { echo '' [ "${cvs#$}" != 'FreeBSD$' ] && - { echo "===>>> FreeBSD version $cvs" ; return 0; } + { echo "===>>> Version $cvs" ; return 0; } echo "===>>> Development version $rcs" } @@ -453,7 +453,6 @@ done set -- $newopts unset var newopts -unset -f packages_init #=============== Begin functions relevant to --features and main =============== @@ -745,6 +744,13 @@ ports_by_category () { done } +delete_empty_dist_subdirs () { + # Get back to somewhere safe so we do not + # delete our CWD out from under ourselves + pm_cd $distdir || fail "Cannot cd into $distdir" + find -d $distdir -type d \( -empty -and ! -path \*\.zfs/\* \) -delete +} + #=============== End functions relevant to --features and main =============== #=============== Begin code relevant only to --features =============== @@ -771,7 +777,7 @@ if [ -n "$CLEAN_DISTFILES" ]; then fi done - find -d $distdir -type d -empty -delete + delete_empty_dist_subdirs safe_exit fi @@ -936,6 +942,7 @@ while getopts 'BCDFGHKLPRabde:fghilm:nop esac done shift $(( $OPTIND - 1 )) +unset -f packages_init [ -n "$PM_EXCL" ] && export PM_EXCL @@ -1112,20 +1119,42 @@ init_packages () { export packages pbu } -backup_package () { - echo "===>>> Creating a backup package for old version $1" - pm_cd $pbu || fail "Cannot cd into $pbu to create a package" - if $PM_SU_CMD pkg_create -b $1; then - if [ -n "$BACKUP" ]; then - echo " ===>>> Package saved to $pbu" - echo '' - else - local pkg ; pkg=`echo $1.*` - NB_DELETE="${NB_DELETE}${pkg} " +pm_pkg_create () { + local pkgdir + + if [ "$1" = "${packages}" ]; then + pm_mkdir_s ${1}/All ${1}/Latest ${1}/${portdir%/*} + pkgdir=${packages}/All + echo "===>>> Creating a package for new version $2" + else + pkgdir=$1 + echo "===>>> Creating a backup package for old version $2" + fi + + pm_cd $pkgdir || fail "Cannot cd into $pkgdir to create a package" + if $PM_SU_CMD pkg_create -b $2; then + if [ "$1" = "$pbu" ]; then + if [ -n "$BACKUP" ]; then + echo " ===>>> Package saved to $1" ; echo '' + else + local pkg ; pkg=`echo $2.*` + NB_DELETE="${NB_DELETE}${pkg} " + fi + elif [ "$1" = "${packages}" ]; then + local pkg latest_link + + pkg=`echo $2.*` + pm_cd $pd/$portdir + latest_link=`pm_make -V LATEST_LINK` + pm_cd ${1}/Latest + ln -sf ../All/$pkg ${latest_link}.tbz + cd ${1}/${portdir%/*} + ln -sf ../All/$pkg $pkg + echo " ===>>> Package saved to ${1}/All" ; echo '' fi else echo '' - echo "===>>> Backup package creation failed for ${1}!" + echo "===>>> Package creation failed for ${2}!" echo '' echo "===>>> Ignore this error [i]" echo "===>>> Abort [a]" @@ -1133,7 +1162,7 @@ backup_package () { echo -n "===>>> How would you like to proceed? [i] " local answer ; read answer case "$answer" in - a) fail "Backup package creation failed for $1" ;; + a) fail "Package creation failed for $2" ;; esac fi } @@ -1378,12 +1407,7 @@ delete_all_distfiles () { [ -n "$distfiles" ] && eval rm -f $distfiles fi delete_stale_distfiles $origin - - # Get back to somewhere safe so we do not - # delete our CWD out from under ourselves - pm_cd $distdir || fail "Cannot cd into $distdir" - - find -d $distdir -type d -empty -delete + delete_empty_dist_subdirs } #=============== End functions for getopts features and main =============== @@ -1461,7 +1485,7 @@ if [ -n "$EXPUNGE" ]; then esac fi - [ -n "$BACKUP" ] && { init_packages ; backup_package $EXPUNGE; } + [ -n "$BACKUP" ] && { init_packages ; pm_pkg_create $pbu $EXPUNGE; } [ -z "$DONT_SCRUB_DISTFILES" ] && delete_all_distfiles $origin delete_dist_list @@ -1502,7 +1526,7 @@ if [ -n "$CLEAN_STALE" ]; then case "$answer" in [yY]) if [ -n "$BACKUP" ]; then [ -z "$packages" ] && init_packages - backup_package $iport + pm_pkg_create $pbu $iport fi [ -z "$DONT_SCRUB_DISTFILES" ] && @@ -2657,7 +2681,7 @@ if [ -n "$upg_port" ]; then UPGRADE_PORT_VER=`echo $UPGRADE_PORT | sed 's#.*-\(.*\)#\1#'` export UPGRADE_PORT UPGRADE_PORT_VER - [ -z "$NO_BACKUP" ] && backup_package $upg_port + [ -z "$NO_BACKUP" ] && pm_pkg_create $pbu $upg_port if [ -n "$SAVE_SHARED" ]; then ldconfig_out=`pm_mktemp ldconfig` @@ -2718,7 +2742,6 @@ install_failed () { fail "Installation of $1 ($portdir) failed" } -echo '' if [ -z "$use_package" ]; then [ -n "$PM_SU_VERBOSE" ] && echo "===>>> Running make install" @@ -2816,20 +2839,28 @@ if [ -n "$distfiles" ]; then pm_unlink $dist_list_temp ; unset ds dist_list_temp fi +if [ -n "$use_package" ]; then + if grep -q DEPORIGIN $pdb/$new_port/+CONTENTS; then +echo "===>>> Updating dependencies for $new_port to match installed versions" + update_contents $pdb/$new_port/+CONTENTS + echo '' + fi +fi + if [ -n "$MAKE_PACKAGE" ]; then - echo "===>>> Creating a package for new version $new_port" - pm_make_s package >/dev/null || - fail "Package creation of $new_port failed" - echo " ===>>> Package saved to $packages/All" ; echo '' + if [ -z "$use_package" ]; then + echo "===>>> Creating a package for new version $new_port" + pm_make_s package >/dev/null || + fail "Package creation of $new_port failed" + echo " ===>>> Package saved to $packages/All" ; echo '' + else + pm_pkg_create $packages $new_port + fi fi if [ -z "$use_package" ]; then [ -z "$DONT_POST_CLEAN" ] && { pm_make clean NOCLEANDEPENDS=ncd2 ; echo ''; } -elif grep -q DEPORIGIN $pdb/$new_port/+CONTENTS; then -echo "===>>> Updating dependencies for $new_port to match installed versions" - update_contents $pdb/$new_port/+CONTENTS - echo '' fi check_dependency_files $portdir $new_port From owner-svn-src-user@FreeBSD.ORG Tue Dec 8 01:57:00 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 532101065672; Tue, 8 Dec 2009 01:57:00 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 427328FC0A; Tue, 8 Dec 2009 01:57:00 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nB81v0DQ072353; Tue, 8 Dec 2009 01:57:00 GMT (envelope-from dougb@svn.freebsd.org) Received: (from dougb@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nB81v0Rs072351; Tue, 8 Dec 2009 01:57:00 GMT (envelope-from dougb@svn.freebsd.org) Message-Id: <200912080157.nB81v0Rs072351@svn.freebsd.org> From: Doug Barton Date: Tue, 8 Dec 2009 01:57:00 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r200245 - user/dougb/portmaster X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Dec 2009 01:57:00 -0000 Author: dougb Date: Tue Dec 8 01:56:59 2009 New Revision: 200245 URL: http://svn.freebsd.org/changeset/base/200245 Log: With the new parser if you use -v with --check-port-dbdir it's going to be $1, not $2. Include the [-v] in the detailed usage() for --check-port-dbdir too Modified: user/dougb/portmaster/portmaster Modified: user/dougb/portmaster/portmaster ============================================================================== --- user/dougb/portmaster/portmaster Tue Dec 8 01:29:52 2009 (r200244) +++ user/dougb/portmaster/portmaster Tue Dec 8 01:56:59 2009 (r200245) @@ -331,7 +331,7 @@ usage () { echo '' echo '--check-depends cross-check and update dependency information for all ports' echo '' - echo "--check-port-dbdir check for stale entries in $port_dbdir" + echo "--check-port-dbdir [-v] check for stale entries in $port_dbdir" echo '' echo "--list-origins list directories from $pd for root and leaf ports" echo '' @@ -835,7 +835,7 @@ IFS=' fi if [ -n "$CHECK_PORT_DBDIR" ]; then - if [ "$2" = "-v" ]; then PM_VERBOSE=vopt; fi + if [ "$1" = "-v" ]; then PM_VERBOSE=vopt; fi unique_list=':' From owner-svn-src-user@FreeBSD.ORG Tue Dec 8 13:25:01 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C44C1106566B; Tue, 8 Dec 2009 13:25:01 +0000 (UTC) (envelope-from eri@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id AF1918FC1B; Tue, 8 Dec 2009 13:25:01 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nB8DP17e089117; Tue, 8 Dec 2009 13:25:01 GMT (envelope-from eri@svn.freebsd.org) Received: (from eri@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nB8DP1s9089088; Tue, 8 Dec 2009 13:25:01 GMT (envelope-from eri@svn.freebsd.org) Message-Id: <200912081325.nB8DP1s9089088@svn.freebsd.org> From: Ermal Luçi Date: Tue, 8 Dec 2009 13:25:01 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r200256 - in user/eri/pf45/head: . bin/date contrib/bind9/lib/isc/ia64/include/isc contrib/bsnmp/snmp_mibII contrib/gcc/config contrib/groff/tmac crypto/openssl/ssl etc etc/defaults etc... X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Dec 2009 13:25:01 -0000 Author: eri Date: Tue Dec 8 13:25:00 2009 New Revision: 200256 URL: http://svn.freebsd.org/changeset/base/200256 Log: MFH to include netstat fixes. Added: user/eri/pf45/head/lib/csu/i386-elf/crt1_c.c - copied unchanged from r200254, head/lib/csu/i386-elf/crt1_c.c user/eri/pf45/head/lib/csu/i386-elf/crt1_s.S - copied unchanged from r200254, head/lib/csu/i386-elf/crt1_s.S user/eri/pf45/head/lib/libulog/ - copied from r200254, head/lib/libulog/ user/eri/pf45/head/libexec/ulog-helper/ - copied from r200254, head/libexec/ulog-helper/ user/eri/pf45/head/share/man/man4/man4.powerpc/adb.4 - copied unchanged from r200254, head/share/man/man4/man4.powerpc/adb.4 user/eri/pf45/head/share/man/man4/man4.powerpc/akbd.4 - copied unchanged from r200254, head/share/man/man4/man4.powerpc/akbd.4 user/eri/pf45/head/share/man/man4/man4.powerpc/ams.4 - copied unchanged from r200254, head/share/man/man4/man4.powerpc/ams.4 user/eri/pf45/head/share/man/man4/man4.powerpc/cuda.4 - copied unchanged from r200254, head/share/man/man4/man4.powerpc/cuda.4 user/eri/pf45/head/sys/compat/linux/linux_videodev.h - copied unchanged from r200254, head/sys/compat/linux/linux_videodev.h user/eri/pf45/head/sys/compat/linux/linux_videodev_compat.h - copied unchanged from r200254, head/sys/compat/linux/linux_videodev_compat.h Deleted: user/eri/pf45/head/etc/rc.d/ip6fw user/eri/pf45/head/etc/rc.firewall6 user/eri/pf45/head/lib/csu/i386-elf/crt1.c Modified: user/eri/pf45/head/MAINTAINERS user/eri/pf45/head/ObsoleteFiles.inc user/eri/pf45/head/bin/date/Makefile user/eri/pf45/head/bin/date/date.c user/eri/pf45/head/contrib/bind9/lib/isc/ia64/include/isc/atomic.h user/eri/pf45/head/contrib/bsnmp/snmp_mibII/BEGEMOT-MIB2-MIB.txt user/eri/pf45/head/contrib/bsnmp/snmp_mibII/mibII.c user/eri/pf45/head/contrib/bsnmp/snmp_mibII/mibII.h user/eri/pf45/head/contrib/bsnmp/snmp_mibII/mibII_begemot.c user/eri/pf45/head/contrib/bsnmp/snmp_mibII/mibII_tree.def user/eri/pf45/head/contrib/gcc/config/freebsd-spec.h user/eri/pf45/head/contrib/groff/tmac/doc-syms user/eri/pf45/head/crypto/openssl/ssl/s3_lib.c user/eri/pf45/head/crypto/openssl/ssl/s3_pkt.c user/eri/pf45/head/crypto/openssl/ssl/s3_srvr.c user/eri/pf45/head/etc/Makefile user/eri/pf45/head/etc/defaults/rc.conf user/eri/pf45/head/etc/mtree/BSD.var.dist user/eri/pf45/head/etc/rc.d/Makefile user/eri/pf45/head/etc/rc.d/ipfw user/eri/pf45/head/etc/rc.firewall user/eri/pf45/head/lib/Makefile user/eri/pf45/head/lib/csu/amd64/Makefile user/eri/pf45/head/lib/csu/arm/Makefile user/eri/pf45/head/lib/csu/i386-elf/Makefile user/eri/pf45/head/lib/csu/ia64/Makefile user/eri/pf45/head/lib/csu/mips/Makefile user/eri/pf45/head/lib/csu/powerpc/Makefile user/eri/pf45/head/lib/csu/sparc64/Makefile user/eri/pf45/head/lib/libc/gen/exec.c user/eri/pf45/head/lib/libc/gen/fdevname.c user/eri/pf45/head/lib/libc/gen/getlogin.c user/eri/pf45/head/lib/libc/gen/getttyent.c user/eri/pf45/head/lib/libc/gen/nlist.c user/eri/pf45/head/lib/libc/gen/pause.c user/eri/pf45/head/lib/libc/gen/pwcache.c user/eri/pf45/head/lib/libc/gen/raise.c user/eri/pf45/head/lib/libc/gen/sleep.c user/eri/pf45/head/lib/libc/gen/termios.c user/eri/pf45/head/lib/libc/gen/timezone.c user/eri/pf45/head/lib/libc/gen/usleep.c user/eri/pf45/head/lib/libc/gmon/gmon.c user/eri/pf45/head/lib/libc/nls/es_ES.ISO8859-1.msg user/eri/pf45/head/lib/libc/posix1e/acl_to_text.c user/eri/pf45/head/lib/libc/rpc/svc.c user/eri/pf45/head/lib/libc/stdio/findfp.c user/eri/pf45/head/lib/libc/stdio/funopen.c user/eri/pf45/head/lib/libc/stdio/printf.3 user/eri/pf45/head/lib/libc/stdlib/getenv.3 user/eri/pf45/head/lib/libc/stdlib/getenv.c user/eri/pf45/head/lib/libc/stdlib/system.c user/eri/pf45/head/lib/libc/string/strcmp.3 user/eri/pf45/head/lib/libc/sys/__error.c user/eri/pf45/head/lib/libutil/libutil.h user/eri/pf45/head/libexec/Makefile user/eri/pf45/head/release/i386/fixit_crunch.conf user/eri/pf45/head/release/pc98/fixit-small_crunch.conf user/eri/pf45/head/release/pc98/fixit_crunch.conf user/eri/pf45/head/rescue/rescue/Makefile user/eri/pf45/head/sbin/atacontrol/atacontrol.c user/eri/pf45/head/sbin/init/Makefile user/eri/pf45/head/sbin/init/init.c user/eri/pf45/head/sbin/ipfw/dummynet.c user/eri/pf45/head/sbin/ipfw/ipfw2.c user/eri/pf45/head/sbin/reboot/Makefile user/eri/pf45/head/sbin/reboot/reboot.c user/eri/pf45/head/share/man/man4/man4.powerpc/Makefile user/eri/pf45/head/share/man/man4/man4.powerpc/pmu.4 user/eri/pf45/head/share/man/man9/VOP_OPENCLOSE.9 user/eri/pf45/head/share/mk/bsd.libnames.mk user/eri/pf45/head/share/mk/bsd.subdir.mk user/eri/pf45/head/sys/amd64/amd64/mca.c user/eri/pf45/head/sys/amd64/include/mca.h user/eri/pf45/head/sys/arm/conf/AVILA user/eri/pf45/head/sys/arm/conf/CAMBRIA user/eri/pf45/head/sys/arm/mv/mv_sata.c user/eri/pf45/head/sys/boot/i386/libi386/biosmem.c user/eri/pf45/head/sys/boot/i386/libi386/libi386.h user/eri/pf45/head/sys/boot/i386/loader/main.c user/eri/pf45/head/sys/boot/pc98/btx/btx/btx.S user/eri/pf45/head/sys/boot/pc98/libpc98/biosmem.c user/eri/pf45/head/sys/boot/pc98/loader/main.c user/eri/pf45/head/sys/cam/ata/ata_all.c user/eri/pf45/head/sys/cam/ata/ata_all.h user/eri/pf45/head/sys/cam/ata/ata_da.c user/eri/pf45/head/sys/cam/ata/ata_pmp.c user/eri/pf45/head/sys/cam/ata/ata_xpt.c user/eri/pf45/head/sys/cam/cam_periph.h user/eri/pf45/head/sys/cam/scsi/scsi_cd.c user/eri/pf45/head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c user/eri/pf45/head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c user/eri/pf45/head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zvol.c user/eri/pf45/head/sys/compat/freebsd32/freebsd32_proto.h user/eri/pf45/head/sys/compat/freebsd32/freebsd32_syscall.h user/eri/pf45/head/sys/compat/freebsd32/freebsd32_syscalls.c user/eri/pf45/head/sys/compat/freebsd32/freebsd32_sysent.c user/eri/pf45/head/sys/compat/freebsd32/syscalls.master user/eri/pf45/head/sys/compat/linux/linux_ioctl.c user/eri/pf45/head/sys/compat/linux/linux_ioctl.h user/eri/pf45/head/sys/conf/NOTES user/eri/pf45/head/sys/conf/options user/eri/pf45/head/sys/dev/aac/aac.c user/eri/pf45/head/sys/dev/aac/aac_cam.c user/eri/pf45/head/sys/dev/aac/aacvar.h user/eri/pf45/head/sys/dev/ahci/ahci.c user/eri/pf45/head/sys/dev/ata/ata-all.c user/eri/pf45/head/sys/dev/ata/ata-all.h user/eri/pf45/head/sys/dev/ata/ata-disk.c user/eri/pf45/head/sys/dev/ata/ata-dma.c user/eri/pf45/head/sys/dev/ata/ata-lowlevel.c user/eri/pf45/head/sys/dev/ata/ata-pci.c user/eri/pf45/head/sys/dev/ata/ata-pci.h user/eri/pf45/head/sys/dev/ata/ata-queue.c user/eri/pf45/head/sys/dev/ata/ata-sata.c user/eri/pf45/head/sys/dev/ata/ata_if.m user/eri/pf45/head/sys/dev/ata/atapi-cd.c user/eri/pf45/head/sys/dev/ata/atapi-fd.c user/eri/pf45/head/sys/dev/ata/atapi-tape.c user/eri/pf45/head/sys/dev/ata/chipsets/ata-acard.c user/eri/pf45/head/sys/dev/ata/chipsets/ata-acerlabs.c user/eri/pf45/head/sys/dev/ata/chipsets/ata-ahci.c user/eri/pf45/head/sys/dev/ata/chipsets/ata-amd.c user/eri/pf45/head/sys/dev/ata/chipsets/ata-ati.c user/eri/pf45/head/sys/dev/ata/chipsets/ata-cenatek.c user/eri/pf45/head/sys/dev/ata/chipsets/ata-cypress.c user/eri/pf45/head/sys/dev/ata/chipsets/ata-cyrix.c user/eri/pf45/head/sys/dev/ata/chipsets/ata-highpoint.c user/eri/pf45/head/sys/dev/ata/chipsets/ata-intel.c user/eri/pf45/head/sys/dev/ata/chipsets/ata-ite.c user/eri/pf45/head/sys/dev/ata/chipsets/ata-jmicron.c user/eri/pf45/head/sys/dev/ata/chipsets/ata-marvell.c user/eri/pf45/head/sys/dev/ata/chipsets/ata-micron.c user/eri/pf45/head/sys/dev/ata/chipsets/ata-national.c user/eri/pf45/head/sys/dev/ata/chipsets/ata-netcell.c user/eri/pf45/head/sys/dev/ata/chipsets/ata-nvidia.c user/eri/pf45/head/sys/dev/ata/chipsets/ata-promise.c user/eri/pf45/head/sys/dev/ata/chipsets/ata-serverworks.c user/eri/pf45/head/sys/dev/ata/chipsets/ata-siliconimage.c user/eri/pf45/head/sys/dev/ata/chipsets/ata-sis.c user/eri/pf45/head/sys/dev/ata/chipsets/ata-via.c user/eri/pf45/head/sys/dev/bge/if_bge.c user/eri/pf45/head/sys/dev/cxgb/common/cxgb_common.h user/eri/pf45/head/sys/dev/cxgb/cxgb_t3fw.h user/eri/pf45/head/sys/dev/e1000/LICENSE user/eri/pf45/head/sys/dev/e1000/e1000_80003es2lan.c user/eri/pf45/head/sys/dev/e1000/e1000_80003es2lan.h user/eri/pf45/head/sys/dev/e1000/e1000_82541.c user/eri/pf45/head/sys/dev/e1000/e1000_82571.c user/eri/pf45/head/sys/dev/e1000/e1000_82575.c user/eri/pf45/head/sys/dev/e1000/e1000_82575.h user/eri/pf45/head/sys/dev/e1000/e1000_api.c user/eri/pf45/head/sys/dev/e1000/e1000_defines.h user/eri/pf45/head/sys/dev/e1000/e1000_hw.h user/eri/pf45/head/sys/dev/e1000/e1000_ich8lan.c user/eri/pf45/head/sys/dev/e1000/e1000_ich8lan.h user/eri/pf45/head/sys/dev/e1000/e1000_mac.c user/eri/pf45/head/sys/dev/e1000/e1000_manage.c user/eri/pf45/head/sys/dev/e1000/e1000_osdep.h user/eri/pf45/head/sys/dev/e1000/e1000_phy.c user/eri/pf45/head/sys/dev/e1000/e1000_phy.h user/eri/pf45/head/sys/dev/e1000/e1000_regs.h user/eri/pf45/head/sys/dev/e1000/if_em.c user/eri/pf45/head/sys/dev/e1000/if_em.h user/eri/pf45/head/sys/dev/e1000/if_igb.c user/eri/pf45/head/sys/dev/e1000/if_igb.h user/eri/pf45/head/sys/dev/hwpmc/hwpmc_x86.c user/eri/pf45/head/sys/dev/ichsmb/ichsmb_pci.c user/eri/pf45/head/sys/dev/if_ndis/if_ndis.c user/eri/pf45/head/sys/dev/iir/iir_ctrl.c user/eri/pf45/head/sys/dev/isp/isp_freebsd.c user/eri/pf45/head/sys/dev/isp/isp_freebsd.h user/eri/pf45/head/sys/dev/ixgbe/ixgbe.c user/eri/pf45/head/sys/dev/ixgbe/ixgbe.h user/eri/pf45/head/sys/dev/ixgbe/ixgbe_82598.c user/eri/pf45/head/sys/dev/ixgbe/ixgbe_82599.c user/eri/pf45/head/sys/dev/ixgbe/ixgbe_api.c user/eri/pf45/head/sys/dev/ixgbe/ixgbe_api.h user/eri/pf45/head/sys/dev/ixgbe/ixgbe_common.c user/eri/pf45/head/sys/dev/ixgbe/ixgbe_common.h user/eri/pf45/head/sys/dev/ixgbe/ixgbe_osdep.h user/eri/pf45/head/sys/dev/ixgbe/ixgbe_phy.c user/eri/pf45/head/sys/dev/ixgbe/ixgbe_phy.h user/eri/pf45/head/sys/dev/ixgbe/ixgbe_type.h user/eri/pf45/head/sys/dev/puc/pucdata.c user/eri/pf45/head/sys/dev/siis/siis.c user/eri/pf45/head/sys/dev/siis/siis.h user/eri/pf45/head/sys/dev/uart/uart_bus_pci.c user/eri/pf45/head/sys/dev/usb/controller/ehci_pci.c user/eri/pf45/head/sys/dev/usb/controller/uhci_pci.c user/eri/pf45/head/sys/dev/usb/input/atp.c user/eri/pf45/head/sys/fs/nfs/nfs_commonacl.c user/eri/pf45/head/sys/fs/nfs/nfs_commonport.c user/eri/pf45/head/sys/fs/nfs/nfs_commonsubs.c user/eri/pf45/head/sys/fs/nfs/nfs_var.h user/eri/pf45/head/sys/fs/nfs/nfsport.h user/eri/pf45/head/sys/fs/nfsclient/nfs_clrpcops.c user/eri/pf45/head/sys/fs/nfsclient/nfs_clvnops.c user/eri/pf45/head/sys/fs/ntfs/ntfs.h user/eri/pf45/head/sys/fs/ntfs/ntfs_subr.c user/eri/pf45/head/sys/fs/ntfs/ntfs_vfsops.c user/eri/pf45/head/sys/fs/portalfs/portal_vnops.c user/eri/pf45/head/sys/geom/mirror/g_mirror.c user/eri/pf45/head/sys/geom/mirror/g_mirror.h user/eri/pf45/head/sys/gnu/fs/reiserfs/reiserfs_vfsops.c user/eri/pf45/head/sys/i386/conf/NOTES user/eri/pf45/head/sys/i386/i386/mca.c user/eri/pf45/head/sys/i386/include/mca.h user/eri/pf45/head/sys/ia64/ia64/clock.c user/eri/pf45/head/sys/ia64/ia64/exception.S user/eri/pf45/head/sys/ia64/ia64/genassym.c user/eri/pf45/head/sys/ia64/ia64/interrupt.c user/eri/pf45/head/sys/ia64/ia64/machdep.c user/eri/pf45/head/sys/ia64/ia64/mp_machdep.c user/eri/pf45/head/sys/ia64/ia64/pmap.c user/eri/pf45/head/sys/ia64/include/bus.h user/eri/pf45/head/sys/ia64/include/cpufunc.h user/eri/pf45/head/sys/ia64/include/ia64_cpu.h user/eri/pf45/head/sys/ia64/include/kdb.h user/eri/pf45/head/sys/ia64/include/param.h user/eri/pf45/head/sys/ia64/include/pcpu.h user/eri/pf45/head/sys/ia64/include/pmap.h user/eri/pf45/head/sys/kern/kern_sig.c user/eri/pf45/head/sys/kern/vfs_acl.c user/eri/pf45/head/sys/net80211/ieee80211_hostap.c user/eri/pf45/head/sys/netinet/in.h user/eri/pf45/head/sys/netinet/ip_carp.c user/eri/pf45/head/sys/netinet/ip_fw.h user/eri/pf45/head/sys/netinet/ipfw/ip_dummynet.c user/eri/pf45/head/sys/netinet/ipfw/ip_fw2.c user/eri/pf45/head/sys/netinet/raw_ip.c user/eri/pf45/head/sys/nfsserver/nfs_serv.c user/eri/pf45/head/sys/pc98/conf/NOTES user/eri/pf45/head/sys/powerpc/aim/machdep.c user/eri/pf45/head/sys/powerpc/include/vmparam.h user/eri/pf45/head/sys/powerpc/powermac/ata_dbdma.c user/eri/pf45/head/sys/powerpc/powermac/ata_kauai.c user/eri/pf45/head/sys/powerpc/powermac/ata_macio.c user/eri/pf45/head/sys/powerpc/psim/ata_iobus.c user/eri/pf45/head/sys/sparc64/sparc64/trap.c user/eri/pf45/head/sys/sys/ata.h user/eri/pf45/head/sys/sys/param.h user/eri/pf45/head/sys/vm/uma_int.h user/eri/pf45/head/tools/regression/bin/sh/parser/for1.0 user/eri/pf45/head/tools/tools/sysbuild/sysbuild.sh user/eri/pf45/head/usr.bin/Makefile user/eri/pf45/head/usr.bin/make/proc.c user/eri/pf45/head/usr.bin/systat/Makefile user/eri/pf45/head/usr.bin/systat/vmstat.c user/eri/pf45/head/usr.bin/users/Makefile user/eri/pf45/head/usr.bin/users/users.c user/eri/pf45/head/usr.bin/w/Makefile user/eri/pf45/head/usr.bin/w/w.c user/eri/pf45/head/usr.bin/wall/Makefile user/eri/pf45/head/usr.bin/wall/wall.c user/eri/pf45/head/usr.bin/who/Makefile user/eri/pf45/head/usr.bin/who/who.c user/eri/pf45/head/usr.bin/write/Makefile user/eri/pf45/head/usr.bin/write/write.c user/eri/pf45/head/usr.sbin/bsnmpd/modules/snmp_pf/pf_snmp.c user/eri/pf45/head/usr.sbin/freebsd-update/freebsd-update.sh user/eri/pf45/head/usr.sbin/mountd/exports.5 user/eri/pf45/head/usr.sbin/nfsd/nfsv4.4 user/eri/pf45/head/usr.sbin/nfsd/stablerestart.5 user/eri/pf45/head/usr.sbin/pw/pw_user.c user/eri/pf45/head/usr.sbin/repquota/repquota.c user/eri/pf45/head/usr.sbin/ypserv/yp_main.c user/eri/pf45/head/usr.sbin/ypserv/yp_server.c Directory Properties: user/eri/pf45/head/ (props changed) Modified: user/eri/pf45/head/MAINTAINERS ============================================================================== --- user/eri/pf45/head/MAINTAINERS Tue Dec 8 13:06:35 2009 (r200255) +++ user/eri/pf45/head/MAINTAINERS Tue Dec 8 13:25:00 2009 (r200256) @@ -124,6 +124,8 @@ usr.bin/bluetooth emax Pre-commit review usr.sbin/bluetooth emax Pre-commit review preferred. gnu/usr.bin/send-pr bugmaster Pre-commit review requested. ncurses rafan Heads-up appreciated, try not to break it. +*env(3) secteam Due to the problematic security history of this + code, please have patches reviewed by secteam. Following are the entries from the Makefiles, and a few other sources. Please remove stale entries from both their origin, and this file. Modified: user/eri/pf45/head/ObsoleteFiles.inc ============================================================================== --- user/eri/pf45/head/ObsoleteFiles.inc Tue Dec 8 13:06:35 2009 (r200255) +++ user/eri/pf45/head/ObsoleteFiles.inc Tue Dec 8 13:25:00 2009 (r200256) @@ -14,8 +14,14 @@ # The file is partitioned: OLD_FILES first, then OLD_LIBS and OLD_DIRS last. # +# 20091202: unify rc.firewall and rc.firewall6. +OLD_FILES+=etc/rc.d/ip6fw +OLD_FILES+=etc/rc.firewall6 +OLD_FILES+=usr/share/examples/etc/rc.firewall6 # 20091117: removal of rc.early(8) link OLD_FILES+=usr/share/man/man8/rc.early.8.gz +# 20091117: usr/share/zoneinfo/GMT link removed +OLD_FILES+=usr/share/zoneinfo/GMT # 20091027: pselect.3 implemented as syscall OLD_FILES+=usr/share/man/man3/pselect.3.gz # 20091005: fusword.9 and susword.9 removed Modified: user/eri/pf45/head/bin/date/Makefile ============================================================================== --- user/eri/pf45/head/bin/date/Makefile Tue Dec 8 13:06:35 2009 (r200255) +++ user/eri/pf45/head/bin/date/Makefile Tue Dec 8 13:25:00 2009 (r200256) @@ -3,7 +3,7 @@ PROG= date SRCS= date.c netdate.c vary.c -DPADD= ${LIBUTIL} -LDADD= -lutil +DPADD= ${LIBULOG} +LDADD= -lulog .include Modified: user/eri/pf45/head/bin/date/date.c ============================================================================== --- user/eri/pf45/head/bin/date/date.c Tue Dec 8 13:06:35 2009 (r200255) +++ user/eri/pf45/head/bin/date/date.c Tue Dec 8 13:25:00 2009 (r200256) @@ -48,11 +48,12 @@ __FBSDID("$FreeBSD$"); #include #include #include -#include #include #include #include #include +#define _ULOG_POSIX_NAMES +#include #include #include "extern.h" @@ -181,6 +182,7 @@ main(int argc, char *argv[]) static void setthetime(const char *fmt, const char *p, int jflag, int nflag) { + struct utmpx utx; struct tm *lt; struct timeval tv; const char *dot, *t; @@ -271,12 +273,16 @@ setthetime(const char *fmt, const char * if (!jflag) { /* set the time */ if (nflag || netsettime(tval)) { - logwtmp("|", "date", ""); + utx.ut_type = OLD_TIME; + gettimeofday(&utx.ut_tv, NULL); + pututxline(&utx); tv.tv_sec = tval; tv.tv_usec = 0; if (settimeofday(&tv, (struct timezone *)NULL)) err(1, "settimeofday (timeval)"); - logwtmp("{", "date", ""); + utx.ut_type = NEW_TIME; + gettimeofday(&utx.ut_tv, NULL); + pututxline(&utx); } if ((p = getlogin()) == NULL) Modified: user/eri/pf45/head/contrib/bind9/lib/isc/ia64/include/isc/atomic.h ============================================================================== --- user/eri/pf45/head/contrib/bind9/lib/isc/ia64/include/isc/atomic.h Tue Dec 8 13:06:35 2009 (r200255) +++ user/eri/pf45/head/contrib/bind9/lib/isc/ia64/include/isc/atomic.h Tue Dec 8 13:25:00 2009 (r200256) @@ -41,7 +41,7 @@ isc_atomic_xadd(isc_int32_t *p, isc_int3 for (prev = *(volatile isc_int32_t *)p; ; prev = swapped) { swapped = prev + val; __asm__ volatile( - "mov ar.ccv=%2;" + "mov ar.ccv=%2;;" "cmpxchg4.acq %0=%4,%3,ar.ccv" : "=r" (swapped), "=m" (*p) : "r" (prev), "r" (swapped), "m" (*p) @@ -84,7 +84,7 @@ isc_atomic_cmpxchg(isc_int32_t *p, isc_i isc_int32_t ret; __asm__ volatile( - "mov ar.ccv=%2;" + "mov ar.ccv=%2;;" "cmpxchg4.acq %0=%4,%3,ar.ccv" : "=r" (ret), "=m" (*p) : "r" (cmpval), "r" (val), "m" (*p) Modified: user/eri/pf45/head/contrib/bsnmp/snmp_mibII/BEGEMOT-MIB2-MIB.txt ============================================================================== --- user/eri/pf45/head/contrib/bsnmp/snmp_mibII/BEGEMOT-MIB2-MIB.txt Tue Dec 8 13:06:35 2009 (r200255) +++ user/eri/pf45/head/contrib/bsnmp/snmp_mibII/BEGEMOT-MIB2-MIB.txt Tue Dec 8 13:25:00 2009 (r200256) @@ -39,7 +39,7 @@ IMPORTS FROM BEGEMOT-IP-MIB; begemotMib2 MODULE-IDENTITY - LAST-UPDATED "200602130000Z" + LAST-UPDATED "200908030000Z" ORGANIZATION "German Aerospace Center" CONTACT-INFO " Hartmut Brandt @@ -54,6 +54,12 @@ begemotMib2 MODULE-IDENTITY E-mail: harti@freebsd.org" DESCRIPTION "The MIB for private mib2 stuff." + REVISION "200908030000Z" + DESCRIPTION + "Second edition adds begemotIfDataPoll object." + REVISION "200602130000Z" + DESCRIPTION + "Initial revision." ::= { begemotIp 1 } begemotIfMaxspeed OBJECT-TYPE @@ -87,4 +93,14 @@ begemotIfForcePoll OBJECT-TYPE bit rate in its MIB." ::= { begemotMib2 3 } +begemotIfDataPoll OBJECT-TYPE + SYNTAX TimeTicks + UNITS "deciseconds" + MAX-ACCESS read-write + STATUS current + DESCRIPTION + "The rate at which the mib2 module will poll interface data." + DEFVAL { 100 } + ::= { begemotMib2 4 } + END Modified: user/eri/pf45/head/contrib/bsnmp/snmp_mibII/mibII.c ============================================================================== --- user/eri/pf45/head/contrib/bsnmp/snmp_mibII/mibII.c Tue Dec 8 13:06:35 2009 (r200255) +++ user/eri/pf45/head/contrib/bsnmp/snmp_mibII/mibII.c Tue Dec 8 13:25:00 2009 (r200256) @@ -117,6 +117,15 @@ u_int mibif_hc_update_interval; /* HC update timer handle */ static void *hc_update_timer; +/* Idle poll timer */ +static void *mibII_poll_timer; + +/* interfaces' data poll interval */ +u_int mibII_poll_ticks; + +/* Idle poll hook */ +static void mibII_idle(void *arg __unused); + /*****************************/ static const struct asn_oid oid_ifMIB = OIDX_ifMIB; @@ -410,6 +419,20 @@ mibif_reset_hc_timer(void) mibif_hc_update_interval = ticks; } +/** + * Restart the idle poll timer. + */ +void +mibif_restart_mibII_poll_timer(void) +{ + if (mibII_poll_timer != NULL) + timer_stop(mibII_poll_timer); + + if ((mibII_poll_timer = timer_start_repeat(mibII_poll_ticks * 10, + mibII_poll_ticks * 10, mibII_idle, NULL, module)) == NULL) + syslog(LOG_ERR, "timer_start(%u): %m", mibII_poll_ticks); +} + /* * Fetch new MIB data. */ @@ -1553,7 +1576,7 @@ get_cloners(void) * Idle function */ static void -mibII_idle(void) +mibII_idle(void *arg __unused) { struct mibifa *ifa; @@ -1608,6 +1631,10 @@ mibII_start(void) ipForward_reg = or_register(&oid_ipForward, "The MIB module for the display of CIDR multipath IP Routes.", module); + + mibII_poll_timer = NULL; + mibII_poll_ticks = MIBII_POLL_TICKS; + mibif_restart_mibII_poll_timer(); } /* @@ -1651,6 +1678,11 @@ mibII_init(struct lmodule *mod, int argc static int mibII_fini(void) { + if (mibII_poll_timer != NULL ) { + timer_stop(mibII_poll_timer); + mibII_poll_timer = NULL; + } + if (route_fd != NULL) fd_deselect(route_fd); if (route != -1) @@ -1690,7 +1722,7 @@ const struct snmp_module config = { "This module implements the interface and ip groups.", mibII_init, mibII_fini, - mibII_idle, /* idle */ + NULL, /* idle */ NULL, /* dump */ NULL, /* config */ mibII_start, Modified: user/eri/pf45/head/contrib/bsnmp/snmp_mibII/mibII.h ============================================================================== --- user/eri/pf45/head/contrib/bsnmp/snmp_mibII/mibII.h Tue Dec 8 13:06:35 2009 (r200255) +++ user/eri/pf45/head/contrib/bsnmp/snmp_mibII/mibII.h Tue Dec 8 13:25:00 2009 (r200256) @@ -211,6 +211,14 @@ extern u_int mibif_hc_update_interval; /* re-compute update interval */ void mibif_reset_hc_timer(void); +/* interfaces' data poll interval */ +extern u_int mibII_poll_ticks; + +/* restart the data poll timer */ +void mibif_restart_mibII_poll_timer(void); + +#define MIBII_POLL_TICKS 100 + /* get interfaces and interface addresses. */ void mib_fetch_interfaces(void); Modified: user/eri/pf45/head/contrib/bsnmp/snmp_mibII/mibII_begemot.c ============================================================================== --- user/eri/pf45/head/contrib/bsnmp/snmp_mibII/mibII_begemot.c Tue Dec 8 13:06:35 2009 (r200255) +++ user/eri/pf45/head/contrib/bsnmp/snmp_mibII/mibII_begemot.c Tue Dec 8 13:25:00 2009 (r200256) @@ -59,6 +59,11 @@ op_begemot_mibII(struct snmp_context *ct ctx->scratch->int1 = mibif_force_hc_update_interval; mibif_force_hc_update_interval = value->v.uint32; return (SNMP_ERR_NOERROR); + + case LEAF_begemotIfDataPoll: + ctx->scratch->int1 = mibII_poll_ticks; + mibII_poll_ticks = value->v.uint32; + return (SNMP_ERR_NOERROR); } abort(); @@ -68,6 +73,10 @@ op_begemot_mibII(struct snmp_context *ct case LEAF_begemotIfForcePoll: mibif_force_hc_update_interval = ctx->scratch->int1; return (SNMP_ERR_NOERROR); + + case LEAF_begemotIfDataPoll: + mibII_poll_ticks = ctx->scratch->int1; + return (SNMP_ERR_NOERROR); } abort(); @@ -78,6 +87,10 @@ op_begemot_mibII(struct snmp_context *ct mibif_force_hc_update_interval = ctx->scratch->int1; mibif_reset_hc_timer(); return (SNMP_ERR_NOERROR); + + case LEAF_begemotIfDataPoll: + mibif_restart_mibII_poll_timer(); + return (SNMP_ERR_NOERROR); } abort(); } @@ -98,6 +111,10 @@ op_begemot_mibII(struct snmp_context *ct case LEAF_begemotIfForcePoll: value->v.uint32 = mibif_force_hc_update_interval; return (SNMP_ERR_NOERROR); + + case LEAF_begemotIfDataPoll: + value->v.uint32 = mibII_poll_ticks; + return (SNMP_ERR_NOERROR); } abort(); } Modified: user/eri/pf45/head/contrib/bsnmp/snmp_mibII/mibII_tree.def ============================================================================== --- user/eri/pf45/head/contrib/bsnmp/snmp_mibII/mibII_tree.def Tue Dec 8 13:06:35 2009 (r200255) +++ user/eri/pf45/head/contrib/bsnmp/snmp_mibII/mibII_tree.def Tue Dec 8 13:25:00 2009 (r200256) @@ -240,6 +240,7 @@ (1 begemotIfMaxspeed COUNTER64 op_begemot_mibII GET) (2 begemotIfPoll TIMETICKS op_begemot_mibII GET) (3 begemotIfForcePoll TIMETICKS op_begemot_mibII GET SET) + (4 begemotIfDataPoll TIMETICKS op_begemot_mibII GET SET) ) ) ) Modified: user/eri/pf45/head/contrib/gcc/config/freebsd-spec.h ============================================================================== --- user/eri/pf45/head/contrib/gcc/config/freebsd-spec.h Tue Dec 8 13:06:35 2009 (r200255) +++ user/eri/pf45/head/contrib/gcc/config/freebsd-spec.h Tue Dec 8 13:25:00 2009 (r200256) @@ -103,9 +103,10 @@ Boston, MA 02110-1301, USA. */ %{p:gcrt1.o%s} \ %{!p: \ %{profile:gcrt1.o%s} \ - %{!profile:crt1.o%s}}}} \ + %{!profile: \ + %{pie: Scrt1.o%s;:crt1.o%s}}}}} \ crti.o%s \ - %{static:crtbeginT.o%s;shared:crtbeginS.o%s;:crtbegin.o%s}" + %{static:crtbeginT.o%s;shared|pie:crtbeginS.o%s;:crtbegin.o%s}" /* Provide an ENDFILE_SPEC appropriate for FreeBSD/i386. Here we tack on our own magical crtend.o file (see crtstuff.c) which provides part of @@ -113,8 +114,7 @@ Boston, MA 02110-1301, USA. */ entering `main', followed by the normal "finalizer" file, `crtn.o'. */ #define FBSD_ENDFILE_SPEC "\ - %{!shared:crtend.o%s} \ - %{shared:crtendS.o%s} \ + %{shared|pie:crtendS.o%s;:crtend.o%s} \ crtn.o%s " /* Provide a LIB_SPEC appropriate for FreeBSD as configured and as Modified: user/eri/pf45/head/contrib/groff/tmac/doc-syms ============================================================================== --- user/eri/pf45/head/contrib/groff/tmac/doc-syms Tue Dec 8 13:06:35 2009 (r200255) +++ user/eri/pf45/head/contrib/groff/tmac/doc-syms Tue Dec 8 13:25:00 2009 (r200256) @@ -777,6 +777,7 @@ .ds doc-str-Lb-librt \*[Px] \*[doc-str-Lb]Real-time Library (librt, \-lrt) .ds doc-str-Lb-libtermcap Termcap Access Library (libtermcap, \-ltermcap) .ds doc-str-Lb-libusbhid USB Human Interface Devices Library (libusbhid, \-lusbhid) +.ds doc-str-Lb-libulog User Login Record Library (libulog, \-lulog) .ds doc-str-Lb-libutil System Utilities Library (libutil, \-lutil) .ds doc-str-Lb-libx86_64 x86_64 Architecture Library (libx86_64, \-lx86_64) .ds doc-str-Lb-libz Compression Library (libz, \-lz) Modified: user/eri/pf45/head/crypto/openssl/ssl/s3_lib.c ============================================================================== --- user/eri/pf45/head/crypto/openssl/ssl/s3_lib.c Tue Dec 8 13:06:35 2009 (r200255) +++ user/eri/pf45/head/crypto/openssl/ssl/s3_lib.c Tue Dec 8 13:25:00 2009 (r200256) @@ -2592,6 +2592,9 @@ int ssl3_renegotiate(SSL *s) if (s->s3->flags & SSL3_FLAGS_NO_RENEGOTIATE_CIPHERS) return(0); + if (1) + return(0); + s->s3->renegotiate=1; return(1); } Modified: user/eri/pf45/head/crypto/openssl/ssl/s3_pkt.c ============================================================================== --- user/eri/pf45/head/crypto/openssl/ssl/s3_pkt.c Tue Dec 8 13:06:35 2009 (r200255) +++ user/eri/pf45/head/crypto/openssl/ssl/s3_pkt.c Tue Dec 8 13:25:00 2009 (r200256) @@ -983,9 +983,7 @@ start: if (s->msg_callback) s->msg_callback(0, s->version, SSL3_RT_HANDSHAKE, s->s3->handshake_fragment, 4, s, s->msg_callback_arg); - if (SSL_is_init_finished(s) && - !(s->s3->flags & SSL3_FLAGS_NO_RENEGOTIATE_CIPHERS) && - !s->s3->renegotiate) + if (0) { ssl3_renegotiate(s); if (ssl3_renegotiate_check(s)) @@ -1116,8 +1114,7 @@ start: /* Unexpected handshake message (Client Hello, or protocol violation) */ if ((s->s3->handshake_fragment_len >= 4) && !s->in_handshake) { - if (((s->state&SSL_ST_MASK) == SSL_ST_OK) && - !(s->s3->flags & SSL3_FLAGS_NO_RENEGOTIATE_CIPHERS)) + if (0) { #if 0 /* worked only because C operator preferences are not as expected (and * because this is not really needed for clients except for detecting Modified: user/eri/pf45/head/crypto/openssl/ssl/s3_srvr.c ============================================================================== --- user/eri/pf45/head/crypto/openssl/ssl/s3_srvr.c Tue Dec 8 13:06:35 2009 (r200255) +++ user/eri/pf45/head/crypto/openssl/ssl/s3_srvr.c Tue Dec 8 13:25:00 2009 (r200256) @@ -718,6 +718,13 @@ int ssl3_get_client_hello(SSL *s) #endif STACK_OF(SSL_CIPHER) *ciphers=NULL; + if (s->new_session) + { + al=SSL_AD_HANDSHAKE_FAILURE; + SSLerr(SSL_F_SSL3_GET_CLIENT_HELLO, ERR_R_INTERNAL_ERROR); + goto f_err; + } + /* We do this so that we will respond with our native type. * If we are TLSv1 and we get SSLv3, we will respond with TLSv1, * This down switching should be handled by a different method. Modified: user/eri/pf45/head/etc/Makefile ============================================================================== --- user/eri/pf45/head/etc/Makefile Tue Dec 8 13:06:35 2009 (r200255) +++ user/eri/pf45/head/etc/Makefile Tue Dec 8 13:25:00 2009 (r200256) @@ -15,7 +15,7 @@ BIN1= auth.conf \ inetd.conf libalias.conf login.access login.conf mac.conf motd \ netconfig network.subr networks newsyslog.conf nsswitch.conf \ phones profile protocols \ - rc rc.bsdextended rc.firewall rc.firewall6 rc.initdiskless \ + rc rc.bsdextended rc.firewall rc.initdiskless \ rc.sendmail rc.shutdown \ rc.subr remote rpc services shells \ sysctl.conf syslog.conf Modified: user/eri/pf45/head/etc/defaults/rc.conf ============================================================================== --- user/eri/pf45/head/etc/defaults/rc.conf Tue Dec 8 13:06:35 2009 (r200255) +++ user/eri/pf45/head/etc/defaults/rc.conf Tue Dec 8 13:25:00 2009 (r200256) @@ -118,7 +118,10 @@ firewall_type="UNKNOWN" # Firewall type firewall_quiet="NO" # Set to YES to suppress rule display firewall_logging="NO" # Set to YES to enable events logging firewall_flags="" # Flags passed to ipfw when type is a file -firewall_client_net="192.0.2.0/24" # Network address for "client" firewall. +firewall_client_net="192.0.2.0/24" # IPv4 Network address for "client" + # firewall. +#firewall_client_net_ipv6="2001:db8:2:1::/64" # IPv6 network prefix for + # "client" firewall. firewall_simple_iif="ed1" # Inside network interface for "simple" # firewall. firewall_simple_inet="192.0.2.16/28" # Inside network address for "simple" @@ -127,12 +130,22 @@ firewall_simple_oif="ed0" # Outside netw # firewall. firewall_simple_onet="192.0.2.0/28" # Outside network address for "simple" # firewall. +#firewall_simple_iif_ipv6="ed1" # Inside IPv6 network interface for "simple" + # firewall. +#firewall_simple_inet_ipv6="2001:db8:2:800::/56" # Inside IPv6 network prefix + # for "simple" firewall. +#firewall_simple_oif_ipv6="ed0" # Outside IPv6 network interface for "simple" + # firewall. +#firewall_simple_onet_ipv6="2001:db8:2:0::/56" # Outside IPv6 network prefix + # for "simple" firewall. firewall_myservices="" # List of TCP ports on which this host # offers services for "workstation" firewall. firewall_allowservices="" # List of IPs which have access to # $firewall_myservices for "workstation" # firewall. -firewall_trusted="" # List of IPs which have full access to this +firewall_trusted="" # List of IPv4s which have full access to this + # host for "workstation" firewall. +firewall_trusted_ipv6="" # List of IPv6s which have full access to this # host for "workstation" firewall. firewall_logdeny="NO" # Set to YES to log default denied incoming # packets for "workstation" firewall. @@ -472,13 +485,6 @@ ipv6_faith_prefix="NO" # Set faith pref # faithd(8) setup. ipv6_ipv4mapping="NO" # Set to "YES" to enable IPv4 mapped IPv6 addr # communication. (like ::ffff:a.b.c.d) -ipv6_firewall_enable="NO" # Set to YES to enable IPv6 firewall - # functionality -ipv6_firewall_script="/etc/rc.firewall6" # Which script to run to set up the IPv6 firewall -ipv6_firewall_type="UNKNOWN" # IPv6 Firewall type (see /etc/rc.firewall6) -ipv6_firewall_quiet="NO" # Set to YES to suppress rule display -ipv6_firewall_logging="NO" # Set to YES to enable events logging -ipv6_firewall_flags="" # Flags passed to ip6fw when type is a file ipv6_ipfilter_rules="/etc/ipf6.rules" # rules definition file for ipfilter, # see /usr/src/contrib/ipfilter/rules # for examples Modified: user/eri/pf45/head/etc/mtree/BSD.var.dist ============================================================================== --- user/eri/pf45/head/etc/mtree/BSD.var.dist Tue Dec 8 13:06:35 2009 (r200255) +++ user/eri/pf45/head/etc/mtree/BSD.var.dist Tue Dec 8 13:25:00 2009 (r200256) @@ -32,7 +32,7 @@ db entropy uname=operator gname=operator mode=0700 .. - freebsd-update + freebsd-update mode=0700 .. ipf mode=0700 .. Modified: user/eri/pf45/head/etc/rc.d/Makefile ============================================================================== --- user/eri/pf45/head/etc/rc.d/Makefile Tue Dec 8 13:06:35 2009 (r200255) +++ user/eri/pf45/head/etc/rc.d/Makefile Tue Dec 8 13:25:00 2009 (r200256) @@ -15,7 +15,7 @@ FILES= DAEMON FILESYSTEMS LOGIN NETWORKI hcsecd \ hostapd hostid hostid_save hostname \ inetd initrandom \ - ip6addrctl ip6fw ipfilter ipfs ipfw ipmon \ + ip6addrctl ipfilter ipfs ipfw ipmon \ ipnat ipsec ipxrouted \ jail \ kadmind kerberos keyserv kldxref kpasswdd \ Modified: user/eri/pf45/head/etc/rc.d/ipfw ============================================================================== --- user/eri/pf45/head/etc/rc.d/ipfw Tue Dec 8 13:06:35 2009 (r200255) +++ user/eri/pf45/head/etc/rc.d/ipfw Tue Dec 8 13:25:00 2009 (r200256) @@ -17,6 +17,8 @@ start_precmd="ipfw_prestart" stop_cmd="ipfw_stop" required_modules="ipfw" +set_rcvar_obsolete ipv6_firewall_enable + ipfw_prestart() { if checkyesno dummynet_enable; then @@ -61,7 +63,13 @@ ipfw_start() # Enable the firewall # if ! ${SYSCTL_W} net.inet.ip.fw.enable=1 1>/dev/null 2>&1; then - warn "failed to enable firewall" + warn "failed to enable IPv4 firewall" + fi + if afexists inet6; then + if ! ${SYSCTL_W} net.inet6.ip6.fw.enable=1 1>/dev/null 2>&1 + then + warn "failed to enable IPv6 firewall" + fi fi } @@ -70,6 +78,9 @@ ipfw_stop() # Disable the firewall # ${SYSCTL_W} net.inet.ip.fw.enable=0 + if afexists inet6; then + ${SYSCTL_W} net.inet6.ip6.fw.enable=0 + fi if [ -f /etc/rc.d/natd ] ; then /etc/rc.d/natd quietstop fi Modified: user/eri/pf45/head/etc/rc.firewall ============================================================================== --- user/eri/pf45/head/etc/rc.firewall Tue Dec 8 13:06:35 2009 (r200255) +++ user/eri/pf45/head/etc/rc.firewall Tue Dec 8 13:25:00 2009 (r200256) @@ -85,12 +85,42 @@ setup_loopback () { ${fwcmd} add 100 pass all from any to any via lo0 ${fwcmd} add 200 deny all from any to 127.0.0.0/8 ${fwcmd} add 300 deny ip from 127.0.0.0/8 to any + if [ $ipv6_available -eq 0 ]; then + ${fwcmd} add 400 deny all from any to ::1 + ${fwcmd} add 500 deny all from ::1 to any + fi +} + +setup_ipv6_mandatory () { + [ $ipv6_available -eq 0 ] || return 0 + + ############ + # Only in rare cases do you want to change these rules + # + # ND + # + # DAD + ${fwcmd} add pass ipv6-icmp from :: to ff02::/16 + # RS, RA, NS, NA, redirect... + ${fwcmd} add pass ipv6-icmp from fe80::/10 to fe80::/10 + ${fwcmd} add pass ipv6-icmp from fe80::/10 to ff02::/16 + + # Allow ICMPv6 destination unreach + ${fwcmd} add pass ipv6-icmp from any to any icmp6types 1 + + # Allow NS/NA/toobig (don't filter it out) + ${fwcmd} add pass ipv6-icmp from any to any icmp6types 2,135,136 } if [ -n "${1}" ]; then firewall_type="${1}" fi +. /etc/rc.subr +. /etc/network.subr +afexists inet6 +ipv6_available=$? + ############ # Set quiet mode if requested # @@ -109,6 +139,7 @@ esac ${fwcmd} -f flush setup_loopback +setup_ipv6_mandatory ############ # Network Address Translation. All packets are passed to natd(8) @@ -166,11 +197,13 @@ case ${firewall_type} in # against people from outside your own network. # # Configuration: - # firewall_client_net: Network address of local network. + # firewall_client_net: Network address of local IPv4 network. + # firewall_client_net_ipv6: Network address of local IPv6 network. ############ # set this to your local network net="$firewall_client_net" + net6="$firewall_client_net_ipv6" # Allow limited broadcast traffic from my own net. ${fwcmd} add pass all from ${net} to 255.255.255.255 @@ -178,6 +211,16 @@ case ${firewall_type} in # Allow any traffic to or from my own net. ${fwcmd} add pass all from me to ${net} ${fwcmd} add pass all from ${net} to me + if [ -n "$net6" ]; then + ${fwcmd} add pass all from me6 to ${net6} + ${fwcmd} add pass all from ${net6} to me6 + fi + + if [ -n "$net6" ]; then + # Allow any link-local multicast traffic + ${fwcmd} add pass all from fe80::/10 to ff02::/16 + ${fwcmd} add pass all from ${net6} to ff02::/16 + fi # Allow TCP through if setup succeeded ${fwcmd} add pass tcp from any to any established @@ -212,23 +255,38 @@ case ${firewall_type} in # on the inside at this machine for those services. # # Configuration: - # firewall_simple_iif: Inside network interface. - # firewall_simple_inet: Inside network address. - # firewall_simple_oif: Outside network interface. - # firewall_simple_onet: Outside network address. + # firewall_simple_iif: Inside IPv4 network interface. + # firewall_simple_inet: Inside IPv4 network address. + # firewall_simple_oif: Outside IPv4 network interface. + # firewall_simple_onet: Outside IPv4 network address. + # firewall_simple_iif_ipv6: Inside IPv6 network interface. + # firewall_simple_inet_ipv6: Inside IPv6 network prefix. + # firewall_simple_oif_ipv6: Outside IPv6 network interface. + # firewall_simple_onet_ipv6: Outside IPv6 network prefix. ############ # set these to your outside interface network oif="$firewall_simple_oif" onet="$firewall_simple_onet" + oif6="${firewall_simple_oif_ipv6:-$firewall_simple_oif}" + onet6="$firewall_simple_onet_ipv6" # set these to your inside interface network iif="$firewall_simple_iif" inet="$firewall_simple_inet" + iif6="${firewall_simple_iif_ipv6:-$firewall_simple_iif}" + inet6="$firewall_simple_inet_ipv6" # Stop spoofing ${fwcmd} add deny all from ${inet} to any in via ${oif} ${fwcmd} add deny all from ${onet} to any in via ${iif} + if [ -n "$inet6" ]; then + ${fwcmd} add deny all from ${inet6} to any in via ${oif6} + if [ -n "$onet6" ]; then + ${fwcmd} add deny all from ${onet6} to any in \ + via ${iif6} + fi + fi # Stop RFC1918 nets on the outside interface ${fwcmd} add deny all from any to 10.0.0.0/8 via ${oif} @@ -254,7 +312,7 @@ case ${firewall_type} in case ${natd_enable} in [Yy][Ee][Ss]) if [ -n "${natd_interface}" ]; then - ${fwcmd} add divert natd all from any to any via ${natd_interface} + ${fwcmd} add divert natd ip4 from any to any via ${natd_interface} fi ;; esac @@ -273,6 +331,55 @@ case ${firewall_type} in ${fwcmd} add deny all from 224.0.0.0/4 to any via ${oif} ${fwcmd} add deny all from 240.0.0.0/4 to any via ${oif} + if [ -n "$inet6" ]; then + # Stop unique local unicast address on the outside interface + ${fwcmd} add deny all from fc00::/7 to any via ${oif6} + ${fwcmd} add deny all from any to fc00::/7 via ${oif6} + + # Stop site-local on the outside interface + ${fwcmd} add deny all from fec0::/10 to any via ${oif6} + ${fwcmd} add deny all from any to fec0::/10 via ${oif6} + + # Disallow "internal" addresses to appear on the wire. + ${fwcmd} add deny all from ::ffff:0.0.0.0/96 to any \ + via ${oif6} + ${fwcmd} add deny all from any to ::ffff:0.0.0.0/96 \ + via ${oif6} + + # Disallow packets to malicious IPv4 compatible prefix. + ${fwcmd} add deny all from ::224.0.0.0/100 to any via ${oif6} + ${fwcmd} add deny all from any to ::224.0.0.0/100 via ${oif6} + ${fwcmd} add deny all from ::127.0.0.0/104 to any via ${oif6} + ${fwcmd} add deny all from any to ::127.0.0.0/104 via ${oif6} + ${fwcmd} add deny all from ::0.0.0.0/104 to any via ${oif6} + ${fwcmd} add deny all from any to ::0.0.0.0/104 via ${oif6} + ${fwcmd} add deny all from ::255.0.0.0/104 to any via ${oif6} + ${fwcmd} add deny all from any to ::255.0.0.0/104 via ${oif6} + + ${fwcmd} add deny all from ::0.0.0.0/96 to any via ${oif6} + ${fwcmd} add deny all from any to ::0.0.0.0/96 via ${oif6} + + # Disallow packets to malicious 6to4 prefix. + ${fwcmd} add deny all from 2002:e000::/20 to any via ${oif6} + ${fwcmd} add deny all from any to 2002:e000::/20 via ${oif6} + ${fwcmd} add deny all from 2002:7f00::/24 to any via ${oif6} + ${fwcmd} add deny all from any to 2002:7f00::/24 via ${oif6} + ${fwcmd} add deny all from 2002:0000::/24 to any via ${oif6} + ${fwcmd} add deny all from any to 2002:0000::/24 via ${oif6} + ${fwcmd} add deny all from 2002:ff00::/24 to any via ${oif6} + ${fwcmd} add deny all from any to 2002:ff00::/24 via ${oif6} + + ${fwcmd} add deny all from 2002:0a00::/24 to any via ${oif6} + ${fwcmd} add deny all from any to 2002:0a00::/24 via ${oif6} + ${fwcmd} add deny all from 2002:ac10::/28 to any via ${oif6} + ${fwcmd} add deny all from any to 2002:ac10::/28 via ${oif6} + ${fwcmd} add deny all from 2002:c0a8::/32 to any via ${oif6} + ${fwcmd} add deny all from any to 2002:c0a8::/32 via ${oif6} + + ${fwcmd} add deny all from ff05::/16 to any via ${oif6} + ${fwcmd} add deny all from any to ff05::/16 via ${oif6} + fi + # Allow TCP through if setup succeeded ${fwcmd} add pass tcp from any to any established @@ -291,7 +398,11 @@ case ${firewall_type} in ${fwcmd} add pass tcp from any to me 80 setup # Reject&Log all setup of incoming connections from the outside - ${fwcmd} add deny log tcp from any to any in via ${oif} setup + ${fwcmd} add deny log ip4 from any to any in via ${oif} setup proto tcp + if [ -n "$inet6" ]; then + ${fwcmd} add deny log ip6 from any to any in via ${oif6} \ + setup proto tcp + fi # Allow setup of any other TCP connection ${fwcmd} add pass tcp from any to any setup @@ -313,7 +424,7 @@ case ${firewall_type} in # offers services. # firewall_allowservices: List of IPs which has access to # $firewall_myservices. - # firewall_trusted: List of IPs which has full access + # firewall_trusted: List of IPv4s which has full access # to this host. Be very carefull # when setting this. This option can # seriously degrade the level of @@ -324,25 +435,44 @@ case ${firewall_type} in # firewall_nologports: List of TCP/UDP ports for which # denied incomming packets are not # logged. - + # firewall_trusted_ipv6: List of IPv6s which has full access + # to this host. Be very carefull + # when setting this. This option can + # seriously degrade the level of + # protection provided by the firewall. + # Allow packets for which a state has been built. ${fwcmd} add check-state # For services permitted below. ${fwcmd} add pass tcp from me to any established + if [ $ipv6_available -eq 0 ]; then + ${fwcmd} add pass tcp from me6 to any established + fi # Allow any connection out, adding state for each. ${fwcmd} add pass tcp from me to any setup keep-state ${fwcmd} add pass udp from me to any keep-state ${fwcmd} add pass icmp from me to any keep-state + if [ $ipv6_available -eq 0 ]; then + ${fwcmd} add pass tcp from me6 to any setup keep-state + ${fwcmd} add pass udp from me6 to any keep-state + ${fwcmd} add pass ipv6-icmp from me6 to any keep-state + fi # Allow DHCP. ${fwcmd} add pass udp from 0.0.0.0 68 to 255.255.255.255 67 out ${fwcmd} add pass udp from any 67 to me 68 in ${fwcmd} add pass udp from any 67 to 255.255.255.255 68 in + if [ $ipv6_available -eq 0 ]; then + ${fwcmd} add pass udp from fe80::/10 to me6 546 in + fi # Some servers will ping the IP while trying to decide if it's # still in use. ${fwcmd} add pass icmp from any to any icmptype 8 + if [ $ipv6_available -eq 0 ]; then + ${fwcmd} add pass ipv6-icmp from any to any icmp6type 128,129 + fi # Allow "mandatory" ICMP in. ${fwcmd} add pass icmp from any to any icmptype 3,4,11 @@ -361,6 +491,9 @@ case ${firewall_type} in for i in ${firewall_allowservices} ; do for j in ${firewall_myservices} ; do ${fwcmd} add pass tcp from $i to me $j + if [ $ipv6_available -eq 0 ]; then + ${fwcmd} add pass tcp from $i to me6 $j + fi done done @@ -370,7 +503,10 @@ case ${firewall_type} in for i in ${firewall_trusted} ; do ${fwcmd} add pass ip from $i to me done - + for i in ${firewall_trusted_ipv6} ; do + ${fwcmd} add pass all from $i to me6 + done + ${fwcmd} add 65000 count ip from any to any # Drop packets to ports where we don't want logging Modified: user/eri/pf45/head/lib/Makefile ============================================================================== --- user/eri/pf45/head/lib/Makefile Tue Dec 8 13:06:35 2009 (r200255) +++ user/eri/pf45/head/lib/Makefile Tue Dec 8 13:25:00 2009 (r200256) @@ -40,8 +40,8 @@ SUBDIR= ${_csu} libc libbsm libauditd li ${_libpmc} libproc librt ${_libsdp} ${_libsm} ${_libsmb} \ ${_libsmdb} \ ${_libsmutil} libstand ${_libtelnet} ${_libthr} libthread_db libufs \ - libugidfw ${_libusbhid} ${_libusb} ${_libvgl} libwrap liby libz \ - ${_bind} + libugidfw libulog ${_libusbhid} ${_libusb} ${_libvgl} libwrap \ + liby libz ${_bind} .if exists(${.CURDIR}/csu/${MACHINE_ARCH}-elf) _csu=csu/${MACHINE_ARCH}-elf Modified: user/eri/pf45/head/lib/csu/amd64/Makefile ============================================================================== --- user/eri/pf45/head/lib/csu/amd64/Makefile Tue Dec 8 13:06:35 2009 (r200255) +++ user/eri/pf45/head/lib/csu/amd64/Makefile Tue Dec 8 13:25:00 2009 (r200256) @@ -4,7 +4,7 @@ SRCS= crt1.c crti.S crtn.S OBJS= ${SRCS:N*.h:R:S/$/.o/g} -OBJS+= gcrt1.o +OBJS+= Scrt1.o gcrt1.o CFLAGS+= -I${.CURDIR}/../common \ -I${.CURDIR}/../../libc/include CFLAGS+= -fno-omit-frame-pointer @@ -16,6 +16,9 @@ CLEANFILES= ${OBJS} gcrt1.o: crt1.c ${CC} ${CFLAGS} -DGCRT -c -o gcrt1.o ${.CURDIR}/crt1.c +Scrt1.o: crt1.c + ${CC} ${CFLAGS} -fPIC -DPIC -c -o Scrt1.o ${.CURDIR}/crt1.c + realinstall: ${INSTALL} -o ${LIBOWN} -g ${LIBGRP} -m ${LIBMODE} \ ${OBJS} ${DESTDIR}${LIBDIR} Modified: user/eri/pf45/head/lib/csu/arm/Makefile ============================================================================== --- user/eri/pf45/head/lib/csu/arm/Makefile Tue Dec 8 13:06:35 2009 (r200255) +++ user/eri/pf45/head/lib/csu/arm/Makefile Tue Dec 8 13:25:00 2009 (r200256) @@ -4,7 +4,7 @@ SRCS= crt1.c crti.S crtn.S OBJS= ${SRCS:N*.h:R:S/$/.o/g} -OBJS+= gcrt1.o +OBJS+= Scrt1.o gcrt1.o CFLAGS+= -Wall -Wno-unused \ -I${.CURDIR}/../common \ -I${.CURDIR}/../../libc/include @@ -16,6 +16,9 @@ CLEANFILES= ${OBJS} gcrt1.o: crt1.c ${CC} ${CFLAGS} -DGCRT -c -o gcrt1.o ${.ALLSRC} +Scrt1.o: crt1.c + ${CC} ${CFLAGS} -fPIC -DPIC -c -o Scrt1.o ${.ALLSRC} + realinstall: ${INSTALL} -o ${LIBOWN} -g ${LIBGRP} -m ${LIBMODE} \ ${OBJS} ${DESTDIR}${LIBDIR} Modified: user/eri/pf45/head/lib/csu/i386-elf/Makefile ============================================================================== --- user/eri/pf45/head/lib/csu/i386-elf/Makefile Tue Dec 8 13:06:35 2009 (r200255) +++ user/eri/pf45/head/lib/csu/i386-elf/Makefile Tue Dec 8 13:25:00 2009 (r200256) @@ -2,8 +2,8 @@ .PATH: ${.CURDIR}/../common -SRCS= crt1.c crti.S crtn.S -FILES= ${SRCS:N*.h:R:S/$/.o/g} gcrt1.o +SRCS= crti.S crtn.S +FILES= ${SRCS:N*.h:R:S/$/.o/g} gcrt1.o crt1.o Scrt1.o FILESOWN= ${LIBOWN} FILESGRP= ${LIBGRP} FILESMODE= ${LIBMODE} @@ -11,9 +11,23 @@ FILESDIR= ${LIBDIR} WARNS?= 6 CFLAGS+= -I${.CURDIR}/../common \ -I${.CURDIR}/../../libc/include -CLEANFILES= ${FILES} +CLEANFILES= ${FILES} crt1_c.o crt1_s.o gcrt1_c.o Scrt1_c.o -gcrt1.o: crt1.c - ${CC} ${CFLAGS} -DGCRT -c -o gcrt1.o ${.CURDIR}/crt1.c +gcrt1_c.o: crt1_c.c + ${CC} ${CFLAGS} -DGCRT -c -o gcrt1_c.o ${.CURDIR}/crt1_c.c + +gcrt1.o: gcrt1_c.o crt1_s.o + ${LD} ${LDFLAGS} -o gcrt1.o -r crt1_s.o gcrt1_c.o + +crt1.o: crt1_c.o crt1_s.o + ${LD} ${LDFLAGS} -o crt1.o -r crt1_s.o crt1_c.o + objcopy --localize-symbol _start1 crt1.o + +Scrt1_c.o: crt1_c.c + ${CC} ${CFLAGS} -DGCRT -fPIC -DPIC -c -o Scrt1_c.o ${.CURDIR}/crt1_c.c + +Scrt1.o: Scrt1_c.o crt1_s.o + ${LD} ${LDFLAGS} -o Scrt1.o -r crt1_s.o Scrt1_c.o + objcopy --localize-symbol _start1 Scrt1.o .include Copied: user/eri/pf45/head/lib/csu/i386-elf/crt1_c.c (from r200254, head/lib/csu/i386-elf/crt1_c.c) ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ user/eri/pf45/head/lib/csu/i386-elf/crt1_c.c Tue Dec 8 13:25:00 2009 (r200256, copy of r200254, head/lib/csu/i386-elf/crt1_c.c) @@ -0,0 +1,95 @@ +/* LINTLIBRARY */ +/*- + * Copyright 1996-1998 John D. Polstra. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR + * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES + * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. + * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, + * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT + * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF + * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * + * $FreeBSD$ + */ + +#ifndef lint +#ifndef __GNUC__ +#error "GCC is needed to compile this file" +#endif +#endif /* lint */ + +#include + +#include "libc_private.h" +#include "crtbrand.c" + +extern int _DYNAMIC; +#pragma weak _DYNAMIC + +typedef void (*fptr)(void); + +extern void _fini(void); +extern void _init(void); +extern int main(int, char **, char **); +extern void _start(char *, ...); + +#ifdef GCRT +extern void _mcleanup(void); +extern void monstartup(void *, void *); +extern int eprol; +extern int etext; +#endif + +char **environ; +const char *__progname = ""; + +void _start1(fptr, int, char *[]) __dead2; + +/* The entry function, C part. */ +void +_start1(fptr cleanup, int argc, char *argv[]) +{ + char **env; + const char *s; + + env = argv + argc + 1; + environ = env; + if (argc > 0 && argv[0] != NULL) { + __progname = argv[0]; + for (s = __progname; *s != '\0'; s++) + if (*s == '/') + __progname = s + 1; + } + + if (&_DYNAMIC != NULL) + atexit(cleanup); + else + _init_tls(); + +#ifdef GCRT + atexit(_mcleanup); +#endif + atexit(_fini); +#ifdef GCRT + monstartup(&eprol, &etext); +__asm__("eprol:"); +#endif + _init(); + exit( main(argc, argv, env) ); +} + +__asm(".hidden _start1"); Copied: user/eri/pf45/head/lib/csu/i386-elf/crt1_s.S (from r200254, head/lib/csu/i386-elf/crt1_s.S) ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ user/eri/pf45/head/lib/csu/i386-elf/crt1_s.S Tue Dec 8 13:25:00 2009 (r200256, copy of r200254, head/lib/csu/i386-elf/crt1_s.S) @@ -0,0 +1,44 @@ +/*- + * Copyright 2009 Konstantin Belousov. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR + * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES + * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. + * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, + * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** From owner-svn-src-user@FreeBSD.ORG Tue Dec 8 22:35:39 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 63DBB1065672; Tue, 8 Dec 2009 22:35:39 +0000 (UTC) (envelope-from eri@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 5089E8FC0A; Tue, 8 Dec 2009 22:35:39 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nB8MZdvc003319; Tue, 8 Dec 2009 22:35:39 GMT (envelope-from eri@svn.freebsd.org) Received: (from eri@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nB8MZddC003317; Tue, 8 Dec 2009 22:35:39 GMT (envelope-from eri@svn.freebsd.org) Message-Id: <200912082235.nB8MZddC003317@svn.freebsd.org> From: Ermal Luçi Date: Tue, 8 Dec 2009 22:35:39 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r200278 - user/eri/pf45/head/sys/contrib/pf/net X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Dec 2009 22:35:39 -0000 Author: eri Date: Tue Dec 8 22:35:39 2009 New Revision: 200278 URL: http://svn.freebsd.org/changeset/base/200278 Log: First pass at VIMAGE for pfsync(4). TODO: Set context in some other needed areas and 'virtualize' statistic. Modified: user/eri/pf45/head/sys/contrib/pf/net/if_pfsync.c Modified: user/eri/pf45/head/sys/contrib/pf/net/if_pfsync.c ============================================================================== --- user/eri/pf45/head/sys/contrib/pf/net/if_pfsync.c Tue Dec 8 22:28:55 2009 (r200277) +++ user/eri/pf45/head/sys/contrib/pf/net/if_pfsync.c Tue Dec 8 22:35:39 2009 (r200278) @@ -251,6 +251,9 @@ struct pfsync_softc { struct in_addr sc_sync_peer; u_int8_t sc_maxupdates; +#ifdef __FreeBSD__ + int pfsync_sync_ok; +#endif struct ip sc_template; @@ -295,9 +298,15 @@ struct pfsync_softc { }; +#ifndef __FreeBSD__ struct pfsync_softc *pfsyncif = NULL; struct pfsyncstats pfsyncstats; +#endif #ifdef __FreeBSD__ +VNET_DEFINE(struct pfsync_softc *, pfsyncif) = NULL; +#define V_pfsyncif VNET(pfsyncif) +VNET_DEFINE(struct pfsyncstats, pfsyncstats); +#define V_pfsyncstats VNET(pfsyncstats) #ifdef notyet SYSCTL_DECL(_net_inet_pfsync); SYSCTL_STRUCT(_net_inet_pfsync, 0, stats, CTLFLAG_RW, @@ -367,7 +376,9 @@ void pfsync_ifdetach(void *, struct ifne #endif #define PFSYNC_MAX_BULKTRIES 12 +#ifndef __FreeBSD__ int pfsync_sync_ok; +#endif #ifdef __FreeBSD__ IFC_SIMPLE_DECLARE(pfsync, 1); @@ -395,7 +406,9 @@ pfsync_clone_create(struct if_clone *ifc if (unit != 0) return (EINVAL); +#ifndef __FreeBSD__ pfsync_sync_ok = 1; +#endif sc = malloc(sizeof(struct pfsync_softc), M_DEVBUF, M_NOWAIT | M_ZERO); if (sc == NULL) @@ -405,6 +418,7 @@ pfsync_clone_create(struct if_clone *ifc TAILQ_INIT(&sc->sc_qs[q]); #ifdef __FreeBSD__ + sc->pfsync_sync_ok = 1; sc->sc_pool = uma_zcreate("pfsync", PFSYNC_PLSIZE, NULL, NULL, NULL, NULL, UMA_ALIGN_PTR, 0); if (sc->sc_pool == NULL) { @@ -447,7 +461,11 @@ pfsync_clone_create(struct if_clone *ifc if_initname(ifp, ifc->ifc_name, unit); sc->sc_detachtag = EVENTHANDLER_REGISTER(ifnet_departure_event, +#ifdef __FreeBSD__ + pfsync_ifdetach, V_pfsyncif, EVENTHANDLER_PRI_ANY); +#else pfsync_ifdetach, pfsyncif, EVENTHANDLER_PRI_ANY); +#endif if (sc->sc_detachtag == NULL) { if_free(ifp); free(sc->sc_imo.imo_membership, M_DEVBUF); @@ -495,7 +513,11 @@ pfsync_clone_create(struct if_clone *ifc #endif #endif +#ifdef __FreeBSD__ + V_pfsyncif = sc; +#else pfsyncif = sc; +#endif return (0); } @@ -516,7 +538,11 @@ pfsync_clone_destroy(struct ifnet *ifp) timeout_del(&sc->sc_tmo); #if NCARP > 0 #ifdef notyet +#ifdef __FreeBSD__ + if (!sc->pfsync_sync_ok) +#else if (!pfsync_sync_ok) +#endif carp_group_demote_adj(&sc->sc_if, -1); #endif #endif @@ -543,7 +569,11 @@ pfsync_clone_destroy(struct ifnet *ifp) #endif free(sc, M_DEVBUF); +#ifdef __FreeBSD__ + V_pfsyncif = NULL; +#else pfsyncif = NULL; +#endif #ifndef __FreeBSD__ return (0); @@ -867,7 +897,11 @@ pfsync_input(struct mbuf *m, __unused in pfsync_input(struct mbuf *m, ...) #endif { +#ifdef __FreeBSD__ + struct pfsync_softc *sc = V_pfsyncif; +#else struct pfsync_softc *sc = pfsyncif; +#endif struct pfsync_pkt pkt; struct ip *ip = mtod(m, struct ip *); struct pfsync_header *ph; @@ -1528,7 +1562,11 @@ pfsync_in_del_c(struct pfsync_pkt *pkt, int pfsync_in_bus(struct pfsync_pkt *pkt, struct mbuf *m, int offset, int count) { +#ifdef __FreeBSD__ + struct pfsync_softc *sc = V_pfsyncif; +#else struct pfsync_softc *sc = pfsyncif; +#endif struct pfsync_bus *bus; struct mbuf *mp; int len = count * sizeof(*bus); @@ -1549,7 +1587,7 @@ pfsync_in_bus(struct pfsync_pkt *pkt, st case PFSYNC_BUS_START: #ifdef __FreeBSD__ callout_reset(&sc->sc_bulkfail_tmo, 5 * hz, pfsync_bulk_fail, - pfsyncif); + V_pfsyncif); #else timeout_add_sec(&sc->sc_bulkfail_tmo, 5); /* XXX magic */ #endif @@ -1574,11 +1612,19 @@ pfsync_in_bus(struct pfsync_pkt *pkt, st timeout_del(&sc->sc_bulkfail_tmo); #if NCARP > 0 #ifdef notyet +#ifdef __FreeBSD__ + if (!sc->pfsync_sync_ok) +#else if (!pfsync_sync_ok) +#endif carp_group_demote_adj(&sc->sc_if, -1); #endif #endif +#ifdef __FreeBSD__ + sc->pfsync_sync_ok = 1; +#else pfsync_sync_ok = 1; +#endif #ifdef __FreeBSD__ if (V_pf_status.debug >= PF_DEBUG_MISC) #else @@ -1914,11 +1960,19 @@ pfsyncioctl(struct ifnet *ifp, u_long cm sc->sc_ureq_sent = time_uptime; #if NCARP > 0 #ifdef notyet +#ifdef __FreeBSD__ + if (sc->pfsync_sync_ok) +#else if (pfsync_sync_ok) +#endif carp_group_demote_adj(&sc->sc_if, 1); #endif #endif +#ifdef __FreeBSD__ + sc->pfsync_sync_ok = 0; +#else pfsync_sync_ok = 0; +#endif #ifdef __FreeBSD__ if (V_pf_status.debug >= PF_DEBUG_MISC) #else @@ -1927,7 +1981,7 @@ pfsyncioctl(struct ifnet *ifp, u_long cm printf("pfsync: requesting bulk update\n"); #ifdef __FreeBSD__ callout_reset(&sc->sc_bulkfail_tmo, 5 * hz, - pfsync_bulk_fail, pfsyncif); + pfsync_bulk_fail, V_pfsyncif); #else timeout_add_sec(&sc->sc_bulkfail_tmo, 5); #endif @@ -2055,7 +2109,11 @@ pfsync_drop(struct pfsync_softc *sc) void pfsync_sendout(void) { +#ifdef __FreeBSD__ + struct pfsync_softc *sc = V_pfsyncif; +#else struct pfsync_softc *sc = pfsyncif; +#endif #if NBPFILTER > 0 #ifdef __FreeBSD__ struct ifnet *ifp = sc->sc_ifp; @@ -2307,7 +2365,11 @@ pfsync_sendout(void) void pfsync_insert_state(struct pf_state *st) { +#ifdef __FreeBSD__ + struct pfsync_softc *sc = V_pfsyncif; +#else struct pfsync_softc *sc = pfsyncif; +#endif #ifdef __FreeBSD__ PF_ASSERT(MA_OWNED); @@ -2336,7 +2398,7 @@ pfsync_insert_state(struct pf_state *st) if (sc->sc_len == PFSYNC_MINPKT) #ifdef __FreeBSD__ callout_reset(&sc->sc_tmo, 1 * hz, pfsync_timeout, - pfsyncif); + V_pfsyncif); #else timeout_add_sec(&sc->sc_tmo, 1); #endif @@ -2354,7 +2416,11 @@ int defer = 10; int pfsync_defer(struct pf_state *st, struct mbuf *m) { +#ifdef __FreeBSD__ + struct pfsync_softc *sc = V_pfsyncif; +#else struct pfsync_softc *sc = pfsyncif; +#endif struct pfsync_deferral *pd; #ifdef __FreeBSD__ @@ -2397,7 +2463,11 @@ pfsync_defer(struct pf_state *st, struct void pfsync_undefer(struct pfsync_deferral *pd, int drop) { +#ifdef __FreeBSD__ + struct pfsync_softc *sc = V_pfsyncif; +#else struct pfsync_softc *sc = pfsyncif; +#endif int s; #ifdef __FreeBSD__ @@ -2433,15 +2503,20 @@ pfsync_undefer(struct pfsync_deferral *p void pfsync_defer_tmo(void *arg) { +#if defined(__FreeBSD__) && defined(VIMAGE) + struct pfsync_deferral *pd = arg; +#endif int s; s = splsoftnet(); #ifdef __FreeBSD__ + CURVNET_SET(pd->pd_m->m_pkthdr.rcvif); /* XXX */ PF_LOCK(); #endif pfsync_undefer(arg, 0); #ifdef __FreeBSD__ PF_UNLOCK(); + CURVNET_RESTORE(); #endif splx(s); } @@ -2449,7 +2524,11 @@ pfsync_defer_tmo(void *arg) void pfsync_deferred(struct pf_state *st, int drop) { +#ifdef __FreeBSD__ + struct pfsync_softc *sc = V_pfsyncif; +#else struct pfsync_softc *sc = pfsyncif; +#endif struct pfsync_deferral *pd; TAILQ_FOREACH(pd, &sc->sc_deferrals, pd_entry) { @@ -2467,7 +2546,11 @@ u_int pfsync_upds = 0; void pfsync_update_state(struct pf_state *st) { +#ifdef __FreeBSD__ + struct pfsync_softc *sc = V_pfsyncif; +#else struct pfsync_softc *sc = pfsyncif; +#endif int sync = 0; #ifdef __FreeBSD__ @@ -2490,7 +2573,7 @@ pfsync_update_state(struct pf_state *st) if (sc->sc_len == PFSYNC_MINPKT) #ifdef __FreeBSD__ callout_reset(&sc->sc_tmo, 1 * hz, pfsync_timeout, - pfsyncif); + V_pfsyncif); #else timeout_add_sec(&sc->sc_tmo, 1); #endif @@ -2527,7 +2610,11 @@ pfsync_update_state(struct pf_state *st) void pfsync_request_update(u_int32_t creatorid, u_int64_t id) { +#ifdef __FreeBSD__ + struct pfsync_softc *sc = V_pfsyncif; +#else struct pfsync_softc *sc = pfsyncif; +#endif struct pfsync_upd_req_item *item; size_t nlen = sizeof(struct pfsync_upd_req); int s; @@ -2571,7 +2658,11 @@ pfsync_request_update(u_int32_t creatori void pfsync_update_state_req(struct pf_state *st) { +#ifdef __FreeBSD__ + struct pfsync_softc *sc = V_pfsyncif; +#else struct pfsync_softc *sc = pfsyncif; +#endif if (sc == NULL) panic("pfsync_update_state_req: nonexistant instance"); @@ -2606,7 +2697,11 @@ pfsync_update_state_req(struct pf_state void pfsync_delete_state(struct pf_state *st) { +#ifdef __FreeBSD__ + struct pfsync_softc *sc = V_pfsyncif; +#else struct pfsync_softc *sc = pfsyncif; +#endif #ifdef __FreeBSD__ PF_ASSERT(MA_OWNED); @@ -2628,7 +2723,7 @@ pfsync_delete_state(struct pf_state *st) if (sc->sc_len == PFSYNC_MINPKT) #ifdef __FreeBSD__ callout_reset(&sc->sc_tmo, 1 * hz, pfsync_timeout, - pfsyncif); + V_pfsyncif); #else timeout_add_sec(&sc->sc_tmo, 1); #endif @@ -2663,7 +2758,11 @@ pfsync_clear_states(u_int32_t creatorid, struct pfsync_clr clr; } __packed r; +#ifdef __FreeBSD__ + struct pfsync_softc *sc = V_pfsyncif; +#else struct pfsync_softc *sc = pfsyncif; +#endif #ifdef __FreeBSD__ PF_ASSERT(MA_OWNED); @@ -2688,7 +2787,11 @@ pfsync_clear_states(u_int32_t creatorid, void pfsync_q_ins(struct pf_state *st, int q) { +#ifdef __FreeBSD__ + struct pfsync_softc *sc = V_pfsyncif; +#else struct pfsync_softc *sc = pfsyncif; +#endif size_t nlen = pfsync_qs[q].len; int s; @@ -2732,7 +2835,11 @@ pfsync_q_ins(struct pf_state *st, int q) void pfsync_q_del(struct pf_state *st) { +#ifdef __FreeBSD__ + struct pfsync_softc *sc = V_pfsyncif; +#else struct pfsync_softc *sc = pfsyncif; +#endif int q = st->sync_state; #ifdef __FreeBSD__ @@ -2754,7 +2861,11 @@ pfsync_q_del(struct pf_state *st) void pfsync_update_tdb(struct tdb *t, int output) { +#ifdef __FreeBSD__ + struct pfsync_softc *sc = V_pfsyncif; +#else struct pfsync_softc *sc = pfsyncif; +#endif size_t nlen = sizeof(struct pfsync_tdb); int s; @@ -2792,7 +2903,11 @@ pfsync_update_tdb(struct tdb *t, int out void pfsync_delete_tdb(struct tdb *t) { +#ifdef __FreeBSD__ + struct pfsync_softc *sc = V_pfsyncif; +#else struct pfsync_softc *sc = pfsyncif; +#endif if (sc == NULL || !ISSET(t->tdb_flags, TDBF_PFSYNC)) return; @@ -2843,7 +2958,11 @@ pfsync_out_tdb(struct tdb *t, struct mbu void pfsync_bulk_start(void) { +#ifdef __FreeBSD__ + struct pfsync_softc *sc = V_pfsyncif; +#else struct pfsync_softc *sc = pfsyncif; +#endif sc->sc_ureq_received = time_uptime; @@ -2876,6 +2995,7 @@ pfsync_bulk_update(void *arg) s = splsoftnet(); #ifdef __FreeBSD__ + CURVNET_SET(sc->sc_ifp->if_vnet); PF_LOCK(); #endif do { @@ -2914,6 +3034,7 @@ pfsync_bulk_update(void *arg) out: #ifdef __FreeBSD__ PF_UNLOCK(); + CURVNET_RESTORE(); #endif splx(s); } @@ -2926,7 +3047,11 @@ pfsync_bulk_status(u_int8_t status) struct pfsync_bus bus; } __packed r; +#ifdef __FreeBSD__ + struct pfsync_softc *sc = V_pfsyncif; +#else struct pfsync_softc *sc = pfsyncif; +#endif bzero(&r, sizeof(r)); @@ -2949,11 +3074,15 @@ pfsync_bulk_fail(void *arg) { struct pfsync_softc *sc = arg; +#ifdef __FreeBSD__ + CURVNET_SET(sc->sc_ifp->if_vnet); +#endif + if (sc->sc_bulk_tries++ < PFSYNC_MAX_BULKTRIES) { /* Try again */ #ifdef __FreeBSD__ callout_reset(&sc->sc_bulkfail_tmo, 5 * hz, - pfsync_bulk_fail, pfsyncif); + pfsync_bulk_fail, V_pfsyncif); #else timeout_add_sec(&sc->sc_bulkfail_tmo, 5); #endif @@ -2964,11 +3093,19 @@ pfsync_bulk_fail(void *arg) sc->sc_bulk_tries = 0; #if NCARP > 0 #ifdef notyet +#ifdef __FreeBSD__ + if (!sc->pfsync_sync_ok) +#else if (!pfsync_sync_ok) +#endif carp_group_demote_adj(&sc->sc_if, -1); #endif #endif +#ifdef __FreeBSD__ + sc->pfsync_sync_ok = 1; +#else pfsync_sync_ok = 1; +#endif #ifdef __FreeBSD__ if (V_pf_status.debug >= PF_DEBUG_MISC) #else @@ -2976,12 +3113,20 @@ pfsync_bulk_fail(void *arg) #endif printf("pfsync: failed to receive bulk update\n"); } + +#ifdef __FreeBSD__ + CURVNET_RESTORE(); +#endif } void pfsync_send_plus(void *plus, size_t pluslen) { +#ifdef __FreeBSD__ + struct pfsync_softc *sc = V_pfsyncif; +#else struct pfsync_softc *sc = pfsyncif; +#endif int s; #ifdef __FreeBSD__ @@ -3017,7 +3162,11 @@ pfsync_send_plus(void *plus, size_t plus int pfsync_up(void) { +#ifdef __FreeBSD__ + struct pfsync_softc *sc = V_pfsyncif; +#else struct pfsync_softc *sc = pfsyncif; +#endif #ifdef __FreeBSD__ if (sc == NULL || !ISSET(sc->sc_ifp->if_flags, IFF_DRV_RUNNING)) @@ -3032,7 +3181,11 @@ pfsync_up(void) int pfsync_state_in_use(struct pf_state *st) { +#ifdef __FreeBSD__ + struct pfsync_softc *sc = V_pfsyncif; +#else struct pfsync_softc *sc = pfsyncif; +#endif if (sc == NULL) return (0); @@ -3052,8 +3205,15 @@ u_int pfsync_tmos; void pfsync_timeout(void *arg) { +#if defined(__FreeBSD__) && defined(VIMAGE) + struct pfsync_softc *sc = arg; +#endif int s; +#ifdef __FreeBSD__ + CURVNET_SET(sc->sc_ifp->if_vnet); +#endif + pfsync_tmos++; s = splnet(); @@ -3065,18 +3225,31 @@ pfsync_timeout(void *arg) PF_UNLOCK(); #endif splx(s); + +#ifdef __FreeBSD__ + CURVNET_RESTORE(); +#endif } /* this is a softnet/netisr handler */ void #ifdef __FreeBSD__ -pfsyncintr(void *unused) +pfsyncintr(void *arg) #else pfsyncintr(void) #endif { +#ifdef __FreeBSD__ + struct pfsync_softc *sc = arg; +#endif int s; +#ifdef __FreeBSD__ + if (sc == NULL) + return; + + CURVNET_SET(sc->sc_ifp->if_vnet); +#endif pfsync_ints++; s = splnet(); @@ -3088,6 +3261,10 @@ pfsyncintr(void) PF_UNLOCK(); #endif splx(s); + +#ifdef __FreeBSD__ + CURVNET_RESTORE(); +#endif } #ifdef notyet @@ -3121,6 +3298,8 @@ pfsync_ifdetach(void *arg, struct ifnet if (sc == NULL || sc->sc_sync_if != ifp) return; /* not for us; unlocked read */ + CURVNET_SET(sc->sc_ifp->if_vnet); + PF_LOCK(); /* Deal with a member interface going away from under us. */ @@ -3142,8 +3321,73 @@ pfsync_ifdetach(void *arg, struct ifnet } PF_UNLOCK(); + + CURVNET_RESTORE(); +} + +static int +vnet_pfsync_init(const void *unused) +{ + int error = 0; + + pfsyncattach(0); + + error = swi_add(NULL, "pfsync", pfsyncintr, pfsyncif, + SWI_NET, INTR_MPSAFE, &pfsync_swi.pfsync_swi_cookie); + if (error) + panic("%s: swi_add %d", __func__, error); + + pfsync_state_import_ptr = pfsync_state_import; + pfsync_up_ptr = pfsync_up; + pfsync_insert_state_ptr = pfsync_insert_state; + pfsync_update_state_ptr = pfsync_update_state; + pfsync_delete_state_ptr = pfsync_delete_state; + pfsync_clear_states_ptr = pfsync_clear_states; + pfsync_state_in_use_ptr = pfsync_state_in_use; + pfsync_defer_ptr = pfsync_defer; + + return (0); +} + +static int +vnet_pfsync_uninit(const void *unused) +{ + + swi_remove(pfsync_swi.pfsync_swi_cookie); + + pfsync_state_import_ptr = NULL; + pfsync_up_ptr = NULL; + pfsync_insert_state_ptr = NULL; + pfsync_update_state_ptr = NULL; + pfsync_delete_state_ptr = NULL; + pfsync_clear_states_ptr = NULL; + pfsync_state_in_use_ptr = NULL; + pfsync_defer_ptr = NULL; + + if_clone_detach(&pfsync_cloner); + + return (0); } +/* Define startup order. */ +#define PFSYNC_SYSINIT_ORDER SI_SUB_PROTO_BEGIN +#define PFSYNC_MODEVENT_ORDER (SI_ORDER_FIRST) /* On boot slot in here. */ +#define PFSYNC_VNET_ORDER (PFSYNC_MODEVENT_ORDER + 2) /* Later still. */ + +/* + * Starting up. + * VNET_SYSINIT is called for each existing vnet and each new vnet. + */ +VNET_SYSINIT(vnet_pfsync_init, PFSYNC_SYSINIT_ORDER, PFSYNC_VNET_ORDER, + vnet_pfsync_init, NULL); + +/* + * Closing up shop. These are done in REVERSE ORDER, + * Not called on reboot. + * VNET_SYSUNINIT is called for each exiting vnet as it exits. + */ +VNET_SYSUNINIT(vnet_pfsync_uninit, PFSYNC_SYSINIT_ORDER, PFSYNC_VNET_ORDER, + vnet_pfsync_uninit, NULL); static int pfsync_modevent(module_t mod, int type, void *data) { @@ -3151,37 +3395,14 @@ pfsync_modevent(module_t mod, int type, switch (type) { case MOD_LOAD: +#ifndef __FreeBSD__ pfsyncattach(0); -#ifdef __FreeBSD__ - error = swi_add(NULL, "pfsync", pfsyncintr, NULL, - SWI_NET, INTR_MPSAFE, &pfsync_swi.pfsync_swi_cookie); - if (error) - panic("%s: swi_add %d", __func__, error); - - pfsync_state_import_ptr = pfsync_state_import; - pfsync_up_ptr = pfsync_up; - pfsync_insert_state_ptr = pfsync_insert_state; - pfsync_update_state_ptr = pfsync_update_state; - pfsync_delete_state_ptr = pfsync_delete_state; - pfsync_clear_states_ptr = pfsync_clear_states; - pfsync_state_in_use_ptr = pfsync_state_in_use; - pfsync_defer_ptr = pfsync_defer; #endif break; case MOD_UNLOAD: -#ifdef __FreeBSD__ - swi_remove(pfsync_swi.pfsync_swi_cookie); - - pfsync_state_import_ptr = NULL; - pfsync_up_ptr = NULL; - pfsync_insert_state_ptr = NULL; - pfsync_update_state_ptr = NULL; - pfsync_delete_state_ptr = NULL; - pfsync_clear_states_ptr = NULL; - pfsync_state_in_use_ptr = NULL; - pfsync_defer_ptr = NULL; -#endif +#ifndef __FreeBSD__ if_clone_detach(&pfsync_cloner); +#endif break; default: error = EINVAL; From owner-svn-src-user@FreeBSD.ORG Tue Dec 8 22:49:03 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C1337106566C; Tue, 8 Dec 2009 22:49:03 +0000 (UTC) (envelope-from eri@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id B066C8FC15; Tue, 8 Dec 2009 22:49:03 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nB8Mn3gx003909; Tue, 8 Dec 2009 22:49:03 GMT (envelope-from eri@svn.freebsd.org) Received: (from eri@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nB8Mn3q6003907; Tue, 8 Dec 2009 22:49:03 GMT (envelope-from eri@svn.freebsd.org) Message-Id: <200912082249.nB8Mn3q6003907@svn.freebsd.org> From: Ermal Luçi Date: Tue, 8 Dec 2009 22:49:03 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r200283 - user/eri/pf45/head/sys/modules/pfsync X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Dec 2009 22:49:03 -0000 Author: eri Date: Tue Dec 8 22:49:03 2009 New Revision: 200283 URL: http://svn.freebsd.org/changeset/base/200283 Log: Teach pfsync(4) makefile how to build a VIMAGE enabled module. Modified: user/eri/pf45/head/sys/modules/pfsync/Makefile Modified: user/eri/pf45/head/sys/modules/pfsync/Makefile ============================================================================== --- user/eri/pf45/head/sys/modules/pfsync/Makefile Tue Dec 8 22:43:29 2009 (r200282) +++ user/eri/pf45/head/sys/modules/pfsync/Makefile Tue Dec 8 22:49:03 2009 (r200283) @@ -22,6 +22,12 @@ opt_inet6.h: opt_bpf.h: echo "#define DEV_BPF 1" > ${.TARGET} + +.if defined(VIMAGE) +opt_global.h: + echo "#define VIMAGE 1" > ${.TARGET} +CFLAGS+= -I${.CURDIR}/opt_global.h +.endif .endif .include From owner-svn-src-user@FreeBSD.ORG Thu Dec 10 20:06:38 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 00D44106566B; Thu, 10 Dec 2009 20:06:38 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id E4E6D8FC15; Thu, 10 Dec 2009 20:06:37 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nBAK6bOY073591; Thu, 10 Dec 2009 20:06:37 GMT (envelope-from dougb@svn.freebsd.org) Received: (from dougb@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nBAK6bLq073589; Thu, 10 Dec 2009 20:06:37 GMT (envelope-from dougb@svn.freebsd.org) Message-Id: <200912102006.nBAK6bLq073589@svn.freebsd.org> From: Doug Barton Date: Thu, 10 Dec 2009 20:06:37 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r200363 - user/dougb/portmaster X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Dec 2009 20:06:38 -0000 Author: dougb Date: Thu Dec 10 20:06:37 2009 New Revision: 200363 URL: http://svn.freebsd.org/changeset/base/200363 Log: In the test for -PP being alone on the command line I forgot the -dash at the beginning of the second test so it was falsely picking up ports with names like p5-Devel-PPPort. If you are building more than one port on the command line, and you use --packages-build and/or --delete-build-only, and one of the ports on the command line would otherwise qualfiy as a build-only dep, it was falsely being treated as such. Solve this problem by adding all ports on the command line to the run_dl_g list in multiport(). Modified: user/dougb/portmaster/portmaster Modified: user/dougb/portmaster/portmaster ============================================================================== --- user/dougb/portmaster/portmaster Thu Dec 10 18:38:40 2009 (r200362) +++ user/dougb/portmaster/portmaster Thu Dec 10 20:06:37 2009 (r200363) @@ -414,7 +414,7 @@ e2="The --packages-build option and the for var in "$@" ; do case "$var" in - -PP[A-Za-z0-9]*|*[A-Za-z0-9]PP*) + -PP[A-Za-z0-9]*|-*[A-Za-z0-9]PP*) fail "The -PP option must stand alone" ;; --packages) packages_init first PM_PACKAGES=first ; export PM_PACKAGES ;; @@ -1912,7 +1912,14 @@ multiport () { if [ -n "$CONFIG_ONLY" ]; then if [ -n "$PM_BUILD_ONLY_LIST" ]; then PM_BUILD_ONLY_LIST=pmp_doing_build_deps - run_dl_g='' ; build_only_dl_g='' + for port in $worklist; do + case "$port" in + */*) run_dl_g="$run_dl_g ${pd}/${port} " ;; + *) + run_dl_g="$run_dl_g ${pd}/`origin_from_pdb $port` " ;; + esac + done + build_only_dl_g='' export run_dl_g build_only_dl_g fi if [ -n "$PM_DEL_BUILD_ONLY" ]; then From owner-svn-src-user@FreeBSD.ORG Thu Dec 10 23:16:51 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 46FB010656AE; Thu, 10 Dec 2009 23:16:51 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 36BFA8FC27; Thu, 10 Dec 2009 23:16:51 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nBANGpoH077757; Thu, 10 Dec 2009 23:16:51 GMT (envelope-from dougb@svn.freebsd.org) Received: (from dougb@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nBANGpGC077755; Thu, 10 Dec 2009 23:16:51 GMT (envelope-from dougb@svn.freebsd.org) Message-Id: <200912102316.nBANGpGC077755@svn.freebsd.org> From: Doug Barton Date: Thu, 10 Dec 2009 23:16:51 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r200370 - user/dougb/portmaster X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Dec 2009 23:16:51 -0000 Author: dougb Date: Thu Dec 10 23:16:50 2009 New Revision: 200370 URL: http://svn.freebsd.org/changeset/base/200370 Log: In the procedure for using portmaster to re-install after a major version upgrade, list "re-install portmaster" as an actual step. Submitted by: Kevin Oberman Modified: user/dougb/portmaster/portmaster.8 Modified: user/dougb/portmaster/portmaster.8 ============================================================================== --- user/dougb/portmaster/portmaster.8 Thu Dec 10 22:25:53 2009 (r200369) +++ user/dougb/portmaster/portmaster.8 Thu Dec 10 23:16:50 2009 (r200370) @@ -24,7 +24,7 @@ .\" .\" $FreeBSD$ .\" -.Dd December 1, 2009 +.Dd December 2, 2009 .Dt PORTMASTER 8 .Os .Sh NAME @@ -770,7 +770,8 @@ to do a complete reinstallation of all y .Dl "7. rm -rf /usr/local/lib/compat/pkg" .Dl "8. Manually check /usr/local and /var/db/pkg" .Dl " to make sure that they are really empty" -.Dl "9. portmaster `cat ~/installed-port-list`" +.Dl "9. Re-install portmaster" +.Dl "10. portmaster `cat ~/installed-port-list`" .Pp You probably want to use the -D option for the installation and then run --clean-distfiles[-all] again when you are done. From owner-svn-src-user@FreeBSD.ORG Thu Dec 10 23:28:54 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C62DC106566C; Thu, 10 Dec 2009 23:28:54 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id B57C68FC19; Thu, 10 Dec 2009 23:28:54 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nBANSsEM077999; Thu, 10 Dec 2009 23:28:54 GMT (envelope-from dougb@svn.freebsd.org) Received: (from dougb@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nBANSsD1077997; Thu, 10 Dec 2009 23:28:54 GMT (envelope-from dougb@svn.freebsd.org) Message-Id: <200912102328.nBANSsD1077997@svn.freebsd.org> From: Doug Barton Date: Thu, 10 Dec 2009 23:28:54 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r200371 - user/dougb/portmaster X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Dec 2009 23:28:54 -0000 Author: dougb Date: Thu Dec 10 23:28:54 2009 New Revision: 200371 URL: http://svn.freebsd.org/changeset/base/200371 Log: Indicate that -v can be used with --check-port-dbdir Modified: user/dougb/portmaster/portmaster.8 Modified: user/dougb/portmaster/portmaster.8 ============================================================================== --- user/dougb/portmaster/portmaster.8 Thu Dec 10 23:16:50 2009 (r200370) +++ user/dougb/portmaster/portmaster.8 Thu Dec 10 23:28:54 2009 (r200371) @@ -108,6 +108,7 @@ and/or multiple globs from /var/db/pkg .Fl -check-depends .Nm .Fl -check-port-dbdir +.Op Fl v .Nm .Fl -list-origins .Nm @@ -440,7 +441,7 @@ If not, offer to delete the stale file. does the same as above, but deletes all files without prompting. .It Fl -check-depends cross-check and update dependency information for all ports -.It Fl -check-port-dbdir +.It [-v] Fl -check-port-dbdir check for stale entries in .Pa /var/db/ports .It Fl -list-origins From owner-svn-src-user@FreeBSD.ORG Fri Dec 11 07:54:38 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C21171065672; Fri, 11 Dec 2009 07:54:38 +0000 (UTC) (envelope-from luigi@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 98C728FC16; Fri, 11 Dec 2009 07:54:38 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nBB7sccb099478; Fri, 11 Dec 2009 07:54:38 GMT (envelope-from luigi@svn.freebsd.org) Received: (from luigi@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nBB7scF8099477; Fri, 11 Dec 2009 07:54:38 GMT (envelope-from luigi@svn.freebsd.org) Message-Id: <200912110754.nBB7scF8099477@svn.freebsd.org> From: Luigi Rizzo Date: Fri, 11 Dec 2009 07:54:38 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r200400 - user/luigi/ipfw_80 X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Dec 2009 07:54:38 -0000 Author: luigi Date: Fri Dec 11 07:54:38 2009 New Revision: 200400 URL: http://svn.freebsd.org/changeset/base/200400 Log: remove now unused branch Deleted: user/luigi/ipfw_80/ From owner-svn-src-user@FreeBSD.ORG Fri Dec 11 07:54:56 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1C11F1065692; Fri, 11 Dec 2009 07:54:56 +0000 (UTC) (envelope-from luigi@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id E6E998FC0C; Fri, 11 Dec 2009 07:54:55 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nBB7stvI099517; Fri, 11 Dec 2009 07:54:55 GMT (envelope-from luigi@svn.freebsd.org) Received: (from luigi@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nBB7stXK099516; Fri, 11 Dec 2009 07:54:55 GMT (envelope-from luigi@svn.freebsd.org) Message-Id: <200912110754.nBB7stXK099516@svn.freebsd.org> From: Luigi Rizzo Date: Fri, 11 Dec 2009 07:54:55 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r200401 - user/luigi/geom_sched X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Dec 2009 07:54:56 -0000 Author: luigi Date: Fri Dec 11 07:54:55 2009 New Revision: 200401 URL: http://svn.freebsd.org/changeset/base/200401 Log: remove another unused branch Deleted: user/luigi/geom_sched/ From owner-svn-src-user@FreeBSD.ORG Fri Dec 11 07:56:37 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E5B08106566B; Fri, 11 Dec 2009 07:56:37 +0000 (UTC) (envelope-from luigi@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id BC8FF8FC15; Fri, 11 Dec 2009 07:56:37 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nBB7ubnd099605; Fri, 11 Dec 2009 07:56:37 GMT (envelope-from luigi@svn.freebsd.org) Received: (from luigi@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nBB7ubAQ099604; Fri, 11 Dec 2009 07:56:37 GMT (envelope-from luigi@svn.freebsd.org) Message-Id: <200912110756.nBB7ubAQ099604@svn.freebsd.org> From: Luigi Rizzo Date: Fri, 11 Dec 2009 07:56:37 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r200402 - user/luigi/ipfw3-head X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Dec 2009 07:56:38 -0000 Author: luigi Date: Fri Dec 11 07:56:37 2009 New Revision: 200402 URL: http://svn.freebsd.org/changeset/base/200402 Log: create a working area for ipfw+dummynet v3. Base version is head-200401 Added: - copied from r200401, head/ Directory Properties: user/luigi/ipfw3-head/ (props changed) From owner-svn-src-user@FreeBSD.ORG Sat Dec 12 03:35:49 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E5D4B106566B; Sat, 12 Dec 2009 03:35:49 +0000 (UTC) (envelope-from kmacy@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id D2F788FC08; Sat, 12 Dec 2009 03:35:49 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nBC3ZnoV029046; Sat, 12 Dec 2009 03:35:49 GMT (envelope-from kmacy@svn.freebsd.org) Received: (from kmacy@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nBC3Zn0Z029042; Sat, 12 Dec 2009 03:35:49 GMT (envelope-from kmacy@svn.freebsd.org) Message-Id: <200912120335.nBC3Zn0Z029042@svn.freebsd.org> From: Kip Macy Date: Sat, 12 Dec 2009 03:35:49 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r200428 - in user/kmacy/releng_8_fcs_buf_xen/sys/cddl/contrib/opensolaris/uts/common/fs/zfs: . sys X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Dec 2009 03:35:50 -0000 Author: kmacy Date: Sat Dec 12 03:35:49 2009 New Revision: 200428 URL: http://svn.freebsd.org/changeset/base/200428 Log: checkpoint mostly complety state of ARC / VM integration Modified: user/kmacy/releng_8_fcs_buf_xen/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zfs_bio.h user/kmacy/releng_8_fcs_buf_xen/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_bio.c user/kmacy/releng_8_fcs_buf_xen/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c Modified: user/kmacy/releng_8_fcs_buf_xen/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zfs_bio.h ============================================================================== --- user/kmacy/releng_8_fcs_buf_xen/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zfs_bio.h Sat Dec 12 02:34:00 2009 (r200427) +++ user/kmacy/releng_8_fcs_buf_xen/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zfs_bio.h Sat Dec 12 03:35:49 2009 (r200428) @@ -34,7 +34,7 @@ $FreeBSD$ #define ZBIO_BUF_CLONING (1 << 30) /* is being cloned */ -void zbio_sync_cache(spa_t *spa, blkptr_t *bp, uint64_t txg, uint64_t size); +void zbio_sync_cache(spa_t *spa, blkptr_t *bp, uint64_t txg, void *data, uint64_t size, int bio_op); void zbio_getblk(arc_buf_t *buf); void zbio_data_getblk(arc_buf_t *buf); void zbio_relse(arc_buf_t *buf, size_t size); Modified: user/kmacy/releng_8_fcs_buf_xen/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_bio.c ============================================================================== --- user/kmacy/releng_8_fcs_buf_xen/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_bio.c Sat Dec 12 02:34:00 2009 (r200427) +++ user/kmacy/releng_8_fcs_buf_xen/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_bio.c Sat Dec 12 03:35:49 2009 (r200428) @@ -27,6 +27,57 @@ POSSIBILITY OF SUCH DAMAGE. ***************************************************************************/ +/************************************************************************** +This module integrates the caching af pages associated with ARC buffers in a +per-SPA vm object. Each SPA also has an associated "zbio_state_t" which +tracks bufs allocated for the SPA in two splay trees. + +The first splay tree tracks bufs by the data pointer's virtual address. +It is used for malloc'ed buffers, and buffers that are VMIO but do not have +any pages in the SPA's vm object(s). + +Buffers are malloced if: + 1) the size is not a multiple of PAGE_SIZE + 2) the buffer is cloned + +There are two reasons why a VMIO buf would not have any pages in the vm object: + 1) the buffer has not yet been assigned an address on disk (and thus + has no offset in the vm object) + 2) the buffer did have pages in the vm object, but they were evicted + and replaced by a newer + +The second splay tree tracks buffers by block address and is only used +to track buffers whose pages are referenced by the vm object. It is used to +ensure that buffers that belong to an older transaction group don't have their +pages mapped by buffers belonging to a newer transaction group. + +zfs_bio assumes that buffers that are cloned and buffers whose pages +are evicted from the vm object are not used for I/O (will not be referenced +from zfs_bio_sync_cache). + +Pages in the vm object are marked valid on completion of a read or before the +initiation of a write. + + + +There are two places where we synchronize the ARC with the vm object's +page cache: getblk and sync_cache. + +In getblk for a malloced buffer we check if the page at the corresponding offset +is valid, if it is map it in and copy it in to the new buffer. For a VMIO buffer +we need to remove the pages for any existing overlapping buffers and free any +other pages in the vm object. + +In sync_cache for a malloced buffer we need to evict pages belonging to overlapping +VMIO buffers, then copy to/from any pages still in the vm object. For an unmapped +VMIO buffer, we need to remove pages belonging to any existing buffers and free +any remaining overlapping pages in the vm object. We then add the VMIO buffers +pages to a VM object. If the buffer is already mapped we mark the pages valid on a +write, on a read we set a flag in the zio and mark the pages valid before calling +the io_done I/O completion function. + + +**************************************************************************/ #include __FBSDID("$FreeBSD$"); @@ -42,6 +93,7 @@ __FBSDID("$FreeBSD$"); #include #include +#include #include #ifdef _KERNEL @@ -58,180 +110,734 @@ SYSCTL_INT(_vfs_zfs, OID_AUTO, page_cach &zfs_page_cache_disable, 0, "Disable backing ARC with page cache "); static eventhandler_tag zbio_event_shutdown = NULL; - +struct zbio_state; +typedef struct zbio_state zbio_state_t; +typedef struct buf buf_t; +typedef uint64_t zbio_pindex_t; + +MALLOC_DEFINE(M_ZFS_BIO, "zfs_bio", "zfs buffer cache / vm"); + +#define B_EVICTED B_00000800 +#define B_CLONED B_00001000 +#define B_ASSIGNED B_00004000 + +#define ZB_EVICT_ALL 0x1 + +#define btos(nbytes) ((nbytes)>>DEV_BSHIFT) +#define stob(nsectors) ((nsectors)<mtx) +#define ZBIO_STATE_UNLOCK(zs) mtx_unlock(&(zs)->mtx) + +#define spa_get_bio_state(spa) ((zbio_state_t *)spa_get_vnode((spa))->v_data) +#define spa_get_vm_object(spa) spa_get_vnode((spa))->v_object +#define zbio_buf_get_spa(bp) (((zbio_buf_hdr_t *)((arc_buf_t *)(bp->b_arc_buf))->b_hdr)->b_spa) + +static void zbio_buf_blkno_remove(buf_t *bp); +static void zbio_buf_va_insert(buf_t *bp, zbio_state_t *object); + +/* + * zbio_buf_blkno_splay: [ internal use only ] + * + * Implements Sleator and Tarjan's top-down splay algorithm. Returns + * the buf containing the given lblkno. If, however, that + * lblkno is not found in the tree, returns a buf that is + * adjacent to the pindex, coming before or after it. + */ +static buf_t * +zbio_buf_blkno_splay(daddr_t blkno, buf_t *root) +{ + buf_t dummy; + buf_t *lefttreemax, *righttreemin, *y; + + if (root == NULL) + return (root); + lefttreemax = righttreemin = &dummy; + for (;; root = y) { + if (blkno < root->b_blkno) { + if ((y = root->b_left) == NULL) + break; + if (blkno < y->b_blkno) { + /* Rotate right. */ + root->b_left = y->b_right; + y->b_right = root; + root = y; + if ((y = root->b_left) == NULL) + break; + } + /* Link into the new root's right tree. */ + righttreemin->b_left = root; + righttreemin = root; + } else if (blkno > root->b_blkno) { + if ((y = root->b_right) == NULL) + break; + if (blkno > y->b_blkno) { + /* Rotate left. */ + root->b_right = y->b_left; + y->b_left = root; + root = y; + if ((y = root->b_right) == NULL) + break; + } + /* Link into the new root's left tree. */ + lefttreemax->b_right = root; + lefttreemax = root; + } else + break; + } + /* Assemble the new root. */ + lefttreemax->b_right = root->b_left; + righttreemin->b_left = root->b_right; + root->b_left = dummy.b_right; + root->b_right = dummy.b_left; + return (root); +} + +static buf_t * +zbio_buf_va_splay(caddr_t va, buf_t *root) +{ + buf_t dummy; + buf_t *lefttreemax, *righttreemin, *y; + + if (root == NULL) + return (root); + lefttreemax = righttreemin = &dummy; + for (;; root = y) { + if (va < root->b_data) { + if ((y = root->b_left) == NULL) + break; + if (va < y->b_data) { + /* Rotate right. */ + root->b_left = y->b_right; + y->b_right = root; + root = y; + if ((y = root->b_left) == NULL) + break; + } + /* Link into the new root's right tree. */ + righttreemin->b_left = root; + righttreemin = root; + } else if (va > root->b_data) { + if ((y = root->b_right) == NULL) + break; + if (va > y->b_data) { + /* Rotate left. */ + root->b_right = y->b_left; + y->b_left = root; + root = y; + if ((y = root->b_right) == NULL) + break; + } + /* Link into the new root's left tree. */ + lefttreemax->b_right = root; + lefttreemax = root; + } else + break; + } + /* Assemble the new root. */ + lefttreemax->b_right = root->b_left; + righttreemin->b_left = root->b_right; + root->b_left = dummy.b_right; + root->b_right = dummy.b_left; + return (root); +} + +/* + * zbio_buf_blkno_insert: [ internal use only ] + * + * Inserts the given buf into the state splay tree and state list. + * + * The object and page must be locked. + * This routine may not block. + */ static void -_zbio_getblk(arc_buf_t *buf, int flags) +zbio_buf_blkno_insert(buf_t *bp, zbio_state_t *object) { - zbio_buf_hdr_t *hdr = (zbio_buf_hdr_t *)buf->b_hdr; - uint64_t size = hdr->b_size; - spa_t *spa = hdr->b_spa; - uint64_t blkno = hdr->b_dva.dva_word[1] & ~(1ULL<<63); - void *data; - struct vnode *vp; - struct buf *newbp; - struct bufobj *bo; - - vp = spa_get_vnode(spa); - bo = &vp->v_bufobj; - newbp = NULL; - if ((size < PAGE_SIZE) || (hdr->b_flags & ZBIO_BUF_CLONING) || - zfs_page_cache_disable) { - data = zio_buf_alloc(size); - hdr->b_flags &= ~ZBIO_BUF_CLONING; - } else if (BUF_EMPTY(hdr)) { - newbp = geteblk(size, flags); - data = newbp->b_data; + buf_t *root; + daddr_t root_blkno_end, blkno, blkno_end; + + blkno = bp->b_blkno; + blkno_end = bp->b_blkno + btos(bp->b_bcount); + + root = object->blkno_root; + if (root == NULL) { + bp->b_left = NULL; + bp->b_right = NULL; + TAILQ_INSERT_TAIL(&object->blkno_memq, bp, b_bobufs); } else { - newbp = getblk(vp, blkno, size, 0, 0, flags | GB_LOCK_NOWAIT); - if (newbp == NULL) - newbp = geteblk(size, flags); - else - brelvp(newbp); - data = newbp->b_data; - } + root = zbio_buf_blkno_splay(bp->b_blkno, root); + root_blkno_end = root->b_blkno + btos(root->b_bcount); - if (newbp != NULL) { - BUF_KERNPROC(newbp); - newbp->b_bufobj = bo; - CTR4(KTR_SPARE2, "arc_getblk() bp=%p flags %X " - "blkno %ld npages %d", - newbp, newbp->b_flags, blkno, newbp->b_npages); + if (blkno < root->b_blkno) { + KASSERT(blkno_end <= root->b_blkno, ("buffer overlap!")); + bp->b_left = root->b_left; + bp->b_right = root; + root->b_left = NULL; + TAILQ_INSERT_BEFORE(root, bp, b_bobufs); + } else if (blkno == root->b_blkno) { + panic("zbio_buf_blkno_insert: blkno already allocated"); + } else { + KASSERT(root_blkno_end <= blkno, ("buffer overlap!")); + + bp->b_right = root->b_right; + bp->b_left = root; + root->b_right = NULL; + TAILQ_INSERT_AFTER(&object->blkno_memq, root, bp, b_bobufs); + } } + object->blkno_root = bp; + object->generation++; - buf->b_bp = newbp; - buf->b_data = data; + /* + * show that the object has one more resident buffer. + */ + object->resident_count++; } -void -zbio_getblk(arc_buf_t *buf) +/* + * zbio_buf_insert: [ internal use only ] + * + * Inserts the given buf into the state splay tree and state list. + * + * The object and page must be locked. + * This routine may not block. + */ +static void +zbio_buf_va_insert(buf_t *bp, zbio_state_t *object) { + buf_t *root; + caddr_t va = bp->b_data; - _zbio_getblk(buf, 0); + bp->b_state = object; + root = object->va_root; + if (root == NULL) { + bp->b_left = NULL; + bp->b_right = NULL; + TAILQ_INSERT_TAIL(&object->va_memq, bp, b_bobufs); + } else { + root = zbio_buf_va_splay(bp->b_data, root); + if (va < root->b_data) { + bp->b_left = root->b_left; + bp->b_right = root; + root->b_left = NULL; + TAILQ_INSERT_BEFORE(root, bp, b_bobufs); + } else if (va == root->b_data) { + panic("zbio_buf_va_insert: address already allocated"); + } else { + bp->b_right = root->b_right; + bp->b_left = root; + root->b_right = NULL; + TAILQ_INSERT_AFTER(&object->va_memq, root, bp, b_bobufs); + } + } + object->va_root = bp; + object->generation++; + + /* + * show that the object has one more resident buffer. + */ + object->resident_count++; } -void -zbio_data_getblk(arc_buf_t *buf) +/* + * zbio_buf_remove: + * + * Removes the given buf from the spa's state tree + * buf list + * + * The state and buf must be locked. + * This routine may not block. + */ +static void +zbio_buf_blkno_remove(buf_t *bp) { + zbio_state_t *state; + buf_t *root; + daddr_t blkno, blkno_end; - _zbio_getblk(buf, GB_NODUMP); + if ((state = bp->b_state) == NULL) + return; + + /* + * Now remove from the object's list of backed pages. + */ + if (bp != state->blkno_root) + zbio_buf_blkno_splay(bp->b_blkno, state->blkno_root); + if (bp->b_left == NULL) + root = bp->b_right; + else { + root = zbio_buf_blkno_splay(bp->b_blkno, bp->b_left); + root->b_right = bp->b_right; + } + state->blkno_root = root; + TAILQ_REMOVE(&state->blkno_memq, bp, b_bobufs); + + /* + * And show that the object has one fewer resident page. + */ + state->resident_count--; + state->generation++; } -void -zbio_relse(arc_buf_t *buf, size_t size) +/* + * zbio_buf_va_remove: + * + * Removes the given buf from the spa's state tree + * buf list + * + * The state and buf must be locked. + * This routine may not block. + */ +static void +zbio_buf_va_remove(buf_t *bp) { - struct buf *bp = buf->b_bp; - void * data = buf->b_data; + zbio_state_t *state; + buf_t *root; + vm_offset_t va; - if (bp == NULL) { - zio_buf_free(data, size); + if ((state = bp->b_state) == NULL) return; + + /* + * Now remove from the object's list of backed pages. + */ + if (bp != state->va_root) + zbio_buf_va_splay(bp->b_data, state->va_root); + if (bp->b_left == NULL) + root = bp->b_right; + else { + root = zbio_buf_va_splay(bp->b_data, bp->b_left); + root->b_right = bp->b_right; } + state->va_root = root; + TAILQ_REMOVE(&state->va_memq, bp, b_bobufs); - CTR4(KTR_SPARE2, "arc_brelse() bp=%p flags %X" - " size %ld blkno=%ld", - bp, bp->b_flags, size, bp->b_blkno); + /* + * And show that the object has one fewer resident page. + */ + state->resident_count--; + state->generation++; +} - bp->b_flags |= B_ZFS; - brelse(bp); +/* + * zbio_buf_va_lookup: + * + * Returns the range associated with the object/offset + * pair specified; if none is found, NULL is returned. + * + * The object must be locked. + * This routine may not block. + * This is a critical path routine + */ +static buf_t * +zbio_buf_va_lookup(zbio_state_t *state, caddr_t va) +{ + buf_t *bp; + + if ((bp = state->va_root) != NULL && bp->b_data != va) { + bp = zbio_buf_va_splay(va, bp); + if ((state->va_root = bp)->b_data != va) + bp = NULL; + } + return (bp); } -void -zbio_sync_cache(spa_t *spa, blkptr_t *bp, uint64_t txg, uint64_t size) + +/* + * zbio_buf_blkno_lookup: + * + * Returns the range associated with the object/offset + * pair specified; if none is found, NULL is returned. + * + * The object must be locked. + * This routine may not block. + * This is a critical path routine + */ +static buf_t * +zbio_buf_blkno_lookup(zbio_state_t *state, daddr_t blkno) +{ + buf_t *bp; + + if ((bp = state->blkno_root) != NULL && bp->b_blkno != blkno) { + bp = zbio_buf_blkno_splay(blkno, bp); + if ((state->blkno_root = bp)->b_blkno != blkno) + bp = NULL; + } + return (bp); +} + +static void +zbio_buf_vm_object_copyin(buf_t *bp) { -#ifdef notyet - uint64_t blkno, blkno_lookup; - struct vnode *vp; - struct bufobj *bo; - struct buf *bp; - vm_pindex_t start, end; - vm_object_t object; - vm_page_t m; - int i; - if (zfs_page_cache_disable) - return; - blkno_lookup = blkno = dva->dva_word[1] & ~(1ULL<<63); - vp = spa_get_vnode(spa); - bo = &vp->v_bufobj; + +} - if (dva == NULL || spa == NULL || blkno == 0 || size == 0) - return; +static void +zbio_buf_vm_object_copyout(buf_t *bp) +{ - start = OFF_TO_IDX((blkno_lookup << 9)); - end = start + OFF_TO_IDX(size + PAGE_MASK); - object = vp->v_object; + +} - VM_OBJECT_LOCK(object); - vm_page_cache_free(object, start, end); - vm_object_page_remove(object, start, end, FALSE); -#ifdef INVARIANTS - for (i = 0; i < OFF_TO_IDX(size); i++) { - KASSERT(vm_page_lookup(object, start + i) == NULL, - ("found page at %ld blkno %ld blkno_lookup %ld", - start + i, blkno, blkno_lookup)); - } -#endif - VM_OBJECT_UNLOCK(object); -#endif +static void +zbio_buf_vm_object_evict(buf_t *bp) +{ + int i; + + /* + * remove pages from backing vm_object + */ + for (i = 0; i < bp->b_npages; i++) + vm_page_remove(bp->b_pages[i]); } -#if 0 static void -arc_pcache(struct vnode *vp, struct buf *bp, uint64_t blkno) +zbio_buf_vm_object_insert(buf_t *bp, int valid) { - vm_pindex_t start = OFF_TO_IDX((blkno << 9)); - vm_object_t object = vp->v_object; - struct bufobj *bo = &vp->v_bufobj; vm_page_t m; + vm_pindex_t start = OFF_TO_IDX(stob(bp->b_blkno)); + spa_t *spa = zbio_buf_get_spa(bp); + struct vnode *vp = spa_get_vnode(spa); + struct vm_object *object = vp->v_object; int i; - CTR3(KTR_SPARE2, "arc_pcache() bp=%p blkno %ld npages %d", - bp, blkno, bp->b_npages); VM_OBJECT_LOCK(object); - vm_page_lock_queues(); + /* + * Insert buffer pages in the object + */ for (i = 0; i < bp->b_npages; i++) { m = bp->b_pages[i]; - m->valid = VM_PAGE_BITS_ALL; + if (valid) + m->valid = VM_PAGE_BITS_ALL; vm_page_insert(m, object, start + i); m->flags &= ~PG_UNMANAGED; - vm_page_enqueue(PQ_INACTIVE, m); vdrop(vp); } + vm_page_lock_queues(); + for (i = 0; i < bp->b_npages; i++) { + m = bp->b_pages[i]; + vm_page_enqueue(PQ_INACTIVE, m); + } vm_page_unlock_queues(); VM_OBJECT_UNLOCK(object); - bp->b_bufobj = bo; - bp->b_flags |= B_VMIO; + } +/* + * zbio_buf_evict_overlap: [ internal use only ] + * + * Evict the pages of any buffers overlapping with this range + * + * If ZB_EVICT_ALL is passed then evict all the pages in that range + * from the vm object + * + * The object and page must be locked. + * This routine may not block. + */ static void -arc_bcache(arc_buf_t *buf) -{ - uint64_t blkno = buf->b_hdr->b_dva.dva_word[1] & ~(1ULL<<63); - struct buf *bp; - struct vnode *vp = spa_get_vnode(buf->b_hdr->b_spa); - arc_buf_hdr_t *hdr = buf->b_hdr; - int cachebuf; +zbio_buf_blkno_evict_overlap(daddr_t blkno, int size, zbio_state_t *state, + uint64_t txg, int evict_op, int locked) +{ + buf_t *root, *tmpbp; + daddr_t blkno_end, tmpblkno, tmpblkno_end; + struct cluster_list_head clh; + int i, collisions; + uint64_t tmptxg; + vm_pindex_t start, end; + vm_object_t object = spa_get_vm_object(state->spa); - if (zfs_page_cache_disable) + if (!locked) + VM_OBJECT_LOCK(object); + if ((root = state->blkno_root) == NULL) + goto done; + + collisions = 0; + root = zbio_buf_blkno_splay(blkno, root); + TAILQ_INIT(&clh); + if (blkno < root->b_blkno) + tmpbp = TAILQ_PREV(root, cluster_list_head, b_bobufs); + + /* + * Find all existing buffers that overlap with this range + */ + tmpbp = tmpbp != NULL ? tmpbp : root; + while (tmpbp != NULL && tmpbp->b_blkno < blkno_end) { + tmpblkno = tmpbp->b_blkno; + tmpblkno_end = tmpblkno + btos(tmpbp->b_bcount); + tmptxg = ((zbio_buf_hdr_t *)((arc_buf_t *)tmpbp->b_arc_buf)->b_hdr)->b_birth; + + if (((tmpblkno >= blkno) && (tmpblkno < blkno_end)) || + (tmpblkno_end > blkno) && (tmpblkno_end <= blkno_end) && + ((txg == 0) || (tmptxg < txg))) { + TAILQ_INSERT_TAIL(&clh, tmpbp, b_freelist); + collisions++; + } + tmpbp = TAILQ_NEXT(tmpbp, b_bobufs); + } + while (!TAILQ_EMPTY(&clh)) { + tmpbp = TAILQ_FIRST(&clh); + TAILQ_REMOVE(&clh, tmpbp, b_freelist); + zbio_buf_vm_object_evict(tmpbp); + + KASSERT(tmpbp->b_flags & B_EVICTED == 0, + ("buffer has already been evicted")); + tmpbp->b_flags |= B_EVICTED; + state->blkno_root = tmpbp; + /* + * move buffer to the unmanaged tree + */ + zbio_buf_blkno_remove(tmpbp); + zbio_buf_va_insert(tmpbp, state); + } +done: + if (!(collisions == 1 && tmpbp->b_blkno == blkno && tmpbp->b_bcount == size) + && (evict_op == ZB_EVICT_ALL)) { + start = OFF_TO_IDX(stob(blkno)); + end = start + OFF_TO_IDX(size); + vm_page_cache_free(object, start, end); + vm_object_page_remove(object, start, end, FALSE); +#ifdef INVARIANTS + for (i = 0; i < OFF_TO_IDX(size); i++) { + KASSERT(vm_page_lookup(object, start + i) == NULL, + ("found page at %ld blkno %ld ",start + i, blkno)); + } +#endif + } + if (!locked) + VM_OBJECT_UNLOCK(object); +} + +/* +Cases: + +A) B_MALLOC / address is known + 1) getblk: + a) page cached: copyin + mark B_CACHE + b) buffer+page cached: copyin + mark B_CACHE + c) default: N/A + 2) sync_cache: + a) page cached: copy{in, out} + b) buffer+page cached: evict overlapping pages + c) default: N/A +B) B_MALLOC / address is !known + 1) getblk: N/A + 2) sync_cache: + a) page cached: copy{in, out} + b) buffer+page cached: evict overlapping pages + c) default: N/A + +C) !B_MALLOC / address is !known + 2) sync_cache: + a) page cached: evict/free old pages + replace + b) buffer+page cached: evict overlapping pages from object + replace + c) default: add pages to vm object + +D) !B_MALLOC / address is known + 1) getblk: + a) buffer+page cached: evict pages belonging to older buffer + b) default: N/A + 2) sync_cache: N/A - we should only be doing I/O on valid B_VMIO buffers + +*/ + +static buf_t * +_zbio_getblk_malloc(zbio_buf_hdr_t *hdr, int flags) +{ + buf_t *newbp, *tmpbp; + void *data; + daddr_t blkno; + uint64_t size = hdr->b_size; + uint64_t txg = hdr->b_birth; + zbio_state_t *state = spa_get_bio_state(hdr->b_spa); + + if (flags & GB_NODUMP) + data = zio_data_buf_alloc(size); + else + data = zio_buf_alloc(size); + newbp = malloc(sizeof(struct buf), M_ZFS_BIO, M_WAITOK|M_ZERO); + newbp->b_data = data; + newbp->b_flags = (B_MALLOC|B_INVAL); + newbp->b_bcount = size; + if (!BUF_EMPTY(hdr) && !(hdr->b_flags & ZBIO_BUF_CLONING)) { + blkno = hdr->b_dva.dva_word[1] & ~(1ULL<<63); + zbio_buf_blkno_evict_overlap(blkno, size, state, txg, 0, FALSE); + newbp->b_blkno = blkno; + /* + * Copy in from the page cache if found & valid + * and mark B_CACHE + */ + zbio_buf_vm_object_copyin(newbp); + } + + if (hdr->b_flags & ZBIO_BUF_CLONING) { + newbp->b_flags |= B_CLONED; + hdr->b_flags &= ~ZBIO_BUF_CLONING; + } + zbio_buf_va_insert(newbp, state); +} + +static buf_t * +_zbio_getblk_vmio(zbio_buf_hdr_t *hdr, int flags) +{ + buf_t *newbp; + daddr_t blkno; + uint64_t size = hdr->b_size; + spa_t *spa = hdr->b_spa; + zbio_state_t *state = spa_get_bio_state(spa); + struct vnode *vp = spa_get_vnode(spa); + struct bufobj *bo = &vp->v_bufobj; + + if (BUF_EMPTY(hdr)) { + newbp = geteblk(size, flags); + zbio_buf_va_insert(newbp, state); + } else { + blkno = hdr->b_dva.dva_word[1] & ~(1ULL<<63); + zbio_buf_blkno_evict_overlap(blkno, size, state, 0, 0, FALSE); + + while (newbp == NULL) + newbp = getblk(vp, blkno, size, 0, 0, flags | GB_LOCK_NOWAIT); + brelvp(newbp); + newbp->b_flags |= B_ASSIGNED; + zbio_buf_blkno_insert(newbp, state); + } + newbp->b_bufobj = bo; + BUF_KERNPROC(newbp); + CTR4(KTR_SPARE2, "arc_getblk() bp=%p flags %X " + "blkno %ld npages %d", + newbp, newbp->b_flags, blkno, newbp->b_npages); + + return (newbp); +} + +static void +_zbio_getblk(arc_buf_t *buf, int flags) +{ + zbio_buf_hdr_t *hdr = (zbio_buf_hdr_t *)buf->b_hdr; + uint64_t size = hdr->b_size; + buf_t *newbp; + + if (zfs_page_cache_disable) { + buf->b_data = zio_buf_alloc(size); + hdr->b_flags &= ~ZBIO_BUF_CLONING; + return; + } + + if ((size & PAGE_MASK) || (hdr->b_flags & ZBIO_BUF_CLONING)) + newbp = _zbio_getblk_malloc(hdr, flags); + else + newbp = _zbio_getblk_vmio(hdr, flags); + + buf->b_bp = newbp; + buf->b_data = newbp->b_data; + newbp->b_arc_buf = buf; +} + +void +zbio_getblk(arc_buf_t *buf) +{ + + _zbio_getblk(buf, 0); +} + +void +zbio_data_getblk(arc_buf_t *buf) +{ + + _zbio_getblk(buf, GB_NODUMP); +} + +void +zbio_relse(arc_buf_t *buf, size_t size) +{ + struct buf *bp = buf->b_bp; + + if (zfs_page_cache_disable) { + zio_buf_free(buf->b_data, size); return; + } + + if (bp->b_flags & B_ASSIGNED) + zbio_buf_blkno_remove(bp); + else + zbio_buf_va_remove(bp); + + if (bp->b_flags & B_MALLOC) { + zio_buf_free(bp->b_data, size); + free(bp, M_ZFS_BIO); + } else { + CTR4(KTR_SPARE2, "arc_brelse() bp=%p flags %X" + " size %ld blkno=%ld", + bp, bp->b_flags, size, bp->b_blkno); + + bp->b_flags |= B_ZFS; + brelse(bp); + } +} - if (blkno == 0 || hdr->b_birth == 0) +void +zbio_sync_cache(spa_t *spa, blkptr_t *blkp, uint64_t txg, void *data, uint64_t size, int bio_op) +{ + buf_t *bp; + zbio_state_t *state = spa_get_bio_state(spa); + dva_t dva = *BP_IDENTITY(blkp); + daddr_t blkno = dva.dva_word[1] & ~(1ULL<<63); + struct vnode *vp = spa_get_vnode(spa); + vm_object_t object = vp->v_object; + vm_pindex_t start; + vm_page_t m; + int i; + + if (zfs_page_cache_disable) return; + /* + * XXX incomplete + */ - bp = buf->b_bp; - bp->b_birth = hdr->b_birth; - bp->b_blkno = bp->b_lblkno = blkno; - bp->b_offset = (blkno << 9); - cachebuf = ((hdr->b_datacnt == 1) && - !(hdr->b_flags & ARC_IO_ERROR) && - ((bp->b_flags & (B_INVAL|B_CACHE)) == B_CACHE) && - (blkno & 0x7) == 0); - - arc_binval(hdr->b_spa, &hdr->b_dva, hdr->b_size); - if (cachebuf) - arc_pcache(vp, bp, blkno); + + if ((bp = zbio_buf_va_lookup(state, data)) != NULL) { + KASSERT(bp->b_flags & (B_CLONED|B_EVICTED) == 0, + ("doing I/O with cloned or evicted buffer 0x%x", bp->b_flags)); + + if (bp->b_flags & B_MALLOC) { + zbio_buf_blkno_evict_overlap(blkno, size, state, txg, 0, FALSE); + + if (bio_op == BIO_READ) { + /* + * if page resident - copy in + * update zio pipeline + */ + zbio_buf_vm_object_copyin(bp); + if (bp->b_flags & B_CACHE) { + /* update zio pipeline */ + } + } else + zbio_buf_vm_object_copyout(bp); + } else { + zbio_buf_blkno_evict_overlap(blkno, size, state, 0, ZB_EVICT_ALL, TRUE); + bp->b_blkno = bp->b_lblkno = blkno; + bp->b_flags |= (B_VMIO|B_ASSIGNED); + zbio_buf_vm_object_insert(bp, bio_op == BIO_WRITE); + } + } else { + bp = zbio_buf_blkno_lookup(state, blkno); + KASSERT(bp != NULL, ("blkno=%ld data=%p unmanaged", blkno, bp->b_data)); + } } -#endif static void zbio_shutdown(void *arg __unused, int howto __unused) @@ -275,6 +881,9 @@ void zbio_init(void) { + if (zfs_page_cache_disable) + return; + zbio_event_shutdown = EVENTHANDLER_REGISTER(shutdown_pre_sync, zbio_shutdown, NULL, EVENTHANDLER_PRI_FIRST); } @@ -285,7 +894,9 @@ zbio_fini(void) if (zbio_event_shutdown != NULL) EVENTHANDLER_DEREGISTER(shutdown_pre_sync, zbio_event_shutdown); } -#else + + +#else /* !_KERNEL */ void zbio_getblk(arc_buf_t *buf) @@ -319,5 +930,5 @@ zbio_sync_cache(spa_t *spa, blkptr_t *bp { ; } - #endif + Modified: user/kmacy/releng_8_fcs_buf_xen/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c ============================================================================== --- user/kmacy/releng_8_fcs_buf_xen/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c Sat Dec 12 02:34:00 2009 (r200427) +++ user/kmacy/releng_8_fcs_buf_xen/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c Sat Dec 12 03:35:49 2009 (r200428) @@ -436,8 +436,9 @@ zio_create(zio_t *pio, spa_t *spa, uint6 if (bp != NULL) { - if ((vd == NULL) || (vd->vdev_parent == NULL)) - zbio_sync_cache(spa, bp, txg, size); + if (((vd == NULL) || (vd->vdev_parent == NULL)) && + ((type == ZIO_TYPE_WRITE) || (type == ZIO_TYPE_READ))) + zbio_sync_cache(spa, bp, txg, data, size, type == ZIO_TYPE_WRITE ? BIO_WRITE : BIO_READ); zio->io_bp = bp; zio->io_bp_copy = *bp; From owner-svn-src-user@FreeBSD.ORG Sat Dec 12 04:50:05 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 304211065670; Sat, 12 Dec 2009 04:50:05 +0000 (UTC) (envelope-from kmacy@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 1E91C8FC0A; Sat, 12 Dec 2009 04:50:05 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nBC4o5NI041698; Sat, 12 Dec 2009 04:50:05 GMT (envelope-from kmacy@svn.freebsd.org) Received: (from kmacy@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nBC4o5ET041696; Sat, 12 Dec 2009 04:50:05 GMT (envelope-from kmacy@svn.freebsd.org) Message-Id: <200912120450.nBC4o5ET041696@svn.freebsd.org> From: Kip Macy Date: Sat, 12 Dec 2009 04:50:05 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r200430 - user/kmacy/releng_8_fcs_buf_xen/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Dec 2009 04:50:05 -0000 Author: kmacy Date: Sat Dec 12 04:50:04 2009 New Revision: 200430 URL: http://svn.freebsd.org/changeset/base/200430 Log: functions for - synchronizing B_MALLOC buffers with the page cache - eviction of pages from the backing object Modified: user/kmacy/releng_8_fcs_buf_xen/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_bio.c Modified: user/kmacy/releng_8_fcs_buf_xen/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_bio.c ============================================================================== --- user/kmacy/releng_8_fcs_buf_xen/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_bio.c Sat Dec 12 04:34:22 2009 (r200429) +++ user/kmacy/releng_8_fcs_buf_xen/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_bio.c Sat Dec 12 04:50:04 2009 (r200430) @@ -93,7 +93,8 @@ __FBSDID("$FreeBSD$"); #include #include -#include + +#include #include #ifdef _KERNEL @@ -122,6 +123,8 @@ MALLOC_DEFINE(M_ZFS_BIO, "zfs_bio", "zfs #define B_ASSIGNED B_00004000 #define ZB_EVICT_ALL 0x1 +#define ZB_COPYIN 0x2 +#define ZB_COPYOUT 0x3 #define btos(nbytes) ((nbytes)>>DEV_BSHIFT) #define stob(nsectors) ((nsectors)<mtx) #define ZBIO_STATE_UNLOCK(zs) mtx_unlock(&(zs)->mtx) -#define spa_get_bio_state(spa) ((zbio_state_t *)spa_get_vnode((spa))->v_data) -#define spa_get_vm_object(spa) spa_get_vnode((spa))->v_object -#define zbio_buf_get_spa(bp) (((zbio_buf_hdr_t *)((arc_buf_t *)(bp->b_arc_buf))->b_hdr)->b_spa) +#define spa_get_bio_state(spa) ((zbio_state_t *)spa_get_vnode((spa))->v_data) +#define spa_get_vm_object(spa) spa_get_vnode((spa))->v_object +#define zbio_buf_get_spa(bp) (((zbio_buf_hdr_t *)((arc_buf_t *)(bp->b_arc_buf))->b_hdr)->b_spa) +#define zbio_buf_get_vm_object(bp) spa_get_vm_object(zbio_buf_get_spa((bp))) static void zbio_buf_blkno_remove(buf_t *bp); static void zbio_buf_va_insert(buf_t *bp, zbio_state_t *object); @@ -481,29 +485,87 @@ zbio_buf_blkno_lookup(zbio_state_t *stat } static void -zbio_buf_vm_object_copyin(buf_t *bp) +zbio_buf_vm_object_copy(buf_t *bp, int direction) { + vm_object_t object; + vm_pindex_t start, end; + vm_offset_t offset; + uint64_t byte_offset; + vm_offset_t page_offset; + int i, size; + caddr_t va; + vm_page_t m; + struct sf_buf *sf; + object = zbio_buf_get_vm_object(bp); + byte_offset = stob(bp->b_blkno); + page_offset = byte_offset & PAGE_MASK; + start = OFF_TO_IDX(byte_offset); + end = OFF_TO_IDX(byte_offset + bp->b_bcount); + + VM_OBJECT_LOCK(object); + for (bp->b_npages = i = 0; start + i < end; i++) { + m = vm_page_lookup(object, start + i); + + if ((m == NULL) || (m->valid != VM_PAGE_BITS_ALL)) + goto done; + + bp->b_pages[i] = m; + bp->b_npages++; + } + for (i = 0; i < bp->b_npages; i++) { + sf = sf_buf_alloc(bp->b_pages[i], 0); + va = (caddr_t)sf_buf_kva(sf); + size = PAGE_SIZE; + + if (i == 0) + va += page_offset; + if (i == bp->b_npages - 1) + size = PAGE_SIZE - page_offset; + + if (direction == ZB_COPYIN) + memcpy(bp->b_data + PAGE_SIZE*i, va, size); + else + memcpy(va, bp->b_data + PAGE_SIZE*i, size); + sf_buf_free(sf); + } +done: + bp->b_npages = 0; + VM_OBJECT_UNLOCK(object); } static void zbio_buf_vm_object_copyout(buf_t *bp) { + + zbio_buf_vm_object_copy(bp, ZB_COPYOUT); +} +static void +zbio_buf_vm_object_copyin(buf_t *bp) +{ + zbio_buf_vm_object_copy(bp, ZB_COPYIN); } static void zbio_buf_vm_object_evict(buf_t *bp) { int i; + vm_page_t m; + VM_OBJECT_LOCK_ASSERT(zbio_buf_get_vm_object(bp), MA_OWNED); /* * remove pages from backing vm_object */ - for (i = 0; i < bp->b_npages; i++) - vm_page_remove(bp->b_pages[i]); + for (i = 0; i < bp->b_npages; i++) { + m = bp->b_pages[i]; + vm_pageq_remove(m); + vm_page_remove(m); + m->valid = 0; + m->flags |= PG_UNMANAGED; + } } static void @@ -513,7 +575,7 @@ zbio_buf_vm_object_insert(buf_t *bp, int vm_pindex_t start = OFF_TO_IDX(stob(bp->b_blkno)); spa_t *spa = zbio_buf_get_spa(bp); struct vnode *vp = spa_get_vnode(spa); - struct vm_object *object = vp->v_object; + vm_object_t object = vp->v_object; int i; VM_OBJECT_LOCK(object); From owner-svn-src-user@FreeBSD.ORG Sat Dec 12 13:08:48 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 086F7106566C; Sat, 12 Dec 2009 13:08:48 +0000 (UTC) (envelope-from nyan@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id EAF678FC0C; Sat, 12 Dec 2009 13:08:47 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nBCD8lb5053533; Sat, 12 Dec 2009 13:08:47 GMT (envelope-from nyan@svn.freebsd.org) Received: (from nyan@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nBCD8lLI053528; Sat, 12 Dec 2009 13:08:47 GMT (envelope-from nyan@svn.freebsd.org) Message-Id: <200912121308.nBCD8lLI053528@svn.freebsd.org> From: Takahashi Yoshihiro Date: Sat, 12 Dec 2009 13:08:47 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r200436 - in user/nyan/pc98/sys/boot/pc98: libpc98 loader X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Dec 2009 13:08:48 -0000 Author: nyan Date: Sat Dec 12 13:08:47 2009 New Revision: 200436 URL: http://svn.freebsd.org/changeset/base/200436 Log: Add setting machine type support to the loader. Added: user/nyan/pc98/sys/boot/pc98/libpc98/libpc98.h (contents, props changed) user/nyan/pc98/sys/boot/pc98/libpc98/pc98_sys.c (contents, props changed) Modified: user/nyan/pc98/sys/boot/pc98/libpc98/Makefile user/nyan/pc98/sys/boot/pc98/loader/main.c Modified: user/nyan/pc98/sys/boot/pc98/libpc98/Makefile ============================================================================== --- user/nyan/pc98/sys/boot/pc98/libpc98/Makefile Sat Dec 12 12:36:41 2009 (r200435) +++ user/nyan/pc98/sys/boot/pc98/libpc98/Makefile Sat Dec 12 13:08:47 2009 (r200436) @@ -7,7 +7,7 @@ INTERNALLIB= SRCS= bioscd.c biosdisk.c biosmem.c biospnp.c biospci.c biossmap.c \ bootinfo.c bootinfo32.c comconsole.c devicename.c elf32_freebsd.c \ - i386_copy.c i386_module.c nullconsole.c pxe.c pxetramp.s \ + i386_copy.c i386_module.c nullconsole.c pc98_sys.c pxe.c pxetramp.s \ time.c vidconsole.c # Enable PXE TFTP or NFS support, not both. Added: user/nyan/pc98/sys/boot/pc98/libpc98/libpc98.h ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ user/nyan/pc98/sys/boot/pc98/libpc98/libpc98.h Sat Dec 12 13:08:47 2009 (r200436) @@ -0,0 +1,28 @@ +/*- + * Copyright (c) 2009 TAKAHASHI Yoshihiro + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + */ + +void set_machine_type(void); Added: user/nyan/pc98/sys/boot/pc98/libpc98/pc98_sys.c ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ user/nyan/pc98/sys/boot/pc98/libpc98/pc98_sys.c Sat Dec 12 13:08:47 2009 (r200436) @@ -0,0 +1,78 @@ +/*- + * Copyright (c) 2009 TAKAHASHI Yoshihiro + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + */ + +#include +__FBSDID("$FreeBSD$"); + +#include +#include +#define _KERNEL +#include + +/* + * Set machine type to PC98_SYSTEM_PARAMETER. + */ +void +set_machine_type(void) +{ + int i; + u_long ret, data; + + /* PC98_SYSTEM_PARAMETER (0x501) */ + ret = ((*(u_char *)PTOV(0xA1501)) & 0x08) >> 3; + + /* Wait V-SYNC */ + while (inb(0x60) & 0x20) {} + while (!(inb(0x60) & 0x20)) {} + + /* ANK 'A' font */ + outb(0xa1, 0x00); + outb(0xa3, 0x41); + + /* M_NORMAL, use CG window (all NEC OK) */ + for (i = data = 0; i < 4; i++) + data += *((u_long *)PTOV(0xA4000) + i); /* 0xa4000 */ + if (data == 0x6efc58fc) /* DA data */ + ret |= M_NEC_PC98; + else + ret |= M_EPSON_PC98; + ret |= (inb(0x42) & 0x20) ? M_8M : 0; + + /* PC98_SYSTEM_PARAMETER(0x400) */ + if ((*(u_char *)PTOV(0xA1400)) & 0x80) + ret |= M_NOTE; + if (ret & M_NEC_PC98) { + /* PC98_SYSTEM_PARAMETER(0x458) */ + if ((*(u_char *)PTOV(0xA1458)) & 0x80) + ret |= M_H98; + else + ret |= M_NOT_H98; + } else + ret |= M_NOT_H98; + + (*(u_long *)PTOV(0xA1620)) = ret; +} Modified: user/nyan/pc98/sys/boot/pc98/loader/main.c ============================================================================== --- user/nyan/pc98/sys/boot/pc98/loader/main.c Sat Dec 12 12:36:41 2009 (r200435) +++ user/nyan/pc98/sys/boot/pc98/loader/main.c Sat Dec 12 13:08:47 2009 (r200436) @@ -40,6 +40,7 @@ __FBSDID("$FreeBSD$"); #include "bootstrap.h" #include "libi386/libi386.h" +#include "libpc98/libpc98.h" #include "btxv86.h" #define KARGS_FLAGS_CD 0x1 @@ -81,6 +82,9 @@ main(void) { int i; + /* Set machine type to PC98_SYSTEM_PARAMETER. */ + set_machine_type(); + /* Pick up arguments */ kargs = (void *)__args; initial_howto = kargs->howto; From owner-svn-src-user@FreeBSD.ORG Sat Dec 12 13:24:07 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 32B6C1065693; Sat, 12 Dec 2009 13:24:07 +0000 (UTC) (envelope-from nyan@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 175CC8FC1D; Sat, 12 Dec 2009 13:24:07 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nBCDO64U053892; Sat, 12 Dec 2009 13:24:06 GMT (envelope-from nyan@svn.freebsd.org) Received: (from nyan@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nBCDO6QJ053889; Sat, 12 Dec 2009 13:24:06 GMT (envelope-from nyan@svn.freebsd.org) Message-Id: <200912121324.nBCDO6QJ053889@svn.freebsd.org> From: Takahashi Yoshihiro Date: Sat, 12 Dec 2009 13:24:06 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r200437 - user/nyan/pc98/sys/boot/pc98/boot2 X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Dec 2009 13:24:07 -0000 Author: nyan Date: Sat Dec 12 13:24:06 2009 New Revision: 200437 URL: http://svn.freebsd.org/changeset/base/200437 Log: - Disable setting machine type. - Add scrolling screen support. Modified: user/nyan/pc98/sys/boot/pc98/boot2/Makefile user/nyan/pc98/sys/boot/pc98/boot2/boot1.S Modified: user/nyan/pc98/sys/boot/pc98/boot2/Makefile ============================================================================== --- user/nyan/pc98/sys/boot/pc98/boot2/Makefile Sat Dec 12 13:08:47 2009 (r200436) +++ user/nyan/pc98/sys/boot/pc98/boot2/Makefile Sat Dec 12 13:24:06 2009 (r200437) @@ -38,6 +38,9 @@ CFLAGS= -Os \ -Wpointer-arith -Wshadow -Wstrict-prototypes -Wwrite-strings \ -Winline --param max-inline-insns-single=100 +# Set machine type to PC98_SYSTEM_PARAMETER +#CFLAGS+= -DSET_MACHINE_TYPE + # Initialize the bi_bios_geom using the BIOS geometry #CFLAGS+= -DGET_BIOSGEOM Modified: user/nyan/pc98/sys/boot/pc98/boot2/boot1.S ============================================================================== --- user/nyan/pc98/sys/boot/pc98/boot2/boot1.S Sat Dec 12 13:08:47 2009 (r200436) +++ user/nyan/pc98/sys/boot/pc98/boot2/boot1.S Sat Dec 12 13:24:06 2009 (r200437) @@ -104,8 +104,10 @@ main: cld and $0x00ffffff,%eax mov %eax,%es:(EPSON_ID) +#ifdef SET_MACHINE_TYPE /* Set machine type to PC98_SYSTEM_PARAMETER */ - call machine_check + call set_machine_type +#endif /* Setup graphic screen */ mov $0x42,%ah /* 640x400 */ @@ -210,11 +212,14 @@ putstr: lodsb /* * Display a single char. */ -putc: push %ds +putc: pusha xor %dx,%dx mov %dx,%ds mov MEM_REL+cursor-start,%di - mov $160,%cx + mov $0xa000,%bx + mov %bx,%es + mov $(80*2),%cx + cmp $0x08,%al je putc.bs cmp $0x0d,%al @@ -224,41 +229,55 @@ putc: push %ds cmp $0x5c,%al /* \ */ jne 1f mov $0xfc,%al -1: mov $0xa000,%bx - mov %bx,%es - mov $0xe1,%bl - mov %bl,%es:0x2000(%di) /* Attribute */ - stosb - inc %di - jmp putc.move +1: movb $0xe1,%es:0x2000(%di) + stosw + jmp putc.scr putc.bs: test %di,%di jz putc.move dec %di dec %di + movb $0xe1,%es:0x2000(%di) + movw $0x20,%es:(%di) jmp putc.move -putc.cr: /* xor %dx,%dx */ - mov %di,%ax +putc.cr: mov %di,%ax div %cx sub %dx,%di jmp putc.move putc.lf: add %cx,%di +putc.scr: cmp $(80*2*25),%di /* Scroll screen */ + jb putc.move + push %ds + mov %bx,%ds + mov $(80*2),%si + xor %di,%di + mov $(80*24/2),%cx + rep + movsl + xor %ax,%ax + mov $0x20,%al + mov $80,%cl + rep + stosw + pop %ds + mov $(80*24*2),%di putc.move: mov %di,MEM_REL+cursor-start /* Move cursor */ mov $0x13,%ah mov %di,%dx int $0x18 - pop %ds + popa lret cursor: .word 0 +#ifdef SET_MACHINE_TYPE /* * Set machine type to PC98_SYSTEM_PARAMETER. */ -machine_check: xor %edx,%edx +set_machine_type: + xor %edx,%edx mov %dx,%ds -# mov $MEM_SYS,%ax -# mov %ax,%es - +// mov $MEM_SYS,%ax +// mov %ax,%es /* Wait V-SYNC */ vsync.1: inb $0x60,%al test $0x20,%al @@ -310,8 +329,8 @@ m_not_h98: or $M_NOT_H98,%edx 1: mov $PC98_MACHINE_TYPE,%bx mov %edx,%es:(%bx) - ret +#endif /* Messages */ From owner-svn-src-user@FreeBSD.ORG Sat Dec 12 23:41:15 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B1CDF106568D; Sat, 12 Dec 2009 23:41:15 +0000 (UTC) (envelope-from kmacy@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 9F81E8FC18; Sat, 12 Dec 2009 23:41:15 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nBCNfFu5067309; Sat, 12 Dec 2009 23:41:15 GMT (envelope-from kmacy@svn.freebsd.org) Received: (from kmacy@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nBCNfFsI067305; Sat, 12 Dec 2009 23:41:15 GMT (envelope-from kmacy@svn.freebsd.org) Message-Id: <200912122341.nBCNfFsI067305@svn.freebsd.org> From: Kip Macy Date: Sat, 12 Dec 2009 23:41:15 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r200455 - in user/kmacy/releng_8_fcs_buf_xen/sys/cddl/contrib/opensolaris/uts/common/fs/zfs: . sys X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Dec 2009 23:41:15 -0000 Author: kmacy Date: Sat Dec 12 23:41:15 2009 New Revision: 200455 URL: http://svn.freebsd.org/changeset/base/200455 Log: - create _locked versions of zbio_buf_evict_overlap and zbio_buf_vm_object_insert - minimize hold time of vm page queue lock - mark B_MALLOC buffers B_CACHE if they reflect contents of cached pages - add symbolic defines to clarify intent of calls to _evict_overlap - update io pipeline to bypass io to disk if read from page cache succeeded or getblk successfully obtained all cached pages Modified: user/kmacy/releng_8_fcs_buf_xen/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zfs_bio.h user/kmacy/releng_8_fcs_buf_xen/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_bio.c user/kmacy/releng_8_fcs_buf_xen/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c Modified: user/kmacy/releng_8_fcs_buf_xen/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zfs_bio.h ============================================================================== --- user/kmacy/releng_8_fcs_buf_xen/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zfs_bio.h Sat Dec 12 23:16:47 2009 (r200454) +++ user/kmacy/releng_8_fcs_buf_xen/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zfs_bio.h Sat Dec 12 23:41:15 2009 (r200455) @@ -34,7 +34,7 @@ $FreeBSD$ #define ZBIO_BUF_CLONING (1 << 30) /* is being cloned */ -void zbio_sync_cache(spa_t *spa, blkptr_t *bp, uint64_t txg, void *data, uint64_t size, int bio_op); +int zbio_sync_cache(spa_t *spa, blkptr_t *bp, uint64_t txg, void *data, uint64_t size, int bio_op); void zbio_getblk(arc_buf_t *buf); void zbio_data_getblk(arc_buf_t *buf); void zbio_relse(arc_buf_t *buf, size_t size); Modified: user/kmacy/releng_8_fcs_buf_xen/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_bio.c ============================================================================== --- user/kmacy/releng_8_fcs_buf_xen/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_bio.c Sat Dec 12 23:16:47 2009 (r200454) +++ user/kmacy/releng_8_fcs_buf_xen/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_bio.c Sat Dec 12 23:41:15 2009 (r200455) @@ -122,9 +122,13 @@ MALLOC_DEFINE(M_ZFS_BIO, "zfs_bio", "zfs #define B_CLONED B_00001000 #define B_ASSIGNED B_00004000 -#define ZB_EVICT_ALL 0x1 -#define ZB_COPYIN 0x2 -#define ZB_COPYOUT 0x3 +#define ZB_EVICT_ALL 0x1 +#define ZB_EVICT_BUFFERED 0x2 + +#define ZB_COPYIN 0x2 +#define ZB_COPYOUT 0x3 + +#define NO_TXG 0x0 #define btos(nbytes) ((nbytes)>>DEV_BSHIFT) #define stob(nsectors) ((nsectors)<b_data + PAGE_SIZE*i, size); sf_buf_free(sf); } - + bp->b_flags &= ~B_INVAL; + bp->b_flags |= B_CACHE; + done: bp->b_npages = 0; VM_OBJECT_UNLOCK(object); @@ -556,12 +562,17 @@ zbio_buf_vm_object_evict(buf_t *bp) vm_page_t m; VM_OBJECT_LOCK_ASSERT(zbio_buf_get_vm_object(bp), MA_OWNED); + vm_page_lock_queues(); + for (i = 0; i < bp->b_npages; i++) { + m = bp->b_pages[i]; + vm_pageq_remove(m); + } + vm_page_unlock_queues(); /* * remove pages from backing vm_object */ for (i = 0; i < bp->b_npages; i++) { m = bp->b_pages[i]; - vm_pageq_remove(m); vm_page_remove(m); m->valid = 0; m->flags |= PG_UNMANAGED; @@ -569,16 +580,13 @@ zbio_buf_vm_object_evict(buf_t *bp) } static void -zbio_buf_vm_object_insert(buf_t *bp, int valid) +zbio_buf_vm_object_insert_locked(buf_t *bp, struct vnode *vp, + vm_object_t object, int valid) { vm_page_t m; vm_pindex_t start = OFF_TO_IDX(stob(bp->b_blkno)); - spa_t *spa = zbio_buf_get_spa(bp); - struct vnode *vp = spa_get_vnode(spa); - vm_object_t object = vp->v_object; int i; - VM_OBJECT_LOCK(object); /* * Insert buffer pages in the object */ @@ -588,7 +596,6 @@ zbio_buf_vm_object_insert(buf_t *bp, int m->valid = VM_PAGE_BITS_ALL; vm_page_insert(m, object, start + i); m->flags &= ~PG_UNMANAGED; - vdrop(vp); } vm_page_lock_queues(); for (i = 0; i < bp->b_npages; i++) { @@ -596,8 +603,18 @@ zbio_buf_vm_object_insert(buf_t *bp, int vm_page_enqueue(PQ_INACTIVE, m); } vm_page_unlock_queues(); +} + +static void +zbio_buf_vm_object_insert(buf_t *bp, int valid) +{ + spa_t *spa = zbio_buf_get_spa(bp); + struct vnode *vp = spa_get_vnode(spa); + vm_object_t object = vp->v_object; + + VM_OBJECT_LOCK(object); + zbio_buf_vm_object_insert_locked(bp, vp, object, valid); VM_OBJECT_UNLOCK(object); - } /* @@ -612,8 +629,8 @@ zbio_buf_vm_object_insert(buf_t *bp, int * This routine may not block. */ static void -zbio_buf_blkno_evict_overlap(daddr_t blkno, int size, zbio_state_t *state, - uint64_t txg, int evict_op, int locked) +zbio_buf_evict_overlap_locked(daddr_t blkno, int size, zbio_state_t *state, + uint64_t txg, int evict_op, vm_object_t object) { buf_t *root, *tmpbp; daddr_t blkno_end, tmpblkno, tmpblkno_end; @@ -621,10 +638,7 @@ zbio_buf_blkno_evict_overlap(daddr_t blk int i, collisions; uint64_t tmptxg; vm_pindex_t start, end; - vm_object_t object = spa_get_vm_object(state->spa); - if (!locked) - VM_OBJECT_LOCK(object); if ((root = state->blkno_root) == NULL) goto done; @@ -645,7 +659,7 @@ zbio_buf_blkno_evict_overlap(daddr_t blk if (((tmpblkno >= blkno) && (tmpblkno < blkno_end)) || (tmpblkno_end > blkno) && (tmpblkno_end <= blkno_end) && - ((txg == 0) || (tmptxg < txg))) { + ((txg == NO_TXG) || (tmptxg < txg))) { TAILQ_INSERT_TAIL(&clh, tmpbp, b_freelist); collisions++; } @@ -680,10 +694,20 @@ done: } #endif } - if (!locked) - VM_OBJECT_UNLOCK(object); } +static void +zbio_buf_evict_overlap(daddr_t blkno, int size, zbio_state_t *state, + uint64_t txg, int evict_op) +{ + vm_object_t object = spa_get_vm_object(state->spa); + + VM_OBJECT_LOCK(object); + zbio_buf_evict_overlap_locked(blkno, size, state, txg, evict_op, object); + VM_OBJECT_UNLOCK(object); +} + + /* Cases: @@ -737,7 +761,7 @@ _zbio_getblk_malloc(zbio_buf_hdr_t *hdr, newbp->b_bcount = size; if (!BUF_EMPTY(hdr) && !(hdr->b_flags & ZBIO_BUF_CLONING)) { blkno = hdr->b_dva.dva_word[1] & ~(1ULL<<63); - zbio_buf_blkno_evict_overlap(blkno, size, state, txg, 0, FALSE); + zbio_buf_evict_overlap(blkno, size, state, txg, ZB_EVICT_BUFFERED); newbp->b_blkno = blkno; /* * Copy in from the page cache if found & valid @@ -769,7 +793,7 @@ _zbio_getblk_vmio(zbio_buf_hdr_t *hdr, i zbio_buf_va_insert(newbp, state); } else { blkno = hdr->b_dva.dva_word[1] & ~(1ULL<<63); - zbio_buf_blkno_evict_overlap(blkno, size, state, 0, 0, FALSE); + zbio_buf_evict_overlap(blkno, size, state, NO_TXG, ZB_EVICT_BUFFERED); while (newbp == NULL) newbp = getblk(vp, blkno, size, 0, 0, flags | GB_LOCK_NOWAIT); @@ -851,7 +875,7 @@ zbio_relse(arc_buf_t *buf, size_t size) } } -void +int zbio_sync_cache(spa_t *spa, blkptr_t *blkp, uint64_t txg, void *data, uint64_t size, int bio_op) { buf_t *bp; @@ -862,10 +886,11 @@ zbio_sync_cache(spa_t *spa, blkptr_t *bl vm_object_t object = vp->v_object; vm_pindex_t start; vm_page_t m; - int i; + int i, io_bypass = FALSE; if (zfs_page_cache_disable) - return; + return (FALSE); + /* * XXX incomplete */ @@ -876,7 +901,7 @@ zbio_sync_cache(spa_t *spa, blkptr_t *bl ("doing I/O with cloned or evicted buffer 0x%x", bp->b_flags)); if (bp->b_flags & B_MALLOC) { - zbio_buf_blkno_evict_overlap(blkno, size, state, txg, 0, FALSE); + zbio_buf_evict_overlap(blkno, size, state, txg, ZB_EVICT_BUFFERED); if (bio_op == BIO_READ) { /* @@ -886,19 +911,28 @@ zbio_sync_cache(spa_t *spa, blkptr_t *bl zbio_buf_vm_object_copyin(bp); if (bp->b_flags & B_CACHE) { /* update zio pipeline */ + io_bypass = TRUE; } - } else + } else { zbio_buf_vm_object_copyout(bp); + } } else { - zbio_buf_blkno_evict_overlap(blkno, size, state, 0, ZB_EVICT_ALL, TRUE); + VM_OBJECT_LOCK(object); + zbio_buf_evict_overlap_locked(blkno, size, state, NO_TXG, + ZB_EVICT_ALL, object); bp->b_blkno = bp->b_lblkno = blkno; bp->b_flags |= (B_VMIO|B_ASSIGNED); - zbio_buf_vm_object_insert(bp, bio_op == BIO_WRITE); + zbio_buf_vm_object_insert_locked(bp, vp, object, bio_op == BIO_WRITE); + VM_OBJECT_UNLOCK(object); } } else { bp = zbio_buf_blkno_lookup(state, blkno); + if (bio_op == BIO_READ && (bp->b_flags & (B_CACHE|B_INVAL)) == B_CACHE) + io_bypass = TRUE; KASSERT(bp != NULL, ("blkno=%ld data=%p unmanaged", blkno, bp->b_data)); } + + return (io_bypass); } static void Modified: user/kmacy/releng_8_fcs_buf_xen/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c ============================================================================== --- user/kmacy/releng_8_fcs_buf_xen/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c Sat Dec 12 23:16:47 2009 (r200454) +++ user/kmacy/releng_8_fcs_buf_xen/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c Sat Dec 12 23:41:15 2009 (r200455) @@ -413,6 +413,7 @@ zio_create(zio_t *pio, spa_t *spa, uint6 const zbookmark_t *zb, uint8_t stage, uint32_t pipeline) { zio_t *zio; + int io_bypass; ASSERT3U(size, <=, SPA_MAXBLOCKSIZE); ASSERT(P2PHASE(size, SPA_MINBLOCKSIZE) == 0); @@ -436,9 +437,12 @@ zio_create(zio_t *pio, spa_t *spa, uint6 if (bp != NULL) { + io_bypass = 0; + if (((vd == NULL) || (vd->vdev_parent == NULL)) && ((type == ZIO_TYPE_WRITE) || (type == ZIO_TYPE_READ))) - zbio_sync_cache(spa, bp, txg, data, size, type == ZIO_TYPE_WRITE ? BIO_WRITE : BIO_READ); + io_bypass = zbio_sync_cache(spa, bp, txg, data, size, + type == ZIO_TYPE_WRITE ? BIO_WRITE : BIO_READ); zio->io_bp = bp; zio->io_bp_copy = *bp; @@ -450,6 +454,8 @@ zio_create(zio_t *pio, spa_t *spa, uint6 pipeline |= ZIO_GANG_STAGES; zio->io_logical = zio; } + if (io_bypass) + pipeline = ZIO_INTERLOCK_STAGES; } zio->io_spa = spa; From owner-svn-src-user@FreeBSD.ORG Sat Dec 12 23:47:31 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BAA0E1065679; Sat, 12 Dec 2009 23:47:31 +0000 (UTC) (envelope-from kmacy@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id A9A6F8FC14; Sat, 12 Dec 2009 23:47:31 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nBCNlVAv067466; Sat, 12 Dec 2009 23:47:31 GMT (envelope-from kmacy@svn.freebsd.org) Received: (from kmacy@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nBCNlVC6067464; Sat, 12 Dec 2009 23:47:31 GMT (envelope-from kmacy@svn.freebsd.org) Message-Id: <200912122347.nBCNlVC6067464@svn.freebsd.org> From: Kip Macy Date: Sat, 12 Dec 2009 23:47:31 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r200456 - user/kmacy/releng_8_fcs_buf_xen/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Dec 2009 23:47:31 -0000 Author: kmacy Date: Sat Dec 12 23:47:31 2009 New Revision: 200456 URL: http://svn.freebsd.org/changeset/base/200456 Log: don't check for buffer cache hit in ARC, this is now handled by zbio_sync_cache in zio_create Modified: user/kmacy/releng_8_fcs_buf_xen/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Modified: user/kmacy/releng_8_fcs_buf_xen/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c ============================================================================== --- user/kmacy/releng_8_fcs_buf_xen/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Sat Dec 12 23:41:15 2009 (r200455) +++ user/kmacy/releng_8_fcs_buf_xen/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Sat Dec 12 23:47:31 2009 (r200456) @@ -2918,21 +2918,6 @@ top: rzio = zio_read(pio, spa, bp, buf->b_data, size, arc_read_done, buf, priority, zio_flags, zb); - /* - * We hit in the page cache - can bypass the I/O stages - * - */ -#ifdef _KERNEL - if ((buf->b_bp != NULL) && - ((buf->b_bp->b_flags & (B_CACHE|B_INVAL)) == B_CACHE)) { - /* - * track the number of times - * the buffer was found in the cache - */ - ARCSTAT_BUMP(arcstat_page_cache_hits); - rzio->io_pipeline = ZIO_INTERLOCK_STAGES; - } -#endif if (*arc_flags & ARC_WAIT) return (zio_wait(rzio)); From owner-svn-src-user@FreeBSD.ORG Sat Dec 12 23:52:21 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 53CFB1065679; Sat, 12 Dec 2009 23:52:21 +0000 (UTC) (envelope-from kmacy@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 430B28FC1C; Sat, 12 Dec 2009 23:52:21 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nBCNqLNL067593; Sat, 12 Dec 2009 23:52:21 GMT (envelope-from kmacy@svn.freebsd.org) Received: (from kmacy@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nBCNqLg0067591; Sat, 12 Dec 2009 23:52:21 GMT (envelope-from kmacy@svn.freebsd.org) Message-Id: <200912122352.nBCNqLg0067591@svn.freebsd.org> From: Kip Macy Date: Sat, 12 Dec 2009 23:52:21 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r200457 - user/kmacy/releng_8_fcs_buf_xen/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Dec 2009 23:52:21 -0000 Author: kmacy Date: Sat Dec 12 23:52:20 2009 New Revision: 200457 URL: http://svn.freebsd.org/changeset/base/200457 Log: add comment clarifying call to zbio_sync_cache Modified: user/kmacy/releng_8_fcs_buf_xen/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c Modified: user/kmacy/releng_8_fcs_buf_xen/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c ============================================================================== --- user/kmacy/releng_8_fcs_buf_xen/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c Sat Dec 12 23:47:31 2009 (r200456) +++ user/kmacy/releng_8_fcs_buf_xen/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c Sat Dec 12 23:52:20 2009 (r200457) @@ -435,10 +435,15 @@ zio_create(zio_t *pio, spa_t *spa, uint6 else zio->io_child_type = ZIO_CHILD_LOGICAL; - if (bp != NULL) { io_bypass = 0; + /* + * Synchronize buffer with page cache - making sure that + * the page cache only holds the most recent txg's pages. + * This also allows us to skip disk I/O if we hit in the + * page cache. + */ if (((vd == NULL) || (vd->vdev_parent == NULL)) && ((type == ZIO_TYPE_WRITE) || (type == ZIO_TYPE_READ))) io_bypass = zbio_sync_cache(spa, bp, txg, data, size, From owner-svn-src-user@FreeBSD.ORG Sat Dec 12 23:57:19 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F2D621065676; Sat, 12 Dec 2009 23:57:19 +0000 (UTC) (envelope-from kmacy@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id E1E2F8FC22; Sat, 12 Dec 2009 23:57:19 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nBCNvJ7b067727; Sat, 12 Dec 2009 23:57:19 GMT (envelope-from kmacy@svn.freebsd.org) Received: (from kmacy@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nBCNvJEA067724; Sat, 12 Dec 2009 23:57:19 GMT (envelope-from kmacy@svn.freebsd.org) Message-Id: <200912122357.nBCNvJEA067724@svn.freebsd.org> From: Kip Macy Date: Sat, 12 Dec 2009 23:57:19 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r200458 - user/kmacy/releng_8_fcs_buf_xen/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Dec 2009 23:57:20 -0000 Author: kmacy Date: Sat Dec 12 23:57:19 2009 New Revision: 200458 URL: http://svn.freebsd.org/changeset/base/200458 Log: - remove unused vnode reference - don't try to sync cache if the vdev has no backing vnode Modified: user/kmacy/releng_8_fcs_buf_xen/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c user/kmacy/releng_8_fcs_buf_xen/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c Modified: user/kmacy/releng_8_fcs_buf_xen/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c ============================================================================== --- user/kmacy/releng_8_fcs_buf_xen/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c Sat Dec 12 23:52:20 2009 (r200457) +++ user/kmacy/releng_8_fcs_buf_xen/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c Sat Dec 12 23:57:19 2009 (r200458) @@ -1060,8 +1060,6 @@ vdev_open(vdev_t *vd) * inconsistently account for existing bp's. */ if (vd->vdev_top == vd) { - struct vnode *vp; - vd->vdev_deflate_ratio = (1<<17) / (vdev_psize_to_asize(vd, 1<<17) >> SPA_MINBLOCKSHIFT); Modified: user/kmacy/releng_8_fcs_buf_xen/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c ============================================================================== --- user/kmacy/releng_8_fcs_buf_xen/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c Sat Dec 12 23:52:20 2009 (r200457) +++ user/kmacy/releng_8_fcs_buf_xen/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c Sat Dec 12 23:57:19 2009 (r200458) @@ -444,7 +444,7 @@ zio_create(zio_t *pio, spa_t *spa, uint6 * This also allows us to skip disk I/O if we hit in the * page cache. */ - if (((vd == NULL) || (vd->vdev_parent == NULL)) && + if (((vd != NULL) && (vd->vdev_vnode != NULL)) && ((type == ZIO_TYPE_WRITE) || (type == ZIO_TYPE_READ))) io_bypass = zbio_sync_cache(spa, bp, txg, data, size, type == ZIO_TYPE_WRITE ? BIO_WRITE : BIO_READ);