From owner-svn-src-user@FreeBSD.ORG Mon Mar 15 09:40:23 2010 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9CFD6106566B; Mon, 15 Mar 2010 09:40:23 +0000 (UTC) (envelope-from luigi@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 8BAC58FC12; Mon, 15 Mar 2010 09:40:23 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id o2F9eNFd020434; Mon, 15 Mar 2010 09:40:23 GMT (envelope-from luigi@svn.freebsd.org) Received: (from luigi@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id o2F9eNDJ020430; Mon, 15 Mar 2010 09:40:23 GMT (envelope-from luigi@svn.freebsd.org) Message-Id: <201003150940.o2F9eNDJ020430@svn.freebsd.org> From: Luigi Rizzo Date: Mon, 15 Mar 2010 09:40:23 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r205166 - in user/luigi/ipfw3-r8/sys: netgraph netinet X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Mar 2010 09:40:23 -0000 Author: luigi Date: Mon Mar 15 09:40:23 2010 New Revision: 205166 URL: http://svn.freebsd.org/changeset/base/205166 Log: missing changes to let the kernel compile Modified: user/luigi/ipfw3-r8/sys/netgraph/ng_ipfw.h user/luigi/ipfw3-r8/sys/netinet/in.h user/luigi/ipfw3-r8/sys/netinet/ip_var.h Modified: user/luigi/ipfw3-r8/sys/netgraph/ng_ipfw.h ============================================================================== --- user/luigi/ipfw3-r8/sys/netgraph/ng_ipfw.h Mon Mar 15 08:58:35 2010 (r205165) +++ user/luigi/ipfw3-r8/sys/netgraph/ng_ipfw.h Mon Mar 15 09:40:23 2010 (r205166) @@ -26,26 +26,8 @@ * $FreeBSD$ */ +#ifndef _NG_IPFW_H +#define _NG_IPFW_H #define NG_IPFW_NODE_TYPE "ipfw" #define NGM_IPFW_COOKIE 1105988990 - -#ifdef _KERNEL - -typedef int ng_ipfw_input_t(struct mbuf **, int, struct ip_fw_args *, int); -extern ng_ipfw_input_t *ng_ipfw_input_p; -#define NG_IPFW_LOADED (ng_ipfw_input_p != NULL) - -struct ng_ipfw_tag { - struct m_tag mt; /* tag header */ - struct ip_fw *rule; /* matching rule */ - uint32_t rule_id; /* matching rule id */ - uint32_t chain_id; /* ruleset id */ - struct ifnet *ifp; /* interface, for ip_output */ - int dir; -#define NG_IPFW_OUT 0 -#define NG_IPFW_IN 1 -}; - -#define TAGSIZ (sizeof(struct ng_ipfw_tag) - sizeof(struct m_tag)) - -#endif /* _KERNEL */ +#endif /* _NG_IPFW_H */ Modified: user/luigi/ipfw3-r8/sys/netinet/in.h ============================================================================== --- user/luigi/ipfw3-r8/sys/netinet/in.h Mon Mar 15 08:58:35 2010 (r205165) +++ user/luigi/ipfw3-r8/sys/netinet/in.h Mon Mar 15 09:40:23 2010 (r205166) @@ -754,6 +754,32 @@ void in_ifdetach(struct ifnet *); #define sintosa(sin) ((struct sockaddr *)(sin)) #define ifatoia(ifa) ((struct in_ifaddr *)(ifa)) +/* + * Historically, BSD keeps ip_len and ip_off in host format + * when doing layer 3 processing, and this often requires + * to translate the format back and forth. + * To make the process explicit, we define a couple of macros + * that also take into account the fact that at some point + * we may want to keep those fields always in net format. + */ + +#if (BYTE_ORDER == BIG_ENDIAN) || defined(HAVE_NET_IPLEN) +#define SET_NET_IPLEN(p) do {} while (0) +#define SET_HOST_IPLEN(p) do {} while (0) +#else +#define SET_NET_IPLEN(p) do { \ + struct ip *h_ip = (p); \ + h_ip->ip_len = htons(h_ip->ip_len); \ + h_ip->ip_off = htons(h_ip->ip_off); \ + } while (0) + +#define SET_HOST_IPLEN(p) do { \ + struct ip *h_ip = (p); \ + h_ip->ip_len = ntohs(h_ip->ip_len); \ + h_ip->ip_off = ntohs(h_ip->ip_off); \ + } while (0) +#endif /* !HAVE_NET_IPLEN */ + #endif /* _KERNEL */ /* INET6 stuff */ Modified: user/luigi/ipfw3-r8/sys/netinet/ip_var.h ============================================================================== --- user/luigi/ipfw3-r8/sys/netinet/ip_var.h Mon Mar 15 08:58:35 2010 (r205165) +++ user/luigi/ipfw3-r8/sys/netinet/ip_var.h Mon Mar 15 09:40:23 2010 (r205166) @@ -249,7 +249,43 @@ VNET_DECLARE(struct pfil_head, inet_pfil void in_delayed_cksum(struct mbuf *m); -/* ipfw and dummynet hooks. Most are declared in raw_ip.c */ +/* Hooks for ipfw, dummynet, divert etc. Most are declared in raw_ip.c */ +/* + * Reference to an ipfw or packet filter rule that can be carried + * outside critical sections. + * A rule is identified by rulenum:rule_id which is ordered. + * In version chain_id the rule can be found in slot 'slot', so + * we don't need a lookup if chain_id == chain->id. + * + * On exit from the firewall this structure refers to the rule after + * the matching one (slot points to the new rule; rulenum:rule_id-1 + * is the matching rule), and additional info (e.g. info often contains + * the insn argument or tablearg in the low 16 bits, in host format). + * On entry, the structure is valid if slot>0, and refers to the starting + * rules. 'info' contains the reason for reinject, e.g. divert port, + * divert direction, and so on. + */ +struct ipfw_rule_ref { + uint32_t slot; /* slot for matching rule */ + uint32_t rulenum; /* matching rule number */ + uint32_t rule_id; /* matching rule id */ + uint32_t chain_id; /* ruleset id */ + uint32_t info; /* see below */ +}; + +enum { + IPFW_INFO_MASK = 0x0000ffff, + IPFW_INFO_OUT = 0x00000000, /* outgoing, just for convenience */ + IPFW_INFO_IN = 0x80000000, /* incoming, overloads dir */ + IPFW_ONEPASS = 0x40000000, /* One-pass, do not reinject */ + IPFW_IS_MASK = 0x30000000, /* which source ? */ + IPFW_IS_DIVERT = 0x20000000, + IPFW_IS_DUMMYNET =0x10000000, + IPFW_IS_PIPE = 0x08000000, /* pip1=1, queue = 0 */ +}; +#define MTAG_IPFW 1148380143 /* IPFW-tagged cookie */ +#define MTAG_IPFW_RULE 1262273568 /* rule reference */ + struct ip_fw_args; typedef int (*ip_fw_chk_ptr_t)(struct ip_fw_args *args); typedef int (*ip_fw_ctl_ptr_t)(struct sockopt *); @@ -258,9 +294,14 @@ VNET_DECLARE(ip_fw_ctl_ptr_t, ip_fw_ctl_ #define V_ip_fw_chk_ptr VNET(ip_fw_chk_ptr) #define V_ip_fw_ctl_ptr VNET(ip_fw_ctl_ptr) +/* Divert hooks. */ +extern void (*ip_divert_ptr)(struct mbuf *m, int incoming); +/* ng_ipfw hooks -- XXX make it the same as divert and dummynet */ +extern int (*ng_ipfw_input_p)(struct mbuf **, int, + struct ip_fw_args *, int); + extern int (*ip_dn_ctl_ptr)(struct sockopt *); -extern int (*ip_dn_io_ptr)(struct mbuf **m, int dir, struct ip_fw_args *fwa); -extern void (*ip_dn_ruledel_ptr)(void *); /* in ip_fw2.c */ +extern int (*ip_dn_io_ptr)(struct mbuf **, int, struct ip_fw_args *); VNET_DECLARE(int, ip_do_randomid); #define V_ip_do_randomid VNET(ip_do_randomid) From owner-svn-src-user@FreeBSD.ORG Mon Mar 15 17:54:54 2010 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 28DDC1065670; Mon, 15 Mar 2010 17:54:54 +0000 (UTC) (envelope-from luigi@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 16AF78FC22; Mon, 15 Mar 2010 17:54:54 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id o2FHsrGd033518; Mon, 15 Mar 2010 17:54:53 GMT (envelope-from luigi@svn.freebsd.org) Received: (from luigi@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id o2FHsrIO033517; Mon, 15 Mar 2010 17:54:53 GMT (envelope-from luigi@svn.freebsd.org) Message-Id: <201003151754.o2FHsrIO033517@svn.freebsd.org> From: Luigi Rizzo Date: Mon, 15 Mar 2010 17:54:53 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r205176 - user/luigi/ipfw3-r8/release/picobsd/floppy.tree X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Mar 2010 17:54:54 -0000 Author: luigi Date: Mon Mar 15 17:54:53 2010 New Revision: 205176 URL: http://svn.freebsd.org/changeset/base/205176 Log: MFStable Added: - copied unchanged from r205081, user/luigi/ipfw3-r8/release/picobsd/floppy.tree/sbin Directory Properties: user/luigi/ipfw3-r8/release/picobsd/floppy.tree/dhclient-script (props changed) Deleted: user/luigi/ipfw3-r8/release/picobsd/floppy.tree/sbin Copied: user/luigi/ipfw3-r8/release/picobsd/floppy.tree/dhclient-script (from r205081, user/luigi/ipfw3-r8/release/picobsd/floppy.tree/sbin) ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ user/luigi/ipfw3-r8/release/picobsd/floppy.tree/dhclient-script Mon Mar 15 17:54:53 2010 (r205176, copy of r205081, user/luigi/ipfw3-r8/release/picobsd/floppy.tree/sbin) @@ -0,0 +1,384 @@ +#!/bin/sh +# +# $OpenBSD: dhclient-script,v 1.6 2004/05/06 18:22:41 claudio Exp $ +# $FreeBSD$ +# +# Copyright (c) 2003 Kenneth R Westerback +# +# Permission to use, copy, modify, and distribute this software for any +# purpose with or without fee is hereby granted, provided that the above +# copyright notice and this permission notice appear in all copies. +# +# THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES +# WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF +# MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR +# ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES +# WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN +# ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF +# OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. +# +# + +ARP=/usr/sbin/arp +HOSTNAME=/bin/hostname +IFCONFIG='/sbin/ifconfig -n' + +LOCALHOST=127.0.0.1 + +if [ -x /usr/bin/logger ]; then + LOGGER="/usr/bin/logger -s -p user.notice -t dhclient" +else + LOGGER=echo +fi + +# +# Helper functions that implement common actions. +# + +check_hostname() { + current_hostname=`$HOSTNAME` + if [ -z "$current_hostname" ]; then + $LOGGER "New Hostname ($interface): $new_host_name" + $HOSTNAME $new_host_name + elif [ "$current_hostname" = "$old_host_name" -a \ + "$new_host_name" != "$old_host_name" ]; then + $LOGGER "New Hostname ($interface): $new_host_name" + $HOSTNAME $new_host_name + fi +} + +arp_flush() { + arp -an -i $interface | \ + sed -n -e 's/^.*(\(.*\)) at .*$/arp -d \1/p' | \ + sh >/dev/null 2>&1 +} + +delete_old_address() { + eval "$IFCONFIG $interface inet -alias $old_ip_address $medium" +} + +add_new_address() { + eval "$IFCONFIG $interface \ + inet $new_ip_address \ + netmask $new_subnet_mask \ + broadcast $new_broadcast_address \ + $medium" + + $LOGGER "New IP Address ($interface): $new_ip_address" + $LOGGER "New Subnet Mask ($interface): $new_subnet_mask" + $LOGGER "New Broadcast Address ($interface): $new_broadcast_address" + $LOGGER "New Routers ($interface): $new_routers" +} + +delete_old_alias() { + if [ -n "$alias_ip_address" ]; then + $IFCONFIG $interface inet -alias $alias_ip_address > /dev/null 2>&1 + #route delete $alias_ip_address $LOCALHOST > /dev/null 2>&1 + fi +} + +add_new_alias() { + if [ -n "$alias_ip_address" ]; then + $IFCONFIG $interface inet alias $alias_ip_address netmask \ + $alias_subnet_mask + #route add $alias_ip_address $LOCALHOST + fi +} + +fill_classless_routes() { + set $1 + while [ $# -ge 5 ]; do + if [ $1 -eq 0 ]; then + route="default" + elif [ $1 -le 8 ]; then + route="$2.0.0.0/$1" + shift + elif [ $1 -le 16 ]; then + route="$2.$3.0.0/$1" + shift; shift + elif [ $1 -le 24 ]; then + route="$2.$3.$4.0/$1" + shift; shift; shift + else + route="$2.$3.$4.$5/$1" + shift; shift; shift; shift + fi + shift + router="$1.$2.$3.$4" + classless_routes="$classless_routes $route $router" + shift; shift; shift; shift + done +} + +delete_old_routes() { + #route delete "$old_ip_address" $LOCALHOST >/dev/null 2>&1 + if [ -n "$old_classless_routes" ]; then + fill_classless_routes "$old_classless_routes" + set $classless_routes + while [ $# -gt 1 ]; do + route delete "$1" "$2" + shift; shift + done + return 0; + fi + + # If we supported multiple default routes, we'd be removing each + # one here. We don't so just delete the default route if it's + # through our interface. + if is_default_interface; then + route delete default >/dev/null 2>&1 + fi + + if [ -n "$old_static_routes" ]; then + set $old_static_routes + while [ $# -gt 1 ]; do + route delete "$1" "$2" + shift; shift + done + fi + + arp_flush +} + +add_new_routes() { + #route add $new_ip_address $LOCALHOST >/dev/null 2>&1 + + # RFC 3442: If the DHCP server returns both a Classless Static + # Routes option and a Router option, the DHCP client MUST ignore + # the Router option. + # + # DHCP clients that support this option (Classless Static Routes) + # MUST NOT install the routes specified in the Static Routes + # option (option code 33) if both a Static Routes option and the + # Classless Static Routes option are provided. + + if [ -n "$new_classless_routes" ]; then + fill_classless_routes "$new_classless_routes" + $LOGGER "New Classless Static Routes ($interface): $classless_routes" + set $classless_routes + while [ $# -gt 1 ]; do + if [ "0.0.0.0" = "$2" ]; then + route add "$1" -iface "$interface" + else + route add "$1" "$2" + fi + shift; shift + done + return + fi + + for router in $new_routers; do + if is_default_interface; then + + if [ "$new_ip_address" = "$router" ]; then + route add default -iface $router >/dev/null 2>&1 + else + route add default $router >/dev/null 2>&1 + fi + fi + # 2nd and subsequent default routers error out, so explicitly + # stop processing the list after the first one. + break + done + + if [ -n "$new_static_routes" ]; then + $LOGGER "New Static Routes ($interface): $new_static_routes" + set $new_static_routes + while [ $# -gt 1 ]; do + route add $1 $2 + shift; shift + done + fi +} + +add_new_resolv_conf() { + # XXX Old code did not create/update resolv.conf unless both + # $new_domain_name and $new_domain_name_servers were provided. PR + # #3135 reported some ISP's only provide $new_domain_name_servers and + # thus broke the script. This code creates the resolv.conf if either + # are provided. + + local tmpres=/var/run/resolv.conf.${interface} + rm -f $tmpres + + if [ -n "$new_domain_name" ]; then + echo "search $new_domain_name" >>$tmpres + fi + + if [ -n "$new_domain_name_servers" ]; then + for nameserver in $new_domain_name_servers; do + echo "nameserver $nameserver" >>$tmpres + done + fi + + if [ -f $tmpres ]; then + if [ -f /etc/resolv.conf.tail ]; then + cat /etc/resolv.conf.tail >>$tmpres + fi + + # When resolv.conf is not changed actually, we don't + # need to update it. + # If /usr is not mounted yet, we cannot use cmp, then + # the following test fails. In such case, we simply + # ignore an error and do update resolv.conf. + if cmp -s $tmpres /etc/resolv.conf; then + rm -f $tmpres + return 0 + fi 2>/dev/null + + # In case (e.g. during OpenBSD installs) /etc/resolv.conf + # is a symbolic link, take care to preserve the link and write + # the new data in the correct location. + + if [ -f /etc/resolv.conf ]; then + cat /etc/resolv.conf > /etc/resolv.conf.save + fi + cat $tmpres > /etc/resolv.conf + rm -f $tmpres + + # Try to ensure correct ownership and permissions. + chown -RL root:wheel /etc/resolv.conf + chmod -RL 644 /etc/resolv.conf + + return 0 + fi + + return 1 +} + +# Must be used on exit. Invokes the local dhcp client exit hooks, if any. +exit_with_hooks() { + exit_status=$1 + if [ -f /etc/dhclient-exit-hooks ]; then + . /etc/dhclient-exit-hooks + fi + # probably should do something with exit status of the local script + exit $exit_status +} + +# Get the interface with the current ipv4 default route on it using only +# commands that are available prior to /usr being mounted. +is_default_interface() +{ + routeget="`route -n get -inet default`" + oldifs="$IFS" + IFS=" +" + defif= + for line in $routeget ; do + case $line in + *interface:*) + defif=${line##*: } + ;; + esac + done + IFS=${oldifs} + + if [ -z "$defif" -o "$defif" = "$interface" ]; then + return 0 + else + return 1 + fi +} + +# +# Start of active code. +# + +# Invoke the local dhcp client enter hooks, if they exist. +if [ -f /etc/dhclient-enter-hooks ]; then + exit_status=0 + . /etc/dhclient-enter-hooks + # allow the local script to abort processing of this state + # local script must set exit_status variable to nonzero. + if [ $exit_status -ne 0 ]; then + exit $exit_status + fi +fi + +case $reason in +MEDIUM) + eval "$IFCONFIG $interface $medium" + eval "$IFCONFIG $interface inet -alias 0.0.0.0 $medium" >/dev/null 2>&1 + sleep 1 + ;; + +PREINIT) + delete_old_alias + $IFCONFIG $interface inet alias 0.0.0.0 netmask 0.0.0.0 broadcast 255.255.255.255 up + ;; + +ARPCHECK|ARPSEND) + ;; + +BOUND|RENEW|REBIND|REBOOT) + check_hostname + if [ -n "$old_ip_address" ]; then + if [ "$old_ip_address" != "$alias_ip_address" ]; then + delete_old_alias + fi + if [ "$old_ip_address" != "$new_ip_address" ]; then + delete_old_address + delete_old_routes + fi + fi + if [ "$reason" = BOUND ] || \ + [ "$reason" = REBOOT ] || \ + [ -z "$old_ip_address" ] || \ + [ "$old_ip_address" != "$new_ip_address" ]; then + add_new_address + add_new_routes + fi + if [ "$new_ip_address" != "$alias_ip_address" ]; then + add_new_alias + fi + if is_default_interface; then + add_new_resolv_conf + fi + ;; + +EXPIRE|FAIL) + delete_old_alias + if [ -n "$old_ip_address" ]; then + delete_old_address + delete_old_routes + fi + if [ -x $ARP ]; then + $ARP -d -a -i $interface + fi + # XXX Why add alias we just deleted above? + add_new_alias + if is_default_interface; then + if [ -f /etc/resolv.conf.save ]; then + cat /etc/resolv.conf.save > /etc/resolv.conf + fi + fi + ;; + +TIMEOUT) + delete_old_alias + add_new_address + sleep 1 + if [ -n "$new_routers" ]; then + $LOGGER "New Routers ($interface): $new_routers" + set "$new_routers" + if ping -q -c 1 -t 1 "$1"; then + if [ "$new_ip_address" != "$alias_ip_address" ]; then + add_new_alias + fi + add_new_routes + if ! is_default_interface; then + exit_with_hooks 0 + fi + if add_new_resolv_conf; then + exit_with_hooks 0 + fi + fi + fi + eval "$IFCONFIG $interface inet -alias $new_ip_address $medium" + delete_old_routes + exit_with_hooks 1 + ;; +esac + +exit_with_hooks 0 From owner-svn-src-user@FreeBSD.ORG Mon Mar 15 17:55:46 2010 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E04231065689; Mon, 15 Mar 2010 17:55:46 +0000 (UTC) (envelope-from luigi@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id CDB9C8FC1D; Mon, 15 Mar 2010 17:55:46 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id o2FHtkvC033736; Mon, 15 Mar 2010 17:55:46 GMT (envelope-from luigi@svn.freebsd.org) Received: (from luigi@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id o2FHtkcb033734; Mon, 15 Mar 2010 17:55:46 GMT (envelope-from luigi@svn.freebsd.org) Message-Id: <201003151755.o2FHtkcb033734@svn.freebsd.org> From: Luigi Rizzo Date: Mon, 15 Mar 2010 17:55:46 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r205177 - in user/luigi/ipfw3-r8/release/picobsd/floppy.tree: . sbin X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Mar 2010 17:55:47 -0000 Author: luigi Date: Mon Mar 15 17:55:46 2010 New Revision: 205177 URL: http://svn.freebsd.org/changeset/base/205177 Log: MFStable Added: user/luigi/ipfw3-r8/release/picobsd/floppy.tree/sbin/ user/luigi/ipfw3-r8/release/picobsd/floppy.tree/sbin/dhclient-script - copied unchanged from r205176, user/luigi/ipfw3-r8/release/picobsd/floppy.tree/dhclient-script Deleted: user/luigi/ipfw3-r8/release/picobsd/floppy.tree/dhclient-script Copied: user/luigi/ipfw3-r8/release/picobsd/floppy.tree/sbin/dhclient-script (from r205176, user/luigi/ipfw3-r8/release/picobsd/floppy.tree/dhclient-script) ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ user/luigi/ipfw3-r8/release/picobsd/floppy.tree/sbin/dhclient-script Mon Mar 15 17:55:46 2010 (r205177, copy of r205176, user/luigi/ipfw3-r8/release/picobsd/floppy.tree/dhclient-script) @@ -0,0 +1,384 @@ +#!/bin/sh +# +# $OpenBSD: dhclient-script,v 1.6 2004/05/06 18:22:41 claudio Exp $ +# $FreeBSD$ +# +# Copyright (c) 2003 Kenneth R Westerback +# +# Permission to use, copy, modify, and distribute this software for any +# purpose with or without fee is hereby granted, provided that the above +# copyright notice and this permission notice appear in all copies. +# +# THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES +# WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF +# MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR +# ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES +# WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN +# ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF +# OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. +# +# + +ARP=/usr/sbin/arp +HOSTNAME=/bin/hostname +IFCONFIG='/sbin/ifconfig -n' + +LOCALHOST=127.0.0.1 + +if [ -x /usr/bin/logger ]; then + LOGGER="/usr/bin/logger -s -p user.notice -t dhclient" +else + LOGGER=echo +fi + +# +# Helper functions that implement common actions. +# + +check_hostname() { + current_hostname=`$HOSTNAME` + if [ -z "$current_hostname" ]; then + $LOGGER "New Hostname ($interface): $new_host_name" + $HOSTNAME $new_host_name + elif [ "$current_hostname" = "$old_host_name" -a \ + "$new_host_name" != "$old_host_name" ]; then + $LOGGER "New Hostname ($interface): $new_host_name" + $HOSTNAME $new_host_name + fi +} + +arp_flush() { + arp -an -i $interface | \ + sed -n -e 's/^.*(\(.*\)) at .*$/arp -d \1/p' | \ + sh >/dev/null 2>&1 +} + +delete_old_address() { + eval "$IFCONFIG $interface inet -alias $old_ip_address $medium" +} + +add_new_address() { + eval "$IFCONFIG $interface \ + inet $new_ip_address \ + netmask $new_subnet_mask \ + broadcast $new_broadcast_address \ + $medium" + + $LOGGER "New IP Address ($interface): $new_ip_address" + $LOGGER "New Subnet Mask ($interface): $new_subnet_mask" + $LOGGER "New Broadcast Address ($interface): $new_broadcast_address" + $LOGGER "New Routers ($interface): $new_routers" +} + +delete_old_alias() { + if [ -n "$alias_ip_address" ]; then + $IFCONFIG $interface inet -alias $alias_ip_address > /dev/null 2>&1 + #route delete $alias_ip_address $LOCALHOST > /dev/null 2>&1 + fi +} + +add_new_alias() { + if [ -n "$alias_ip_address" ]; then + $IFCONFIG $interface inet alias $alias_ip_address netmask \ + $alias_subnet_mask + #route add $alias_ip_address $LOCALHOST + fi +} + +fill_classless_routes() { + set $1 + while [ $# -ge 5 ]; do + if [ $1 -eq 0 ]; then + route="default" + elif [ $1 -le 8 ]; then + route="$2.0.0.0/$1" + shift + elif [ $1 -le 16 ]; then + route="$2.$3.0.0/$1" + shift; shift + elif [ $1 -le 24 ]; then + route="$2.$3.$4.0/$1" + shift; shift; shift + else + route="$2.$3.$4.$5/$1" + shift; shift; shift; shift + fi + shift + router="$1.$2.$3.$4" + classless_routes="$classless_routes $route $router" + shift; shift; shift; shift + done +} + +delete_old_routes() { + #route delete "$old_ip_address" $LOCALHOST >/dev/null 2>&1 + if [ -n "$old_classless_routes" ]; then + fill_classless_routes "$old_classless_routes" + set $classless_routes + while [ $# -gt 1 ]; do + route delete "$1" "$2" + shift; shift + done + return 0; + fi + + # If we supported multiple default routes, we'd be removing each + # one here. We don't so just delete the default route if it's + # through our interface. + if is_default_interface; then + route delete default >/dev/null 2>&1 + fi + + if [ -n "$old_static_routes" ]; then + set $old_static_routes + while [ $# -gt 1 ]; do + route delete "$1" "$2" + shift; shift + done + fi + + arp_flush +} + +add_new_routes() { + #route add $new_ip_address $LOCALHOST >/dev/null 2>&1 + + # RFC 3442: If the DHCP server returns both a Classless Static + # Routes option and a Router option, the DHCP client MUST ignore + # the Router option. + # + # DHCP clients that support this option (Classless Static Routes) + # MUST NOT install the routes specified in the Static Routes + # option (option code 33) if both a Static Routes option and the + # Classless Static Routes option are provided. + + if [ -n "$new_classless_routes" ]; then + fill_classless_routes "$new_classless_routes" + $LOGGER "New Classless Static Routes ($interface): $classless_routes" + set $classless_routes + while [ $# -gt 1 ]; do + if [ "0.0.0.0" = "$2" ]; then + route add "$1" -iface "$interface" + else + route add "$1" "$2" + fi + shift; shift + done + return + fi + + for router in $new_routers; do + if is_default_interface; then + + if [ "$new_ip_address" = "$router" ]; then + route add default -iface $router >/dev/null 2>&1 + else + route add default $router >/dev/null 2>&1 + fi + fi + # 2nd and subsequent default routers error out, so explicitly + # stop processing the list after the first one. + break + done + + if [ -n "$new_static_routes" ]; then + $LOGGER "New Static Routes ($interface): $new_static_routes" + set $new_static_routes + while [ $# -gt 1 ]; do + route add $1 $2 + shift; shift + done + fi +} + +add_new_resolv_conf() { + # XXX Old code did not create/update resolv.conf unless both + # $new_domain_name and $new_domain_name_servers were provided. PR + # #3135 reported some ISP's only provide $new_domain_name_servers and + # thus broke the script. This code creates the resolv.conf if either + # are provided. + + local tmpres=/var/run/resolv.conf.${interface} + rm -f $tmpres + + if [ -n "$new_domain_name" ]; then + echo "search $new_domain_name" >>$tmpres + fi + + if [ -n "$new_domain_name_servers" ]; then + for nameserver in $new_domain_name_servers; do + echo "nameserver $nameserver" >>$tmpres + done + fi + + if [ -f $tmpres ]; then + if [ -f /etc/resolv.conf.tail ]; then + cat /etc/resolv.conf.tail >>$tmpres + fi + + # When resolv.conf is not changed actually, we don't + # need to update it. + # If /usr is not mounted yet, we cannot use cmp, then + # the following test fails. In such case, we simply + # ignore an error and do update resolv.conf. + if cmp -s $tmpres /etc/resolv.conf; then + rm -f $tmpres + return 0 + fi 2>/dev/null + + # In case (e.g. during OpenBSD installs) /etc/resolv.conf + # is a symbolic link, take care to preserve the link and write + # the new data in the correct location. + + if [ -f /etc/resolv.conf ]; then + cat /etc/resolv.conf > /etc/resolv.conf.save + fi + cat $tmpres > /etc/resolv.conf + rm -f $tmpres + + # Try to ensure correct ownership and permissions. + chown -RL root:wheel /etc/resolv.conf + chmod -RL 644 /etc/resolv.conf + + return 0 + fi + + return 1 +} + +# Must be used on exit. Invokes the local dhcp client exit hooks, if any. +exit_with_hooks() { + exit_status=$1 + if [ -f /etc/dhclient-exit-hooks ]; then + . /etc/dhclient-exit-hooks + fi + # probably should do something with exit status of the local script + exit $exit_status +} + +# Get the interface with the current ipv4 default route on it using only +# commands that are available prior to /usr being mounted. +is_default_interface() +{ + routeget="`route -n get -inet default`" + oldifs="$IFS" + IFS=" +" + defif= + for line in $routeget ; do + case $line in + *interface:*) + defif=${line##*: } + ;; + esac + done + IFS=${oldifs} + + if [ -z "$defif" -o "$defif" = "$interface" ]; then + return 0 + else + return 1 + fi +} + +# +# Start of active code. +# + +# Invoke the local dhcp client enter hooks, if they exist. +if [ -f /etc/dhclient-enter-hooks ]; then + exit_status=0 + . /etc/dhclient-enter-hooks + # allow the local script to abort processing of this state + # local script must set exit_status variable to nonzero. + if [ $exit_status -ne 0 ]; then + exit $exit_status + fi +fi + +case $reason in +MEDIUM) + eval "$IFCONFIG $interface $medium" + eval "$IFCONFIG $interface inet -alias 0.0.0.0 $medium" >/dev/null 2>&1 + sleep 1 + ;; + +PREINIT) + delete_old_alias + $IFCONFIG $interface inet alias 0.0.0.0 netmask 0.0.0.0 broadcast 255.255.255.255 up + ;; + +ARPCHECK|ARPSEND) + ;; + +BOUND|RENEW|REBIND|REBOOT) + check_hostname + if [ -n "$old_ip_address" ]; then + if [ "$old_ip_address" != "$alias_ip_address" ]; then + delete_old_alias + fi + if [ "$old_ip_address" != "$new_ip_address" ]; then + delete_old_address + delete_old_routes + fi + fi + if [ "$reason" = BOUND ] || \ + [ "$reason" = REBOOT ] || \ + [ -z "$old_ip_address" ] || \ + [ "$old_ip_address" != "$new_ip_address" ]; then + add_new_address + add_new_routes + fi + if [ "$new_ip_address" != "$alias_ip_address" ]; then + add_new_alias + fi + if is_default_interface; then + add_new_resolv_conf + fi + ;; + +EXPIRE|FAIL) + delete_old_alias + if [ -n "$old_ip_address" ]; then + delete_old_address + delete_old_routes + fi + if [ -x $ARP ]; then + $ARP -d -a -i $interface + fi + # XXX Why add alias we just deleted above? + add_new_alias + if is_default_interface; then + if [ -f /etc/resolv.conf.save ]; then + cat /etc/resolv.conf.save > /etc/resolv.conf + fi + fi + ;; + +TIMEOUT) + delete_old_alias + add_new_address + sleep 1 + if [ -n "$new_routers" ]; then + $LOGGER "New Routers ($interface): $new_routers" + set "$new_routers" + if ping -q -c 1 -t 1 "$1"; then + if [ "$new_ip_address" != "$alias_ip_address" ]; then + add_new_alias + fi + add_new_routes + if ! is_default_interface; then + exit_with_hooks 0 + fi + if add_new_resolv_conf; then + exit_with_hooks 0 + fi + fi + fi + eval "$IFCONFIG $interface inet -alias $new_ip_address $medium" + delete_old_routes + exit_with_hooks 1 + ;; +esac + +exit_with_hooks 0 From owner-svn-src-user@FreeBSD.ORG Mon Mar 15 18:27:51 2010 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DA6231065670; Mon, 15 Mar 2010 18:27:51 +0000 (UTC) (envelope-from luigi@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id C79AA8FC0A; Mon, 15 Mar 2010 18:27:51 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id o2FIRpi2041112; Mon, 15 Mar 2010 18:27:51 GMT (envelope-from luigi@svn.freebsd.org) Received: (from luigi@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id o2FIRp27041092; Mon, 15 Mar 2010 18:27:51 GMT (envelope-from luigi@svn.freebsd.org) Message-Id: <201003151827.o2FIRp27041092@svn.freebsd.org> From: Luigi Rizzo Date: Mon, 15 Mar 2010 18:27:51 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r205182 - in user/luigi/ipfw3-r8: sbin/ipfw sys/conf sys/net sys/netinet sys/netinet/ipfw X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Mar 2010 18:27:52 -0000 Author: luigi Date: Mon Mar 15 18:27:51 2010 New Revision: 205182 URL: http://svn.freebsd.org/changeset/base/205182 Log: almost completed merge of ipfw3 from HEAD into my working tree Modified: user/luigi/ipfw3-r8/sbin/ipfw/dummynet.c user/luigi/ipfw3-r8/sbin/ipfw/ipfw2.c user/luigi/ipfw3-r8/sbin/ipfw/ipfw2.h user/luigi/ipfw3-r8/sys/conf/files user/luigi/ipfw3-r8/sys/net/if_bridge.c user/luigi/ipfw3-r8/sys/net/if_ethersubr.c user/luigi/ipfw3-r8/sys/net/radix.c user/luigi/ipfw3-r8/sys/net/radix.h user/luigi/ipfw3-r8/sys/net/route.c user/luigi/ipfw3-r8/sys/netinet/ip_divert.c user/luigi/ipfw3-r8/sys/netinet/ip_divert.h user/luigi/ipfw3-r8/sys/netinet/ip_fw.h user/luigi/ipfw3-r8/sys/netinet/ipfw/ip_dn_io.c user/luigi/ipfw3-r8/sys/netinet/ipfw/ip_dn_private.h user/luigi/ipfw3-r8/sys/netinet/ipfw/ip_dummynet.c user/luigi/ipfw3-r8/sys/netinet/ipfw/ip_fw2.c user/luigi/ipfw3-r8/sys/netinet/ipfw/ip_fw_dynamic.c user/luigi/ipfw3-r8/sys/netinet/ipfw/ip_fw_log.c user/luigi/ipfw3-r8/sys/netinet/ipfw/ip_fw_table.c user/luigi/ipfw3-r8/sys/netinet/raw_ip.c Modified: user/luigi/ipfw3-r8/sbin/ipfw/dummynet.c ============================================================================== --- user/luigi/ipfw3-r8/sbin/ipfw/dummynet.c Mon Mar 15 18:26:09 2010 (r205181) +++ user/luigi/ipfw3-r8/sbin/ipfw/dummynet.c Mon Mar 15 18:27:51 2010 (r205182) @@ -141,7 +141,8 @@ print_mask(struct ipfw_flow_id *id) { if (!IS_IP6_FLOW_ID(id)) { printf(" " - "mask: 0x%02x 0x%08x/0x%04x -> 0x%08x/0x%04x\n", + "mask: %s 0x%02x 0x%08x/0x%04x -> 0x%08x/0x%04x\n", + id->extra ? "queue," : "", id->proto, id->src_ip, id->src_port, id->dst_ip, id->dst_port); @@ -151,7 +152,8 @@ print_mask(struct ipfw_flow_id *id) "Tot_pkt/bytes Pkt/Byte Drp\n"); } else { char buf[255]; - printf("\n mask: proto: 0x%02x, flow_id: 0x%08x, ", + printf("\n mask: %sproto: 0x%02x, flow_id: 0x%08x, ", + id->extra ? "queue," : "", id->proto, id->flow_id6); inet_ntop(AF_INET6, &(id->src_ip6), buf, sizeof(buf)); printf("%s/0x%04x -> ", buf, id->src_port); @@ -175,7 +177,8 @@ list_flow(struct dn_flow *ni) pe = getprotobynumber(id->proto); /* XXX: Should check for IPv4 flows */ - printf("%3u ", (ni->oid.id) & 0xff); + printf("%3u%c", (ni->oid.id) & 0xff, + id->extra ? '*' : ' '); if (!IS_IP6_FLOW_ID(id)) { if (pe) printf("%-4s ", pe->p_name); @@ -910,6 +913,7 @@ ipfw_config_pipe(int ac, char **av) case TOK_ALL: /* * special case, all bits significant + * except 'extra' (the queue number) */ mask->dst_ip = ~0; mask->src_ip = ~0; @@ -922,6 +926,11 @@ ipfw_config_pipe(int ac, char **av) *flags |= DN_HAVE_MASK; goto end_mask; + case TOK_QUEUE: + mask->extra = ~0; + *flags |= DN_HAVE_MASK; + goto end_mask; + case TOK_DSTIP: mask->addr_type = 4; p32 = &mask->dst_ip; @@ -992,7 +1001,7 @@ ipfw_config_pipe(int ac, char **av) if (a > 0xFF) errx(EX_DATAERR, "proto mask must be 8 bit"); - fs->flow_mask.proto = (uint8_t)a; + mask->proto = (uint8_t)a; } if (a != 0) *flags |= DN_HAVE_MASK; Modified: user/luigi/ipfw3-r8/sbin/ipfw/ipfw2.c ============================================================================== --- user/luigi/ipfw3-r8/sbin/ipfw/ipfw2.c Mon Mar 15 18:26:09 2010 (r205181) +++ user/luigi/ipfw3-r8/sbin/ipfw/ipfw2.c Mon Mar 15 18:27:51 2010 (r205182) @@ -231,7 +231,7 @@ static struct _s_x rule_action_params[] */ static int lookup_key[] = { TOK_DSTIP, TOK_SRCIP, TOK_DSTPORT, TOK_SRCPORT, - TOK_UID, TOK_JAIL, -1 }; + TOK_UID, TOK_JAIL, TOK_DSCP, -1 }; static struct _s_x rule_options[] = { { "tagged", TOK_TAGGED }, @@ -258,6 +258,7 @@ static struct _s_x rule_options[] = { { "iplen", TOK_IPLEN }, { "ipid", TOK_IPID }, { "ipprecedence", TOK_IPPRECEDENCE }, + { "dscp", TOK_DSCP }, { "iptos", TOK_IPTOS }, { "ipttl", TOK_IPTTL }, { "ipversion", TOK_IPVER }, @@ -3519,7 +3520,7 @@ read_options: char *p; int j; - if (av[0] && av[1]) + if (!av[0] || !av[1]) errx(EX_USAGE, "format: lookup argument tablenum"); cmd->opcode = O_IP_DST_LOOKUP; cmd->len |= F_INSN_SIZE(ipfw_insn) + 2; Modified: user/luigi/ipfw3-r8/sbin/ipfw/ipfw2.h ============================================================================== --- user/luigi/ipfw3-r8/sbin/ipfw/ipfw2.h Mon Mar 15 18:26:09 2010 (r205181) +++ user/luigi/ipfw3-r8/sbin/ipfw/ipfw2.h Mon Mar 15 18:27:51 2010 (r205182) @@ -125,6 +125,7 @@ enum tokens { TOK_IPLEN, TOK_IPID, TOK_IPPRECEDENCE, + TOK_DSCP, TOK_IPTOS, TOK_IPTTL, TOK_IPVER, Modified: user/luigi/ipfw3-r8/sys/conf/files ============================================================================== --- user/luigi/ipfw3-r8/sys/conf/files Mon Mar 15 18:26:09 2010 (r205181) +++ user/luigi/ipfw3-r8/sys/conf/files Mon Mar 15 18:27:51 2010 (r205182) @@ -2474,13 +2474,24 @@ netinet/in_proto.c optional inet \ compile-with "${NORMAL_C} -I$S/contrib/pf" netinet/in_rmx.c optional inet netinet/ip_divert.c optional inet ipdivert ipfirewall +netinet/ipfw/dn_heap.c optional inet dummynet +netinet/ipfw/dn_sched_fifo.c optional inet dummynet +netinet/ipfw/dn_sched_rr.c optional inet dummynet +netinet/ipfw/dn_sched_wf2q.c optional inet dummynet +netinet/ipfw/dn_sched_qfq.c optional inet dummynet netinet/ipfw/ip_dummynet.c optional inet dummynet +netinet/ipfw/ip_dn_io.c optional inet dummynet +netinet/ipfw/ip_dn_glue.c optional inet dummynet netinet/ip_ecn.c optional inet | inet6 netinet/ip_encap.c optional inet | inet6 netinet/ip_fastfwd.c optional inet netinet/ipfw/ip_fw2.c optional inet ipfirewall \ compile-with "${NORMAL_C} -I$S/contrib/pf" +netinet/ipfw/ip_fw_dynamic.c optional inet ipfirewall +netinet/ipfw/ip_fw_log.c optional inet ipfirewall netinet/ipfw/ip_fw_pfil.c optional inet ipfirewall +netinet/ipfw/ip_fw_sockopt.c optional inet ipfirewall +netinet/ipfw/ip_fw_table.c optional inet ipfirewall netinet/ipfw/ip_fw_nat.c optional inet ipfirewall_nat netinet/ip_icmp.c optional inet netinet/ip_input.c optional inet Modified: user/luigi/ipfw3-r8/sys/net/if_bridge.c ============================================================================== --- user/luigi/ipfw3-r8/sys/net/if_bridge.c Mon Mar 15 18:26:09 2010 (r205181) +++ user/luigi/ipfw3-r8/sys/net/if_bridge.c Mon Mar 15 18:27:51 2010 (r205182) @@ -134,7 +134,7 @@ __FBSDID("$FreeBSD$"); #include #include -#include +#include /* * Size of the route hash table. Must be a power of two. @@ -3038,20 +3038,28 @@ bridge_pfil(struct mbuf **mp, struct ifn goto bad; } - if (V_ip_fw_chk_ptr && pfil_ipfw != 0 && dir == PFIL_OUT && ifp != NULL) { - struct dn_pkt_tag *dn_tag; + /* XXX this section is also in if_ethersubr.c */ + // XXX PFIL_OUT or DIR_OUT ? + if (V_ip_fw_chk_ptr && pfil_ipfw != 0 && + dir == PFIL_OUT && ifp != NULL) { + struct m_tag *mtag; error = -1; - dn_tag = ip_dn_claim_tag(*mp); - if (dn_tag != NULL) { - if (dn_tag->rule != NULL && V_fw_one_pass) - /* packet already partially processed */ + /* fetch the start point from existing tags, if any */ + mtag = m_tag_locate(*mp, MTAG_IPFW_RULE, 0, NULL); + if (mtag == NULL) { + args.rule.slot = 0; + } else { + struct ipfw_rule_ref *r; + + /* XXX can we free the tag after use ? */ + mtag->m_tag_id = PACKET_TAG_NONE; + r = (struct ipfw_rule_ref *)(mtag + 1); + /* packet already partially processed ? */ + if (r->info & IPFW_ONEPASS) goto ipfwpass; - args.rule = dn_tag->rule; /* matching rule to restart */ - args.rule_id = dn_tag->rule_id; - args.chain_id = dn_tag->chain_id; - } else - args.rule = NULL; + args.rule = *r; + } args.m = *mp; args.oif = ifp; @@ -3077,7 +3085,7 @@ bridge_pfil(struct mbuf **mp, struct ifn * packet will return to us via bridge_dummynet(). */ args.oif = ifp; - ip_dn_io_ptr(mp, DN_TO_IFB_FWD, &args); + ip_dn_io_ptr(mp, DIR_FWD | PROTO_IFB, &args); return (error); } Modified: user/luigi/ipfw3-r8/sys/net/if_ethersubr.c ============================================================================== --- user/luigi/ipfw3-r8/sys/net/if_ethersubr.c Mon Mar 15 18:26:09 2010 (r205181) +++ user/luigi/ipfw3-r8/sys/net/if_ethersubr.c Mon Mar 15 18:27:51 2010 (r205182) @@ -70,9 +70,9 @@ #include #include #include -#include -#include #include +#include +#include #endif #ifdef INET6 #include @@ -466,19 +466,23 @@ ether_ipfw_chk(struct mbuf **m0, struct struct mbuf *m; int i; struct ip_fw_args args; - struct dn_pkt_tag *dn_tag; - - dn_tag = ip_dn_claim_tag(*m0); + struct m_tag *mtag; - if (dn_tag != NULL) { - if (dn_tag->rule != NULL && V_fw_one_pass) + /* fetch start point from rule, if any */ + mtag = m_tag_locate(*m0, MTAG_IPFW_RULE, 0, NULL); + if (mtag == NULL) { + args.rule.slot = 0; + } else { /* dummynet packet, already partially processed */ + struct ipfw_rule_ref *r; + + /* XXX can we free it after use ? */ + mtag->m_tag_id = PACKET_TAG_NONE; + r = (struct ipfw_rule_ref *)(mtag + 1); + if (r->info & IPFW_ONEPASS) return (1); - args.rule = dn_tag->rule; /* matching rule to restart */ - args.rule_id = dn_tag->rule_id; - args.chain_id = dn_tag->chain_id; - } else - args.rule = NULL; + args.rule = *r; + } /* * I need some amt of data to be contiguous, and in case others need @@ -529,6 +533,7 @@ ether_ipfw_chk(struct mbuf **m0, struct return 1; if (ip_dn_io_ptr && (i == IP_FW_DUMMYNET)) { + int dir; /* * Pass the pkt to dummynet, which consumes it. * If shared, make a copy and keep the original. @@ -544,7 +549,8 @@ ether_ipfw_chk(struct mbuf **m0, struct */ *m0 = NULL ; } - ip_dn_io_ptr(&m, dst ? DN_TO_ETH_OUT: DN_TO_ETH_DEMUX, &args); + dir = PROTO_LAYER2 | (dst ? DIR_OUT : DIR_IN); + ip_dn_io_ptr(&m, dir, &args); return 0; } /* Modified: user/luigi/ipfw3-r8/sys/net/radix.c ============================================================================== --- user/luigi/ipfw3-r8/sys/net/radix.c Mon Mar 15 18:26:09 2010 (r205181) +++ user/luigi/ipfw3-r8/sys/net/radix.c Mon Mar 15 18:27:51 2010 (r205182) @@ -33,7 +33,6 @@ /* * Routines to build and maintain radix trees for routing lookups. */ -#ifndef _RADIX_H_ #include #ifdef _KERNEL #include @@ -41,20 +40,21 @@ #include #include #include -#include -#else -#include -#endif #include #include -#endif - #include "opt_mpath.h" - #ifdef RADIX_MPATH #include #endif - +#else /* !_KERNEL */ +#include +#include +#include +#define log(x, arg...) fprintf(stderr, ## arg) +#define panic(x) fprintf(stderr, "PANIC: %s", x), exit(1) +#define min(a, b) ((a) < (b) ? (a) : (b) ) +#include +#endif /* !_KERNEL */ static int rn_walktree_from(struct radix_node_head *h, void *a, void *m, walktree_f_t *f, void *w); @@ -72,6 +72,8 @@ static struct radix_node_head *mask_rnhe /* * Work area -- the following point to 3 buffers of size max_keylen, * allocated in this order in a block of memory malloc'ed by rn_init. + * rn_zeros, rn_ones are set in rn_init and used in readonly afterwards. + * addmask_key is used in rn_addmask in rw mode and not thread-safe. */ static char *rn_zeros, *rn_ones, *addmask_key; @@ -135,8 +137,9 @@ static int rn_satisfies_leaf(char *trial * To make the assumption more explicit, we use the LEN() macro to access * this field. It is safe to pass an expression with side effects * to LEN() as the argument is evaluated only once. + * We cast the result to int as this is the dominant usage. */ -#define LEN(x) (*(const u_char *)(x)) +#define LEN(x) ( (int) (*(const u_char *)(x)) ) /* * XXX THIS NEEDS TO BE FIXED @@ -197,7 +200,7 @@ rn_refines(m_arg, n_arg) { register caddr_t m = m_arg, n = n_arg; register caddr_t lim, lim2 = lim = n + LEN(n); - int longer = LEN(n++) - (int)LEN(m++); + int longer = LEN(n++) - LEN(m++); int masks_are_equal = 1; if (longer > 0) @@ -250,10 +253,10 @@ rn_satisfies_leaf(trial, leaf, skip) char *cplim; int length = min(LEN(cp), LEN(cp2)); - if (cp3 == 0) + if (cp3 == NULL) cp3 = rn_ones; else - length = min(length, *(u_char *)cp3); + length = min(length, LEN(cp3)); cplim = cp + length; cp3 += skip; cp2 += skip; for (cp += skip; cp < cplim; cp++, cp2++, cp3++) if ((*cp ^ *cp2) & *cp3) @@ -424,7 +427,7 @@ rn_insert(v_arg, head, dupentry, nodes) { caddr_t v = v_arg; struct radix_node *top = head->rnh_treetop; - int head_off = top->rn_offset, vlen = (int)LEN(v); + int head_off = top->rn_offset, vlen = LEN(v); register struct radix_node *t = rn_search(v_arg, top); register caddr_t cp = v + head_off; register int b; @@ -933,7 +936,7 @@ on1: if (m) log(LOG_ERR, "rn_delete: Orphaned Mask %p at %p\n", - (void *)m, (void *)x); + m, x); } } /* @@ -1158,17 +1161,28 @@ rn_inithead(head, off) return (1); } +#if 0 +int rn_detachhead(void **head) +{ + struct radix_node_head *rnh; + + KASSERT((head != NULL && *head != NULL), + ("%s: head already freed", __func__)); + rnh = *head; + + /* Free nodes. */ + Free(rnh); + + *head = NULL; + return (1); +} +#endif // XXX void -rn_init() +rn_init(int maxk) { char *cp, *cplim; -#ifdef _KERNEL - struct domain *dom; - for (dom = domains; dom; dom = dom->dom_next) - if (dom->dom_maxrtkey > max_keylen) - max_keylen = dom->dom_maxrtkey; -#endif + max_keylen = maxk; if (max_keylen == 0) { log(LOG_ERR, "rn_init: radix functions require max_keylen be set\n"); Modified: user/luigi/ipfw3-r8/sys/net/radix.h ============================================================================== --- user/luigi/ipfw3-r8/sys/net/radix.h Mon Mar 15 18:26:09 2010 (r205181) +++ user/luigi/ipfw3-r8/sys/net/radix.h Mon Mar 15 18:27:51 2010 (r205182) @@ -160,8 +160,9 @@ struct radix_node_head { #define RADIX_NODE_HEAD_WLOCK_ASSERT(rnh) rw_assert(&(rnh)->rnh_lock, RA_WLOCKED) #endif /* _KERNEL */ -void rn_init(void); +void rn_init(int); int rn_inithead(void **, int); +int rn_detachhead(void **); int rn_refines(void *, void *); struct radix_node *rn_addmask(void *, int, int), Modified: user/luigi/ipfw3-r8/sys/net/route.c ============================================================================== --- user/luigi/ipfw3-r8/sys/net/route.c Mon Mar 15 18:26:09 2010 (r205181) +++ user/luigi/ipfw3-r8/sys/net/route.c Mon Mar 15 18:27:51 2010 (r205182) @@ -169,13 +169,20 @@ rt_tables_get_rnh(int table, int fam) static void route_init(void) { + struct domain *dom; + int max_keylen = 0; /* whack the tunable ints into line. */ if (rt_numfibs > RT_MAXFIBS) rt_numfibs = RT_MAXFIBS; if (rt_numfibs == 0) rt_numfibs = 1; - rn_init(); /* initialize all zeroes, all ones, mask table */ + + for (dom = domains; dom; dom = dom->dom_next) + if (dom->dom_maxrtkey > max_keylen) + max_keylen = dom->dom_maxrtkey; + + rn_init(max_keylen); /* init all zeroes, all ones, mask table */ } SYSINIT(route_init, SI_SUB_PROTO_DOMAIN, SI_ORDER_THIRD, route_init, 0); Modified: user/luigi/ipfw3-r8/sys/netinet/ip_divert.c ============================================================================== --- user/luigi/ipfw3-r8/sys/netinet/ip_divert.c Mon Mar 15 18:26:09 2010 (r205181) +++ user/luigi/ipfw3-r8/sys/netinet/ip_divert.c Mon Mar 15 18:27:51 2010 (r205182) @@ -32,14 +32,10 @@ __FBSDID("$FreeBSD$"); #if !defined(KLD_MODULE) #include "opt_inet.h" -#include "opt_ipfw.h" #include "opt_sctp.h" #ifndef INET #error "IPDIVERT requires INET." #endif -#ifndef IPFIREWALL -#error "IPDIVERT requires IPFIREWALL" -#endif #endif #include @@ -72,9 +68,7 @@ __FBSDID("$FreeBSD$"); #include #include #include -#include #include -#include #ifdef SCTP #include #endif @@ -92,27 +86,29 @@ __FBSDID("$FreeBSD$"); #define DIVRCVQ (65536 + 100) /* - * Divert sockets work in conjunction with ipfw, see the divert(4) - * manpage for features. - * Internally, packets selected by ipfw in ip_input() or ip_output(), - * and never diverted before, are passed to the input queue of the - * divert socket with a given 'divert_port' number (as specified in - * the matching ipfw rule), and they are tagged with a 16 bit cookie - * (representing the rule number of the matching ipfw rule), which - * is passed to process reading from the socket. + * Divert sockets work in conjunction with ipfw or other packet filters, + * see the divert(4) manpage for features. + * Packets are selected by the packet filter and tagged with an + * MTAG_IPFW_RULE tag carrying the 'divert port' number (as set by + * the packet filter) and information on the matching filter rule for + * subsequent reinjection. The divert_port is used to put the packet + * on the corresponding divert socket, while the rule number is passed + * up (at least partially) as the sin_port in the struct sockaddr. * - * Packets written to the divert socket are again tagged with a cookie - * (usually the same as above) and a destination address. - * If the destination address is INADDR_ANY then the packet is - * treated as outgoing and sent to ip_output(), otherwise it is - * treated as incoming and sent to ip_input(). - * In both cases, the packet is tagged with the cookie. + * Packets written to the divert socket carry in sin_addr a + * destination address, and in sin_port the number of the filter rule + * after which to continue processing. + * If the destination address is INADDR_ANY, the packet is treated as + * as outgoing and sent to ip_output(); otherwise it is treated as + * incoming and sent to ip_input(). + * Further, sin_zero carries some information on the interface, + * which can be used in the reinject -- see comments in the code. * * On reinjection, processing in ip_input() and ip_output() * will be exactly the same as for the original packet, except that - * ipfw processing will start at the rule number after the one - * written in the cookie (so, tagging a packet with a cookie of 0 - * will cause it to be effectively considered as a standard packet). + * packet filter processing will start at the rule number after the one + * written in the sin_port (ipfw does not allow a rule #0, so sin_port=0 + * will apply the entire ruleset to the packet). */ /* Internal variables. */ @@ -193,7 +189,7 @@ div_destroy(void) * IPPROTO_DIVERT is not in the real IP protocol number space; this * function should never be called. Just in case, drop any packets. */ -void +static void div_input(struct mbuf *m, int off) { @@ -217,9 +213,8 @@ divert_packet(struct mbuf *m, int incomi struct sockaddr_in divsrc; struct m_tag *mtag; - mtag = m_tag_find(m, PACKET_TAG_DIVERT, NULL); + mtag = m_tag_locate(m, MTAG_IPFW_RULE, 0, NULL); if (mtag == NULL) { - printf("%s: no divert tag\n", __func__); m_freem(m); return; } @@ -244,14 +239,15 @@ divert_packet(struct mbuf *m, int incomi ip->ip_len = htons(ip->ip_len); } #endif + bzero(&divsrc, sizeof(divsrc)); + divsrc.sin_len = sizeof(divsrc); + divsrc.sin_family = AF_INET; + /* record matching rule, in host format */ + divsrc.sin_port = ((struct ipfw_rule_ref *)(mtag+1))->rulenum; /* * Record receive interface address, if any. * But only for incoming packets. */ - bzero(&divsrc, sizeof(divsrc)); - divsrc.sin_len = sizeof(divsrc); - divsrc.sin_family = AF_INET; - divsrc.sin_port = divert_cookie(mtag); /* record matching rule */ if (incoming) { struct ifaddr *ifa; struct ifnet *ifp; @@ -299,7 +295,7 @@ divert_packet(struct mbuf *m, int incomi /* Put packet on socket queue, if any */ sa = NULL; - nport = htons((u_int16_t)divert_info(mtag)); + nport = htons((u_int16_t)(((struct ipfw_rule_ref *)(mtag+1))->info)); INP_INFO_RLOCK(&V_divcbinfo); LIST_FOREACH(inp, &V_divcb, inp_list) { /* XXX why does only one socket match? */ @@ -338,7 +334,7 @@ div_output(struct socket *so, struct mbu struct mbuf *control) { struct m_tag *mtag; - struct divert_tag *dt; + struct ipfw_rule_ref *dt; int error = 0; struct mbuf *options; @@ -353,23 +349,31 @@ div_output(struct socket *so, struct mbu if (control) m_freem(control); /* XXX */ - if ((mtag = m_tag_find(m, PACKET_TAG_DIVERT, NULL)) == NULL) { - mtag = m_tag_get(PACKET_TAG_DIVERT, sizeof(struct divert_tag), - M_NOWAIT | M_ZERO); + mtag = m_tag_locate(m, MTAG_IPFW_RULE, 0, NULL); + if (mtag == NULL) { + /* this should be normal */ + mtag = m_tag_alloc(MTAG_IPFW_RULE, 0, + sizeof(struct ipfw_rule_ref), M_NOWAIT | M_ZERO); if (mtag == NULL) { error = ENOBUFS; goto cantsend; } - dt = (struct divert_tag *)(mtag+1); m_tag_prepend(m, mtag); - } else - dt = (struct divert_tag *)(mtag+1); + } + dt = (struct ipfw_rule_ref *)(mtag+1); /* Loopback avoidance and state recovery */ if (sin) { int i; - dt->cookie = sin->sin_port; + /* set the starting point. We provide a non-zero slot, + * but a non_matching chain_id to skip that info and use + * the rulenum/rule_id. + */ + dt->slot = 1; /* dummy, chain_id is invalid */ + dt->chain_id = 0; + dt->rulenum = sin->sin_port+1; /* host format ? */ + dt->rule_id = 0; /* * Find receive interface with the given name, stuffed * (if it exists) in the sin_zero[] field. @@ -387,7 +391,7 @@ div_output(struct socket *so, struct mbu struct ip *const ip = mtod(m, struct ip *); struct inpcb *inp; - dt->info |= IP_FW_DIVERT_OUTPUT_FLAG; + dt->info |= IPFW_IS_DIVERT | IPFW_INFO_OUT; INP_INFO_WLOCK(&V_divcbinfo); inp = sotoinpcb(so); INP_RLOCK(inp); @@ -453,7 +457,7 @@ div_output(struct socket *so, struct mbu m_freem(options); } } else { - dt->info |= IP_FW_DIVERT_LOOPBACK_FLAG; + dt->info |= IPFW_IS_DIVERT | IPFW_INFO_IN; if (m->m_pkthdr.rcvif == NULL) { /* * No luck with the name, check by IP address. @@ -587,7 +591,7 @@ div_send(struct socket *so, int flags, s return div_output(so, m, (struct sockaddr_in *)nam, control); } -void +static void div_ctlinput(int cmd, struct sockaddr *sa, void *vip) { struct in_addr faddr; @@ -800,5 +804,5 @@ static moduledata_t ipdivertmod = { }; DECLARE_MODULE(ipdivert, ipdivertmod, SI_SUB_PROTO_IFATTACHDOMAIN, SI_ORDER_ANY); -MODULE_DEPEND(dummynet, ipfw, 2, 2, 2); +MODULE_DEPEND(ipdivert, ipfw, 2, 2, 2); MODULE_VERSION(ipdivert, 1); Modified: user/luigi/ipfw3-r8/sys/netinet/ip_divert.h ============================================================================== --- user/luigi/ipfw3-r8/sys/netinet/ip_divert.h Mon Mar 15 18:26:09 2010 (r205181) +++ user/luigi/ipfw3-r8/sys/netinet/ip_divert.h Mon Mar 15 18:27:51 2010 (r205182) @@ -36,53 +36,20 @@ #define _NETINET_IP_DIVERT_H_ /* - * Sysctl declaration. + * divert has no custom kernel-userland API. + * + * All communication occurs through a sockaddr_in socket where + * + * kernel-->userland + * sin_port = matching rule, host format; + * sin_addr = IN: first address of the incoming interface; + * OUT: INADDR_ANY + * sin_zero = if fits, the interface name (max 7 bytes + NUL) + * + * userland->kernel + * sin_port = restart-rule - 1, host order + * (we restart at sin_port + 1) + * sin_addr = IN: address of the incoming interface; + * OUT: INADDR_ANY */ -#ifdef SYSCTL_DECL -SYSCTL_DECL(_net_inet_divert); -#endif - -/* - * Divert socket definitions. - */ -struct divert_tag { - u_int32_t info; /* port & flags */ - u_int16_t cookie; /* ipfw rule number */ -}; - -/* - * Return the divert cookie associated with the mbuf; if any. - */ -static __inline u_int16_t -divert_cookie(struct m_tag *mtag) -{ - return ((struct divert_tag *)(mtag+1))->cookie; -} -static __inline u_int16_t -divert_find_cookie(struct mbuf *m) -{ - struct m_tag *mtag = m_tag_find(m, PACKET_TAG_DIVERT, NULL); - return mtag ? divert_cookie(mtag) : 0; -} - -/* - * Return the divert info associated with the mbuf; if any. - */ -static __inline u_int32_t -divert_info(struct m_tag *mtag) -{ - return ((struct divert_tag *)(mtag+1))->info; -} -static __inline u_int32_t -divert_find_info(struct mbuf *m) -{ - struct m_tag *mtag = m_tag_find(m, PACKET_TAG_DIVERT, NULL); - return mtag ? divert_info(mtag) : 0; -} - -typedef void ip_divert_packet_t(struct mbuf *m, int incoming); -extern ip_divert_packet_t *ip_divert_ptr; - -extern void div_input(struct mbuf *, int); -extern void div_ctlinput(int, struct sockaddr *, void *); #endif /* _NETINET_IP_DIVERT_H_ */ Modified: user/luigi/ipfw3-r8/sys/netinet/ip_fw.h ============================================================================== --- user/luigi/ipfw3-r8/sys/netinet/ip_fw.h Mon Mar 15 18:26:09 2010 (r205181) +++ user/luigi/ipfw3-r8/sys/netinet/ip_fw.h Mon Mar 15 18:27:51 2010 (r205182) @@ -487,24 +487,27 @@ struct ip_fw { #define RULESIZE(rule) (sizeof(struct ip_fw) + \ ((struct ip_fw *)(rule))->cmd_len * 4 - 4) -#if 1 // moved to in.h +#if 1 // should be moved to in.h /* * This structure is used as a flow mask and a flow id for various * parts of the code. + * addr_type is used in userland and kernel to mark the address type. + * fib is used in the kernel to record the fib in use. + * _flags is used in the kernel to store tcp flags for dynamic rules. */ struct ipfw_flow_id { uint32_t dst_ip; uint32_t src_ip; uint16_t dst_port; uint16_t src_port; - uint8_t fib; - uint8_t proto; - uint8_t flags; /* protocol-specific flags */ - uint8_t addr_type; /* 4 = ipv4, 6 = ipv6, 1=ether ? */ + uint8_t fib; + uint8_t proto; + uint8_t _flags; /* protocol-specific flags */ + uint8_t addr_type; /* 4=ip4, 6=ip6, 1=ether ? */ struct in6_addr dst_ip6; struct in6_addr src_ip6; uint32_t flow_id6; - uint32_t frag_id6; + uint32_t extra; /* queue/pipe or frag_id */ }; #endif Modified: user/luigi/ipfw3-r8/sys/netinet/ipfw/ip_dn_io.c ============================================================================== --- user/luigi/ipfw3-r8/sys/netinet/ipfw/ip_dn_io.c Mon Mar 15 18:26:09 2010 (r205181) +++ user/luigi/ipfw3-r8/sys/netinet/ipfw/ip_dn_io.c Mon Mar 15 18:27:51 2010 (r205182) @@ -113,6 +113,10 @@ SYSCTL_INT(_net_inet_ip_dummynet, OID_AU CTLFLAG_RW, &dn_cfg.io_fast, 0, "Enable fast dummynet io."); SYSCTL_INT(_net_inet_ip_dummynet, OID_AUTO, debug, CTLFLAG_RW, &dn_cfg.debug, 0, "Dummynet debug level"); +SYSCTL_INT(_net_inet_ip_dummynet, OID_AUTO, expire, + CTLFLAG_RW, &dn_cfg.expire, 0, "Expire empty queues/pipes"); +SYSCTL_INT(_net_inet_ip_dummynet, OID_AUTO, expire_cycle, + CTLFLAG_RD, &dn_cfg.expire_cycle, 0, "Expire cycle for queues/pipes"); /* RED parameters */ SYSCTL_INT(_net_inet_ip_dummynet, OID_AUTO, red_lookup_depth, @@ -546,8 +550,11 @@ dummynet_task(void *context, int pending transmit_event(&q, (struct delay_line *)p, dn_cfg.curr_time); } } - dn_drain_scheduler(); - dn_drain_queue(); + if (dn_cfg.expire && ++dn_cfg.expire_cycle >= dn_cfg.expire) { + dn_cfg.expire_cycle = 0; + dn_drain_scheduler(); + dn_drain_queue(); + } DN_BH_WUNLOCK(); dn_reschedule(); Modified: user/luigi/ipfw3-r8/sys/netinet/ipfw/ip_dn_private.h ============================================================================== --- user/luigi/ipfw3-r8/sys/netinet/ipfw/ip_dn_private.h Mon Mar 15 18:26:09 2010 (r205181) +++ user/luigi/ipfw3-r8/sys/netinet/ipfw/ip_dn_private.h Mon Mar 15 18:27:51 2010 (r205182) @@ -141,10 +141,14 @@ struct dn_parms { struct dn_alg_head schedlist; /* list of algorithms */ /* Store the fs/sch to scan when draining. The value is the - * bucket number of the hash table + * bucket number of the hash table. Expire can be disabled + * with net.inet.ip.dummynet.expire=0, or it happens every + * expire ticks. **/ int drain_fs; int drain_sch; + uint32_t expire; + uint32_t expire_cycle; /* tick count */ /* if the upper half is busy doing something long, * can set the busy flag and we will enqueue packets in Modified: user/luigi/ipfw3-r8/sys/netinet/ipfw/ip_dummynet.c ============================================================================== --- user/luigi/ipfw3-r8/sys/netinet/ipfw/ip_dummynet.c Mon Mar 15 18:26:09 2010 (r205181) +++ user/luigi/ipfw3-r8/sys/netinet/ipfw/ip_dummynet.c Mon Mar 15 18:27:51 2010 (r205182) @@ -122,6 +122,12 @@ ipdn_bound_var(int *v, int dflt, int lo, } /*---- flow_id mask, hash and compare functions ---*/ +/* + * The flow_id includes the 5-tuple, the queue/pipe number + * which we store in the extra area in host order, + * and for ipv6 also the flow_id6. + * XXX see if we want the tos byte (can store in 'flags') + */ static struct ipfw_flow_id * flow_id_mask(struct ipfw_flow_id *mask, struct ipfw_flow_id *id) { @@ -130,7 +136,7 @@ flow_id_mask(struct ipfw_flow_id *mask, id->dst_port &= mask->dst_port; id->src_port &= mask->src_port; id->proto &= mask->proto; - id->flags = 0; /* we don't care about this one */ + id->extra &= mask->extra; if (is_v6) { APPLY_MASK(&id->dst_ip6, &mask->dst_ip6); APPLY_MASK(&id->src_ip6, &mask->src_ip6); @@ -151,7 +157,7 @@ flow_id_or(struct ipfw_flow_id *src, str dst->dst_port |= src->dst_port; dst->src_port |= src->src_port; dst->proto |= src->proto; - dst->flags = 0; /* we don't care about this one */ + dst->extra |= src->extra; if (is_v6) { #define OR_MASK(_d, _s) \ (_d)->__u6_addr.__u6_addr32[0] |= (_s)->__u6_addr.__u6_addr32[0]; \ @@ -172,7 +178,7 @@ flow_id_or(struct ipfw_flow_id *src, str static int nonzero_mask(struct ipfw_flow_id *m) { - if (m->dst_port || m->src_port || m->proto) + if (m->dst_port || m->src_port || m->proto || m->extra) return 1; if (IS_IP6_FLOW_ID(m)) { return @@ -208,10 +214,12 @@ flow_id_hash(struct ipfw_flow_id *id) (s[0] << 16) ^ (s[1] << 16) ^ (s[2] << 16) ^ (s[3] << 16) ^ (id->dst_port << 1) ^ (id->src_port) ^ + (id->extra) ^ (id->proto ) ^ (id->flow_id6); } else { i = (id->dst_ip) ^ (id->dst_ip >> 15) ^ (id->src_ip << 1) ^ (id->src_ip >> 16) ^ + (id->extra) ^ (id->dst_port << 1) ^ (id->src_port) ^ (id->proto); } return i; @@ -223,29 +231,26 @@ flow_id_cmp(struct ipfw_flow_id *id1, st { int is_v6 = IS_IP6_FLOW_ID(id1); - if (is_v6 != IS_IP6_FLOW_ID(id2)) - return 1; /* a ipv4 and a ipv6 flow */ - - if (!is_v6 && id1->dst_ip == id2->dst_ip && - id1->src_ip == id2->src_ip && - id1->dst_port == id2->dst_port && - id1->src_port == id2->src_port && - id1->proto == id2->proto && - id1->flags == id2->flags) - return 0; - - if (is_v6 && + if (!is_v6) { + if (IS_IP6_FLOW_ID(id2)) + return 1; /* different address families */ + + return (id1->dst_ip == id2->dst_ip && + id1->src_ip == id2->src_ip && + id1->dst_port == id2->dst_port && + id1->src_port == id2->src_port && + id1->proto == id2->proto && + id1->extra == id2->extra) ? 0 : 1; + } + /* the ipv6 case */ + return ( !bcmp(&id1->dst_ip6,&id2->dst_ip6, sizeof(id1->dst_ip6)) && !bcmp(&id1->src_ip6,&id2->src_ip6, sizeof(id1->src_ip6)) && id1->dst_port == id2->dst_port && id1->src_port == id2->src_port && id1->proto == id2->proto && - id1->flags == id2->flags && - id1->flow_id6 == id2->flow_id6) - return 0; - - /* Masks differ */ - return 1; + id1->extra == id2->extra && + id1->flow_id6 == id2->flow_id6) ? 0 : 1; } /*--------- end of flow-id mask, hash and compare ---------*/ @@ -2111,10 +2116,13 @@ ip_dn_init(void) if (bootverbose) printf("DUMMYNET with IPv6 initialized (100131)\n"); - /* init defaults here, MSVC does not accept initializers */ + /* Set defaults here. MSVC does not accept initializers, + * and this is also useful for vimages + */ /* queue limits */ dn_cfg.slot_limit = 100; /* Foot shooting limit for queues. */ dn_cfg.byte_limit = 1024 * 1024; + dn_cfg.expire = 1; /* RED parameters */ dn_cfg.red_lookup_depth = 256; /* default lookup table depth */ Modified: user/luigi/ipfw3-r8/sys/netinet/ipfw/ip_fw2.c ============================================================================== --- user/luigi/ipfw3-r8/sys/netinet/ipfw/ip_fw2.c Mon Mar 15 18:26:09 2010 (r205181) +++ user/luigi/ipfw3-r8/sys/netinet/ipfw/ip_fw2.c Mon Mar 15 18:27:51 2010 (r205182) @@ -886,10 +886,13 @@ ipfw_chk(struct ip_fw_args *args) * ulp is NULL if not found. */ void *ulp = NULL; /* upper layer protocol pointer. */ + /* XXX ipv6 variables */ int is_ipv6 = 0; - u_int16_t ext_hd = 0; /* bits vector for extension header filtering */ + uint8_t icmp6_type = 0; + uint16_t ext_hd = 0; /* bits vector for extension header filtering */ /* end of ipv6 variables */ + int is_ipv4 = 0; int done = 0; /* flag to exit the outer loop */ @@ -941,14 +944,15 @@ do { \ switch (proto) { case IPPROTO_ICMPV6: PULLUP_TO(hlen, ulp, struct icmp6_hdr); - args->f_id.flags = ICMP6(ulp)->icmp6_type; + icmp6_type = ICMP6(ulp)->icmp6_type; break; case IPPROTO_TCP: PULLUP_TO(hlen, ulp, struct tcphdr); dst_port = TCP(ulp)->th_dport; src_port = TCP(ulp)->th_sport; - args->f_id.flags = TCP(ulp)->th_flags; + /* save flags for dynamic rules */ + args->f_id._flags = TCP(ulp)->th_flags; break; case IPPROTO_SCTP: @@ -1012,7 +1016,7 @@ do { \ return (IP_FW_DENY); break; } - args->f_id.frag_id6 = + args->f_id.extra = ntohl(((struct ip6_frag *)ulp)->ip6f_ident); ulp = NULL; break; @@ -1115,7 +1119,8 @@ do { \ PULLUP_TO(hlen, ulp, struct tcphdr); dst_port = TCP(ulp)->th_dport; src_port = TCP(ulp)->th_sport; - args->f_id.flags = TCP(ulp)->th_flags; + /* save flags for dynamic rules */ + args->f_id._flags = TCP(ulp)->th_flags; break; case IPPROTO_UDP: @@ -1126,7 +1131,7 @@ do { \ case IPPROTO_ICMP: PULLUP_TO(hlen, ulp, struct icmphdr); - args->f_id.flags = ICMP(ulp)->icmp_type; + //args->f_id.flags = ICMP(ulp)->icmp_type; break; default: @@ -1362,6 +1367,8 @@ do { \ key = dst_ip.s_addr; else if (v == 1) key = src_ip.s_addr; + else if (v == 6) /* dscp */ + key = (ip->ip_tos >> 2) & 0x3f; else if (offset != 0) break; else if (proto != IPPROTO_TCP && @@ -2034,7 +2041,7 @@ do { \ if (hlen > 0 && is_ipv6 && ((offset & IP6F_OFF_MASK) == 0) && (proto != IPPROTO_ICMPV6 || - (is_icmp6_query(args->f_id.flags) == 1)) && + (is_icmp6_query(icmp6_type) == 1)) && !(m->m_flags & (M_BCAST|M_MCAST)) && !IN6_IS_ADDR_MULTICAST(&args->f_id.dst_ip6)) { send_reject6( @@ -2392,7 +2399,7 @@ vnet_ipfw_uninit(const void *unused) IPFW_WLOCK(chain); ipfw_dyn_uninit(0); /* run the callout_drain */ - ipfw_destroy_tables(chain); + ipfw_flush_tables(chain); reap = NULL; for (i = 0; i < chain->n_rules; i++) { rule = chain->map[i]; Modified: user/luigi/ipfw3-r8/sys/netinet/ipfw/ip_fw_dynamic.c ============================================================================== --- user/luigi/ipfw3-r8/sys/netinet/ipfw/ip_fw_dynamic.c Mon Mar 15 18:26:09 2010 (r205181) +++ user/luigi/ipfw3-r8/sys/netinet/ipfw/ip_fw_dynamic.c Mon Mar 15 18:27:51 2010 (r205182) @@ -476,7 +476,7 @@ next: V_ipfw_dyn_v[i] = q; } if (pkt->proto == IPPROTO_TCP) { /* update state according to flags */ - u_char flags = pkt->flags & (TH_FIN|TH_SYN|TH_RST); + u_char flags = pkt->_flags & (TH_FIN|TH_SYN|TH_RST); #define BOTH_SYN (TH_SYN | (TH_SYN << 8)) #define BOTH_FIN (TH_FIN | (TH_FIN << 8)) Modified: user/luigi/ipfw3-r8/sys/netinet/ipfw/ip_fw_log.c ============================================================================== --- user/luigi/ipfw3-r8/sys/netinet/ipfw/ip_fw_log.c Mon Mar 15 18:26:09 2010 (r205181) +++ user/luigi/ipfw3-r8/sys/netinet/ipfw/ip_fw_log.c Mon Mar 15 18:27:51 2010 (r205182) *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** From owner-svn-src-user@FreeBSD.ORG Tue Mar 16 01:48:40 2010 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9C53F106564A; Tue, 16 Mar 2010 01:48:40 +0000 (UTC) (envelope-from jmallett@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 8B1DE8FC14; Tue, 16 Mar 2010 01:48:40 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id o2G1meZb038869; Tue, 16 Mar 2010 01:48:40 GMT (envelope-from jmallett@svn.freebsd.org) Received: (from jmallett@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id o2G1mevt038867; Tue, 16 Mar 2010 01:48:40 GMT (envelope-from jmallett@svn.freebsd.org) Message-Id: <201003160148.o2G1mevt038867@svn.freebsd.org> From: Juli Mallett Date: Tue, 16 Mar 2010 01:48:40 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r205201 - user/jmallett/octeon/lib/libc/mips/sys X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Mar 2010 01:48:40 -0000 Author: jmallett Date: Tue Mar 16 01:48:40 2010 New Revision: 205201 URL: http://svn.freebsd.org/changeset/base/205201 Log: Put __sys_brk in text, not data. Modified: user/jmallett/octeon/lib/libc/mips/sys/brk.S Modified: user/jmallett/octeon/lib/libc/mips/sys/brk.S ============================================================================== --- user/jmallett/octeon/lib/libc/mips/sys/brk.S Tue Mar 16 00:50:57 2010 (r205200) +++ user/jmallett/octeon/lib/libc/mips/sys/brk.S Tue Mar 16 01:48:40 2010 (r205201) @@ -49,6 +49,7 @@ __FBSDID("$FreeBSD$"); _C_LABEL(minbrk): .word _C_LABEL(_end) + .text LEAF(__sys_brk) WEAK_ALIAS(brk, __sys_brk) WEAK_ALIAS(_brk, __sys_brk) From owner-svn-src-user@FreeBSD.ORG Tue Mar 16 01:52:04 2010 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0F48C106566B; Tue, 16 Mar 2010 01:52:04 +0000 (UTC) (envelope-from jmallett@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id F1FC38FC12; Tue, 16 Mar 2010 01:52:03 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id o2G1q36D039624; Tue, 16 Mar 2010 01:52:03 GMT (envelope-from jmallett@svn.freebsd.org) Received: (from jmallett@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id o2G1q3Mh039622; Tue, 16 Mar 2010 01:52:03 GMT (envelope-from jmallett@svn.freebsd.org) Message-Id: <201003160152.o2G1q3Mh039622@svn.freebsd.org> From: Juli Mallett Date: Tue, 16 Mar 2010 01:52:03 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r205202 - user/jmallett/octeon/lib/libc/mips/sys X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Mar 2010 01:52:04 -0000 Author: jmallett Date: Tue Mar 16 01:52:03 2010 New Revision: 205202 URL: http://svn.freebsd.org/changeset/base/205202 Log: Properly save and restore the GP on the stack in __cerror, rather than using PIC_PROLOGUE which simply saves and restores it to a temporary register, as is sufficient for syscalls but not for function calls. Submitted by: C. Jayachandran (original version) Modified: user/jmallett/octeon/lib/libc/mips/sys/cerror.S Modified: user/jmallett/octeon/lib/libc/mips/sys/cerror.S ============================================================================== --- user/jmallett/octeon/lib/libc/mips/sys/cerror.S Tue Mar 16 01:48:40 2010 (r205201) +++ user/jmallett/octeon/lib/libc/mips/sys/cerror.S Tue Mar 16 01:52:03 2010 (r205202) @@ -47,21 +47,26 @@ __FBSDID("$FreeBSD$"); .globl _C_LABEL(__error) NESTED_NOPROFILE(__cerror, CALLFRAME_SIZ, ra) .mask 0x80000000, (CALLFRAME_RA - CALLFRAME_SIZ) - PIC_PROLOGUE(__cerror) + SETUP_GP PTR_SUBU sp, sp, CALLFRAME_SIZ - PTR_S ra, CALLFRAME_RA(sp) - INT_S v0, CALLFRAME_S0(sp) # save errno value - + SETUP_GP64(CALLFRAME_GP, __cerror) SAVE_GP(CALLFRAME_GP) + PTR_S ra, CALLFRAME_RA(sp) + REG_S v0, CALLFRAME_S0(sp) # save errno value + PTR_LA t9, _C_LABEL(__error) # locate address of errno jalr t9 - INT_L t0, CALLFRAME_S0(sp) + REG_L t0, CALLFRAME_S0(sp) PTR_L ra, CALLFRAME_RA(sp) INT_S t0, 0(v0) # update errno value + + RESTORE_GP64 PTR_ADDU sp, sp, CALLFRAME_SIZ + li v0, -1 li v1, -1 - PIC_RETURN() + + j ra END(__cerror) From owner-svn-src-user@FreeBSD.ORG Tue Mar 16 04:31:13 2010 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 91DE21065670; Tue, 16 Mar 2010 04:31:13 +0000 (UTC) (envelope-from jmallett@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 80F328FC18; Tue, 16 Mar 2010 04:31:13 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id o2G4VDU0074599; Tue, 16 Mar 2010 04:31:13 GMT (envelope-from jmallett@svn.freebsd.org) Received: (from jmallett@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id o2G4VDUx074597; Tue, 16 Mar 2010 04:31:13 GMT (envelope-from jmallett@svn.freebsd.org) Message-Id: <201003160431.o2G4VDUx074597@svn.freebsd.org> From: Juli Mallett Date: Tue, 16 Mar 2010 04:31:13 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r205203 - user/jmallett/octeon/lib/libc/mips/sys X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Mar 2010 04:31:13 -0000 Author: jmallett Date: Tue Mar 16 04:31:13 2010 New Revision: 205203 URL: http://svn.freebsd.org/changeset/base/205203 Log: Make ptrace save/restore gp properly like cerror. Sponsored by: Packet Forensics Modified: user/jmallett/octeon/lib/libc/mips/sys/ptrace.S Modified: user/jmallett/octeon/lib/libc/mips/sys/ptrace.S ============================================================================== --- user/jmallett/octeon/lib/libc/mips/sys/ptrace.S Tue Mar 16 01:52:03 2010 (r205202) +++ user/jmallett/octeon/lib/libc/mips/sys/ptrace.S Tue Mar 16 04:31:13 2010 (r205203) @@ -43,25 +43,29 @@ __FBSDID("$FreeBSD$"); NESTED_NOPROFILE(ptrace, CALLFRAME_SIZ, ra) .mask 0x80000000, (CALLFRAME_RA - CALLFRAME_SIZ) - PIC_PROLOGUE(ptrace) - PTR_SUBU sp, sp, CALLFRAME_SP - REG_S ra, CALLFRAME_RA(sp) - + SETUP_GP + PTR_SUBU sp, sp, CALLFRAME_SIZ + SETUP_GP64(CALLFRAME_GP, ptrace) SAVE_GP(CALLFRAME_GP) - PTR_LA t9, _C_LABEL(__error) # locate address of errno - jalr t9 + PTR_S ra, CALLFRAME_RA(sp) - INT_S zero, 0(v0) + PTR_LA t9, _C_LABEL(__error) # locate address of errno + jalr t9 - REG_L ra, CALLFRAME_RA(sp) - PTR_ADDU sp, sp, CALLFRAME_SIZ + PTR_L ra, CALLFRAME_RA(sp) + INT_S zero, 0(v0) # update errno value - li v0, SYS_ptrace + li v0, SYS_ptrace syscall - bne a3, zero, 1f - PIC_RETURN() + RESTORE_GP64 + PTR_ADDU sp, sp, CALLFRAME_SIZ + + bne a3, zero, 1f + + j ra 1: - PIC_TAILCALL(__cerror) + PTR_LA t9, __cerror + jalr t9 END(ptrace) From owner-svn-src-user@FreeBSD.ORG Tue Mar 16 04:31:57 2010 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 594E7106566B; Tue, 16 Mar 2010 04:31:57 +0000 (UTC) (envelope-from jmallett@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 490128FC0A; Tue, 16 Mar 2010 04:31:57 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id o2G4Vv1I074799; Tue, 16 Mar 2010 04:31:57 GMT (envelope-from jmallett@svn.freebsd.org) Received: (from jmallett@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id o2G4Vvl3074797; Tue, 16 Mar 2010 04:31:57 GMT (envelope-from jmallett@svn.freebsd.org) Message-Id: <201003160431.o2G4Vvl3074797@svn.freebsd.org> From: Juli Mallett Date: Tue, 16 Mar 2010 04:31:57 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r205204 - user/jmallett/octeon/lib/libc/mips/gen X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Mar 2010 04:31:57 -0000 Author: jmallett Date: Tue Mar 16 04:31:57 2010 New Revision: 205204 URL: http://svn.freebsd.org/changeset/base/205204 Log: Get rid of some bogus REG_PROLOGUE / REG_EPILOGUE. Remove duplicate abort. Sponsored by: Packet Forensics Modified: user/jmallett/octeon/lib/libc/mips/gen/setjmp.S Modified: user/jmallett/octeon/lib/libc/mips/gen/setjmp.S ============================================================================== --- user/jmallett/octeon/lib/libc/mips/gen/setjmp.S Tue Mar 16 04:31:13 2010 (r205203) +++ user/jmallett/octeon/lib/libc/mips/gen/setjmp.S Tue Mar 16 04:31:57 2010 (r205204) @@ -83,7 +83,6 @@ NESTED(setjmp, SETJMP_FRAME_SIZE, ra) REG_L ra, CALLFRAME_RA(sp) # restore RA PTR_ADDU sp, sp, SETJMP_FRAME_SIZE # pop stack frame - REG_PROLOGUE REG_LI v0, _JB_MAGIC_SETJMP REG_S v0, (_JB_MAGIC * SZREG)(a0) REG_S ra, (_JB_REG_RA * SZREG)(a0) @@ -100,7 +99,6 @@ NESTED(setjmp, SETJMP_FRAME_SIZE, ra) #if defined(__mips_n32) || defined(__mips_n64) REG_S gp, (_JB_REG_GP * SZREG)(a0) #endif - REG_EPILOGUE move v0, zero jr ra @@ -134,7 +132,6 @@ NESTED(longjmp, LONGJMP_FRAME_SIZE, ra) REG_L a0, CALLFRAME_SIZ(sp) # restore env REG_L a1, (CALLFRAME_SIZ + SZREG)(sp) # restore return value - REG_PROLOGUE REG_L ra, (_JB_REG_RA * SZREG)(a0) REG_L s0, (_JB_REG_S0 * SZREG)(a0) REG_L s1, (_JB_REG_S1 * SZREG)(a0) @@ -150,7 +147,6 @@ NESTED(longjmp, LONGJMP_FRAME_SIZE, ra) REG_L gp, (_JB_REG_GP * SZREG)(a0) #endif - REG_EPILOGUE move v0, a1 j ra nop @@ -165,7 +161,4 @@ botch: nop PIC_TAILCALL(abort) - PTR_LA t9, _C_LABEL(abort) - jal t9 - nop END(longjmp) From owner-svn-src-user@FreeBSD.ORG Tue Mar 16 04:36:11 2010 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E20A7106564A; Tue, 16 Mar 2010 04:36:11 +0000 (UTC) (envelope-from jmallett@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id D196F8FC13; Tue, 16 Mar 2010 04:36:11 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id o2G4aBob075823; Tue, 16 Mar 2010 04:36:11 GMT (envelope-from jmallett@svn.freebsd.org) Received: (from jmallett@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id o2G4aB9I075821; Tue, 16 Mar 2010 04:36:11 GMT (envelope-from jmallett@svn.freebsd.org) Message-Id: <201003160436.o2G4aB9I075821@svn.freebsd.org> From: Juli Mallett Date: Tue, 16 Mar 2010 04:36:11 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r205205 - in user/jmallett/octeon/lib/libc/mips: . net X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Mar 2010 04:36:12 -0000 Author: jmallett Date: Tue Mar 16 04:36:11 2010 New Revision: 205205 URL: http://svn.freebsd.org/changeset/base/205205 Log: Remove unused assembly hton functions. Deleted: user/jmallett/octeon/lib/libc/mips/net/ Modified: user/jmallett/octeon/lib/libc/mips/Symbol.map Modified: user/jmallett/octeon/lib/libc/mips/Symbol.map ============================================================================== --- user/jmallett/octeon/lib/libc/mips/Symbol.map Tue Mar 16 04:31:57 2010 (r205204) +++ user/jmallett/octeon/lib/libc/mips/Symbol.map Tue Mar 16 04:36:11 2010 (r205205) @@ -24,13 +24,9 @@ FBSD_1.0 { sigsetjmp; siglongjmp; htonl; - __htonl; htons; - __htons; ntohl; - __ntohl; ntohs; - __ntohs; vfork; brk; cerror; /* XXX - Should this be .cerror (see sys/cerror.S)? */ From owner-svn-src-user@FreeBSD.ORG Tue Mar 16 22:28:07 2010 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E799F106564A; Tue, 16 Mar 2010 22:28:07 +0000 (UTC) (envelope-from jmallett@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id D6F2D8FC13; Tue, 16 Mar 2010 22:28:07 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id o2GMS7q0014677; Tue, 16 Mar 2010 22:28:07 GMT (envelope-from jmallett@svn.freebsd.org) Received: (from jmallett@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id o2GMS70m014675; Tue, 16 Mar 2010 22:28:07 GMT (envelope-from jmallett@svn.freebsd.org) Message-Id: <201003162228.o2GMS70m014675@svn.freebsd.org> From: Juli Mallett Date: Tue, 16 Mar 2010 22:28:07 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r205232 - user/jmallett/octeon/lib/libc/mips/sys X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Mar 2010 22:28:08 -0000 Author: jmallett Date: Tue Mar 16 22:28:07 2010 New Revision: 205232 URL: http://svn.freebsd.org/changeset/base/205232 Log: Remove special shmat. NetBSD only has per-arch shmat because one of their architectures uses a different ABI for it. FreeBSD doesn't. Deleted: user/jmallett/octeon/lib/libc/mips/sys/shmat.S Modified: user/jmallett/octeon/lib/libc/mips/sys/Makefile.inc Modified: user/jmallett/octeon/lib/libc/mips/sys/Makefile.inc ============================================================================== --- user/jmallett/octeon/lib/libc/mips/sys/Makefile.inc Tue Mar 16 22:17:21 2010 (r205231) +++ user/jmallett/octeon/lib/libc/mips/sys/Makefile.inc Tue Mar 16 22:28:07 2010 (r205232) @@ -1,7 +1,7 @@ # $FreeBSD$ MDASM= Ovfork.S brk.S cerror.S exect.S \ - fork.S pipe.S ptrace.S sbrk.S shmat.S syscall.S + fork.S pipe.S ptrace.S sbrk.S syscall.S # Don't generate default code for these syscalls: NOASM= break.o exit.o ftruncate.o getlogin.o lseek.o mmap.o \ From owner-svn-src-user@FreeBSD.ORG Wed Mar 17 03:02:49 2010 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EFF67106564A; Wed, 17 Mar 2010 03:02:48 +0000 (UTC) (envelope-from lstewart@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id D3C178FC0A; Wed, 17 Mar 2010 03:02:48 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id o2H32mf5075676; Wed, 17 Mar 2010 03:02:48 GMT (envelope-from lstewart@svn.freebsd.org) Received: (from lstewart@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id o2H32mpw075673; Wed, 17 Mar 2010 03:02:48 GMT (envelope-from lstewart@svn.freebsd.org) Message-Id: <201003170302.o2H32mpw075673@svn.freebsd.org> From: Lawrence Stewart Date: Wed, 17 Mar 2010 03:02:48 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r205237 - in user/lstewart/alq_varlen_head/sys: kern sys X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Mar 2010 03:02:49 -0000 Author: lstewart Date: Wed Mar 17 03:02:48 2010 New Revision: 205237 URL: http://svn.freebsd.org/changeset/base/205237 Log: - Rework the way thread ordering is enforced so that it actually behaves as expected (issue discovered during detailed testing). Ordering is now an off-by-default option that can be enabled at ALQ creation time using the ALQ_ORDERED flag. - Add an alq_open_flags() KPI call to allow the new ALQ_ORDERED flag to be specified. alq_open() is now implemented as a wrapper around alq_open_flags. - Rename alq_postn() to alq_post_flags() to keep the naming consistent. - Keep a record of some useful debugging printf's (will be removed in a later diff). - Remove some no longer relevant assertions. - Introduce the AQ_VARLEN flag, used internally to indicate the ALQ is variable length message capable. - Protect alq_getn/alq_post from 0 length writes so that the "use less than you asked for" feature works in contexts where no data may be generated. Sponsored by: FreeBSD Foundation Modified: user/lstewart/alq_varlen_head/sys/kern/kern_alq.c user/lstewart/alq_varlen_head/sys/sys/alq.h Modified: user/lstewart/alq_varlen_head/sys/kern/kern_alq.c ============================================================================== --- user/lstewart/alq_varlen_head/sys/kern/kern_alq.c Wed Mar 17 02:48:14 2010 (r205236) +++ user/lstewart/alq_varlen_head/sys/kern/kern_alq.c Wed Mar 17 03:02:48 2010 (r205237) @@ -54,15 +54,16 @@ __FBSDID("$FreeBSD$"); /* Async. Logging Queue */ struct alq { + char *aq_entbuf; /* Buffer for stored entries */ int aq_entmax; /* Max entries */ int aq_entlen; /* Entry length */ int aq_freebytes; /* Bytes available in buffer */ int aq_buflen; /* Total length of our buffer */ - char *aq_entbuf; /* Buffer for stored entries */ int aq_writehead; /* Location for next write */ int aq_writetail; /* Flush starts at this location */ int aq_wrapearly; /* # bytes left blank at end of buf */ int aq_flags; /* Queue flags */ + int aq_waiters; /* Num threads waiting for resources */ struct ale aq_getpost; /* ALE for use by get/post */ struct mtx aq_mtx; /* Queue lock */ struct vnode *aq_vp; /* Open vnode handle */ @@ -75,6 +76,8 @@ struct alq { #define AQ_ACTIVE 0x0002 /* on the active list */ #define AQ_FLUSHING 0x0004 /* doing IO */ #define AQ_SHUTDOWN 0x0008 /* Queue no longer valid */ +#define AQ_ORDERED 0x0010 /* Queue enforces ordered writes */ +#define AQ_VARLEN 0x0020 /* Queue is variable length capable */ #define ALQ_LOCK(alq) mtx_lock_spin(&(alq)->aq_mtx) #define ALQ_UNLOCK(alq) mtx_unlock_spin(&(alq)->aq_mtx) @@ -200,7 +203,7 @@ ald_daemon(void) needwakeup = alq_doio(alq); ALQ_UNLOCK(alq); if (needwakeup) - wakeup_one(alq); + wakeup(alq); ALD_LOCK(); } @@ -334,6 +337,9 @@ alq_doio(struct alq *alq) totlen = aiov[0].iov_len + aiov[1].iov_len; } + /*printf("Flushing %d bytes to disk, aq_freebytes==%d\n", totlen, + alq->aq_freebytes);*/ + alq->aq_flags |= AQ_FLUSHING; ALQ_UNLOCK(alq); @@ -366,6 +372,9 @@ alq_doio(struct alq *alq) alq->aq_buflen; alq->aq_freebytes += totlen + wrapearly; + /*printf("Flushed %d bytes to disk, aq_freebytes==%d, AQ_WANTED==%d\n", + totlen, alq->aq_freebytes, alq->aq_flags & AQ_WANTED);*/ + /* * If we just flushed part of the buffer which wrapped, reset the * wrapearly indicator. @@ -374,8 +383,8 @@ alq_doio(struct alq *alq) alq->aq_wrapearly = 0; /* - * If we just flushed the buffer completely, - * reset indexes to 0 to minimise buffer wraps. + * If we just flushed the buffer completely, reset indexes to 0 to + * minimise buffer wraps. * This is also required to ensure alq_getn() can't wedge itself. */ if (!HAS_PENDING_DATA(alq)) @@ -407,14 +416,15 @@ SYSINIT(ald, SI_SUB_LOCK, SI_ORDER_ANY, /* * Create the queue data structure, allocate the buffer, and open the file. */ + int -alq_open(struct alq **alqp, const char *file, struct ucred *cred, int cmode, - int size, int count) +alq_open_flags(struct alq **alqp, const char *file, struct ucred *cred, int cmode, + int size, int count, int flags) { struct thread *td; struct nameidata nd; struct alq *alq; - int flags; + int oflags; int error; int vfslocked; @@ -425,9 +435,9 @@ alq_open(struct alq **alqp, const char * td = curthread; NDINIT(&nd, LOOKUP, NOFOLLOW | MPSAFE, UIO_SYSSPACE, file, td); - flags = FWRITE | O_NOFOLLOW | O_CREAT; + oflags = FWRITE | O_NOFOLLOW | O_CREAT; - error = vn_open_cred(&nd, &flags, cmode, 0, cred, NULL); + error = vn_open_cred(&nd, &oflags, cmode, 0, cred, NULL); if (error) return (error); @@ -453,11 +463,14 @@ alq_open(struct alq **alqp, const char * alq->aq_buflen = size; alq->aq_entmax = 0; alq->aq_entlen = 0; + alq->aq_flags |= AQ_VARLEN; } alq->aq_freebytes = alq->aq_buflen; alq->aq_entbuf = malloc(alq->aq_buflen, M_ALD, M_WAITOK|M_ZERO); alq->aq_writehead = alq->aq_writetail = 0; + if (flags & ALQ_ORDERED) + alq->aq_flags |= AQ_ORDERED; if ((error = ald_add(alq)) != 0) { alq_destroy(alq); @@ -503,27 +516,47 @@ alq_writen(struct alq *alq, void *data, } /* + * If we want ordered writes and there are threads already waiting for + * resources to become available, spin until we're woken. + */ + if (alq->aq_flags & AQ_ORDERED && alq->aq_waiters > 0) { + /*printf("tid %d order sleep, wants %d bytes (%d avail)\n", + curthread->td_tid, len, alq->aq_freebytes);*/ + alq->aq_waiters++; + msleep_spin(&alq->aq_waiters, &alq->aq_mtx, "alqwriten", 0); + alq->aq_waiters--; + /*printf("tid %d order woken, wants %d bytes (%d avail)\n", + curthread->td_tid, len, alq->aq_freebytes);*/ + } + + /* * ALQ_WAITOK or alq->aq_freebytes > len, either spin until - * we have enough free bytes (former) or skip (latter). However in the - * latter case, we can't skip if other threads are already - * waiting (AQ_WANTED is set), otherwise records can get out of order. + * we have enough free bytes (former) or skip (latter). */ - while ((alq->aq_freebytes < len && !(alq->aq_flags & AQ_SHUTDOWN)) - || alq->aq_flags & AQ_WANTED) { + while (alq->aq_freebytes < len && !(alq->aq_flags & AQ_SHUTDOWN)) { + /*printf("tid %d sleep, wants %d bytes (%d avail)\n", + curthread->td_tid, len, alq->aq_freebytes);*/ alq->aq_flags |= AQ_WANTED; + alq->aq_waiters++; msleep_spin(alq, &alq->aq_mtx, "alqwriten", 0); - KASSERT(!(alq->aq_flags & AQ_WANTED), - ("AQ_WANTED should have been unset!")); + alq->aq_waiters--; + /*printf("tid %d woken, wants %d bytes (%d avail)\n", + curthread->td_tid, len, alq->aq_freebytes);*/ } + /*printf("tid %d got %d bytes (%d avail, %d waiters)\n", + curthread->td_tid, len, alq->aq_freebytes, alq->aq_waiters);*/ + /* - * We need to serialise wakeups to ensure records remain in order. - * Therefore, wakeup the next thread in the queue waiting for - * ALQ resources to be available. - * (Technically this is only required if we actually entered the above - * while loop.) + * If there are waiters, wakeup the next thread in the queue waiting for + * ALQ resources. */ - wakeup_one(alq); + if (alq->aq_waiters > 0) { + if (alq->aq_flags & AQ_ORDERED) + wakeup_one(&alq->aq_waiters); + else + wakeup(alq); + } /* Bail if we're shutting down. */ if (alq->aq_flags & AQ_SHUTDOWN) { @@ -569,6 +602,8 @@ alq_writen(struct alq *alq, void *data, activate = 1; } + KASSERT((HAS_PENDING_DATA(alq)), ("%s: queue emtpy!", __func__)); + ALQ_UNLOCK(alq); if (activate) { @@ -584,7 +619,7 @@ int alq_write(struct alq *alq, void *data, int flags) { /* Should only be called in fixed length message (legacy) mode. */ - KASSERT((alq->aq_entmax > 0 && alq->aq_entlen > 0), + KASSERT((!(alq->aq_flags & AQ_VARLEN)), ("%s: fixed length write on variable length queue", __func__)); return (alq_writen(alq, data, alq->aq_entlen, flags)); } @@ -651,19 +686,31 @@ alq_getn(struct alq *alq, int len, int f } /* + * If we want ordered writes and there are threads already waiting for + * resources to become available, spin until we're woken. + */ + if (alq->aq_flags & AQ_ORDERED && alq->aq_waiters > 0) { + /*printf("tid %d order sleep, wants %d bytes (%d avail)\n", + curthread->td_tid, len, alq->aq_freebytes);*/ + alq->aq_waiters++; + msleep_spin(&alq->aq_waiters, &alq->aq_mtx, "alqgetn", 0); + alq->aq_waiters--; + /*printf("tid %d order woken, wants %d bytes (%d avail)\n", + curthread->td_tid, len, alq->aq_freebytes);*/ + } + + /* * ALQ_WAITOK or contigbytes >= len, * either spin until we have enough free contiguous bytes (former) * or skip (latter). However, in the latter case, we can't skip if * other threads are already waiting (AQ_WANTED is set), otherwise * records can get out of order. */ - while ((contigbytes < len && !(alq->aq_flags & AQ_SHUTDOWN)) - || alq->aq_flags & AQ_WANTED) { + while (contigbytes < len && !(alq->aq_flags & AQ_SHUTDOWN)) { alq->aq_flags |= AQ_WANTED; + alq->aq_waiters++; msleep_spin(alq, &alq->aq_mtx, "alqgetn", 0); - - KASSERT(!(alq->aq_flags & AQ_WANTED), - ("AQ_WANTED should have been unset!")); + alq->aq_waiters--; if (alq->aq_writehead <= alq->aq_writetail) contigbytes = alq->aq_freebytes; @@ -672,13 +719,15 @@ alq_getn(struct alq *alq, int len, int f } /* - * We need to serialise wakeups to ensure records remain in order. - * Therefore, wakeup the next thread in the queue waiting for - * ALQ resources to be available. - * (Technically this is only required if we actually entered the above - * while loop.) + * If there are waiters, wakeup the next thread in the queue waiting for + * ALQ resources. */ - wakeup_one(alq); + if (alq->aq_waiters > 0) { + if (alq->aq_flags & AQ_ORDERED) + wakeup_one(&alq->aq_waiters); + else + wakeup(alq); + } /* Bail if we're shutting down. */ if (alq->aq_flags & AQ_SHUTDOWN) { @@ -700,32 +749,39 @@ struct ale * alq_get(struct alq *alq, int flags) { /* Should only be called in fixed length message (legacy) mode. */ - KASSERT((alq->aq_entmax > 0 && alq->aq_entlen > 0), + KASSERT((!(alq->aq_flags & AQ_VARLEN)), ("%s: fixed length get on variable length queue", __func__)); return (alq_getn(alq, alq->aq_entlen, flags)); } void -alq_postn(struct alq *alq, struct ale *ale, int flags) +alq_post_flags(struct alq *alq, struct ale *ale, int flags) { int activate; activate = 0; - if (!(alq->aq_flags & AQ_ACTIVE) && !(flags & ALQ_NOACTIVATE)) { - alq->aq_flags |= AQ_ACTIVE; - activate = 1; - } + if (ale->ae_bytesused > 0) { + if (!(alq->aq_flags & AQ_ACTIVE) && + !(flags & ALQ_NOACTIVATE)) { + alq->aq_flags |= AQ_ACTIVE; + activate = 1; + } - alq->aq_writehead += ale->ae_bytesused; - alq->aq_freebytes -= ale->ae_bytesused; + alq->aq_writehead += ale->ae_bytesused; + alq->aq_freebytes -= ale->ae_bytesused; - /* Wrap aq_writehead if we've filled to the end of the buffer. */ - if (alq->aq_writehead == alq->aq_buflen) - alq->aq_writehead = 0; + /* Wrap aq_writehead if we filled to the end of the buffer. */ + if (alq->aq_writehead == alq->aq_buflen) + alq->aq_writehead = 0; + + KASSERT((alq->aq_writehead >= 0 && + alq->aq_writehead < alq->aq_buflen), + ("%s: aq_writehead < 0 || aq_writehead >= aq_buflen", + __func__)); - KASSERT((alq->aq_writehead >= 0 && alq->aq_writehead < alq->aq_buflen), - ("%s: aq_writehead < 0 || aq_writehead >= aq_buflen", __func__)); + KASSERT((HAS_PENDING_DATA(alq)), ("%s: queue emtpy!", __func__)); + } ALQ_UNLOCK(alq); @@ -761,7 +817,7 @@ alq_flush(struct alq *alq) ALQ_UNLOCK(alq); if (needwakeup) - wakeup_one(alq); + wakeup(alq); } /* Modified: user/lstewart/alq_varlen_head/sys/sys/alq.h ============================================================================== --- user/lstewart/alq_varlen_head/sys/sys/alq.h Wed Mar 17 02:48:14 2010 (r205236) +++ user/lstewart/alq_varlen_head/sys/sys/alq.h Wed Mar 17 03:02:48 2010 (r205237) @@ -56,6 +56,7 @@ struct ale { #define ALQ_NOWAIT 0x0001 #define ALQ_WAITOK 0x0002 #define ALQ_NOACTIVATE 0x0004 +#define ALQ_ORDERED 0x0010 /* Suggested mode for file creation. */ #define ALQ_DEFAULT_CMODE 0600 @@ -77,8 +78,15 @@ struct ale { * error from open or 0 on success */ struct ucred; -int alq_open(struct alq **, const char *file, struct ucred *cred, int cmode, - int size, int count); +int alq_open_flags(struct alq **alqp, const char *file, struct ucred *cred, int cmode, + int size, int count, int flags); + +static __inline int +alq_open(struct alq **alqp, const char *file, struct ucred *cred, int cmode, + int size, int count) +{ + return alq_open_flags(alqp, file, cred, cmode, size, count, 0); +} /* * alq_writen: Write data into the queue @@ -133,12 +141,12 @@ struct ale *alq_get(struct alq *alq, int * ale An asynch logging entry returned by alq_get. * flags ALQ_NOACTIVATE */ -void alq_postn(struct alq *alq, struct ale *ale, int flags); +void alq_post_flags(struct alq *alq, struct ale *ale, int flags); static __inline void alq_post(struct alq *alq, struct ale *ale) { - alq_postn(alq, ale, 0); + alq_post_flags(alq, ale, 0); } #endif /* _SYS_ALQ_H_ */ From owner-svn-src-user@FreeBSD.ORG Wed Mar 17 05:59:21 2010 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 78797106566B; Wed, 17 Mar 2010 05:59:21 +0000 (UTC) (envelope-from lstewart@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 67D4A8FC17; Wed, 17 Mar 2010 05:59:21 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id o2H5xLTl014540; Wed, 17 Mar 2010 05:59:21 GMT (envelope-from lstewart@svn.freebsd.org) Received: (from lstewart@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id o2H5xLCi014538; Wed, 17 Mar 2010 05:59:21 GMT (envelope-from lstewart@svn.freebsd.org) Message-Id: <201003170559.o2H5xLCi014538@svn.freebsd.org> From: Lawrence Stewart Date: Wed, 17 Mar 2010 05:59:21 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r205240 - user/lstewart/alq_varlen_head/share/man/man9 X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Mar 2010 05:59:21 -0000 Author: lstewart Date: Wed Mar 17 05:59:21 2010 New Revision: 205240 URL: http://svn.freebsd.org/changeset/base/205240 Log: Update man page in light of changes in previous commit. Sponsored by: FreeBSD Foundation Modified: user/lstewart/alq_varlen_head/share/man/man9/alq.9 Modified: user/lstewart/alq_varlen_head/share/man/man9/alq.9 ============================================================================== --- user/lstewart/alq_varlen_head/share/man/man9/alq.9 Wed Mar 17 05:54:46 2010 (r205239) +++ user/lstewart/alq_varlen_head/share/man/man9/alq.9 Wed Mar 17 05:59:21 2010 (r205240) @@ -37,6 +37,7 @@ .Sh NAME .Nm alq , .Nm alq_open , +.Nm alq_open_flags , .Nm alq_write , .Nm alq_writen , .Nm alq_flush , @@ -44,7 +45,7 @@ .Nm alq_get , .Nm alq_getn , .Nm alq_post , -.Nm alq_postn +.Nm alq_post_flags .Nd Asynchronous Logging Queues .Sh SYNOPSIS .In sys/alq.h @@ -58,6 +59,16 @@ .Fa "int count" .Fc .Ft int +.Fo alq_open_flags +.Fa "struct alq **app" +.Fa "const char *file" +.Fa "struct ucred *cred" +.Fa "int cmode" +.Fa "int size" +.Fa "int count" +.Fa "int flags" +.Fc +.Ft int .Fn alq_write "struct alq *alq" "void *data" "int flags" .Ft int .Fn alq_writen "struct alq *alq" "void *data" "int len" "int flags" @@ -72,7 +83,7 @@ .Ft void .Fn alq_post "struct alq *alq" "struct ale *ale" .Ft void -.Fn alq_postn "struct alq *alq" "struct ale *ale" "int flags" +.Fn alq_post_flags "struct alq *alq" "struct ale *ale" "int flags" .Sh DESCRIPTION The .Nm @@ -121,7 +132,7 @@ each of fixed size (set at queue creatio Fixed length mode is deprecated in favour of variable length mode. .Sh FUNCTIONS The -.Fn alq_open +.Fn alq_open_flags function creates a new asynchronous logging queue. The .Fa file @@ -159,6 +170,20 @@ argument should be set to the number of byte chunks to reserve capacity for. .Pp The +.Fn alq_open +function is implemented as a wrapper around +.Fn alq_open_flags +to provide backwards compatibility to consumers that have not been updated to +utilise the newer +.Fn alq_open_flags +function. +It simply passes all arguments through to +.Fn alq_open_flags +untouched, and sets +.Fa flags +to 0. +.Pp +The .Fn alq_writen function writes .Fa len @@ -235,7 +260,7 @@ This function leaves in a locked state, until a subsequent .Fn alq_post or -.Fn alq_postn +.Fn alq_post_flags call is made. If .Fn alq_getn @@ -274,7 +299,7 @@ Note that it is an error to call on a variable length mode queue. .Pp The -.Fn alq_postn +.Fn alq_post_flags function schedules the asynchronous log entry .Fa ale (obtained from @@ -294,17 +319,13 @@ in an unlocked state. The .Fn alq_post function is implemented as a wrapper around -.Fn alq_postn +.Fn alq_post_flags to provide backwards compatibility to consumers that have not been updated to utilise the newer -.Fn alq_postn +.Fn alq_post_flags function. -It simply passes -.Fa alq -and -.Fa ale -through to -.Fn alq_postn +It simply passes all arguments through to +.Fn alq_post_flags untouched, and sets .Fa flags to 0. @@ -332,7 +353,7 @@ or and .Fn alq_post or -.Fn alq_postn , +.Fn alq_post_flags , so this method of writing to a queue is unsuitable for situations where the time between calls may be substantial. .Sh LOCKING From owner-svn-src-user@FreeBSD.ORG Wed Mar 17 06:20:40 2010 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9FE41106566B; Wed, 17 Mar 2010 06:20:40 +0000 (UTC) (envelope-from jmallett@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 8F8938FC08; Wed, 17 Mar 2010 06:20:40 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id o2H6Kevn019245; Wed, 17 Mar 2010 06:20:40 GMT (envelope-from jmallett@svn.freebsd.org) Received: (from jmallett@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id o2H6KeIK019243; Wed, 17 Mar 2010 06:20:40 GMT (envelope-from jmallett@svn.freebsd.org) Message-Id: <201003170620.o2H6KeIK019243@svn.freebsd.org> From: Juli Mallett Date: Wed, 17 Mar 2010 06:20:40 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r205241 - user/jmallett/octeon/lib/libc/mips/gen X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Mar 2010 06:20:40 -0000 Author: jmallett Date: Wed Mar 17 06:20:40 2010 New Revision: 205241 URL: http://svn.freebsd.org/changeset/base/205241 Log: Remove another redundant abort call. Modified: user/jmallett/octeon/lib/libc/mips/gen/_setjmp.S Modified: user/jmallett/octeon/lib/libc/mips/gen/_setjmp.S ============================================================================== --- user/jmallett/octeon/lib/libc/mips/gen/_setjmp.S Wed Mar 17 05:59:21 2010 (r205240) +++ user/jmallett/octeon/lib/libc/mips/gen/_setjmp.S Wed Mar 17 06:20:40 2010 (r205241) @@ -123,7 +123,4 @@ botch: nop PIC_TAILCALL(abort) - PTR_LA t9, _C_LABEL(abort) - jal t9 - nop END(_longjmp) From owner-svn-src-user@FreeBSD.ORG Wed Mar 17 06:41:10 2010 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 78A3D106564A; Wed, 17 Mar 2010 06:41:10 +0000 (UTC) (envelope-from lstewart@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 6801A8FC16; Wed, 17 Mar 2010 06:41:10 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id o2H6fAqV023728; Wed, 17 Mar 2010 06:41:10 GMT (envelope-from lstewart@svn.freebsd.org) Received: (from lstewart@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id o2H6fAjD023726; Wed, 17 Mar 2010 06:41:10 GMT (envelope-from lstewart@svn.freebsd.org) Message-Id: <201003170641.o2H6fAjD023726@svn.freebsd.org> From: Lawrence Stewart Date: Wed, 17 Mar 2010 06:41:10 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r205242 - user/lstewart/alq_varlen_head/sys/kern X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Mar 2010 06:41:10 -0000 Author: lstewart Date: Wed Mar 17 06:41:10 2010 New Revision: 205242 URL: http://svn.freebsd.org/changeset/base/205242 Log: Remove debugging printfs. Sponsored by: FreeBSD Foundation Modified: user/lstewart/alq_varlen_head/sys/kern/kern_alq.c Modified: user/lstewart/alq_varlen_head/sys/kern/kern_alq.c ============================================================================== --- user/lstewart/alq_varlen_head/sys/kern/kern_alq.c Wed Mar 17 06:20:40 2010 (r205241) +++ user/lstewart/alq_varlen_head/sys/kern/kern_alq.c Wed Mar 17 06:41:10 2010 (r205242) @@ -337,9 +337,6 @@ alq_doio(struct alq *alq) totlen = aiov[0].iov_len + aiov[1].iov_len; } - /*printf("Flushing %d bytes to disk, aq_freebytes==%d\n", totlen, - alq->aq_freebytes);*/ - alq->aq_flags |= AQ_FLUSHING; ALQ_UNLOCK(alq); @@ -372,9 +369,6 @@ alq_doio(struct alq *alq) alq->aq_buflen; alq->aq_freebytes += totlen + wrapearly; - /*printf("Flushed %d bytes to disk, aq_freebytes==%d, AQ_WANTED==%d\n", - totlen, alq->aq_freebytes, alq->aq_flags & AQ_WANTED);*/ - /* * If we just flushed part of the buffer which wrapped, reset the * wrapearly indicator. @@ -520,13 +514,9 @@ alq_writen(struct alq *alq, void *data, * resources to become available, spin until we're woken. */ if (alq->aq_flags & AQ_ORDERED && alq->aq_waiters > 0) { - /*printf("tid %d order sleep, wants %d bytes (%d avail)\n", - curthread->td_tid, len, alq->aq_freebytes);*/ alq->aq_waiters++; msleep_spin(&alq->aq_waiters, &alq->aq_mtx, "alqwriten", 0); alq->aq_waiters--; - /*printf("tid %d order woken, wants %d bytes (%d avail)\n", - curthread->td_tid, len, alq->aq_freebytes);*/ } /* @@ -534,19 +524,12 @@ alq_writen(struct alq *alq, void *data, * we have enough free bytes (former) or skip (latter). */ while (alq->aq_freebytes < len && !(alq->aq_flags & AQ_SHUTDOWN)) { - /*printf("tid %d sleep, wants %d bytes (%d avail)\n", - curthread->td_tid, len, alq->aq_freebytes);*/ alq->aq_flags |= AQ_WANTED; alq->aq_waiters++; msleep_spin(alq, &alq->aq_mtx, "alqwriten", 0); alq->aq_waiters--; - /*printf("tid %d woken, wants %d bytes (%d avail)\n", - curthread->td_tid, len, alq->aq_freebytes);*/ } - /*printf("tid %d got %d bytes (%d avail, %d waiters)\n", - curthread->td_tid, len, alq->aq_freebytes, alq->aq_waiters);*/ - /* * If there are waiters, wakeup the next thread in the queue waiting for * ALQ resources. @@ -690,13 +673,9 @@ alq_getn(struct alq *alq, int len, int f * resources to become available, spin until we're woken. */ if (alq->aq_flags & AQ_ORDERED && alq->aq_waiters > 0) { - /*printf("tid %d order sleep, wants %d bytes (%d avail)\n", - curthread->td_tid, len, alq->aq_freebytes);*/ alq->aq_waiters++; msleep_spin(&alq->aq_waiters, &alq->aq_mtx, "alqgetn", 0); alq->aq_waiters--; - /*printf("tid %d order woken, wants %d bytes (%d avail)\n", - curthread->td_tid, len, alq->aq_freebytes);*/ } /* From owner-svn-src-user@FreeBSD.ORG Wed Mar 17 07:08:57 2010 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D67A71065672; Wed, 17 Mar 2010 07:08:57 +0000 (UTC) (envelope-from lstewart@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id C45A48FC16; Wed, 17 Mar 2010 07:08:57 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id o2H78v7k029865; Wed, 17 Mar 2010 07:08:57 GMT (envelope-from lstewart@svn.freebsd.org) Received: (from lstewart@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id o2H78vC6029861; Wed, 17 Mar 2010 07:08:57 GMT (envelope-from lstewart@svn.freebsd.org) Message-Id: <201003170708.o2H78vC6029861@svn.freebsd.org> From: Lawrence Stewart Date: Wed, 17 Mar 2010 07:08:57 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r205243 - in user/lstewart/alq_varlen_head/tools/test: . alq X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Mar 2010 07:08:57 -0000 Author: lstewart Date: Wed Mar 17 07:08:57 2010 New Revision: 205243 URL: http://svn.freebsd.org/changeset/base/205243 Log: Commit ALQ testing code. Sponsored by: FreeBSD Foundation Added: user/lstewart/alq_varlen_head/tools/test/alq/ user/lstewart/alq_varlen_head/tools/test/alq/Makefile (contents, props changed) user/lstewart/alq_varlen_head/tools/test/alq/alq_varlen_test.c (contents, props changed) Modified: user/lstewart/alq_varlen_head/tools/test/README Modified: user/lstewart/alq_varlen_head/tools/test/README ============================================================================== --- user/lstewart/alq_varlen_head/tools/test/README Wed Mar 17 06:41:10 2010 (r205242) +++ user/lstewart/alq_varlen_head/tools/test/README Wed Mar 17 07:08:57 2010 (r205243) @@ -7,6 +7,7 @@ and try to break it and/or measuring per Please make a subdir per program, and add a brief description to this file. +alq A kernel module to put ALQ(9) through its paces. devrandom Programs to test /dev/*random. dtrace DTrace test suite malloc A program to test and benchmark malloc(). Added: user/lstewart/alq_varlen_head/tools/test/alq/Makefile ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ user/lstewart/alq_varlen_head/tools/test/alq/Makefile Wed Mar 17 07:08:57 2010 (r205243) @@ -0,0 +1,17 @@ +# $FreeBSD$ + +.include + +KMOD=alqtest +SRCS=alq_varlen_test.c alq_varlen_test.h +CLEANFILES=alq_varlen_test.h + +alq_varlen_test.h: + @awk -F "\n" '{ if(index($$0, "struct alq {") > 0) p=1; if(p == 1) { print $$0; if($$0 == "};") exit; } }' ${.CURDIR}/../../../sys/kern/kern_alq.c >> ${.TARGET} + grep "#define AQ_" ${.CURDIR}/../../../sys/kern/kern_alq.c >> ${.TARGET} + +#KMOD=alqmodtest +#SRCS=alq_mod_test.c + +.include + Added: user/lstewart/alq_varlen_head/tools/test/alq/alq_varlen_test.c ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ user/lstewart/alq_varlen_head/tools/test/alq/alq_varlen_test.c Wed Mar 17 07:08:57 2010 (r205243) @@ -0,0 +1,536 @@ +/*- + * Copyright (c) 2008-2009 Lawrence Stewart + * Copyright (c) 2009-2010, The FreeBSD Foundation + * All rights reserved. + * + * Portions of this software were developed at the Centre for Advanced + * Internet Architectures, Swinburne University of Technology, Melbourne, + * Australia by Lawrence Stewart under sponsorship from the FreeBSD Foundation. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHORS AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +#include "alq_varlen_test.h" + +MALLOC_DECLARE(M_ALQTEST); +MALLOC_DEFINE(M_ALQTEST, "alqtest", "dynamic memory used by alqtest"); + +#define ENABLE 0x01 +#define DISABLE 0x02 + +#define NUM_TEST_RUNS 1 + +static volatile uint8_t run_test_thread; + +static char logfile[PATH_MAX] = "/tmp/alqtest.log"; + +static struct thread *alq_test_thr = NULL; + +static struct mtx alqtestmtx; + +typedef int (*testfunc)(struct sbuf *, struct sbuf *); + +struct writen_thr_data { + char name[20]; + struct mtx dlock; + struct alq *q; + u_int id; + struct thread *thr; + volatile u_int finished; + char *buf; + u_int buflen; + u_int nwrites; +}; + +typedef const enum { + BLACK = 30, + RED, + GREEN, + YELLOW, + BLUE, + MAGENTA, + CYAN, + WHITE +} fgcolor_t; + +static int +alqtest_printf(struct sbuf *s, fgcolor_t c, const char *fmt, ...) +{ + int ret; + va_list ap1, ap2; + + va_start(ap1, fmt); + va_copy(ap2, ap1); + + printf("\033[%dm", c); + vprintf(fmt, ap1); + printf("\033[0m"); + + sbuf_printf(s, "\033[%dm", c); + ret = sbuf_vprintf(s, fmt, ap2); + sbuf_printf(s, "\033[0m"); + + va_end(ap2); + va_end(ap1); + + return (ret); +} + +static char +alqtest_randchar(void) +{ + uint32_t c; + + /* generate a random character in the ascii range [32, 126] */ + while ((c = arc4random() % 126) < 32); + + return ((char)c); +} + +static uint32_t +alqtest_rand(uint32_t lower, uint32_t upper) +{ + uint32_t n; + + while ((n = arc4random() % (upper+1)) < lower); + + return (n); +} + +/*static void +alqtest_doio_callback(void) +{ + printf("doing io baby!\n"); +}*/ + +static void +alqtest_writen_thread(void *arg) +{ + uint32_t n, i; + + struct writen_thr_data *data = (struct writen_thr_data *)arg; + printf("tid %d id %d started\n", data->thr->td_tid, data->id); + + for (i = 0; i < data->nwrites; i++) { + n = alqtest_rand(1, data->buflen); + //sbuf_printf(s, "--- msg==%d,msglen==%d\n", i, n); + //printf("--- thread==%d,msg==%d,msglen==%d\n", data->id, i, n); + alq_writen(data->q, data->buf, n, ALQ_WAITOK); + } + + mtx_lock(&data->dlock); + data->finished = 1; + mtx_unlock(&data->dlock); + kthread_exit(); +} + +static int +alqtest_writen(struct sbuf *s, struct sbuf *debug) +{ +#define NMSGS 10000 +#define MAX_THREADS 50 + struct alq *testalq; + const int buflen = 100; + int i, n, ret, errors, nthreads, threadsactive, nsecs; + char buf[buflen+1]; + struct writen_thr_data *thrdata[MAX_THREADS]; + ret = errors = 0; + nthreads = 1; + + for (i = 0; i < MAX_THREADS; i++) { + thrdata[i] = malloc(sizeof(struct writen_thr_data), M_ALQTEST, + M_WAITOK|M_ZERO); + sprintf(thrdata[i]->name, "writen_thr_data_%d", i); + mtx_init(&thrdata[i]->dlock, thrdata[i]->name, NULL, MTX_DEF); + } + + alqtest_printf(s, 0, "- variable length message writing\n"); + + /* test variable length message writing */ + ret = alq_open_flags(&testalq, logfile, curthread->td_ucred, 0600, buflen, + 0, ALQ_ORDERED); + + /*testalq->doio_debugcallback = &alqtest_doio_callback;*/ + + for (i = 0; i < sizeof(buf); i++) + buf[i] = alqtest_randchar(); + + alqtest_printf(s, 0, "-- nthreads==1,nmsgs==1,msglen==1,buflen=%d," + "flags==ALQ_WAITOK|ALQ_NOACTIVATE\n", buflen); + alq_writen(testalq, buf, 1, ALQ_WAITOK|ALQ_NOACTIVATE); + + if ((buflen-1 != testalq->aq_freebytes) && + (1 != testalq->aq_writehead) && + (0 != testalq->aq_writetail)) { + errors++; + sbuf_printf(debug, + "alq->%-15s\texpected=%d\tactual=%d\n", "aq_freebytes", + buflen-1, testalq->aq_freebytes); + sbuf_printf(debug, "alq->%-15s\texpected=%d\tactual=%d\n", + "aq_writehead", 1, testalq->aq_writehead); + sbuf_printf(debug, "alq->%-15s\texpected=%d\tactual=%d\n", + "aq_writetail", 0, testalq->aq_writetail); + } + + alq_flush(testalq); + + if ((buflen != testalq->aq_freebytes) && + (0 != testalq->aq_writehead) && + (0 != testalq->aq_writetail)) { + errors++; + sbuf_printf(debug, "alq->%-15s\texpected=%d\tactual=%d\n", + "aq_freebytes", buflen, testalq->aq_freebytes); + sbuf_printf(debug, "alq->%-15s\texpected=%d\tactual=%d\n", + "aq_writehead", 0, testalq->aq_writehead); + sbuf_printf(debug, "alq->%-15s\texpected=%d\tactual=%d\n", + "aq_writetail", 0, testalq->aq_writetail); + } + + alqtest_printf(s, 0, "-- nthreads==1,nmsgs==1,msglen==%d,buflen=%d,flags==ALQ_WAITOK|ALQ_NOACTIVATE\n", buflen, buflen); + alq_writen(testalq, buf, buflen, ALQ_WAITOK | ALQ_NOACTIVATE); + + if ((0 != testalq->aq_freebytes) && + (0 != testalq->aq_writehead) && + (0 != testalq->aq_writetail)) { + errors++; + sbuf_printf(debug, "alq->%-15s\texpected=%d\tactual=%d\n", + "aq_freebytes", 0, testalq->aq_freebytes); + sbuf_printf(debug, "alq->%-15s\texpected=%d\tactual=%d\n", + "aq_writehead", 0, testalq->aq_writehead); + sbuf_printf(debug, "alq->%-15s\texpected=%d\tactual=%d\n", + "aq_writetail", 0, testalq->aq_writetail); + } + + alq_flush(testalq); + + if ((buflen != testalq->aq_freebytes) && + (0 != testalq->aq_writehead) && + (0 != testalq->aq_writetail)) { + errors++; + sbuf_printf(debug, "alq->%-15s\texpected=%d\tactual=%d\n", + "aq_freebytes", buflen, testalq->aq_freebytes); + sbuf_printf(debug, "alq->%-15s\texpected=%d\tactual=%d\n", + "aq_writehead", 0, testalq->aq_writehead); + sbuf_printf(debug, "alq->%-15s\texpected=%d\tactual=%d\n", + "aq_writetail", 0, testalq->aq_writetail); + } + + alqtest_printf(s, 0, "-- nthreads==1,nmsgs==%d,buflen=%d,msglen==[1,%d]," + "flags==ALQ_WAITOK|ALQ_NOACTIVATE\n", NMSGS, buflen, buflen); + + for (i = 0; i < NMSGS; i++) { + n = alqtest_rand(1, buflen); + sbuf_printf(debug, "--- msg==%d,msglen==%d\n", i, n); + alq_writen(testalq, buf, n, ALQ_WAITOK|ALQ_NOACTIVATE); + + alq_flush(testalq); + + if ((buflen != testalq->aq_freebytes) && + (0 != testalq->aq_writehead) && + (0 != testalq->aq_writetail)) { + errors++; + sbuf_printf(debug, + "alq->%-15s\texpected=%d\tactual=%d\n", + "aq_freebytes", buflen, testalq->aq_freebytes); + sbuf_printf(debug, + "alq->%-15s\texpected=%d\tactual=%d\n", + "aq_writehead", 0, testalq->aq_writehead); + sbuf_printf(debug, + "alq->%-15s\texpected=%d\tactual=%d\n", + "aq_writetail", 0, testalq->aq_writetail); + } + } + + alqtest_printf(s, 0, + "-- nthreads==1,nmsgs==%d,buflen=%d,msglen==[1,%d],flags==ALQ_WAITOK\n", + NMSGS, buflen, buflen); + + for (i = 0; i < NMSGS; i++) { + n = alqtest_rand(1, buflen); + sbuf_printf(s, "--- msg==%d,msglen==%d\n", i, n); + alq_writen(testalq, buf, n, ALQ_WAITOK); + } + + alq_flush(testalq); + + nthreads = 5; + threadsactive = 0; + alqtest_printf(s, 0, + "-- nthreads=%d,nmsgs==%d,buflen=%d,msglen==[1,%d],flags==ALQ_WAITOK\n", + nthreads, NMSGS, buflen, buflen); + for (i = 0; i < nthreads; i++) { + thrdata[i]->q = testalq; + thrdata[i]->id = i; + thrdata[i]->finished = 0; + thrdata[i]->buf = buf; + thrdata[i]->buflen = buflen; + thrdata[i]->nwrites = NMSGS; + kthread_add(&alqtest_writen_thread, thrdata[i], NULL, + &thrdata[i]->thr, RFNOWAIT, 0, thrdata[i]->name); + threadsactive++; + } + + nsecs = 0; + while (threadsactive) { + //printf("threadsactive: %d\n", threadsactive); + for (i = 0; i < nthreads; i++) { + mtx_lock(&thrdata[i]->dlock); + if (thrdata[i]->thr != NULL && + thrdata[i]->finished) { + printf("tid %d id %d finished\n", + thrdata[i]->thr->td_tid, thrdata[i]->id); + threadsactive--; + thrdata[i]->thr = NULL; + } + mtx_unlock(&thrdata[i]->dlock); + } + pause("alqtest", hz); + nsecs++; + + if (nsecs > 20) { + printf("It is likely that a bug has stalled one or more " + "of the writing threads. Calling alq_flush() to see if " + "we can loosen things up\n"); + alq_flush(testalq); + + if (nsecs > 22) { + printf("Also trying wakeup_one(&aq_waiters) and" + "wakeup_one(alq)\n"); + wakeup_one(&testalq->aq_waiters); + wakeup_one(testalq); + nsecs = 0; + } + } + } + + /* Cleanup. */ + alq_close(testalq); + for (i = 0; i < MAX_THREADS; i++) { + mtx_destroy(&thrdata[i]->dlock); + free(thrdata[i], M_ALQTEST); + } + + return (errors); +} + +static int +alqtest_open(struct sbuf *s, struct sbuf *debug) +{ + struct alq *testalq; + const int buflen = 100; + int ret = 0, errors = 0; + + alqtest_printf(s, 0, "- variable length message queue creation\n"); + + /* test variable length message queue creation */ + ret = alq_open(&testalq, logfile, curthread->td_ucred, 0600, buflen, + 0); + + if (0 != testalq->aq_entmax) { + errors++; + sbuf_printf(debug, "alq->%-15s\texpected=%d\tactual=%d\n", + "aq_entmax", 0, testalq->aq_entmax); + } + + if (0 != testalq->aq_entlen) { + errors++; + sbuf_printf(debug, "alq->%-15s\texpected=%d\tactual=%d\n", + "aq_entlen", 0, testalq->aq_entlen); + } + + if (buflen != testalq->aq_freebytes) { + errors++; + sbuf_printf(debug, "alq->%-15s\texpected=%d\tactual=%d\n", + "aq_freebytes", buflen, testalq->aq_freebytes); + } + + if (buflen != testalq->aq_buflen) { + errors++; + sbuf_printf(debug, "alq->%-15s\texpected=%d\tactual=%d\n", + "aq_buflen", buflen, testalq->aq_buflen); + } + + if (0 != testalq->aq_writehead) { + errors++; + sbuf_printf(debug, "alq->%-15s\texpected=%d\tactual=%d\n", + "aq_writehead", 0, testalq->aq_writehead); + } + + if (0 != testalq->aq_writetail) { + errors++; + sbuf_printf(debug, "alq->%-15s\texpected=%d\tactual=%d\n", + "aq_writetail", 0, testalq->aq_writetail); + } + + if (AQ_VARLEN != testalq->aq_flags) { + errors++; + sbuf_printf(debug, "alq->%-15s\texpected=%d\tactual=%d\n", + "aq_flags", 0, testalq->aq_flags); + } + + alq_close(testalq); + + return (errors); +} + +static void +run_test(struct sbuf *s, const char *test_banner, testfunc test) +{ + struct sbuf *debug = NULL; + + if ((debug = sbuf_new(NULL, NULL, 1024, SBUF_AUTOEXTEND)) != NULL) { + alqtest_printf(s, 0, "########################################\n"); + alqtest_printf(s, GREEN, "%s\n", test_banner); + if (test(s, debug)) { + sbuf_finish(debug); + alqtest_printf(s, RED, "!!ERROR(S) FOUND!!\n"); + alqtest_printf(s, 0, "%s", sbuf_data(debug)); + alqtest_printf(s, RED, "!!ERROR(S) FOUND!!\n"); + } + alqtest_printf(s, 0, "########################################\n\n"); + sbuf_delete(debug); + } +} + +static void +alqtest_thread(void *arg) +{ + struct sbuf *s = NULL; + long runs = 0; + + /* loop until thread is signalled to exit */ + while (run_test_thread && runs < NUM_TEST_RUNS) { + if ((s = sbuf_new(NULL, NULL, 1024, SBUF_AUTOEXTEND)) != NULL) { + alqtest_printf(s, 0, "TEST RUN: %ld\n", ++runs); + + run_test(s, "alq_open", &alqtest_open); + run_test(s, "alq_writen", &alqtest_writen); + + sbuf_finish(s); + /*printf("%s", sbuf_data(s));*/ + sbuf_delete(s); + } + } + + kthread_exit(); +} + +static int +manage_test_ops(uint8_t action) +{ + int error = 0; + //struct sbuf *s = NULL; + + /* init an autosizing sbuf that initially holds 200 chars */ + //if ((s = sbuf_new(NULL, NULL, 200, SBUF_AUTOEXTEND)) == NULL) + // return -1; + + if (action == ENABLE) { + + run_test_thread = 1; + + kthread_add(&alqtest_thread, NULL, NULL, &alq_test_thr, + RFNOWAIT, 0, "alq_test_thr"); + } + else if (action == DISABLE && alq_test_thr != NULL) { + /* tell the test thread that it should exit now */ + run_test_thread = 0; + alq_test_thr = NULL; + } + + return (error); +} + +static int +deinit(void) +{ + manage_test_ops(DISABLE); + mtx_destroy(&alqtestmtx); + return (0); +} + +static int +init(void) +{ + mtx_init(&alqtestmtx, "alqtestmtx", NULL, MTX_DEF); + manage_test_ops(ENABLE); + return (0); +} + +/* + * This is the function that is called to load and unload the module. + * When the module is loaded, this function is called once with + * "what" == MOD_LOAD + * When the module is unloaded, this function is called twice with + * "what" = MOD_QUIESCE first, followed by "what" = MOD_UNLOAD second + * When the system is shut down e.g. CTRL-ALT-DEL or using the shutdown command, + * this function is called once with "what" = MOD_SHUTDOWN + * When the system is shut down, the handler isn't called until the very end + * of the shutdown sequence i.e. after the disks have been synced. + */ +static int alqtest_load_handler(module_t mod, int what, void *arg) +{ + switch(what) { + case MOD_LOAD: + return init(); + break; + + case MOD_QUIESCE: + case MOD_SHUTDOWN: + return deinit(); + break; + + case MOD_UNLOAD: + return (0); + break; + + default: + return (EINVAL); + break; + } +} + +static moduledata_t alqtest_mod = +{ + "alqtest", + alqtest_load_handler, + NULL +}; + +DECLARE_MODULE(alqtest, alqtest_mod, SI_SUB_SMP, SI_ORDER_ANY); +MODULE_DEPEND(alqtest, alq, 1, 1, 1); From owner-svn-src-user@FreeBSD.ORG Wed Mar 17 07:16:28 2010 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 582941065670; Wed, 17 Mar 2010 07:16:28 +0000 (UTC) (envelope-from lstewart@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id DC9FF8FC18; Wed, 17 Mar 2010 07:16:27 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id o2H7GRn1031543; Wed, 17 Mar 2010 07:16:27 GMT (envelope-from lstewart@svn.freebsd.org) Received: (from lstewart@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id o2H7GRtc031541; Wed, 17 Mar 2010 07:16:27 GMT (envelope-from lstewart@svn.freebsd.org) Message-Id: <201003170716.o2H7GRtc031541@svn.freebsd.org> From: Lawrence Stewart Date: Wed, 17 Mar 2010 07:16:27 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r205244 - user/lstewart/alq_varlen_head/sys/sys X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Mar 2010 07:16:28 -0000 Author: lstewart Date: Wed Mar 17 07:16:27 2010 New Revision: 205244 URL: http://svn.freebsd.org/changeset/base/205244 Log: Catch up with code changes. Sponsored by: FreeBSD Foundation Modified: user/lstewart/alq_varlen_head/sys/sys/alq.h Modified: user/lstewart/alq_varlen_head/sys/sys/alq.h ============================================================================== --- user/lstewart/alq_varlen_head/sys/sys/alq.h Wed Mar 17 07:08:57 2010 (r205243) +++ user/lstewart/alq_varlen_head/sys/sys/alq.h Wed Mar 17 07:16:27 2010 (r205244) @@ -62,7 +62,7 @@ struct ale { #define ALQ_DEFAULT_CMODE 0600 /* - * alq_open: Creates a new queue + * alq_open_flags: Creates a new queue * * Arguments: * alq Storage for a pointer to the newly created queue. @@ -74,6 +74,7 @@ struct ale { * count The number of items in the buffer, this should be large enough * to store items over the period of a disk write. For variable * length queues, this should be set to 0. + * flags ALQ_ORDERED * Returns: * error from open or 0 on success */ @@ -136,7 +137,7 @@ struct ale *alq_getn(struct alq *alq, in struct ale *alq_get(struct alq *alq, int flags); /* - * alq_postn: Schedule the ale retrieved by alq_get/alq_getn for writing. + * alq_post_flags: Schedule the ale retrieved by alq_get/alq_getn for writing. * alq The queue to post the entry to. * ale An asynch logging entry returned by alq_get. * flags ALQ_NOACTIVATE From owner-svn-src-user@FreeBSD.ORG Wed Mar 17 08:55:30 2010 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 562371065676; Wed, 17 Mar 2010 08:55:30 +0000 (UTC) (envelope-from jmallett@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 2D2A48FC21; Wed, 17 Mar 2010 08:55:30 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id o2H8tUCw053339; Wed, 17 Mar 2010 08:55:30 GMT (envelope-from jmallett@svn.freebsd.org) Received: (from jmallett@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id o2H8tUsW053337; Wed, 17 Mar 2010 08:55:30 GMT (envelope-from jmallett@svn.freebsd.org) Message-Id: <201003170855.o2H8tUsW053337@svn.freebsd.org> From: Juli Mallett Date: Wed, 17 Mar 2010 08:55:30 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r205247 - user/jmallett/octeon/sys/mips/mips X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Mar 2010 08:55:30 -0000 Author: jmallett Date: Wed Mar 17 08:55:29 2010 New Revision: 205247 URL: http://svn.freebsd.org/changeset/base/205247 Log: Enable 64-bit operations for threads as well. There are a multitude of sins surrounding this that are not yet fixed, including: o) The multiple copies of status register and other coprocessor-related bits. o) The fact that all of these bits are spelled out here at all rather than included in some KERNEL_STATUS and USER_STATUS definitions. o) The fact that these things are under "TARGET_OCTEON" rather than something related to ABI, since the only thing that should restrict whether user programs can use 64-bit instructions is whether the kernel can save and restore 64-bit registers. o) The fact that many things here enable 64-bit virtual addresses for supervisor mode, which FreeBSD has never used (although we can and should consider repurposing those address spaces when the time is right.) o) All of the duplication of logic wrt setting up PCBs and trapframes in this file. (To say nothing of the varying styles that makes it easy to miss these differences.) o) The lack of any assertions about the state of the status register for user threads with INVARIANTS on. Sponsored by: Packet Forensics Modified: user/jmallett/octeon/sys/mips/mips/vm_machdep.c Modified: user/jmallett/octeon/sys/mips/mips/vm_machdep.c ============================================================================== --- user/jmallett/octeon/sys/mips/mips/vm_machdep.c Wed Mar 17 07:26:59 2010 (r205246) +++ user/jmallett/octeon/sys/mips/mips/vm_machdep.c Wed Mar 17 08:55:29 2010 (r205247) @@ -425,7 +425,7 @@ cpu_set_upcall_kse(struct thread *td, vo tf->sr = SR_KSU_USER | SR_EXL | (SR_INT_MASK & mips_rd_status()) | MIPS_SR_INT_IE; #ifdef TARGET_OCTEON - tf->sr |= MIPS_SR_INT_IE | MIPS_SR_COP_0_BIT | MIPS_SR_UX | + tf->sr |= MIPS_SR_INT_IE | MIPS_SR_COP_0_BIT | MIPS32_SR_PX | MIPS_SR_UX | MIPS_SR_KX; #endif /* tf->sr |= (ALL_INT_MASK & idle_mask) | SR_INT_ENAB; */ From owner-svn-src-user@FreeBSD.ORG Wed Mar 17 08:56:14 2010 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 31BE9106564A; Wed, 17 Mar 2010 08:56:14 +0000 (UTC) (envelope-from jmallett@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 070378FC1A; Wed, 17 Mar 2010 08:56:14 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id o2H8uDUC053513; Wed, 17 Mar 2010 08:56:13 GMT (envelope-from jmallett@svn.freebsd.org) Received: (from jmallett@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id o2H8uDYb053511; Wed, 17 Mar 2010 08:56:13 GMT (envelope-from jmallett@svn.freebsd.org) Message-Id: <201003170856.o2H8uDYb053511@svn.freebsd.org> From: Juli Mallett Date: Wed, 17 Mar 2010 08:56:13 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r205248 - user/jmallett/octeon/sys/mips/mips X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Mar 2010 08:56:14 -0000 Author: jmallett Date: Wed Mar 17 08:56:13 2010 New Revision: 205248 URL: http://svn.freebsd.org/changeset/base/205248 Log: As with page faults, for now emit something in the log about reserved instructions. Sponsored by: Packet Forensics Modified: user/jmallett/octeon/sys/mips/mips/trap.c Modified: user/jmallett/octeon/sys/mips/mips/trap.c ============================================================================== --- user/jmallett/octeon/sys/mips/mips/trap.c Wed Mar 17 08:55:29 2010 (r205247) +++ user/jmallett/octeon/sys/mips/mips/trap.c Wed Mar 17 08:56:13 2010 (r205248) @@ -104,6 +104,7 @@ int trap_debug = 1; extern unsigned onfault_table[]; +static void log_reserved_instruction(struct trapframe *); static void log_bad_page_fault(char *, struct trapframe *, int); static void log_frame_dump(struct trapframe *frame); static void get_mapping_info(vm_offset_t, pd_entry_t **, pt_entry_t **); @@ -883,6 +884,7 @@ dofault: } case T_RES_INST + T_USER: + log_reserved_instruction(trapframe); i = SIGILL; addr = trapframe->pc; break; @@ -1306,6 +1308,50 @@ get_mapping_info(vm_offset_t va, pd_entr *ptepp = ptep; } +static void +log_reserved_instruction(struct trapframe *frame) +{ + pt_entry_t *ptep; + pd_entry_t *pdep; + unsigned int *addr; + struct proc *p = curproc; + register_t pc; + +#ifdef SMP + printf("cpuid = %d\n", PCPU_GET(cpuid)); +#endif + pc = frame->pc + (DELAYBRANCH(frame->cause) ? 4 : 0); + log(LOG_ERR, "RES_INST: pid %d (%s), uid %d: pc %p ra %p\n", + p->p_pid, p->p_comm, + p->p_ucred ? p->p_ucred->cr_uid : -1, + (void *)(intptr_t)pc, + (void *)(intptr_t)frame->ra); + + /* log registers in trap frame */ + log_frame_dump(frame); + + get_mapping_info((vm_offset_t)pc, &pdep, &ptep); + + /* + * Dump a few words around faulting instruction, if the addres is + * valid. + */ + if (!(pc & 3) && + useracc((caddr_t)(intptr_t)pc, sizeof(int) * 4, VM_PROT_READ)) { + /* dump page table entry for faulting instruction */ + log(LOG_ERR, "Page table info for pc address %p: pde = %p, pte = 0x%lx\n", + (void *)(intptr_t)pc, *pdep, ptep ? *ptep : 0); + + addr = (unsigned int *)(intptr_t)pc; + log(LOG_ERR, "Dumping 4 words starting at pc address %p: \n", + addr); + log(LOG_ERR, "%08x %08x %08x %08x\n", + addr[0], addr[1], addr[2], addr[3]); + } else { + log(LOG_ERR, "pc address %p is inaccessible, pde = 0x%p, pte = 0x%lx\n", + (void *)(intptr_t)pc, *pdep, ptep ? *ptep : 0); + } +} static void log_bad_page_fault(char *msg, struct trapframe *frame, int trap_type) From owner-svn-src-user@FreeBSD.ORG Fri Mar 19 00:19:55 2010 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CCCFE1065676; Fri, 19 Mar 2010 00:19:55 +0000 (UTC) (envelope-from delphij@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 6B5168FC08; Fri, 19 Mar 2010 00:19:55 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id o2J0JtlE081427; Fri, 19 Mar 2010 00:19:55 GMT (envelope-from delphij@svn.freebsd.org) Received: (from delphij@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id o2J0Jt67081426; Fri, 19 Mar 2010 00:19:55 GMT (envelope-from delphij@svn.freebsd.org) Message-Id: <201003190019.o2J0Jt67081426@svn.freebsd.org> From: Xin LI Date: Fri, 19 Mar 2010 00:19:55 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r205302 - user/delphij/libz X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Mar 2010 00:19:55 -0000 Author: delphij Date: Fri Mar 19 00:19:55 2010 New Revision: 205302 URL: http://svn.freebsd.org/changeset/base/205302 Log: Create a custom branch where I will be able to do the merge. Added: - copied from r205301, head/lib/libz/ Directory Properties: user/delphij/libz/ (props changed) From owner-svn-src-user@FreeBSD.ORG Fri Mar 19 00:33:49 2010 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9E1FD106564A; Fri, 19 Mar 2010 00:33:49 +0000 (UTC) (envelope-from delphij@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 8CBEB8FC18; Fri, 19 Mar 2010 00:33:49 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id o2J0Xnfs084642; Fri, 19 Mar 2010 00:33:49 GMT (envelope-from delphij@svn.freebsd.org) Received: (from delphij@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id o2J0XnE4084636; Fri, 19 Mar 2010 00:33:49 GMT (envelope-from delphij@svn.freebsd.org) Message-Id: <201003190033.o2J0XnE4084636@svn.freebsd.org> From: Xin LI Date: Fri, 19 Mar 2010 00:33:49 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r205305 - in user/delphij/libz: . contrib doc X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Mar 2010 00:33:49 -0000 Author: delphij Date: Fri Mar 19 00:33:49 2010 New Revision: 205305 URL: http://svn.freebsd.org/changeset/base/205305 Log: Merge changes from zlib 1.2.4. Bump shared library version number. Added: user/delphij/libz/contrib/ - copied from r205302, vendor/libz/dist/contrib/ user/delphij/libz/doc/ - copied from r205302, vendor/libz/dist/doc/ user/delphij/libz/gzclose.c - copied unchanged from r205302, vendor/libz/dist/gzclose.c user/delphij/libz/gzguts.h - copied unchanged from r205302, vendor/libz/dist/gzguts.h user/delphij/libz/gzlib.c - copied unchanged from r205302, vendor/libz/dist/gzlib.c user/delphij/libz/gzread.c - copied, changed from r205302, vendor/libz/dist/gzread.c user/delphij/libz/gzwrite.c - copied, changed from r205302, vendor/libz/dist/gzwrite.c Deleted: user/delphij/libz/algorithm.txt user/delphij/libz/gzio.c Modified: user/delphij/libz/ChangeLog user/delphij/libz/FAQ user/delphij/libz/Makefile user/delphij/libz/README user/delphij/libz/adler32.c user/delphij/libz/compress.c user/delphij/libz/crc32.c user/delphij/libz/deflate.c user/delphij/libz/deflate.h user/delphij/libz/example.c user/delphij/libz/infback.c user/delphij/libz/inffast.c user/delphij/libz/inflate.c user/delphij/libz/inflate.h user/delphij/libz/inftrees.c user/delphij/libz/inftrees.h user/delphij/libz/minigzip.c user/delphij/libz/trees.c user/delphij/libz/uncompr.c user/delphij/libz/zconf.h user/delphij/libz/zlib.3 user/delphij/libz/zlib.h user/delphij/libz/zutil.c user/delphij/libz/zutil.h Directory Properties: user/delphij/libz/ (props changed) Modified: user/delphij/libz/ChangeLog ============================================================================== --- user/delphij/libz/ChangeLog Fri Mar 19 00:29:52 2010 (r205304) +++ user/delphij/libz/ChangeLog Fri Mar 19 00:33:49 2010 (r205305) @@ -1,6 +1,281 @@ ChangeLog file for zlib +Changes in 1.2.4 (14 Mar 2010) +- Fix VER3 extraction in configure for no fourth subversion +- Update zlib.3, add docs to Makefile.in to make .pdf out of it +- Add zlib.3.pdf to distribution +- Don't set error code in gzerror() if passed pointer is NULL +- Apply destination directory fixes to CMakeLists.txt [Lowman] +- Move #cmakedefine's to a new zconf.in.cmakein +- Restore zconf.h for builds that don't use configure or cmake +- Add distclean to dummy Makefile for convenience +- Update and improve INDEX, README, and FAQ +- Update CMakeLists.txt for the return of zconf.h [Lowman] +- Update contrib/vstudio/vc9 and vc10 [Vollant] +- Change libz.dll.a back to libzdll.a in win32/Makefile.gcc +- Apply license and readme changes to contrib/asm686 [Raiter] +- Check file name lengths and add -c option in minigzip.c [Li] +- Update contrib/amd64 and contrib/masmx86/ [Vollant] +- Avoid use of "eof" parameter in trees.c to not shadow library variable +- Update make_vms.com for removal of zlibdefs.h [Zinser] +- Update assembler code and vstudio projects in contrib [Vollant] +- Remove outdated assembler code contrib/masm686 and contrib/asm586 +- Remove old vc7 and vc8 from contrib/vstudio +- Update win32/Makefile.msc, add ZLIB_VER_SUBREVISION [Rowe] +- Fix memory leaks in gzclose_r() and gzclose_w(), file leak in gz_open() +- Add contrib/gcc_gvmat64 for longest_match and inflate_fast [Vollant] +- Remove *64 functions from win32/zlib.def (they're not 64-bit yet) +- Fix bug in void-returning vsprintf() case in gzwrite.c +- Fix name change from inflate.h in contrib/inflate86/inffas86.c +- Check if temporary file exists before removing in make_vms.com [Zinser] +- Fix make install and uninstall for --static option +- Fix usage of _MSC_VER in gzguts.h and zutil.h [Truta] +- Update readme.txt in contrib/masmx64 and masmx86 to assemble + +Changes in 1.2.3.9 (21 Feb 2010) +- Expunge gzio.c +- Move as400 build information to old +- Fix updates in contrib/minizip and contrib/vstudio +- Add const to vsnprintf test in configure to avoid warnings [Weigelt] +- Delete zconf.h (made by configure) [Weigelt] +- Change zconf.in.h to zconf.h.in per convention [Weigelt] +- Check for NULL buf in gzgets() +- Return empty string for gzgets() with len == 1 (like fgets()) +- Fix description of gzgets() in zlib.h for end-of-file, NULL return +- Update minizip to 1.1 [Vollant] +- Avoid MSVC loss of data warnings in gzread.c, gzwrite.c +- Note in zlib.h that gzerror() should be used to distinguish from EOF +- Remove use of snprintf() from gzlib.c +- Fix bug in gzseek() +- Update contrib/vstudio, adding vc9 and vc10 [Kuno, Vollant] +- Fix zconf.h generation in CMakeLists.txt [Lowman] +- Improve comments in zconf.h where modified by configure + +Changes in 1.2.3.8 (13 Feb 2010) +- Clean up text files (tabs, trailing whitespace, etc.) [Oberhumer] +- Use z_off64_t in gz_zero() and gz_skip() to match state->skip +- Avoid comparison problem when sizeof(int) == sizeof(z_off64_t) +- Revert to Makefile.in from 1.2.3.6 (live with the clutter) +- Fix missing error return in gzflush(), add zlib.h note +- Add *64 functions to zlib.map [Levin] +- Fix signed/unsigned comparison in gz_comp() +- Use SFLAGS when testing shared linking in configure +- Add --64 option to ./configure to use -m64 with gcc +- Fix ./configure --help to correctly name options +- Have make fail if a test fails [Levin] +- Avoid buffer overrun in contrib/masmx64/gvmat64.asm [Simpson] +- Remove assembler object files from contrib + +Changes in 1.2.3.7 (24 Jan 2010) +- Always gzopen() with O_LARGEFILE if available +- Fix gzdirect() to work immediately after gzopen() or gzdopen() +- Make gzdirect() more precise when the state changes while reading +- Improve zlib.h documentation in many places +- Catch memory allocation failure in gz_open() +- Complete close operation if seek forward in gzclose_w() fails +- Return Z_ERRNO from gzclose_r() if close() fails +- Return Z_STREAM_ERROR instead of EOF for gzclose() being passed NULL +- Return zero for gzwrite() errors to match zlib.h description +- Return -1 on gzputs() error to match zlib.h description +- Add zconf.in.h to allow recovery from configure modification [Weigelt] +- Fix static library permissions in Makefile.in [Weigelt] +- Avoid warnings in configure tests that hide functionality [Weigelt] +- Add *BSD and DragonFly to Linux case in configure [gentoo 123571] +- Change libzdll.a to libz.dll.a in win32/Makefile.gcc [gentoo 288212] +- Avoid access of uninitialized data for first inflateReset2 call [Gomes] +- Keep object files in subdirectories to reduce the clutter somewhat +- Remove default Makefile and zlibdefs.h, add dummy Makefile +- Add new external functions to Z_PREFIX, remove duplicates, z_z_ -> z_ +- Remove zlibdefs.h completely -- modify zconf.h instead + +Changes in 1.2.3.6 (17 Jan 2010) +- Avoid void * arithmetic in gzread.c and gzwrite.c +- Make compilers happier with const char * for gz_error message +- Avoid unused parameter warning in inflate.c +- Avoid signed-unsigned comparison warning in inflate.c +- Indent #pragma's for traditional C +- Fix usage of strwinerror() in glib.c, change to gz_strwinerror() +- Correct email address in configure for system options +- Update make_vms.com and add make_vms.com to contrib/minizip [Zinser] +- Update zlib.map [Brown] +- Fix Makefile.in for Solaris 10 make of example64 and minizip64 [Tšršk] +- Apply various fixes to CMakeLists.txt [Lowman] +- Add checks on len in gzread() and gzwrite() +- Add error message for no more room for gzungetc() +- Remove zlib version check in gzwrite() +- Defer compression of gzprintf() result until need to +- Use snprintf() in gzdopen() if available +- Remove USE_MMAP configuration determination (only used by minigzip) +- Remove examples/pigz.c (available separately) +- Update examples/gun.c to 1.6 + +Changes in 1.2.3.5 (8 Jan 2010) +- Add space after #if in zutil.h for some compilers +- Fix relatively harmless bug in deflate_fast() [Exarevsky] +- Fix same problem in deflate_slow() +- Add $(SHAREDLIBV) to LIBS in Makefile.in [Brown] +- Add deflate_rle() for faster Z_RLE strategy run-length encoding +- Add deflate_huff() for faster Z_HUFFMAN_ONLY encoding +- Change name of "write" variable in inffast.c to avoid library collisions +- Fix premature EOF from gzread() in gzio.c [Brown] +- Use zlib header window size if windowBits is 0 in inflateInit2() +- Remove compressBound() call in deflate.c to avoid linking compress.o +- Replace use of errno in gz* with functions, support WinCE [Alves] +- Provide alternative to perror() in minigzip.c for WinCE [Alves] +- Don't use _vsnprintf on later versions of MSVC [Lowman] +- Add CMake build script and input file [Lowman] +- Update contrib/minizip to 1.1 [Svensson, Vollant] +- Moved nintendods directory from contrib to . +- Replace gzio.c with a new set of routines with the same functionality +- Add gzbuffer(), gzoffset(), gzclose_r(), gzclose_w() as part of above +- Update contrib/minizip to 1.1b +- Change gzeof() to return 0 on error instead of -1 to agree with zlib.h + +Changes in 1.2.3.4 (21 Dec 2009) +- Use old school .SUFFIXES in Makefile.in for FreeBSD compatibility +- Update comments in configure and Makefile.in for default --shared +- Fix test -z's in configure [Marquess] +- Build examplesh and minigzipsh when not testing +- Change NULL's to Z_NULL's in deflate.c and in comments in zlib.h +- Import LDFLAGS from the environment in configure +- Fix configure to populate SFLAGS with discovered CFLAGS options +- Adapt make_vms.com to the new Makefile.in [Zinser] +- Add zlib2ansi script for C++ compilation [Marquess] +- Add _FILE_OFFSET_BITS=64 test to make test (when applicable) +- Add AMD64 assembler code for longest match to contrib [Teterin] +- Include options from $SFLAGS when doing $LDSHARED +- Simplify 64-bit file support by introducing z_off64_t type +- Make shared object files in objs directory to work around old Sun cc +- Use only three-part version number for Darwin shared compiles +- Add rc option to ar in Makefile.in for when ./configure not run +- Add -WI,-rpath,. to LDFLAGS for OSF 1 V4* +- Set LD_LIBRARYN32_PATH for SGI IRIX shared compile +- Protect against _FILE_OFFSET_BITS being defined when compiling zlib +- Rename Makefile.in targets allstatic to static and allshared to shared +- Fix static and shared Makefile.in targets to be independent +- Correct error return bug in gz_open() by setting state [Brown] +- Put spaces before ;;'s in configure for better sh compatibility +- Add pigz.c (parallel implementation of gzip) to examples/ +- Correct constant in crc32.c to UL [Leventhal] +- Reject negative lengths in crc32_combine() +- Add inflateReset2() function to work like inflateEnd()/inflateInit2() +- Include sys/types.h for _LARGEFILE64_SOURCE [Brown] +- Correct typo in doc/algorithm.txt [Janik] +- Fix bug in adler32_combine() [Zhu] +- Catch missing-end-of-block-code error in all inflates and in puff + Assures that random input to inflate eventually results in an error +- Added enough.c (calculation of ENOUGH for inftrees.h) to examples/ +- Update ENOUGH and its usage to reflect discovered bounds +- Fix gzerror() error report on empty input file [Brown] +- Add ush casts in trees.c to avoid pedantic runtime errors +- Fix typo in zlib.h uncompress() description [Reiss] +- Correct inflate() comments with regard to automatic header detection +- Remove deprecation comment on Z_PARTIAL_FLUSH (it stays) +- Put new version of gzlog (2.0) in examples with interruption recovery +- Add puff compile option to permit invalid distance-too-far streams +- Add puff TEST command options, ability to read piped input +- Prototype the *64 functions in zlib.h when _FILE_OFFSET_BITS == 64, but + _LARGEFILE64_SOURCE not defined +- Fix Z_FULL_FLUSH to truly erase the past by resetting s->strstart +- Fix deflateSetDictionary() to use all 32K for output consistency +- Remove extraneous #define MIN_LOOKAHEAD in deflate.c (in deflate.h) +- Clear bytes after deflate lookahead to avoid use of uninitialized data +- Change a limit in inftrees.c to be more transparent to Coverity Prevent +- Update win32/zlib.def with exported symbols from zlib.h +- Correct spelling error in zlib.h [Willem] +- Allow Z_BLOCK for deflate() to force a new block +- Allow negative bits in inflatePrime() to delete existing bit buffer +- Add Z_TREES flush option to inflate() to return at end of trees +- Add inflateMark() to return current state information for random access +- Add Makefile for NintendoDS to contrib [Costa] +- Add -w in configure compile tests to avoid spurious warnings [Beucler] +- Fix typos in zlib.h comments for deflateSetDictionary() +- Fix EOF detection in transparent gzread() [Maier] + +Changes in 1.2.3.3 (2 October 2006) +- Make --shared the default for configure, add a --static option +- Add compile option to permit invalid distance-too-far streams +- Add inflateUndermine() function which is required to enable above +- Remove use of "this" variable name for C++ compatibility [Marquess] +- Add testing of shared library in make test, if shared library built +- Use ftello() and fseeko() if available instead of ftell() and fseek() +- Provide two versions of all functions that use the z_off_t type for + binary compatibility -- a normal version and a 64-bit offset version, + per the Large File Support Extension when _LARGEFILE64_SOURCE is + defined; use the 64-bit versions by default when _FILE_OFFSET_BITS + is defined to be 64 +- Add a --uname= option to configure to perhaps help with cross-compiling + +Changes in 1.2.3.2 (3 September 2006) +- Turn off silly Borland warnings [Hay] +- Use off64_t and define _LARGEFILE64_SOURCE when present +- Fix missing dependency on inffixed.h in Makefile.in +- Rig configure --shared to build both shared and static [Teredesai, Truta] +- Remove zconf.in.h and instead create a new zlibdefs.h file +- Fix contrib/minizip/unzip.c non-encrypted after encrypted [Vollant] +- Add treebuild.xml (see http://treebuild.metux.de/) [Weigelt] + +Changes in 1.2.3.1 (16 August 2006) +- Add watcom directory with OpenWatcom make files [Daniel] +- Remove #undef of FAR in zconf.in.h for MVS [Fedtke] +- Update make_vms.com [Zinser] +- Use -fPIC for shared build in configure [Teredesai, Nicholson] +- Use only major version number for libz.so on IRIX and OSF1 [Reinholdtsen] +- Use fdopen() (not _fdopen()) for Interix in zutil.h [BŠck] +- Add some FAQ entries about the contrib directory +- Update the MVS question in the FAQ +- Avoid extraneous reads after EOF in gzio.c [Brown] +- Correct spelling of "successfully" in gzio.c [Randers-Pehrson] +- Add comments to zlib.h about gzerror() usage [Brown] +- Set extra flags in gzip header in gzopen() like deflate() does +- Make configure options more compatible with double-dash conventions + [Weigelt] +- Clean up compilation under Solaris SunStudio cc [Rowe, Reinholdtsen] +- Fix uninstall target in Makefile.in [Truta] +- Add pkgconfig support [Weigelt] +- Use $(DESTDIR) macro in Makefile.in [Reinholdtsen, Weigelt] +- Replace set_data_type() with a more accurate detect_data_type() in + trees.c, according to the txtvsbin.txt document [Truta] +- Swap the order of #include and #include "zlib.h" in + gzio.c, example.c and minigzip.c [Truta] +- Shut up annoying VS2005 warnings about standard C deprecation [Rowe, + Truta] (where?) +- Fix target "clean" from win32/Makefile.bor [Truta] +- Create .pdb and .manifest files in win32/makefile.msc [Ziegler, Rowe] +- Update zlib www home address in win32/DLL_FAQ.txt [Truta] +- Update contrib/masmx86/inffas32.asm for VS2005 [Vollant, Van Wassenhove] +- Enable browse info in the "Debug" and "ASM Debug" configurations in + the Visual C++ 6 project, and set (non-ASM) "Debug" as default [Truta] +- Add pkgconfig support [Weigelt] +- Add ZLIB_VER_MAJOR, ZLIB_VER_MINOR and ZLIB_VER_REVISION in zlib.h, + for use in win32/zlib1.rc [Polushin, Rowe, Truta] +- Add a document that explains the new text detection scheme to + doc/txtvsbin.txt [Truta] +- Add rfc1950.txt, rfc1951.txt and rfc1952.txt to doc/ [Truta] +- Move algorithm.txt into doc/ [Truta] +- Synchronize FAQ with website +- Fix compressBound(), was low for some pathological cases [Fearnley] +- Take into account wrapper variations in deflateBound() +- Set examples/zpipe.c input and output to binary mode for Windows +- Update examples/zlib_how.html with new zpipe.c (also web site) +- Fix some warnings in examples/gzlog.c and examples/zran.c (it seems + that gcc became pickier in 4.0) +- Add zlib.map for Linux: "All symbols from zlib-1.1.4 remain + un-versioned, the patch adds versioning only for symbols introduced in + zlib-1.2.0 or later. It also declares as local those symbols which are + not designed to be exported." [Levin] +- Update Z_PREFIX list in zconf.in.h, add --zprefix option to configure +- Do not initialize global static by default in trees.c, add a response + NO_INIT_GLOBAL_POINTERS to initialize them if needed [Marquess] +- Don't use strerror() in gzio.c under WinCE [Yakimov] +- Don't use errno.h in zutil.h under WinCE [Yakimov] +- Move arguments for AR to its usage to allow replacing ar [Marot] +- Add HAVE_VISIBILITY_PRAGMA in zconf.in.h for Mozilla [Randers-Pehrson] +- Improve inflateInit() and inflateInit2() documentation +- Fix structure size comment in inflate.h +- Change configure help option from --h* to --help [Santos] + Changes in 1.2.3 (18 July 2005) - Apply security vulnerability fixes to contrib/infback9 as well - Clean up some text files (carriage returns, trailing space) @@ -13,7 +288,7 @@ Changes in 1.2.2.4 (11 July 2005) compile - Fix some spelling errors in comments [Betts] - Correct inflateInit2() error return documentation in zlib.h -- Added zran.c example of compressed data random access to examples +- Add zran.c example of compressed data random access to examples directory, shows use of inflatePrime() - Fix cast for assignments to strm->state in inflate.c and infback.c - Fix zlibCompileFlags() in zutil.c to use 1L for long shifts [Oberhumer] Modified: user/delphij/libz/FAQ ============================================================================== --- user/delphij/libz/FAQ Fri Mar 19 00:29:52 2010 (r205304) +++ user/delphij/libz/FAQ Fri Mar 19 00:33:49 2010 (r205305) @@ -3,8 +3,8 @@ If your question is not there, please check the zlib home page -http://www.zlib.org which may have more recent information. -The lastest zlib FAQ is at http://www.gzip.org/zlib/zlib_faq.html +http://zlib.net/ which may have more recent information. +The lastest zlib FAQ is at http://zlib.net/zlib_faq.html 1. Is zlib Y2K-compliant? @@ -13,54 +13,51 @@ The lastest zlib FAQ is at http://www.gz 2. Where can I get a Windows DLL version? - The zlib sources can be compiled without change to produce a DLL. - See the file win32/DLL_FAQ.txt in the zlib distribution. - Pointers to the precompiled DLL are found in the zlib web site at - http://www.zlib.org. + The zlib sources can be compiled without change to produce a DLL. See the + file win32/DLL_FAQ.txt in the zlib distribution. Pointers to the + precompiled DLL are found in the zlib web site at http://zlib.net/ . 3. Where can I get a Visual Basic interface to zlib? See - * http://www.dogma.net/markn/articles/zlibtool/zlibtool.htm - * contrib/visual-basic.txt in the zlib distribution + * http://marknelson.us/1997/01/01/zlib-engine/ * win32/DLL_FAQ.txt in the zlib distribution 4. compress() returns Z_BUF_ERROR. - Make sure that before the call of compress, the length of the compressed - buffer is equal to the total size of the compressed buffer and not - zero. For Visual Basic, check that this parameter is passed by reference + Make sure that before the call of compress(), the length of the compressed + buffer is equal to the available size of the compressed buffer and not + zero. For Visual Basic, check that this parameter is passed by reference ("as any"), not by value ("as long"). 5. deflate() or inflate() returns Z_BUF_ERROR. - Before making the call, make sure that avail_in and avail_out are not - zero. When setting the parameter flush equal to Z_FINISH, also make sure - that avail_out is big enough to allow processing all pending input. - Note that a Z_BUF_ERROR is not fatal--another call to deflate() or - inflate() can be made with more input or output space. A Z_BUF_ERROR - may in fact be unavoidable depending on how the functions are used, since - it is not possible to tell whether or not there is more output pending - when strm.avail_out returns with zero. + Before making the call, make sure that avail_in and avail_out are not zero. + When setting the parameter flush equal to Z_FINISH, also make sure that + avail_out is big enough to allow processing all pending input. Note that a + Z_BUF_ERROR is not fatal--another call to deflate() or inflate() can be + made with more input or output space. A Z_BUF_ERROR may in fact be + unavoidable depending on how the functions are used, since it is not + possible to tell whether or not there is more output pending when + strm.avail_out returns with zero. See http://zlib.net/zlib_how.html for a + heavily annotated example. 6. Where's the zlib documentation (man pages, etc.)? - It's in zlib.h for the moment, and Francis S. Lin has converted it to a - web page zlib.html. Volunteers to transform this to Unix-style man pages, - please contact us (zlib@gzip.org). Examples of zlib usage are in the files - example.c and minigzip.c. + It's in zlib.h . Examples of zlib usage are in the files example.c and + minigzip.c, with more in examples/ . 7. Why don't you use GNU autoconf or libtool or ...? - Because we would like to keep zlib as a very small and simple - package. zlib is rather portable and doesn't need much configuration. + Because we would like to keep zlib as a very small and simple package. + zlib is rather portable and doesn't need much configuration. 8. I found a bug in zlib. - Most of the time, such problems are due to an incorrect usage of - zlib. Please try to reproduce the problem with a small program and send - the corresponding source to us at zlib@gzip.org . Do not send - multi-megabyte data files without prior agreement. + Most of the time, such problems are due to an incorrect usage of zlib. + Please try to reproduce the problem with a small program and send the + corresponding source to us at zlib@gzip.org . Do not send multi-megabyte + data files without prior agreement. 9. Why do I get "undefined reference to gzputc"? @@ -82,7 +79,7 @@ The lastest zlib FAQ is at http://www.gz 12. Can zlib handle .Z files? - No, sorry. You have to spawn an uncompress or gunzip subprocess, or adapt + No, sorry. You have to spawn an uncompress or gunzip subprocess, or adapt the code of uncompress on your own. 13. How can I make a Unix shared library? @@ -99,8 +96,10 @@ The lastest zlib FAQ is at http://www.gz However, many flavors of Unix come with a shared zlib already installed. Before going to the trouble of compiling a shared version of zlib and - trying to install it, you may want to check if it's already there! If you - can #include , it's there. The -lz option will probably link to it. + trying to install it, you may want to check if it's already there! If you + can #include , it's there. The -lz option will probably link to + it. You can check the version at the top of zlib.h or with the + ZLIB_VERSION symbol defined in zlib.h . 15. I have a question about OttoPDF. @@ -109,8 +108,8 @@ The lastest zlib FAQ is at http://www.gz 16. Can zlib decode Flate data in an Adobe PDF file? - Yes. See http://www.fastio.com/ (ClibPDF), or http://www.pdflib.com/ . - To modify PDF forms, see http://sourceforge.net/projects/acroformtool/ . + Yes. See http://www.pdflib.com/ . To modify PDF forms, see + http://sourceforge.net/projects/acroformtool/ . 17. Why am I getting this "register_frame_info not found" error on Solaris? @@ -121,67 +120,67 @@ The lastest zlib FAQ is at http://www.gz symbol __register_frame_info: referenced symbol not found The symbol __register_frame_info is not part of zlib, it is generated by - the C compiler (cc or gcc). You must recompile applications using zlib - which have this problem. This problem is specific to Solaris. See + the C compiler (cc or gcc). You must recompile applications using zlib + which have this problem. This problem is specific to Solaris. See http://www.sunfreeware.com for Solaris versions of zlib and applications using zlib. 18. Why does gzip give an error on a file I make with compress/deflate? The compress and deflate functions produce data in the zlib format, which - is different and incompatible with the gzip format. The gz* functions in - zlib on the other hand use the gzip format. Both the zlib and gzip - formats use the same compressed data format internally, but have different - headers and trailers around the compressed data. + is different and incompatible with the gzip format. The gz* functions in + zlib on the other hand use the gzip format. Both the zlib and gzip formats + use the same compressed data format internally, but have different headers + and trailers around the compressed data. 19. Ok, so why are there two different formats? - The gzip format was designed to retain the directory information about - a single file, such as the name and last modification date. The zlib - format on the other hand was designed for in-memory and communication - channel applications, and has a much more compact header and trailer and - uses a faster integrity check than gzip. + The gzip format was designed to retain the directory information about a + single file, such as the name and last modification date. The zlib format + on the other hand was designed for in-memory and communication channel + applications, and has a much more compact header and trailer and uses a + faster integrity check than gzip. 20. Well that's nice, but how do I make a gzip file in memory? You can request that deflate write the gzip format instead of the zlib - format using deflateInit2(). You can also request that inflate decode - the gzip format using inflateInit2(). Read zlib.h for more details. + format using deflateInit2(). You can also request that inflate decode the + gzip format using inflateInit2(). Read zlib.h for more details. 21. Is zlib thread-safe? - Yes. However any library routines that zlib uses and any application- - provided memory allocation routines must also be thread-safe. zlib's gz* + Yes. However any library routines that zlib uses and any application- + provided memory allocation routines must also be thread-safe. zlib's gz* functions use stdio library routines, and most of zlib's functions use the - library memory allocation routines by default. zlib's Init functions allow - for the application to provide custom memory allocation routines. + library memory allocation routines by default. zlib's *Init* functions + allow for the application to provide custom memory allocation routines. Of course, you should only operate on any given zlib or gzip stream from a single thread at a time. 22. Can I use zlib in my commercial application? - Yes. Please read the license in zlib.h. + Yes. Please read the license in zlib.h. 23. Is zlib under the GNU license? - No. Please read the license in zlib.h. + No. Please read the license in zlib.h. 24. The license says that altered source versions must be "plainly marked". So what exactly do I need to do to meet that requirement? - You need to change the ZLIB_VERSION and ZLIB_VERNUM #defines in zlib.h. In + You need to change the ZLIB_VERSION and ZLIB_VERNUM #defines in zlib.h. In particular, the final version number needs to be changed to "f", and an - identification string should be appended to ZLIB_VERSION. Version numbers + identification string should be appended to ZLIB_VERSION. Version numbers x.x.x.f are reserved for modifications to zlib by others than the zlib - maintainers. For example, if the version of the base zlib you are altering + maintainers. For example, if the version of the base zlib you are altering is "1.2.3.4", then in zlib.h you should change ZLIB_VERNUM to 0x123f, and - ZLIB_VERSION to something like "1.2.3.f-zachary-mods-v3". You can also + ZLIB_VERSION to something like "1.2.3.f-zachary-mods-v3". You can also update the version strings in deflate.c and inftrees.c. For altered source distributions, you should also note the origin and nature of the changes in zlib.h, as well as in ChangeLog and README, along - with the dates of the alterations. The origin should include at least your + with the dates of the alterations. The origin should include at least your name (or your company's name), and an email address to contact for help or issues with the library. @@ -197,105 +196,112 @@ The lastest zlib FAQ is at http://www.gz 26. Will zlib work on a 64-bit machine? - It should. It has been tested on 64-bit machines, and has no dependence - on any data types being limited to 32-bits in length. If you have any + Yes. It has been tested on 64-bit machines, and has no dependence on any + data types being limited to 32-bits in length. If you have any difficulties, please provide a complete problem report to zlib@gzip.org 27. Will zlib decompress data from the PKWare Data Compression Library? - No. The PKWare DCL uses a completely different compressed data format - than does PKZIP and zlib. However, you can look in zlib's contrib/blast + No. The PKWare DCL uses a completely different compressed data format than + does PKZIP and zlib. However, you can look in zlib's contrib/blast directory for a possible solution to your problem. 28. Can I access data randomly in a compressed stream? - No, not without some preparation. If when compressing you periodically - use Z_FULL_FLUSH, carefully write all the pending data at those points, - and keep an index of those locations, then you can start decompression - at those points. You have to be careful to not use Z_FULL_FLUSH too - often, since it can significantly degrade compression. + No, not without some preparation. If when compressing you periodically use + Z_FULL_FLUSH, carefully write all the pending data at those points, and + keep an index of those locations, then you can start decompression at those + points. You have to be careful to not use Z_FULL_FLUSH too often, since it + can significantly degrade compression. Alternatively, you can scan a + deflate stream once to generate an index, and then use that index for + random access. See examples/zran.c . 29. Does zlib work on MVS, OS/390, CICS, etc.? - We don't know for sure. We have heard occasional reports of success on - these systems. If you do use it on one of these, please provide us with - a report, instructions, and patches that we can reference when we get - these questions. Thanks. + It has in the past, but we have not heard of any recent evidence. There + were working ports of zlib 1.1.4 to MVS, but those links no longer work. + If you know of recent, successful applications of zlib on these operating + systems, please let us know. Thanks. -30. Is there some simpler, easier to read version of inflate I can look at - to understand the deflate format? +30. Is there some simpler, easier to read version of inflate I can look at to + understand the deflate format? - First off, you should read RFC 1951. Second, yes. Look in zlib's + First off, you should read RFC 1951. Second, yes. Look in zlib's contrib/puff directory. 31. Does zlib infringe on any patents? - As far as we know, no. In fact, that was originally the whole point behind - zlib. Look here for some more information: + As far as we know, no. In fact, that was originally the whole point behind + zlib. Look here for some more information: http://www.gzip.org/#faq11 32. Can zlib work with greater than 4 GB of data? - Yes. inflate() and deflate() will process any amount of data correctly. + Yes. inflate() and deflate() will process any amount of data correctly. Each call of inflate() or deflate() is limited to input and output chunks of the maximum value that can be stored in the compiler's "unsigned int" - type, but there is no limit to the number of chunks. Note however that the - strm.total_in and strm_total_out counters may be limited to 4 GB. These + type, but there is no limit to the number of chunks. Note however that the + strm.total_in and strm_total_out counters may be limited to 4 GB. These counters are provided as a convenience and are not used internally by - inflate() or deflate(). The application can easily set up its own counters + inflate() or deflate(). The application can easily set up its own counters updated after each call of inflate() or deflate() to count beyond 4 GB. compress() and uncompress() may be limited to 4 GB, since they operate in a - single call. gzseek() and gztell() may be limited to 4 GB depending on how - zlib is compiled. See the zlibCompileFlags() function in zlib.h. + single call. gzseek() and gztell() may be limited to 4 GB depending on how + zlib is compiled. See the zlibCompileFlags() function in zlib.h. - The word "may" appears several times above since there is a 4 GB limit - only if the compiler's "long" type is 32 bits. If the compiler's "long" - type is 64 bits, then the limit is 16 exabytes. + The word "may" appears several times above since there is a 4 GB limit only + if the compiler's "long" type is 32 bits. If the compiler's "long" type is + 64 bits, then the limit is 16 exabytes. 33. Does zlib have any security vulnerabilities? - The only one that we are aware of is potentially in gzprintf(). If zlib - is compiled to use sprintf() or vsprintf(), then there is no protection - against a buffer overflow of a 4K string space, other than the caller of - gzprintf() assuring that the output will not exceed 4K. On the other - hand, if zlib is compiled to use snprintf() or vsnprintf(), which should - normally be the case, then there is no vulnerability. The ./configure - script will display warnings if an insecure variation of sprintf() will - be used by gzprintf(). Also the zlibCompileFlags() function will return - information on what variant of sprintf() is used by gzprintf(). + The only one that we are aware of is potentially in gzprintf(). If zlib is + compiled to use sprintf() or vsprintf(), then there is no protection + against a buffer overflow of an 8K string space (or other value as set by + gzbuffer()), other than the caller of gzprintf() assuring that the output + will not exceed 8K. On the other hand, if zlib is compiled to use + snprintf() or vsnprintf(), which should normally be the case, then there is + no vulnerability. The ./configure script will display warnings if an + insecure variation of sprintf() will be used by gzprintf(). Also the + zlibCompileFlags() function will return information on what variant of + sprintf() is used by gzprintf(). If you don't have snprintf() or vsnprintf() and would like one, you can find a portable implementation here: http://www.ijs.si/software/snprintf/ - Note that you should be using the most recent version of zlib. Versions - 1.1.3 and before were subject to a double-free vulnerability. + Note that you should be using the most recent version of zlib. Versions + 1.1.3 and before were subject to a double-free vulnerability, and versions + 1.2.1 and 1.2.2 were subject to an access exception when decompressing + invalid compressed data. 34. Is there a Java version of zlib? Probably what you want is to use zlib in Java. zlib is already included as part of the Java SDK in the java.util.zip package. If you really want a version of zlib written in the Java language, look on the zlib home - page for links: http://www.zlib.org/ + page for links: http://zlib.net/ . 35. I get this or that compiler or source-code scanner warning when I crank it up to maximally-pedantic. Can't you guys write proper code? Many years ago, we gave up attempting to avoid warnings on every compiler - in the universe. It just got to be a waste of time, and some compilers - were downright silly. So now, we simply make sure that the code always - works. + in the universe. It just got to be a waste of time, and some compilers + were downright silly as well as contradicted each other. So now, we simply + make sure that the code always works. 36. Valgrind (or some similar memory access checker) says that deflate is performing a conditional jump that depends on an uninitialized value. Isn't that a bug? - No. That is intentional for performance reasons, and the output of - deflate is not affected. This only started showing up recently since - zlib 1.2.x uses malloc() by default for allocations, whereas earlier - versions used calloc(), which zeros out the allocated memory. + No. That is intentional for performance reasons, and the output of deflate + is not affected. This only started showing up recently since zlib 1.2.x + uses malloc() by default for allocations, whereas earlier versions used + calloc(), which zeros out the allocated memory. Even though the code was + correct, versions 1.2.4 and later was changed to not stimulate these + checkers. 37. Will zlib read the (insert any ancient or arcane format here) compressed data format? @@ -305,20 +311,21 @@ The lastest zlib FAQ is at http://www.gz 38. How can I encrypt/decrypt zip files with zlib? - zlib doesn't support encryption. The original PKZIP encryption is very weak - and can be broken with freely available programs. To get strong encryption, - use GnuPG, http://www.gnupg.org/ , which already includes zlib compression. - For PKZIP compatible "encryption", look at http://www.info-zip.org/ + zlib doesn't support encryption. The original PKZIP encryption is very + weak and can be broken with freely available programs. To get strong + encryption, use GnuPG, http://www.gnupg.org/ , which already includes zlib + compression. For PKZIP compatible "encryption", look at + http://www.info-zip.org/ 39. What's the difference between the "gzip" and "deflate" HTTP 1.1 encodings? - "gzip" is the gzip format, and "deflate" is the zlib format. They should - probably have called the second one "zlib" instead to avoid confusion - with the raw deflate compressed data format. While the HTTP 1.1 RFC 2616 + "gzip" is the gzip format, and "deflate" is the zlib format. They should + probably have called the second one "zlib" instead to avoid confusion with + the raw deflate compressed data format. While the HTTP 1.1 RFC 2616 correctly points to the zlib specification in RFC 1950 for the "deflate" transfer encoding, there have been reports of servers and browsers that incorrectly produce or expect raw deflate data per the deflate - specficiation in RFC 1951, most notably Microsoft. So even though the + specficiation in RFC 1951, most notably Microsoft. So even though the "deflate" transfer encoding using the zlib format would be the more efficient approach (and in fact exactly what the zlib format was designed for), using the "gzip" transfer encoding is probably more reliable due to @@ -328,12 +335,32 @@ The lastest zlib FAQ is at http://www.gz 40. Does zlib support the new "Deflate64" format introduced by PKWare? - No. PKWare has apparently decided to keep that format proprietary, since - they have not documented it as they have previous compression formats. - In any case, the compression improvements are so modest compared to other - more modern approaches, that it's not worth the effort to implement. + No. PKWare has apparently decided to keep that format proprietary, since + they have not documented it as they have previous compression formats. In + any case, the compression improvements are so modest compared to other more + modern approaches, that it's not worth the effort to implement. -41. Can you please sign these lengthy legal documents and fax them back to us +41. I'm having a problem with the zip functions in zlib, can you help? + + There are no zip functions in zlib. You are probably using minizip by + Giles Vollant, which is found in the contrib directory of zlib. It is not + part of zlib. In fact none of the stuff in contrib is part of zlib. The + files in there are not supported by the zlib authors. You need to contact + the authors of the respective contribution for help. + +42. The match.asm code in contrib is under the GNU General Public License. + Since it's part of zlib, doesn't that mean that all of zlib falls under the + GNU GPL? + + No. The files in contrib are not part of zlib. They were contributed by + other authors and are provided as a convenience to the user within the zlib + distribution. Each item in contrib has its own license. + +43. Is zlib subject to export controls? What is its ECCN? + + zlib is not subject to export controls, and so is classified as EAR99. + +44. Can you please sign these lengthy legal documents and fax them back to us so that we can use your software in our product? No. Go away. Shoo. Modified: user/delphij/libz/Makefile ============================================================================== --- user/delphij/libz/Makefile Fri Mar 19 00:29:52 2010 (r205304) +++ user/delphij/libz/Makefile Fri Mar 19 00:33:49 2010 (r205305) @@ -4,21 +4,54 @@ LIB= z SHLIBDIR?= /lib +SHLIB_MAJOR= 6 MAN= zlib.3 -#CFLAGS+= -DMAX_WBITS=14 -DMAX_MEM_LEVEL=7 -#CFLAGS+= -g -DDEBUG -#CFLAGS+= -Wall -Wwrite-strings -Wpointer-arith -Wconversion \ -# -Wstrict-prototypes -Wmissing-prototypes +#CFLAGS=-O -DMAX_WBITS=14 -DMAX_MEM_LEVEL=7 +#CFLAGS=-g -DDEBUG +#CFLAGS=-O3 -Wall -Wwrite-strings -Wpointer-arith -Wconversion \ +# -Wstrict-prototypes -Wmissing-prototypes -CFLAGS+= -DHAS_snprintf -DHAS_vsnprintf +CFLAGS+= -DHAS_snprintf -DHAS_vsnprintf -I${.CURDIR} -WARNS?= 3 +WARNS?= 2 CLEANFILES+= example.o example foo.gz minigzip.o minigzip -SRCS = adler32.c compress.c crc32.c gzio.c uncompr.c deflate.c trees.c \ - zutil.c inflate.c inftrees.c inffast.c zopen.c infback.c +SRCS+= adler32.c +SRCS+= compress.c +SRCS+= crc32.c +SRCS+= deflate.c +SRCS+= gzclose.c +SRCS+= gzlib.c +SRCS+= gzread.c +SRCS+= gzwrite.c +SRCS+= infback.c +SRCS+= inflate.c +SRCS+= inftrees.c +SRCS+= trees.c +SRCS+= uncompr.c +SRCS+= zopen.c +SRCS+= zutil.c + +.if ${MACHINE_ARCH} == "i386" && defined(MACHINE_CPU) +.if ${MACHINE_CPU:M*i686*} +.PATH: ${.CURDIR}/contrib/asm686 +SRCS+= match.S +CFLAGS+= -DASMV -DNO_UNDERLINE +.endif +.endif + +.if ${MACHINE_ARCH} == "amd64" +.PATH: ${.CURDIR}/contrib/gcc_gvmat64 +.PATH: ${.CURDIR}/contrib/inflate86 +SRCS+= gvmat64.S +SRCS+= inffas86.c +CFLAGS+= -DASMV -DASMINF -DNO_UNDERLINE +.else +SRCS+= inffast.c +.endif + INCS= zconf.h zlib.h minigzip: all minigzip.o Modified: user/delphij/libz/README ============================================================================== --- user/delphij/libz/README Fri Mar 19 00:29:52 2010 (r205304) +++ user/delphij/libz/README Fri Mar 19 00:33:49 2010 (r205305) @@ -1,56 +1,51 @@ ZLIB DATA COMPRESSION LIBRARY -zlib 1.2.3 is a general purpose data compression library. All the code is +zlib 1.2.4 is a general purpose data compression library. All the code is thread safe. The data format used by the zlib library is described by RFCs (Request for Comments) 1950 to 1952 in the files http://www.ietf.org/rfc/rfc1950.txt (zlib format), rfc1951.txt (deflate format) -and rfc1952.txt (gzip format). These documents are also available in other -formats from ftp://ftp.uu.net/graphics/png/documents/zlib/zdoc-index.html +and rfc1952.txt (gzip format). All functions of the compression library are documented in the file zlib.h -(volunteer to write man pages welcome, contact zlib@gzip.org). A usage example +(volunteer to write man pages welcome, contact zlib@gzip.org). A usage example of the library is given in the file example.c which also tests that the library -is working correctly. Another example is given in the file minigzip.c. The +is working correctly. Another example is given in the file minigzip.c. The compression library itself is composed of all source files except example.c and minigzip.c. To compile all files and run the test program, follow the instructions given at -the top of Makefile. In short "make test; make install" should work for most -machines. For Unix: "./configure; make test; make install". For MSDOS, use one -of the special makefiles such as Makefile.msc. For VMS, use make_vms.com. +the top of Makefile.in. In short "./configure; make test", and if that goes +well, "make install" should work for most flavors of Unix. For Windows, use one +of the special makefiles in win32/ or projects/ . For VMS, use make_vms.com. Questions about zlib should be sent to , or to Gilles Vollant - for the Windows DLL version. The zlib home page is -http://www.zlib.org or http://www.gzip.org/zlib/ Before reporting a problem, -please check this site to verify that you have the latest version of zlib; -otherwise get the latest version and check whether the problem still exists or -not. - -PLEASE read the zlib FAQ http://www.gzip.org/zlib/zlib_faq.html before asking -for help. - -Mark Nelson wrote an article about zlib for the Jan. 1997 -issue of Dr. Dobb's Journal; a copy of the article is available in -http://dogma.net/markn/articles/zlibtool/zlibtool.htm - -The changes made in version 1.2.3 are documented in the file ChangeLog. - -Unsupported third party contributions are provided in directory "contrib". - -A Java implementation of zlib is available in the Java Development Kit -http://java.sun.com/j2se/1.4.2/docs/api/java/util/zip/package-summary.html -See the zlib home page http://www.zlib.org for details. - -A Perl interface to zlib written by Paul Marquess is in the -CPAN (Comprehensive Perl Archive Network) sites -http://www.cpan.org/modules/by-module/Compress/ + for the Windows DLL version. The zlib home page is +http://zlib.net/ . Before reporting a problem, please check this site to +verify that you have the latest version of zlib; otherwise get the latest +version and check whether the problem still exists or not. + +PLEASE read the zlib FAQ http://zlib.net/zlib_faq.html before asking for help. + +Mark Nelson wrote an article about zlib for the Jan. 1997 +issue of Dr. Dobb's Journal; a copy of the article is available at +http://marknelson.us/1997/01/01/zlib-engine/ . + +The changes made in version 1.2.4 are documented in the file ChangeLog. + +Unsupported third party contributions are provided in directory contrib/ . + +zlib is available in Java using the java.util.zip package, documented at +http://java.sun.com/developer/technicalArticles/Programming/compression/ . + +A Perl interface to zlib written by Paul Marquess is available +at CPAN (Comprehensive Perl Archive Network) sites, including +http://search.cpan.org/~pmqs/IO-Compress-Zlib/ . A Python interface to zlib written by A.M. Kuchling is available in Python 1.5 and later versions, see -http://www.python.org/doc/lib/module-zlib.html +http://www.python.org/doc/lib/module-zlib.html . -A zlib binding for TCL written by Andreas Kupries is -availlable at http://www.oche.de/~akupries/soft/trf/trf_zip.html +zlib is built into tcl: http://wiki.tcl.tk/4610 . An experimental package to read and write files in .zip format, written on top of zlib by Gilles Vollant , is available in the @@ -74,25 +69,21 @@ Notes for some targets: - zlib doesn't work on HP-UX 9.05 with some versions of /bin/cc. It works with other compilers. Use "make test" to check your compiler. -- gzdopen is not supported on RISCOS, BEOS and by some Mac compilers. +- gzdopen is not supported on RISCOS or BEOS. - For PalmOs, see http://palmzlib.sourceforge.net/ -- When building a shared, i.e. dynamic library on Mac OS X, the library must be - installed before testing (do "make install" before "make test"), since the - library location is specified in the library. - Acknowledgments: - The deflate format used by zlib was defined by Phil Katz. The deflate - and zlib specifications were written by L. Peter Deutsch. Thanks to all the - people who reported problems and suggested various improvements in zlib; - they are too numerous to cite here. + The deflate format used by zlib was defined by Phil Katz. The deflate and + zlib specifications were written by L. Peter Deutsch. Thanks to all the + people who reported problems and suggested various improvements in zlib; they + are too numerous to cite here. Copyright notice: - (C) 1995-2004 Jean-loup Gailly and Mark Adler + (C) 1995-2010 Jean-loup Gailly and Mark Adler This software is provided 'as-is', without any express or implied warranty. In no event will the authors be held liable for any damages @@ -113,13 +104,11 @@ Copyright notice: Jean-loup Gailly Mark Adler jloup@gzip.org madler@alumni.caltech.edu -If you use the zlib library in a product, we would appreciate *not* -receiving lengthy legal documents to sign. The sources are provided -for free but without warranty of any kind. The library has been -entirely written by Jean-loup Gailly and Mark Adler; it does not -include third-party code. - -If you redistribute modified sources, we would appreciate that you include -in the file ChangeLog history information documenting your changes. Please -read the FAQ for more information on the distribution of modified source -versions. +If you use the zlib library in a product, we would appreciate *not* receiving +lengthy legal documents to sign. The sources are provided for free but without +warranty of any kind. The library has been entirely written by Jean-loup +Gailly and Mark Adler; it does not include third-party code. + +If you redistribute modified sources, we would appreciate that you include in +the file ChangeLog history information documenting your changes. Please read +the FAQ for more information on the distribution of modified source versions. Modified: user/delphij/libz/adler32.c ============================================================================== --- user/delphij/libz/adler32.c Fri Mar 19 00:29:52 2010 (r205304) +++ user/delphij/libz/adler32.c Fri Mar 19 00:33:49 2010 (r205305) @@ -1,12 +1,15 @@ /* adler32.c -- compute the Adler-32 checksum of a data stream - * Copyright (C) 1995-2004 Mark Adler + * Copyright (C) 1995-2007 Mark Adler * For conditions of distribution and use, see copyright notice in zlib.h */ /* @(#) $Id$ */ -#define ZLIB_INTERNAL -#include "zlib.h" +#include "zutil.h" + +#define local static + +local uLong adler32_combine_(uLong adler1, uLong adler2, z_off64_t len2); #define BASE 65521UL /* largest prime smaller than 65536 */ #define NMAX 5552 @@ -125,10 +128,10 @@ uLong ZEXPORT adler32(adler, buf, len) } /* ========================================================================= */ -uLong ZEXPORT adler32_combine(adler1, adler2, len2) +local uLong adler32_combine_(adler1, adler2, len2) uLong adler1; uLong adler2; - z_off_t len2; + z_off64_t len2; { unsigned long sum1; unsigned long sum2; @@ -141,9 +144,26 @@ uLong ZEXPORT adler32_combine(adler1, ad MOD(sum2); sum1 += (adler2 & 0xffff) + BASE - 1; sum2 += ((adler1 >> 16) & 0xffff) + ((adler2 >> 16) & 0xffff) + BASE - rem; - if (sum1 > BASE) sum1 -= BASE; - if (sum1 > BASE) sum1 -= BASE; - if (sum2 > (BASE << 1)) sum2 -= (BASE << 1); - if (sum2 > BASE) sum2 -= BASE; + if (sum1 >= BASE) sum1 -= BASE; + if (sum1 >= BASE) sum1 -= BASE; + if (sum2 >= (BASE << 1)) sum2 -= (BASE << 1); + if (sum2 >= BASE) sum2 -= BASE; return sum1 | (sum2 << 16); } + +/* ========================================================================= */ +uLong ZEXPORT adler32_combine(adler1, adler2, len2) + uLong adler1; + uLong adler2; + z_off_t len2; +{ *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** From owner-svn-src-user@FreeBSD.ORG Fri Mar 19 00:51:48 2010 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 82D23106564A; Fri, 19 Mar 2010 00:51:48 +0000 (UTC) (envelope-from delphij@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 72C1C8FC0C; Fri, 19 Mar 2010 00:51:48 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id o2J0pmlE088613; Fri, 19 Mar 2010 00:51:48 GMT (envelope-from delphij@svn.freebsd.org) Received: (from delphij@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id o2J0pmLM088609; Fri, 19 Mar 2010 00:51:48 GMT (envelope-from delphij@svn.freebsd.org) Message-Id: <201003190051.o2J0pmLM088609@svn.freebsd.org> From: Xin LI Date: Fri, 19 Mar 2010 00:51:48 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r205306 - user/delphij/libz X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Mar 2010 00:51:48 -0000 Author: delphij Date: Fri Mar 19 00:51:48 2010 New Revision: 205306 URL: http://svn.freebsd.org/changeset/base/205306 Log: This is a working version of the zlib version 1.2.4 but there are still some rough edges that needs to be improved. Specifically: - zlib now uses off_t instead of long. So far I am fairly confident that this *should* work just fine but still needs further testing, etc. - The change from long to off_t requires users to rebuild all their binaries if linked with zlib. (Should we avoid the shared library version bump?) - We diveraged a little bit from official zlib's definition for their 64-bit variants. Technically we should have all these stuff without the 64 prefix since our off_t is 64 bit from the day 0 of FreeBSD 2.x, which is derived from 4.4BSD-Lite, while version 7.21 92/05/13 14:44:26 mckusick bumped it to 64-bit. Currently this is done with some direct changes in zlib.h and hack in zconf.h. - We need to import zlib's versioned symbols, perhaps the same of Linux's one provided with the distribution, or our own (since we bumped shared library version). Modified: user/delphij/libz/Makefile user/delphij/libz/zconf.h user/delphij/libz/zlib.h Modified: user/delphij/libz/Makefile ============================================================================== --- user/delphij/libz/Makefile Fri Mar 19 00:33:49 2010 (r205305) +++ user/delphij/libz/Makefile Fri Mar 19 00:51:48 2010 (r205306) @@ -14,7 +14,7 @@ MAN= zlib.3 CFLAGS+= -DHAS_snprintf -DHAS_vsnprintf -I${.CURDIR} -WARNS?= 2 +WARNS?= 3 CLEANFILES+= example.o example foo.gz minigzip.o minigzip Modified: user/delphij/libz/zconf.h ============================================================================== --- user/delphij/libz/zconf.h Fri Mar 19 00:33:49 2010 (r205305) +++ user/delphij/libz/zconf.h Fri Mar 19 00:51:48 2010 (r205306) @@ -360,7 +360,7 @@ typedef uLong FAR uLongf; typedef Byte *voidp; #endif -#ifdef HAVE_UNISTD_H /* may be set to #if 1 by ./configure */ +#if 0 /* was set to #if 0 by ./configure */ # define Z_HAVE_UNISTD_H #endif @@ -386,12 +386,13 @@ typedef uLong FAR uLongf; #endif /* - * This is hard-configured for FreeBSD, since zlib doesn't actually support - * using the system off_t for offsets unless off_t is no longer than long. - * To minimize the diff, we just "undef z_off_t" rather than modifying - * the following lines. + * This is hard-configured for FreeBSD. */ -#undef z_off_t +#include +#define z_off_t off_t +#ifndef _FILE_OFFSET_BITS +#define _FILE_OFFSET_BITS 64 +#endif #ifndef z_off_t # define z_off_t long Modified: user/delphij/libz/zlib.h ============================================================================== --- user/delphij/libz/zlib.h Fri Mar 19 00:33:49 2010 (r205305) +++ user/delphij/libz/zlib.h Fri Mar 19 00:51:48 2010 (r205306) @@ -1563,6 +1563,13 @@ ZEXTERN int ZEXPORT inflateBackInit_ OF( ZEXTERN off64_t ZEXPORT gzoffset64 OF((gzFile)); ZEXTERN uLong ZEXPORT adler32_combine64 OF((uLong, uLong, off64_t)); ZEXTERN uLong ZEXPORT crc32_combine64 OF((uLong, uLong, off64_t)); +#elif _FILE_OFFSET_BITS == 64 + ZEXTERN gzFile ZEXPORT gzopen64 OF((const char *, const char *)); + ZEXTERN off_t ZEXPORT gzseek64 OF((gzFile, off_t, int)); + ZEXTERN off_t ZEXPORT gztell64 OF((gzFile)); + ZEXTERN off_t ZEXPORT gzoffset64 OF((gzFile)); + ZEXTERN uLong ZEXPORT adler32_combine64 OF((uLong, uLong, off_t)); + ZEXTERN uLong ZEXPORT crc32_combine64 OF((uLong, uLong, off_t)); #endif #if !defined(ZLIB_INTERNAL) && _FILE_OFFSET_BITS == 64 @@ -1572,14 +1579,6 @@ ZEXTERN int ZEXPORT inflateBackInit_ OF( # define gzoffset gzoffset64 # define adler32_combine adler32_combine64 # define crc32_combine crc32_combine64 -# ifndef _LARGEFILE64_SOURCE - ZEXTERN gzFile ZEXPORT gzopen64 OF((const char *, const char *)); - ZEXTERN off_t ZEXPORT gzseek64 OF((gzFile, off_t, int)); - ZEXTERN off_t ZEXPORT gztell64 OF((gzFile)); - ZEXTERN off_t ZEXPORT gzoffset64 OF((gzFile)); - ZEXTERN uLong ZEXPORT adler32_combine64 OF((uLong, uLong, off_t)); - ZEXTERN uLong ZEXPORT crc32_combine64 OF((uLong, uLong, off_t)); -# endif #else ZEXTERN gzFile ZEXPORT gzopen OF((const char *, const char *)); ZEXTERN z_off_t ZEXPORT gzseek OF((gzFile, z_off_t, int)); From owner-svn-src-user@FreeBSD.ORG Fri Mar 19 03:30:19 2010 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BF415106566B; Fri, 19 Mar 2010 03:30:19 +0000 (UTC) (envelope-from kmacy@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id AFA738FC0C; Fri, 19 Mar 2010 03:30:19 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id o2J3UJDP023944; Fri, 19 Mar 2010 03:30:19 GMT (envelope-from kmacy@svn.freebsd.org) Received: (from kmacy@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id o2J3UJb5023943; Fri, 19 Mar 2010 03:30:19 GMT (envelope-from kmacy@svn.freebsd.org) Message-Id: <201003190330.o2J3UJb5023943@svn.freebsd.org> From: Kip Macy Date: Fri, 19 Mar 2010 03:30:19 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r205309 - user/kmacy/head_page_lock/sys/vm X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Mar 2010 03:30:19 -0000 Author: kmacy Date: Fri Mar 19 03:30:19 2010 New Revision: 205309 URL: http://svn.freebsd.org/changeset/base/205309 Log: remove gratuitous vm_page_io_start() calls Modified: user/kmacy/head_page_lock/sys/vm/vm_object.c Modified: user/kmacy/head_page_lock/sys/vm/vm_object.c ============================================================================== --- user/kmacy/head_page_lock/sys/vm/vm_object.c Fri Mar 19 01:38:03 2010 (r205308) +++ user/kmacy/head_page_lock/sys/vm/vm_object.c Fri Mar 19 03:30:19 2010 (r205309) @@ -964,8 +964,6 @@ vm_object_page_collect_flush(vm_object_t } vm_page_lock(p); } - vm_page_io_start(p); - pmap_remove_write(p); vm_page_unlock(p); maxf = 0; @@ -985,11 +983,6 @@ vm_object_page_collect_flush(vm_object_t tp->oflags &= ~VPO_CLEANCHK; break; } -#ifdef notyet - /* XXX */ - vm_page_io_start(tp); - pmap_remove_write(tp); -#endif vm_page_unlock(tp); maf[ i - 1 ] = tp; maxf++; @@ -1017,10 +1010,6 @@ vm_object_page_collect_flush(vm_object_t tp->oflags &= ~VPO_CLEANCHK; break; } -#ifdef notyet - vm_page_io_start(tp); - pmap_remove_write(tp); -#endif vm_page_unlock(tp); mab[ i - 1 ] = tp; maxb++; From owner-svn-src-user@FreeBSD.ORG Fri Mar 19 21:29:30 2010 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A5B24106566B; Fri, 19 Mar 2010 21:29:30 +0000 (UTC) (envelope-from delphij@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 944EA8FC08; Fri, 19 Mar 2010 21:29:30 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id o2JLTUiN069925; Fri, 19 Mar 2010 21:29:30 GMT (envelope-from delphij@svn.freebsd.org) Received: (from delphij@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id o2JLTUDt069923; Fri, 19 Mar 2010 21:29:30 GMT (envelope-from delphij@svn.freebsd.org) Message-Id: <201003192129.o2JLTUDt069923@svn.freebsd.org> From: Xin LI Date: Fri, 19 Mar 2010 21:29:30 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r205348 - in user/delphij/libz: . contrib/inflate86 X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Mar 2010 21:29:30 -0000 Author: delphij Date: Fri Mar 19 21:29:30 2010 New Revision: 205348 URL: http://svn.freebsd.org/changeset/base/205348 Log: Remove the optimized inflate code - it doesn't work on i386 (uses %ebx which conflicts with PIC) and doesn't seem to provide benefit during my tests with help from ministat(1) if we compare with the gain provided by gvmat64 code. Deleted: user/delphij/libz/contrib/inflate86/ Modified: user/delphij/libz/Makefile Modified: user/delphij/libz/Makefile ============================================================================== --- user/delphij/libz/Makefile Fri Mar 19 21:15:43 2010 (r205347) +++ user/delphij/libz/Makefile Fri Mar 19 21:29:30 2010 (r205348) @@ -27,6 +27,7 @@ SRCS+= gzlib.c SRCS+= gzread.c SRCS+= gzwrite.c SRCS+= infback.c +SRCS+= inffast.c SRCS+= inflate.c SRCS+= inftrees.c SRCS+= trees.c @@ -34,22 +35,16 @@ SRCS+= uncompr.c SRCS+= zopen.c SRCS+= zutil.c -.if ${MACHINE_ARCH} == "i386" && defined(MACHINE_CPU) -.if ${MACHINE_CPU:M*i686*} +.if ${MACHINE_ARCH} == "i386" && ${MACHINE_CPU:M*i686*} .PATH: ${.CURDIR}/contrib/asm686 SRCS+= match.S CFLAGS+= -DASMV -DNO_UNDERLINE .endif -.endif .if ${MACHINE_ARCH} == "amd64" .PATH: ${.CURDIR}/contrib/gcc_gvmat64 -.PATH: ${.CURDIR}/contrib/inflate86 SRCS+= gvmat64.S -SRCS+= inffas86.c -CFLAGS+= -DASMV -DASMINF -DNO_UNDERLINE -.else -SRCS+= inffast.c +CFLAGS+= -DASMV -DNO_UNDERLINE .endif INCS= zconf.h zlib.h