From owner-freebsd-fs@FreeBSD.ORG Mon Dec 22 07:29:57 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 864BF106564A for ; Mon, 22 Dec 2008 07:29:57 +0000 (UTC) (envelope-from gerryw@compvia.com) Received: from mail01.compvia.com (mail01.compvia.com [12.147.132.91]) by mx1.freebsd.org (Postfix) with ESMTP id 517DE8FC18 for ; Mon, 22 Dec 2008 07:29:57 +0000 (UTC) (envelope-from gerryw@compvia.com) Received: from [10.10.20.11] ([10.10.20.11]) by mail01.compvia.com (Kerio MailServer 6.5.1) for freebsd-fs@freebsd.org; Mon, 22 Dec 2008 00:59:54 -0600 To: freebsd-fs@freebsd.org From: "Gerry Weaver" Message-ID: <20081222065954.8a0760ac@mail01.compvia.com> Date: Mon, 22 Dec 2008 00:59:54 -0600 X-Mailer: Kerio MailServer 6.5.1 WebMail X-User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1b1) Gecko/20081007 Firefox/3.1b1 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Headers files included by vnode.h X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Dec 2008 07:29:57 -0000 Hello All, I hope this is the right place to post this. I've noticed that there are several header files included by /usr/includ= e/sys/vnode.h that are not present in the directory. Are these files sup= posed to be there=3F If not, what is the proper include path to use when= including vnode.h=3F They only appear in the source tree on my system.= =20 FreeBSD 7.0-RELEASE vnode=5Fif.h vnode=5Fif=5Fnewproto.h vnode=5Fif=5Ftypedef.h Thanks, Gerry From owner-freebsd-fs@FreeBSD.ORG Mon Dec 22 10:30:01 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 988AB1065673 for ; Mon, 22 Dec 2008 10:30:01 +0000 (UTC) (envelope-from arnaud.houdelette@tzim.net) Received: from golanth.tzim.net (unknown [IPv6:2001:41d0:1:d91f:21c:c0ff:fe4b:cf32]) by mx1.freebsd.org (Postfix) with ESMTP id 2A6768FC17 for ; Mon, 22 Dec 2008 10:30:00 +0000 (UTC) (envelope-from arnaud.houdelette@tzim.net) Received: from path-wifi.irisa.fr ([131.254.64.80]) by golanth.tzim.net with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.69 (FreeBSD)) (envelope-from ) id 1LEi2w-000CXU-TL; Mon, 22 Dec 2008 11:29:59 +0100 Message-ID: <494F6C21.2000801@tzim.net> Date: Mon, 22 Dec 2008 11:29:53 +0100 From: Arnaud Houdelette User-Agent: Thunderbird 2.0.0.18 (Macintosh/20081105) MIME-Version: 1.0 To: Doug Rabson References: <9461581F-F354-486D-961D-3FD5B1EF007C@rabson.org> <2F0DF92C-4240-48D4-9A5F-8B826D6D6E95@rabson.org> <87E89284-D3BF-4A5A-B6F7-C30709A3F2D9@lassitu.de> <4AC3BEB2-B47E-4280-85E1-C72891412D09@rabson.org> In-Reply-To: <4AC3BEB2-B47E-4280-85E1-C72891412D09@rabson.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit X-Authenticated-User: tzim@tzim.net X-Authenticator: plain Cc: freebsd-fs@freebsd.org, Stefan Bethke Subject: Re: Re: Booting from ZFS raidz X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Dec 2008 10:30:01 -0000 As I'm fairly interrested in this kind of setup, I set up a virtual machine (VirtualBox) with 3 HD. Sources are from a fresh current (csup yesterday). Applied your patch successfully. Done a make installworld / installkernel to the zfs root. Applied the bootcode as Stephan. The seem's the loader gets loaded, but it cant proceed further. I got those kind of errors : ZFS: i/o error - all block copies unavailable ZFS: can't read MOS object directory Can't find root filesystem - giving up Then I get to the loader prompt. ls give same errors. show lists (partial) : currdev=zfs0 loaddev=disk1a lsdev lists : cd devices: disk devices : disk0: BIOS drive A: disk1: BIOS drive C: disk1p1: FreeBSD boot disk1p2: FreeBSD swap disk1p3: FreeBSD ZFS pxe devices: zfs devices: zfs0: ztboot ztboot is the name of my pool. but other physical disks aren't shown... Seem's the loader can't read the ZFS raidz pool. The loader as been built with LOADER_ZFS_SUPPORT option on /etc/make.conf Anything I could get wrong ? Arnaud Houdelette Doug Rabson a écrit : >
> On 20 Dec 2008, at 15:01, Stefan Bethke wrote: > >> Am 20.12.2008 um 15:23 schrieb Doug Rabson: >> >>> On 19 Dec 2008, at 21:46, Stefan Bethke wrote: >>> >>>> gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 $1 >> >>> 1. Are you absolutely sure you are using gptzfsboot built with the >>> patch - the steps you list above show you building it but not >>> installing it on the system which is initialising the pool. >> >> Ugh, sorry. That is in fact the old version from before the patch. I >> will try again tonight, with updated sources and the right gptzfsboot. > > You should be able to re-install gptzfsboot without changing anything > else using something like: > > # dd if=/boot/gptzfsboot of=dap1 conv=osync > > > >
From owner-freebsd-fs@FreeBSD.ORG Mon Dec 22 11:06:51 2008 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3B503106567D for ; Mon, 22 Dec 2008 11:06:51 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 282CB8FC14 for ; Mon, 22 Dec 2008 11:06:51 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id mBMB6pso060561 for ; Mon, 22 Dec 2008 11:06:51 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id mBMB6oMr060557 for freebsd-fs@FreeBSD.org; Mon, 22 Dec 2008 11:06:50 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 22 Dec 2008 11:06:50 GMT Message-Id: <200812221106.mBMB6oMr060557@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-fs@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-fs@FreeBSD.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Dec 2008 11:06:51 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/129760 fs [nfs] after 'umount -f' of a stale NFS share FreeBSD l o kern/129231 fs [ufs] [patch] New UFS mount (norandom) option - mostly o kern/129174 fs [nfs][zfs][panic] NFS v3 Panic when under high load ex o kern/129152 fs [panic] non-userfriendly panic when trying to mount(8) o kern/129084 fs [udf] [panic] udf panic: getblk: size(67584) > MAXBSIZ f kern/128829 fs smbd(8) causes periodic panic on 7-RELEASE o kern/128633 fs [zfs] [lor] lock order reversal in zfs o kern/128514 fs [zfs] [mpt] problems with ZFS and LSILogic SAS/SATA Ad o kern/128173 fs [ext2fs] ls gives "Input/output error" on mounted ext3 o kern/127420 fs [gjournal] [panic] Journal overflow on gmirrored gjour o kern/127213 fs [tmpfs] sendfile on tmpfs data corruption o kern/127029 fs [panic] mount(8): trying to mount a write protected zi o kern/126287 fs [ufs] [panic] Kernel panics while mounting an UFS file o kern/125536 fs [ext2fs] ext 2 mounts cleanly but fails on commands li o kern/125149 fs [nfs][panic] changing into .zfs dir from nfs client ca o kern/124621 fs [ext3] [patch] Cannot mount ext2fs partition o kern/122888 fs [zfs] zfs hang w/ prefetch on, zil off while running t o bin/122172 fs [fs]: amd(8) automount daemon dies on 6.3-STABLE i386, o bin/121072 fs [smbfs] mount_smbfs(8) cannot normally convert the cha o bin/118249 fs mv(1): moving a directory changes its mtime o kern/116170 fs [panic] Kernel panic when mounting /tmp o kern/114955 fs [cd9660] [patch] [request] support for mask,dirmask,ui o kern/114847 fs [ntfs] [patch] [request] dirmask support for NTFS ala o kern/114676 fs [ufs] snapshot creation panics: snapacct_ufs2: bad blo o bin/114468 fs [patch] [request] add -d option to umount(8) to detach o bin/113838 fs [patch] [request] mount(8): add support for relative p o bin/113049 fs [patch] [request] make quot(8) use getopt(3) and show o kern/112658 fs [smbfs] [patch] smbfs and caching problems (resolves b o kern/93942 fs [vfs] [patch] panic: ufs_dirbad: bad dir (patch from D 29 problems total. From owner-freebsd-fs@FreeBSD.ORG Mon Dec 22 13:53:59 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D962D1065670 for ; Mon, 22 Dec 2008 13:53:59 +0000 (UTC) (envelope-from gary.jennejohn@freenet.de) Received: from mout2.freenet.de (mout2.freenet.de [IPv6:2001:748:100:40::2:4]) by mx1.freebsd.org (Postfix) with ESMTP id 706028FC0C for ; Mon, 22 Dec 2008 13:53:59 +0000 (UTC) (envelope-from gary.jennejohn@freenet.de) Received: from [195.4.92.26] (helo=16.mx.freenet.de) by mout2.freenet.de with esmtpa (ID gary.jennejohn@freenet.de) (port 25) (Exim 4.69 #73) id 1LElEL-0002As-Rd; Mon, 22 Dec 2008 14:53:57 +0100 Received: from ta323.t.pppool.de ([89.55.163.35]:28115 helo=ernst.jennejohn.org) by 16.mx.freenet.de with esmtpa (ID gary.jennejohn@freenet.de) (port 25) (Exim 4.69 #73) id 1LElEL-0003x8-JH; Mon, 22 Dec 2008 14:53:57 +0100 Date: Mon, 22 Dec 2008 14:53:56 +0100 From: Gary Jennejohn To: "Gerry Weaver" Message-ID: <20081222145356.56c91d0b@ernst.jennejohn.org> In-Reply-To: <20081222065954.8a0760ac@mail01.compvia.com> References: <20081222065954.8a0760ac@mail01.compvia.com> X-Mailer: Claws Mail 3.5.0 (GTK+ 2.12.11; amd64-portbld-freebsd8.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: Headers files included by vnode.h X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: gary.jennejohn@freenet.de List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Dec 2008 13:53:59 -0000 On Mon, 22 Dec 2008 00:59:54 -0600 "Gerry Weaver" wrote: > Hello All, > > I hope this is the right place to post this. > > I've noticed that there are several header files included by /usr/include/sys/vnode.h that are not present in the directory. Are these files supposed to be there? If not, what is the proper include path to use when including vnode.h? They only appear in the source tree on my system. > > FreeBSD 7.0-RELEASE > > vnode_if.h > vnode_if_newproto.h > vnode_if_typedef.h > These files (among others) are dynamically generated when you make a kernel. See /sys/kern/vnode_if.src and /sys/tools/vnode_if.awk. --- Gary Jennejohn From owner-freebsd-fs@FreeBSD.ORG Mon Dec 22 16:59:03 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3E3F1106564A for ; Mon, 22 Dec 2008 16:59:03 +0000 (UTC) (envelope-from gerryw@compvia.com) Received: from mail01.compvia.com (mail01.compvia.com [12.147.132.91]) by mx1.freebsd.org (Postfix) with ESMTP id EBE8C8FC1B for ; Mon, 22 Dec 2008 16:59:02 +0000 (UTC) (envelope-from gerryw@compvia.com) Received: from [10.10.20.11] ([10.10.20.11]) by mail01.compvia.com (Kerio MailServer 6.5.1); Mon, 22 Dec 2008 10:58:58 -0600 To: gary.jennejohn@freenet.de From: "Gerry Weaver" In-Reply-To: 20081222145356.56c91d0b@ernst.jennejohn.org Message-ID: <20081222165858.be1f1fea@mail01.compvia.com> Date: Mon, 22 Dec 2008 10:58:58 -0600 X-Mailer: Kerio MailServer 6.5.1 WebMail X-User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1b1) Gecko/20081007 Firefox/3.1b1 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-fs@freebsd.org Subject: Re: Headers files included by vnode.h X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Dec 2008 16:59:03 -0000 Hi, Shouldn't these headers be installed/linked as part of the kernel make i= nstall process then=3F It seems odd to use an include path to the kernel= source tree. Thanks, Gerry =5F=5F=5F=5F=5F =20 From: Gary Jennejohn [mailto:gary.jennejohn@freenet.de] To: Gerry Weaver [mailto:gerryw@compvia.com] Cc: freebsd-fs@freebsd.org Sent: Mon, 22 Dec 2008 07:53:56 -0600 Subject: Re: Headers files included by vnode.h On Mon, 22 Dec 2008 00:59:54 -0600 "Gerry Weaver" wrote: =20 > Hello All, >=20 > I hope this is the right place to post this. >=20 > I've noticed that there are several header files included by /usr/in= clude/sys/vnode.h that are not present in the directory. Are these files= supposed to be there=3F If not, what is the proper include path to use = when including vnode.h=3F They only appear in the source tree on my syst= em.=20 >=20 > FreeBSD 7.0-RELEASE >=20 > vnode=5Fif.h > vnode=5Fif=5Fnewproto.h > vnode=5Fif=5Ftypedef.h >=20 =20 These files (among others) are dynamically generated when you make a kernel. See /sys/kern/vnode=5Fif.src and /sys/tools/vnode=5Fif.awk. =20 --- Gary Jennejohn =20 From owner-freebsd-fs@FreeBSD.ORG Mon Dec 22 17:36:03 2008 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8D7181065687; Mon, 22 Dec 2008 17:36:03 +0000 (UTC) (envelope-from hselasky@freebsd.org) Received: from swip.net (mailfe14.swip.net [212.247.155.161]) by mx1.freebsd.org (Postfix) with ESMTP id BA9F68FC25; Mon, 22 Dec 2008 17:36:02 +0000 (UTC) (envelope-from hselasky@freebsd.org) X-Cloudmark-Score: 0.000000 [] X-Cloudmark-Analysis: v=1.0 c=1 a=b-Idb26ITvkA:10 a=AWGMfE17WNHoQZabtIIA:9 a=83yrW3uf-nrSiy0NIu0065aecJcA:4 a=50e4U0PicR4A:10 a=qNfsqeB5S6M8GqLfWEUA:9 a=sHe3vdHnRMbs7Sbp2UMA:7 a=F5YzBXDzJjiwOpYNm4yZFR7I9fIA:4 a=rPt6xJ-oxjAA:10 Received: from [193.217.167.134] (account mc467741@c2i.net HELO [10.0.0.186]) by mailfe14.swip.net (CommuniGate Pro SMTP 5.2.6) with ESMTPA id 427809989; Mon, 22 Dec 2008 17:35:58 +0100 From: Hans Petter Selasky To: Alfred Perlstein User-Agent: KMail/1.9.7 References: <20081221112157.GU18389@elvis.mu.org> <200812221125.58857.hselasky@c2i.net> <20081222121725.GN18389@elvis.mu.org> In-Reply-To: <20081222121725.GN18389@elvis.mu.org> MIME-Version: 1.0 Date: Mon, 22 Dec 2008 17:38:14 +0100 Content-Type: Multipart/Mixed; boundary="Boundary-00=_4J8TJJP/MuRjiyU" Message-Id: <200812221738.16396.hselasky@freebsd.org> Cc: fs@freebsd.org, jhb@freebsd.org Subject: Small patch for vfs_mount.c needs review X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Dec 2008 17:36:03 -0000 --Boundary-00=_4J8TJJP/MuRjiyU Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Hi, Here is a small patch for vfs_mount.c which irons out some problems with USB2 and booting from a memory stick. This patch is just a workaround. I think that an event driven model would be better where the OS is allowed to boot at the moment the root partition disk is plugged in. Typically the USB memory stick will appear within 2 seconds of waiting. The patch has been tested with FreeSBIE. I hope that the mailing list software did not strip away the attached patch. --HPS --Boundary-00=_4J8TJJP/MuRjiyU Content-Type: text/x-diff; charset="iso-8859-1"; name="vfs_mount.c.diff" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="vfs_mount.c.diff" --- vfs_mount.c.orig Mon Dec 22 14:43:36 2008 +++ vfs_mount.c Mon Dec 22 15:09:14 2008 @@ -58,6 +58,7 @@ #include #include #include +#include #include #include @@ -1606,7 +1607,11 @@ vfs_mountroot(void) { char *cp; - int error, i, asked = 0; + const char *rootdevname_orig; + int error; + unsigned int i; + unsigned char asked = 0; /* set if asked for mount point */ + unsigned char timeout = 16; /* seconds */ root_mount_prepare(); @@ -1624,6 +1629,10 @@ asked = 1; } + /* store a copy of the initial root device name */ + rootdevname_orig = ctrootdevname; + retry: + /* * The root filesystem information is compiled in, and we are * booted with instructions to use it. @@ -1674,12 +1683,27 @@ if (!vfs_mountroot_try(ctrootdevname)) goto mounted; /* - * Everything so far has failed, prompt on the console if we haven't - * already tried that. + * Check if we should try more times. + */ + if (timeout != 0) { + timeout--; + pause("WROOT", hz); + if (cncheckc() == -1) { + /* no key press - try again */ + ctrootdevname = rootdevname_orig; + goto retry; + } + } + + /* + * Everything so far has failed, prompt on the console if we + * haven't already tried that. */ - if (!asked) + if (!asked) { + printf("\n"); if (!vfs_mountroot_ask()) goto mounted; + } panic("Root mount failed, startup aborted."); --Boundary-00=_4J8TJJP/MuRjiyU-- From owner-freebsd-fs@FreeBSD.ORG Mon Dec 22 18:16:55 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 85221106564A for ; Mon, 22 Dec 2008 18:16:55 +0000 (UTC) (envelope-from gary.jennejohn@freenet.de) Received: from mout1.freenet.de (mout1.freenet.de [IPv6:2001:748:100:40::2:3]) by mx1.freebsd.org (Postfix) with ESMTP id 1BFC18FC16 for ; Mon, 22 Dec 2008 18:16:54 +0000 (UTC) (envelope-from gary.jennejohn@freenet.de) Received: from [195.4.92.22] (helo=12.mx.freenet.de) by mout1.freenet.de with esmtpa (ID gary.jennejohn@freenet.de) (port 25) (Exim 4.69 #73) id 1LEpKn-000830-BP; Mon, 22 Dec 2008 19:16:53 +0100 Received: from ta323.t.pppool.de ([89.55.163.35]:33959 helo=ernst.jennejohn.org) by 12.mx.freenet.de with esmtpa (ID gary.jennejohn@freenet.de) (port 25) (Exim 4.69 #73) id 1LEpKn-0007Gc-3n; Mon, 22 Dec 2008 19:16:53 +0100 Date: Mon, 22 Dec 2008 19:16:51 +0100 From: Gary Jennejohn To: "Gerry Weaver" Message-ID: <20081222191651.051cb2b6@ernst.jennejohn.org> In-Reply-To: <20081222165858.be1f1fea@mail01.compvia.com> References: <20081222165858.be1f1fea@mail01.compvia.com> X-Mailer: Claws Mail 3.5.0 (GTK+ 2.12.11; amd64-portbld-freebsd8.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: Headers files included by vnode.h X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: gary.jennejohn@freenet.de List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Dec 2008 18:16:55 -0000 On Mon, 22 Dec 2008 10:58:58 -0600 "Gerry Weaver" wrote: > From: Gary Jennejohn [mailto:gary.jennejohn@freenet.de] > > On Mon, 22 Dec 2008 00:59:54 -0600 > "Gerry Weaver" wrote: > > > Hello All, > > > > I hope this is the right place to post this. > > > > I've noticed that there are several header files included by /usr/include/sys/vnode.h that are not present in the directory. Are these files supposed to be there? If not, what is the proper include path to use when including vnode.h? They only appear in the source tree on my system. > > > > FreeBSD 7.0-RELEASE > > > > vnode_if.h > > vnode_if_newproto.h > > vnode_if_typedef.h > > > > These files (among others) are dynamically generated when you make a > kernel. See /sys/kern/vnode_if.src and /sys/tools/vnode_if.awk. > > Shouldn't these headers be installed/linked as part of the kernel make > install process then? It seems odd to use an include path to the kernel > source tree. > Please don't top post and try to wrap your lines. Because these files are dynamically generated it makes no sense to install them. There are quite a few files like these which are used during the kernel generation process to dynamically create include files. This allows greater flexibility. --- Gary Jennejohn From owner-freebsd-fs@FreeBSD.ORG Mon Dec 22 19:44:25 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 848161065674 for ; Mon, 22 Dec 2008 19:44:25 +0000 (UTC) (envelope-from gerryw@compvia.com) Received: from mail01.compvia.com (mail01.compvia.com [12.147.132.91]) by mx1.freebsd.org (Postfix) with ESMTP id 432188FC16 for ; Mon, 22 Dec 2008 19:44:25 +0000 (UTC) (envelope-from gerryw@compvia.com) Received: from [10.10.20.11] ([10.10.20.11]) by mail01.compvia.com (Kerio MailServer 6.5.1); Mon, 22 Dec 2008 13:44:20 -0600 To: gary.jennejohn@freenet.de From: "Gerry Weaver" In-Reply-To: 20081222191651.051cb2b6@ernst.jennejohn.org Message-ID: <20081222194420.98abb8bb@mail01.compvia.com> Date: Mon, 22 Dec 2008 13:44:20 -0600 X-Mailer: Kerio MailServer 6.5.1 WebMail X-User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1b1) Gecko/20081007 Firefox/3.1b1 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-fs@freebsd.org Subject: Re: Headers files included by vnode.h X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Dec 2008 19:44:25 -0000 =5F=5F=5F=5F=5F =20 From: Gary Jennejohn [mailto:gary.jennejohn@freenet.de] To: Gerry Weaver [mailto:gerryw@compvia.com] Cc: freebsd-fs@freebsd.org Sent: Mon, 22 Dec 2008 12:16:51 -0600 Subject: Re: Headers files included by vnode.h On Mon, 22 Dec 2008 10:58:58 -0600 "Gerry Weaver" wrote: =20 > From: Gary Jennejohn [mailto:gary.jennejohn@freenet.de] >=20 > On Mon, 22 Dec 2008 00:59:54 -0600 > "Gerry Weaver" wrote: > =20 > > Hello All, > >=20 > > I hope this is the right place to post this. > >=20 > > I've noticed that there are several header files included by /us= r/include/sys/vnode.h that are not present in the directory. Are these f= iles supposed to be there=3F If not, what is the proper include path to = use when including vnode.h=3F They only appear in the source tree on my = system.=20 > >=20 > > FreeBSD 7.0-RELEASE > >=20 > > vnode=5Fif.h > > vnode=5Fif=5Fnewproto.h > > vnode=5Fif=5Ftypedef.h > >=20 > =20 > These files (among others) are dynamically generated when you make= a > kernel. See /sys/kern/vnode=5Fif.src and /sys/tools/vnode=5Fif.aw= k. > =20 > Shouldn't these headers be installed/linked as part of the kernel ma= ke > install process then=3F It seems odd to use an include path to the k= ernel > source tree. >=20 =20 Please don't top post and try to wrap your lines. =20 Because these files are dynamically generated it makes no sense to install them. There are quite a few files like these which are used during the kernel generation process to dynamically create include files. This allows greater flexibility. =20 --- Gary JennejohnHi, Thanks Gary. I appreciate your help. Thanks, Gerry =20 From owner-freebsd-fs@FreeBSD.ORG Tue Dec 23 11:41:13 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E12071065673 for ; Tue, 23 Dec 2008 11:41:13 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id BD6838FC1D for ; Tue, 23 Dec 2008 11:41:13 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [65.122.17.41]) by cyrus.watson.org (Postfix) with ESMTP id 5561B46B64; Tue, 23 Dec 2008 06:41:13 -0500 (EST) Date: Tue, 23 Dec 2008 11:41:13 +0000 (GMT) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Gerry Weaver In-Reply-To: <20081222194420.98abb8bb@mail01.compvia.com> Message-ID: References: <20081222194420.98abb8bb@mail01.compvia.com> User-Agent: Alpine 1.10 (BSF 962 2008-03-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-fs@freebsd.org Subject: Re: Headers files included by vnode.h X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Dec 2008 11:41:14 -0000 On Mon, 22 Dec 2008, Gerry Weaver wrote: > Because these files are dynamically generated it makes no sense to > install them. There are quite a few files like these which are used > during the kernel generation process to dynamically create include > files. This allows greater flexibility. > > Thanks Gary. I appreciate your help. The usual question here is "what are you trying to do?". Normally we dynamically generate the implementations of the VOP interfaces when the kernel (or a module built independently from the kernel) depends on them. If you're creating a kernel module, you can add a dependency on vnode_if.h, which will cause the kernel module build framework to generate local copies of the file in your build directory during the build, as seen in the Coda module: # $FreeBSD: src/sys/modules/coda/Makefile,v 1.17.2.1 2008/03/14 17:12:40 rwatson Exp $ .PATH: ${.CURDIR}/../../fs/coda KMOD= coda SRCS= vnode_if.h \ coda_fbsd.c coda_psdev.c coda_subr.c coda_venus.c coda_vfsops.c \ coda_vnops.c opt_coda.h .include If it's for the purposes of debugging a kernel, you should be able to find the generated copies of the files in the build directory for the kernel. Robert N M Watson Computer Laboratory University of Cambridge From owner-freebsd-fs@FreeBSD.ORG Tue Dec 23 20:16:27 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D5F8A1065670 for ; Tue, 23 Dec 2008 20:16:27 +0000 (UTC) (envelope-from gerryw@compvia.com) Received: from mail01.compvia.com (mail01.compvia.com [12.147.132.91]) by mx1.freebsd.org (Postfix) with ESMTP id 8EFF08FC19 for ; Tue, 23 Dec 2008 20:16:27 +0000 (UTC) (envelope-from gerryw@compvia.com) Received: from [10.10.20.11] ([10.10.20.11]) by mail01.compvia.com (Kerio MailServer 6.5.1); Tue, 23 Dec 2008 14:16:26 -0600 To: "Robert Watson" From: "Gerry Weaver" In-Reply-To: alpine.BSF.1.10.0812231138272.90302@fledge.watson.org Message-ID: <20081223201626.caa483ad@mail01.compvia.com> Date: Tue, 23 Dec 2008 14:16:26 -0600 X-Mailer: Kerio MailServer 6.5.1 WebMail X-User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1b1) Gecko/20081007 Firefox/3.1b1 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-fs@freebsd.org Subject: Re: Headers files included by vnode.h X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Dec 2008 20:16:28 -0000 =5F=5F=5F=5F=5F =20 From: Robert Watson [mailto:rwatson@FreeBSD.org] To: Gerry Weaver [mailto:gerryw@compvia.com] Cc: gary.jennejohn@freenet.de, freebsd-fs@freebsd.org Sent: Tue, 23 Dec 2008 05:41:13 -0600 Subject: Re: Headers files included by vnode.h On Mon, 22 Dec 2008, Gerry Weaver wrote: =20 > Because these files are dynamically generated it makes no sense to > install them. There are quite a few files like these which are use= d > during the kernel generation process to dynamically create include > files. This allows greater flexibility. > > Thanks Gary. I appreciate your help. =20 The usual question here is "what are you trying to do=3F". Normally w= e=20 dynamically generate the implementations of the VOP interfaces when th= e kernel=20 (or a module built independently from the kernel) depends on them. If= you're=20 creating a kernel module, you can add a dependency on vnode=5Fif.h, wh= ich will=20 cause the kernel module build framework to generate local copies of th= e file=20 in your build directory during the build, as seen in the Coda module: =20 # $FreeBSD: src/sys/modules/coda/Makefile,v 1.17.2.1 2008/03/14 17:= 12:40 rwatson Exp $ =20 .PATH: ${.CURDIR}/../../fs/coda =20 KMOD=3D coda SRCS=3D vnode=5Fif.h \ coda=5Ffbsd.c coda=5Fpsdev.c coda=5Fsubr.c coda=5Fvenus.c c= oda=5Fvfsops.c \ coda=5Fvnops.c opt=5Fcoda.h =20 .include =20 If it's for the purposes of debugging a kernel, you should be able to = find the=20 generated copies of the files in the build directory for the kernel. =20 Robert N M Watson Computer Laboratory University of CambridgeHi Robert, Perfect. Yes, I am building a kernel module. This is exactly what I was = looking for.=20 I really appreciate your help.=20 I am also trying to figure out a zero copy approach to kernel memory acc= ess from user space. Would you happen to know which list I should post t= he question to=3F Have a great holiday all. Thanks, Gerry =20 From owner-freebsd-fs@FreeBSD.ORG Tue Dec 23 20:43:50 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A9CA810656AA for ; Tue, 23 Dec 2008 20:43:50 +0000 (UTC) (envelope-from matt@corp.spry.com) Received: from wf-out-1314.google.com (wf-out-1314.google.com [209.85.200.172]) by mx1.freebsd.org (Postfix) with ESMTP id 865EC8FC1E for ; Tue, 23 Dec 2008 20:43:50 +0000 (UTC) (envelope-from matt@corp.spry.com) Received: by wf-out-1314.google.com with SMTP id 24so4399011wfg.7 for ; Tue, 23 Dec 2008 12:43:50 -0800 (PST) Received: by 10.142.103.2 with SMTP id a2mr3289679wfc.164.1230065029491; Tue, 23 Dec 2008 12:43:49 -0800 (PST) Received: from imac24.simerson.net (c-67-168-10-190.hsd1.wa.comcast.net [67.168.10.190]) by mx.google.com with ESMTPS id 9sm29340525wfc.56.2008.12.23.12.43.48 (version=TLSv1/SSLv3 cipher=RC4-MD5); Tue, 23 Dec 2008 12:43:48 -0800 (PST) Message-Id: <1424BEB3-69FE-4BA2-884F-4862B3D7BCFD@corp.spry.com> From: Matt Simerson To: freebsd-fs@freebsd.org In-Reply-To: <494AE6F4.30506@modulus.org> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Date: Tue, 23 Dec 2008 12:43:47 -0800 References: <22C8092E-210F-4E91-AA09-CFD38966975C@spry.com> <494AE6F4.30506@modulus.org> X-Mailer: Apple Mail (2.930.3) Subject: Re: ZFS performance gains real or imaginary? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Dec 2008 20:43:50 -0000 On Dec 18, 2008, at 4:12 PM, Andrew Snow wrote: >> If so, then I really should be upgrading my production ZFS servers >> to the latest -HEAD. > > Thats correct, that is the only way to get the best working version > of ZFS. Of course, then everything is unstable and broken - eg. > SMBFS became unusable for me and would crash the server. I got bit by the ARP bug (kern/129730) which really annoyed our network manager. After applying a patch for that, I've got a working kernel and upgraded ZFS. Unfortunately, the newer kernel hangs much more frequently. Previously I was getting nearly a month between reboots. Now I don't even get 1 day between hangs. Worse yet, I upgraded the ZFS version of the pools, to see if that would make any difference. It did not, and now I can't revert. :-( I have these settings in /boot/loader.conf vm.kmem_size="1536M" vm.kmem_size_max="1536M" vfs.zfs.arc_max="100M" I have also experimented with vfs.zfs.prefetch_disable, vfs.zfs.arc_min in the past, and I'm open to suggestions on what might help under this workload (multiple concurrent rsync processes from remote systems to this one). Matt Specs: SuperMicro 24 disk, 8-core Xeon, 16GB RAM, 2 x 12 disk HW RAID striped (RAID 0). back01# uname -a FreeBSD back01.int.spry.com 8.0-CURRENT FreeBSD 8.0-CURRENT #2: Fri Dec 19 15:37:12 PST 2008 root@back01.int.spry.com:/usr/obj/usr/src/ sys/BACK01 amd64 back01# zpool list NAME SIZE USED AVAIL CAP HEALTH ALTROOT back01 18.1T 16.5T 1.60T 91% ONLINE - back01# zpool status pool: back01 state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM back01 ONLINE 0 0 0 da0 ONLINE 0 0 0 da1 ONLINE 0 0 0 errors: No known data errors back01# zpool upgrade This system is currently running ZFS pool version 13. All pools are formatted using this version. From owner-freebsd-fs@FreeBSD.ORG Wed Dec 24 04:07:42 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8A7911065673 for ; Wed, 24 Dec 2008 04:07:42 +0000 (UTC) (envelope-from mcdouga9@egr.msu.edu) Received: from mx.egr.msu.edu (surfnturf.egr.msu.edu [35.9.37.164]) by mx1.freebsd.org (Postfix) with ESMTP id 5CEE38FC18 for ; Wed, 24 Dec 2008 04:07:42 +0000 (UTC) (envelope-from mcdouga9@egr.msu.edu) Received: from localhost (localhost [127.0.0.1]) by mx.egr.msu.edu (Postfix) with ESMTP id C80AC71EDBC; Tue, 23 Dec 2008 22:48:12 -0500 (EST) X-Virus-Scanned: amavisd-new at egr.msu.edu Received: from mx.egr.msu.edu ([127.0.0.1]) by localhost (surfnturf.egr.msu.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id WA+uYhJEBLYr; Tue, 23 Dec 2008 22:48:12 -0500 (EST) Received: from localhost (daemon.egr.msu.edu [35.9.44.65]) by mx.egr.msu.edu (Postfix) with ESMTP id 918A371ED5E; Tue, 23 Dec 2008 22:48:12 -0500 (EST) Received: by localhost (Postfix, from userid 21281) id 540B7B6A; Tue, 23 Dec 2008 22:48:12 -0500 (EST) Date: Tue, 23 Dec 2008 22:48:12 -0500 From: Adam McDougall To: Matt Simerson Message-ID: <20081224034812.GR87625@egr.msu.edu> References: <22C8092E-210F-4E91-AA09-CFD38966975C@spry.com> <494AE6F4.30506@modulus.org> <1424BEB3-69FE-4BA2-884F-4862B3D7BCFD@corp.spry.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1424BEB3-69FE-4BA2-884F-4862B3D7BCFD@corp.spry.com> User-Agent: Mutt/1.5.18 (2008-05-17) Cc: freebsd-fs@freebsd.org Subject: Re: ZFS performance gains real or imaginary? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Dec 2008 04:07:42 -0000 On Tue, Dec 23, 2008 at 12:43:47PM -0800, Matt Simerson wrote: On Dec 18, 2008, at 4:12 PM, Andrew Snow wrote: >> If so, then I really should be upgrading my production ZFS servers >> to the latest -HEAD. > > Thats correct, that is the only way to get the best working version > of ZFS. Of course, then everything is unstable and broken - eg. > SMBFS became unusable for me and would crash the server. I got bit by the ARP bug (kern/129730) which really annoyed our network manager. After applying a patch for that, I've got a working kernel and upgraded ZFS. Unfortunately, the newer kernel hangs much more frequently. Previously I was getting nearly a month between reboots. Now I don't even get 1 day between hangs. Worse yet, I upgraded the ZFS version of the pools, to see if that would make any difference. It did not, and now I can't revert. :-( I have these settings in /boot/loader.conf vm.kmem_size="1536M" vm.kmem_size_max="1536M" vfs.zfs.arc_max="100M" I have also experimented with vfs.zfs.prefetch_disable, vfs.zfs.arc_min in the past, and I'm open to suggestions on what might help under this workload (multiple concurrent rsync processes from remote systems to this one). Matt Specs: SuperMicro 24 disk, 8-core Xeon, 16GB RAM, 2 x 12 disk HW RAID striped (RAID 0). back01# uname -a FreeBSD back01.int.spry.com 8.0-CURRENT FreeBSD 8.0-CURRENT #2: Fri Dec 19 15:37:12 PST 2008 root@back01.int.spry.com:/usr/obj/usr/src/ sys/BACK01 amd64 Can you try: vm.kmem_size=2G vm.kmem_size_max=2G vfs.zfs.arc_max=512M This has been working for me on one amd64 system that only has 2G of ram but had similar problem frequency to yours. I don't know if its coincidence with the data that I am rsyncing lately, but: 10:47PM up 22 days, 7:12 From owner-freebsd-fs@FreeBSD.ORG Thu Dec 25 06:35:01 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4ADCD106564A for ; Thu, 25 Dec 2008 06:35:01 +0000 (UTC) (envelope-from matt@corp.spry.com) Received: from wf-out-1314.google.com (wf-out-1314.google.com [209.85.200.169]) by mx1.freebsd.org (Postfix) with ESMTP id 256438FC18 for ; Thu, 25 Dec 2008 06:35:00 +0000 (UTC) (envelope-from matt@corp.spry.com) Received: by wf-out-1314.google.com with SMTP id 24so5244485wfg.7 for ; Wed, 24 Dec 2008 22:35:00 -0800 (PST) Received: by 10.143.18.21 with SMTP id v21mr3888707wfi.336.1230186900481; Wed, 24 Dec 2008 22:35:00 -0800 (PST) Received: from ?10.0.1.193? (c-67-168-10-190.hsd1.wa.comcast.net [67.168.10.190]) by mx.google.com with ESMTPS id 30sm20035717wff.12.2008.12.24.22.34.59 (version=TLSv1/SSLv3 cipher=RC4-MD5); Wed, 24 Dec 2008 22:34:59 -0800 (PST) Message-Id: <5C120CEB-6CEB-4722-BB23-7E4B83F779C2@corp.spry.com> From: Matt Simerson To: freebsd-fs@freebsd.org In-Reply-To: <20081225052903.GC87625@egr.msu.edu> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Date: Wed, 24 Dec 2008 22:35:11 -0800 References: <22C8092E-210F-4E91-AA09-CFD38966975C@spry.com> <494AE6F4.30506@modulus.org> <1424BEB3-69FE-4BA2-884F-4862B3D7BCFD@corp.spry.com> <20081224034812.GR87625@egr.msu.edu> <20081225052903.GC87625@egr.msu.edu> X-Mailer: Apple Mail (2.930.3) Subject: Re: ZFS performance gains real or imaginary? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Dec 2008 06:35:01 -0000 On Dec 24, 2008, at 9:29 PM, Adam McDougall wrote: >> On Wed, Dec 24, 2008 at 01:00:14PM -0800, Matt Simerson wrote: >> >> On Dec 23, 2008, at 7:48 PM, Adam McDougall wrote: >> >>>> On Tue, Dec 23, 2008 at 12:43:47PM -0800, Matt Simerson wrote: >>>> >>>>> On Dec 18, 2008, at 4:12 PM, Andrew Snow wrote: >>>>> >>>>>> If so, then I really should be upgrading my production ZFS >>>>>> servers >>>>>> to the latest -HEAD. >>>>> >>>>> Thats correct, that is the only way to get the best working >>>>> version >>>>> of ZFS. Of course, then everything is unstable and broken - eg. >>>>> SMBFS became unusable for me and would crash the server. >>>> >>>> Unfortunately, the newer kernel hangs much more frequently. >>>> >>>> I have these settings in /boot/loader.conf >>>> >>>> vm.kmem_size="1536M" >>>> vm.kmem_size_max="1536M" >>>> vfs.zfs.arc_max="100M" >>>> >>>> I have also experimented with vfs.zfs.prefetch_disable, >>>> vfs.zfs.arc_min in the past, and I'm open to suggestions on what >>>> might help under this workload (multiple concurrent rsync >>>> processes from remote systems to this one). >>> >>> Can you try: >>> >>> vm.kmem_size=2G >>> vm.kmem_size_max=2G >>> vfs.zfs.arc_max=512M >>> >>> This has been working for me on one amd64 system that only >>> has 2G of ram but had similar problem frequency to yours. I >>> don't know if its coincidence with the data that I am rsyncing >>> lately, but: 10:47PM up 22 days, 7:12 >> >> I made it 23 minutes. I've reduced my rsync concurrency to 1, so I'm >> not hitting the system nearly as hard but it seems not to matter. >> >> Other workloads, like a 'make buildworld' will complete with no >> problems. For whatever reason, rsync sessions of entire unix systems >> to my backup servers are very troublesome. >> >> Matt > > Ok. Since you have 16G of ram, I suppose you could try setting both > kmem > sizes to something like 8G to see if it makes a difference? I'm > getting > a feeling that even if we don't see an outright failure, it might be > deadlocking due to a kmem shortage. back01# w 10:17PM up 40 mins, 2 users, load averages: 4.20, 3.07, 1.74 This is with: vm.kmem_size="4G" vm.kmem_size_max="4G" vfs.zfs.arc_max="512M" I'll let it trundle along with that setting and see how long it lasts. Matt PS: These settings earlier today resulted in 12+ hours of uptime, until I rebooted to test raising kmem_size to 4G. vm.kmem_size="2G" vm.kmem_size_max="2G" vfs.zfs.arc_max="512M" vfs.zfs.zil_disable=1 vfs.zfs.prefetch_disable=1 PPS: If/when it hangs with 4G, I'll raise it again to 6 or 8 GB and see how long it lasts. Whatever pattern emerges might be useful for Pawel. From owner-freebsd-fs@FreeBSD.ORG Sat Dec 27 03:22:14 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E59521065670 for ; Sat, 27 Dec 2008 03:22:14 +0000 (UTC) (envelope-from morganw@chemikals.org) Received: from warped.bluecherry.net (unknown [IPv6:2001:440:eeee:fffb::2]) by mx1.freebsd.org (Postfix) with ESMTP id 031788FC1A for ; Sat, 27 Dec 2008 03:22:14 +0000 (UTC) (envelope-from morganw@chemikals.org) Received: from volatile.chemikals.org (unknown [74.193.182.107]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by warped.bluecherry.net (Postfix) with ESMTPSA id 96702A31A159 for ; Fri, 26 Dec 2008 21:22:12 -0600 (CST) Received: from localhost (morganw@localhost [127.0.0.1]) by volatile.chemikals.org (8.14.3/8.14.3) with ESMTP id mBR3M9pk002366 for ; Fri, 26 Dec 2008 21:22:10 -0600 (CST) (envelope-from morganw@chemikals.org) Date: Fri, 26 Dec 2008 21:22:09 -0600 (CST) From: Wes Morgan To: freebsd-fs@freebsd.org In-Reply-To: <9461581F-F354-486D-961D-3FD5B1EF007C@rabson.org> Message-ID: References: <9461581F-F354-486D-961D-3FD5B1EF007C@rabson.org> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Subject: zpool resilver restarting X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Dec 2008 03:22:15 -0000 I just did a zpool replace on a new drive, and now it's resilvering. Only, when it gets about 20mb resilvered it restarts. I can see all the drive activity simply halting for a period then resuming in gstat. I see some bugs in the opensolaris tracker about this, but no resolutions. It doesn't seem to be related to calling "zpool status" because I can watch gstat and see it restarting... Anyone seen this before, and hopefully have a workaround...? The pool lost a drive on Wednesday and was running with a device missing, however due to the device numbering changing on the scsi bus, I had to export/import the pool to get it to come up, the same for after replacing it. [morganw@volatile:~$]: zpool status media (hangs a bit) pool: media state: DEGRADED status: One or more devices is currently being resilvered. The pool will continue to function, possibly in a degraded state. action: Wait for the resilver to complete. scrub: resilver in progress for 0h1m, 0.01% done, 368h10m to go config: NAME STATE READ WRITE CKSUM media DEGRADED 0 0 0 raidz2 DEGRADED 0 0 0 da2 ONLINE 0 0 0 20.3M resilvered da6 ONLINE 0 0 0 20.3M resilvered da0 ONLINE 0 0 0 20.3M resilvered da7 ONLINE 0 0 0 17.0M resilvered da1 ONLINE 0 0 0 20.2M resilvered da3 ONLINE 0 0 0 20.2M resilvered da5 ONLINE 0 0 0 19.2M resilvered replacing DEGRADED 0 0 0 17628927049345412941 UNAVAIL 3 3.91K 0 was /dev/da4 da4 ONLINE 0 0 0 22.6M resilvered errors: No known data errors [morganw@volatile:~$]: zpool status media pool: media state: DEGRADED status: One or more devices is currently being resilvered. The pool will continue to function, possibly in a degraded state. action: Wait for the resilver to complete. scrub: resilver in progress for 0h0m, 0.00% done, 766h21m to go config: NAME STATE READ WRITE CKSUM media DEGRADED 0 0 0 raidz2 DEGRADED 0 0 0 da2 ONLINE 0 0 0 738K resilvered da6 ONLINE 0 0 0 742K resilvered da0 ONLINE 0 0 0 739K resilvered da7 ONLINE 0 0 0 563K resilvered da1 ONLINE 0 0 0 732K resilvered da3 ONLINE 0 0 0 734K resilvered da5 ONLINE 0 0 0 738K resilvered replacing DEGRADED 0 0 0 17628927049345412941 UNAVAIL 3 4.67K 0 was /dev/da4 da4 ONLINE 0 0 0 848K resilvered errors: No known data errors From owner-freebsd-fs@FreeBSD.ORG Sat Dec 27 03:57:08 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 91F791065673 for ; Sat, 27 Dec 2008 03:57:08 +0000 (UTC) (envelope-from morganw@chemikals.org) Received: from warped.bluecherry.net (unknown [IPv6:2001:440:eeee:fffb::2]) by mx1.freebsd.org (Postfix) with ESMTP id 0B88B8FC13 for ; Sat, 27 Dec 2008 03:57:08 +0000 (UTC) (envelope-from morganw@chemikals.org) Received: from volatile.chemikals.org (unknown [74.193.182.107]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by warped.bluecherry.net (Postfix) with ESMTPSA id 7A09E84CB115 for ; Fri, 26 Dec 2008 21:57:05 -0600 (CST) Received: from localhost (morganw@localhost [127.0.0.1]) by volatile.chemikals.org (8.14.3/8.14.3) with ESMTP id mBR3v33X004269 for ; Fri, 26 Dec 2008 21:57:03 -0600 (CST) (envelope-from morganw@chemikals.org) Date: Fri, 26 Dec 2008 21:57:03 -0600 (CST) From: Wes Morgan To: freebsd-fs@freebsd.org In-Reply-To: Message-ID: References: <9461581F-F354-486D-961D-3FD5B1EF007C@rabson.org> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Subject: Re: zpool resilver restarting X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Dec 2008 03:57:08 -0000 On Fri, 26 Dec 2008, Wes Morgan wrote: > I just did a zpool replace on a new drive, and now it's resilvering. > Only, when it gets about 20mb resilvered it restarts. I can see all the drive > activity simply halting for a period then resuming in gstat. I see some bugs > in the opensolaris tracker about this, but no resolutions. It doesn't seem to > be related to calling "zpool status" because I can watch gstat and see it > restarting... Anyone seen this before, and hopefully have a workaround...? > > The pool lost a drive on Wednesday and was running with a device missing, > however due to the device numbering changing on the scsi bus, I had to > export/import the pool to get it to come up, the same for after replacing it. Replying to myself with some more information. zpool history -l -i shows the scrub loop happening: 2008-12-26.21:39:46 [internal pool scrub done txg:6463875] complete=0 [user root on volatile] 2008-12-26.21:39:46 [internal pool scrub txg:6463875] func=1 mintxg=3 maxtxg=6463720 [user root on volatile] 2008-12-26.21:41:23 [internal pool scrub done txg:6463879] complete=0 [user root on volatile] 2008-12-26.21:41:23 [internal pool scrub txg:6463879] func=1 mintxg=3 maxtxg=6463720 [user root on volatile] 2008-12-26.21:43:00 [internal pool scrub done txg:6463883] complete=0 [user root on volatile] 2008-12-26.21:43:00 [internal pool scrub txg:6463883] func=1 mintxg=3 maxtxg=6463720 [user root on volatile] 2008-12-26.21:44:38 [internal pool scrub done txg:6463887] complete=0 [user root on volatile] 2008-12-26.21:44:38 [internal pool scrub txg:6463887] func=1 mintxg=3 maxtxg=6463720 [user root on volatile] From owner-freebsd-fs@FreeBSD.ORG Sat Dec 27 19:59:59 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5A8371065670 for ; Sat, 27 Dec 2008 19:59:59 +0000 (UTC) (envelope-from morganw@chemikals.org) Received: from warped.bluecherry.net (unknown [IPv6:2001:440:eeee:fffb::2]) by mx1.freebsd.org (Postfix) with ESMTP id 92C998FC16 for ; Sat, 27 Dec 2008 19:59:58 +0000 (UTC) (envelope-from morganw@chemikals.org) Received: from volatile.chemikals.org (unknown [74.193.182.107]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by warped.bluecherry.net (Postfix) with ESMTPSA id 180B8A2FB13A for ; Sat, 27 Dec 2008 13:59:55 -0600 (CST) Received: from localhost (morganw@localhost [127.0.0.1]) by volatile.chemikals.org (8.14.3/8.14.3) with ESMTP id mBRJxqQE001654 for ; Sat, 27 Dec 2008 13:59:52 -0600 (CST) (envelope-from morganw@chemikals.org) Date: Sat, 27 Dec 2008 13:59:52 -0600 (CST) From: Wes Morgan To: freebsd-fs@freebsd.org In-Reply-To: Message-ID: References: <9461581F-F354-486D-961D-3FD5B1EF007C@rabson.org> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Subject: zpool devices "stuck" (was zpool resilver restarting) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Dec 2008 19:59:59 -0000 On Fri, 26 Dec 2008, Wes Morgan wrote: > On Fri, 26 Dec 2008, Wes Morgan wrote: > >> I just did a zpool replace on a new drive, and now it's resilvering. >> Only, when it gets about 20mb resilvered it restarts. I can see all the >> drive activity simply halting for a period then resuming in gstat. I see >> some bugs in the opensolaris tracker about this, but no resolutions. It >> doesn't seem to be related to calling "zpool status" because I can watch >> gstat and see it restarting... Anyone seen this before, and hopefully have >> a workaround...? >> >> The pool lost a drive on Wednesday and was running with a device missing, >> however due to the device numbering changing on the scsi bus, I had to >> export/import the pool to get it to come up, the same for after replacing >> it. > > Replying to myself with some more information. zpool history -l -i shows the > scrub loop happening: > > 2008-12-26.21:39:46 [internal pool scrub done txg:6463875] complete=0 [user > root on volatile] > 2008-12-26.21:39:46 [internal pool scrub txg:6463875] func=1 mintxg=3 > maxtxg=6463720 [user root on volatile] > 2008-12-26.21:41:23 [internal pool scrub done txg:6463879] complete=0 [user > root on volatile] > 2008-12-26.21:41:23 [internal pool scrub txg:6463879] func=1 mintxg=3 > maxtxg=6463720 [user root on volatile] > 2008-12-26.21:43:00 [internal pool scrub done txg:6463883] complete=0 [user > root on volatile] > 2008-12-26.21:43:00 [internal pool scrub txg:6463883] func=1 mintxg=3 > maxtxg=6463720 [user root on volatile] > 2008-12-26.21:44:38 [internal pool scrub done txg:6463887] complete=0 [user > root on volatile] > 2008-12-26.21:44:38 [internal pool scrub txg:6463887] func=1 mintxg=3 > maxtxg=6463720 [user root on volatile] It seems that the resliver and drive replacement were "fighting" each other somehow. Detaching the new drive allowed the resilver to complete, but now I'm stuck with two nonexistent devices trying to replace each other, and I can't replace a device that is being replaced: replacing UNAVAIL 0 36.4K 0 insufficient replicas 17628927049345412941 FAULTED 0 0 0 was /dev/da4 5474360425105728553 FAULTED 0 0 0 was /dev/da4 errors: No known data errors So, how the heck do I cancel that replacement and restart it using /dev/da4?