From owner-freebsd-fs@FreeBSD.ORG Sun May 13 13:46:40 2007 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 192DC16A403; Sun, 13 May 2007 13:46:38 +0000 (UTC) (envelope-from freebsd@ruomad.net) Received: from postfix2-g20.free.fr (postfix2-g20.free.fr [212.27.60.43]) by mx1.freebsd.org (Postfix) with ESMTP id 60B1713C469; Sun, 13 May 2007 13:46:38 +0000 (UTC) (envelope-from freebsd@ruomad.net) Received: from smtp2-g19.free.fr (smtp2-g19.free.fr [212.27.42.28]) by postfix2-g20.free.fr (Postfix) with ESMTP id 4E22EFFD1CE; Sun, 13 May 2007 14:19:54 +0200 (CEST) Received: from [192.168.0.138] (vln78-1-82-238-160-33.fbx.proxad.net [82.238.160.33]) by smtp2-g19.free.fr (Postfix) with ESMTP id E648696AE6; Sun, 13 May 2007 15:19:26 +0200 (CEST) Message-ID: <46471067.1060905@ruomad.net> Date: Sun, 13 May 2007 15:19:35 +0200 From: Bruno Damour User-Agent: Thunderbird 2.0.0.0 (X11/20070429) MIME-Version: 1.0 To: Pawel Jakub Dawidek References: <20070409011723.GB74547@garage.freebsd.pl> <20070409094319.GB76673@garage.freebsd.pl> <46209D21.2010704@FreeBSD.org> <20070414102313.GC10527@garage.freebsd.pl> <462BCF46.6030704@infidyne.com> <20070422195818.GH52622@garage.freebsd.pl> <462BC4ED.1050003@infidyne.com> <20070422203556.GI52622@garage.freebsd.pl> In-Reply-To: <20070422203556.GI52622@garage.freebsd.pl> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-fs@FreeBSD.org, Stefan Esser , freebsd-current@FreeBSD.org Subject: Re: ZFS: amd64, devd, root file system. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 May 2007 13:46:40 -0000 Pawel Jakub Dawidek wrote: > On Sun, Apr 22, 2007 at 10:26:21PM +0200, Peter Schuller wrote: > >>> Try zfs_load="YES". If this is what you had, send me full dmesg. >>> >> Sorry, I had zfs_load="YES". Typo on my part. No typo in loader.conf. >> >> Do you have any suggestion on how to best grab the dmesg, given that >> when triggering this the root fs cannot be mounted? >> >> Or do you just want a dmesg from my system in general, with a matching >> loader.conf minus the vfs.root.mountfromt? >> >> If the latter, it is available here: >> >> http://distfiles.scode.org/mlref/freebsd-dmesg-zfs-glabel-7current.txt >> >> In this dmesg the ZFS pool version output preceeds the glabel tasting. I >> was so sure this was not the case. Either I was misstaken or this was >> different then for some reason. >> > > And this is the problem... ZFS tries only ones. I'll think it over and > see what can be done. Unfortunately ZFS doesn't use GEOM tasting > mechanism and I'd prefer not to modify it to do so, because the changes > may be huge. > > Hello, Sorry about the noise, but I wonder if anyone saw my previous post about zfs root not being found at once (I have to input it manually at mountroot> prompt) ? Thanks Bruno From owner-freebsd-fs@FreeBSD.ORG Sun May 13 17:56:38 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 1847416A404 for ; Sun, 13 May 2007 17:56:38 +0000 (UTC) (envelope-from mashtizadeh@gmail.com) Received: from nz-out-0506.google.com (nz-out-0506.google.com [64.233.162.224]) by mx1.freebsd.org (Postfix) with ESMTP id B719A13C447 for ; Sun, 13 May 2007 17:56:37 +0000 (UTC) (envelope-from mashtizadeh@gmail.com) Received: by nz-out-0506.google.com with SMTP id s1so1628239nze for ; Sun, 13 May 2007 10:56:37 -0700 (PDT) DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:references; b=oFwOdJXV2ytK0n7yw+dL84LVlrq6u8HcW6qWVaZcAdCuP/UsI6qF2c7EQwfq4SKlPWLTuhgvSsMAK8vrLzTw7zo5vZs6a6GuyFiIg7H2rP6gAfVVGDLWVS32RNGeGjqKm3RGCccKQjgr1oxfTIijfxIqkdS/sOaLvBYygRcZ/bM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:references; b=SI8emnv5MwiZmtUMM+fPH/FZK56d/WKkv3AwDqOcydVHBpc30a8IIEQX66bd9pBlOnZYZEuonsY0FoYY/GzIKtYcdZhWBL8qj7JvFykRosF3br3lkSOK0Am3Rp0v4Yw+sdOBZirLML7h41wlC5S8NsVmCmVF3zK9LbW5j26FEgs= Received: by 10.114.103.1 with SMTP id a1mr673757wac.1179077491272; Sun, 13 May 2007 10:31:31 -0700 (PDT) Received: by 10.114.158.15 with HTTP; Sun, 13 May 2007 10:31:30 -0700 (PDT) Message-ID: <440b3e930705131031v5e97db7fq486d8d17aeb9f622@mail.gmail.com> Date: Sun, 13 May 2007 13:31:30 -0400 From: "Ali Mashtizadeh" To: "Pawel Jakub Dawidek" In-Reply-To: <20070409153338.GH76673@garage.freebsd.pl> MIME-Version: 1.0 References: <20070409011723.GB74547@garage.freebsd.pl> <20070409094319.GB76673@garage.freebsd.pl> <70e8236f0704090808y5d305175wdc3cee5be1a26a9@mail.gmail.com> <20070409153338.GH76673@garage.freebsd.pl> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: base64 Content-Disposition: inline X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-fs@freebsd.org, Joao Barros , freebsd-current@freebsd.org Subject: Re: ZFS: amd64, devd, root file system. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 May 2007 17:56:38 -0000 V2VsbCBjYW4ndCB3ZSBidWlsZCBhIGJvb3QgbG9hZGVyIG9mZiBvZiB0aGVyZSBzcGVjaWZpY2F0 aW9uIGRvY3VtZW50PyBBbHNvLAp3ZSBwcm9iYWJseSBuZWVkIHRvIGltcHJvdmUgdGhlIGJvb3Qg bG9hZGVyIEkgdGhpbmsgc29tZXRoaW5nIHRvIHRoYXQgZWZmZWN0CndhcyBpbiB0aGUgc3VnZ2Vz dGVkIHByb2plY3RzIGxpc3QuCgpUaGFua3MgdG8gZXZlcnlvbmUgd2hvIGhlbHBlZCBtYWtlIGEg WkZTIHJvb3QgcG9zc2libGUuIEl0J3Mgbm90IHNvIGJhZApoYXZpbmcgYSBVRlMgYm9vdCBmaWxl IHN5c3RlbS4gSSdtIHN3aXRjaGluZyB0b2RheSA6LSkKCi0tIApBbGkgTWFzaHRpemFkZWgK2LnZ hNuMINmF2LTYqtuMINiy2KfYr9mHCgoKT24gNC85LzA3LCBQYXdlbCBKYWt1YiBEYXdpZGVrIDxw amRAZnJlZWJzZC5vcmc+IHdyb3RlOgo+Cj4gT24gTW9uLCBBcHIgMDksIDIwMDcgYXQgMDQ6MDg6 MjZQTSArMDEwMCwgSm9hbyBCYXJyb3Mgd3JvdGU6Cj4gPiBJIHdhcyBsb29raW5nIGF0IGhvdyBT b2xhcmlzIGdvdCBzdXBwb3J0IGZvciBib290aW5nIG9mZiBaRlMgYW5kIHRoZQo+ID4gcGF0Y2gg dG8gR1JVQiB0byBzdXBwb3J0IGl0Lgo+ID4gSXMgaXQgZmVhc2libGUgZm9yIEZyZWVCU0QncyBi b290IGxvYWRlcj8gV2hhdCB3b3VsZCBiZSB0aGUgbWFpbgo+ID4gaXNzdWU6IHRlY2huaWNhbCBv ciBsaWNlbnNpbmc/Cj4KPiBJIGRvbid0IGtub3cgaWYgdGhlcmUgYXJlIGxpY2Vuc2luZyBpc3N1 ZXMsIHByb2JhYmx5IHllcyBpZiB3ZSB3b3VsZAo+IGxpa2UgdG8gYWRkIFpGUyBzdXBwb3J0IHRv IG91ciBzdGFuZGFyZCBsb2FkZXIuCj4KPiBUaGUgYmlnZ2VzdCBwcm9ibGVtIEkgc2VlIGlzIGxh Y2sgb2YgbW90aXZhdGlvbiBvbiBteSBzaWRlLiBSZWFsbHkuIFdoeQo+IHNvbWVvbmUgd291bGQg bGlrZSB0byBrZWVwIGtlcm5lbCBvbiBaRlMgc28gbXVjaD8gVGhpcyB3b3VsZCBiZSBhIGh1Z2UK PiBhbW91bnQgb2Ygd29yaywgSSBleHBlY3QsIGFuZCB3aGF0IHdlIGdldCBpbiB0dXJuPyBPbiBT b2xhcmlzIHlvdSBjYW4KPiBvbmx5IGJvb3QgZnJvbSBhIHNpbmdsZS1kaXNrIHBvb2wgb3IgZnJv bSBhIG1pcnJvcmVkIHBvb2wuIElzIGl0IHJlYWxseQo+IHdvcnRoIHRoZSBlZmZvcnQgZm9yIHVz PyBJIG11Y2ggbW9yZSBwcmVmZXIgdG8gc3BlbmQgdGhlIHRpbWUgd29ya2luZyBvbgo+IHNvbWV0 aGluZyBtb3JlIHVzZWZ1bCB0aGFuIHRoYXQgYW5kIGtlZXAgbXkgc21hbGwgL2Jvb3QvIGZpbGUg c3lzdGVtCj4gcHJvdGVjdGVkIGJ5IGdtaXJyb3Igb24gVUZTIC0gd2l0aCB0aGlzIGFwcHJvYWNo IHRoZXJlIGFyZSBubwo+IGxpbWl0YXRpb25zIC0gd2UgY2FuIGtlZXAgb3VyIHJvb3QgZmlsZSBz eXN0ZW0gb24gY29tcHJlc3NlZCBSQUlELVoKPiBwb29sLgo+Cj4gT2YgY291cnNlIGlmIHNvbWVv bmUgaXMgd2lsbGluZyB0byB0cnkgd29ya2luZyBvbiB0aGlzLCBJJ20gaGFwcHkgdG8KPiBoZWxw Lgo+Cj4gLS0KPiBQYXdlbCBKYWt1YiBEYXdpZGVrICAgICAgICAgICAgICAgICAgICAgICBodHRw Oi8vd3d3LndoZWVsLnBsCj4gcGpkQEZyZWVCU0Qub3JnICAgICAgICAgICAgICAgICAgICAgICAg ICAgaHR0cDovL3d3dy5GcmVlQlNELm9yZwo+IEZyZWVCU0QgY29tbWl0dGVyICAgICAgICAgICAg ICAgICAgICAgICAgIEFtIEkgRXZpbD8gWWVzLCBJIEFtIQo+Cj4K From owner-freebsd-fs@FreeBSD.ORG Mon May 14 00:20:30 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id F2F9816A400 for ; Mon, 14 May 2007 00:20:29 +0000 (UTC) (envelope-from quetzal@zone3000.net) Received: from mx1.sitevalley.com (sitevalley.com [209.67.60.43]) by mx1.freebsd.org (Postfix) with SMTP id 8DEE313C447 for ; Mon, 14 May 2007 00:20:29 +0000 (UTC) (envelope-from quetzal@zone3000.net) Received: from zone3000.kharkov.ua (HELO localhost) (217.144.69.37) by 209.67.61.254 with SMTP; 13 May 2007 23:53:48 -0000 Date: Mon, 14 May 2007 02:53:21 +0300 From: Nikolay Pavlov To: freebsd-fs@freebsd.org Message-ID: <20070513235321.GA21268@zone3000.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Operating-System: FreeBSD 6.1-RELEASE-p10 User-Agent: mutt-ng/devel-r804 (FreeBSD) Subject: ZFS lock order reversal. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 May 2007 00:20:30 -0000 Hi. I am getting this lock order reversal while compiling ports. I am using a simple disk pool. quetzal@orion:~> zpool list <932> NAME SIZE USED AVAIL CAP HEALTH ALTROOT pool 11,6G 1,37G 10,3G 11% ONLINE - quetzal@orion:~> zpool status <933> pool: pool state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM pool ONLINE 0 0 0 ad0s3 ONLINE 0 0 0 errors: No known data errors quetzal@orion:~> zfs list <935> NAME USED AVAIL REFER MOUNTPOINT pool 1,37G 10,1G 18K /pool pool/distfiles 904M 10,1G 904M /usr/ports/distfiles pool/local 158M 10,1G 158M /usr/local pool/packages 55,0M 10,1G 55,0M /usr/ports/packages pool/ports 285M 10,1G 285M /usr/ports May 14 02:46:02 orion kernel: lock order reversal: May 14 02:46:02 orion kernel: 1st 0xc737da20 zfs:&dr->dt.di.dr_mtx (zfs:&dr->dt.di.dr_mtx) @ /usr/src/sys/modules/zfs/../../contrib/opensolaris/uts/common/ fs/zfs/dbuf.c:1866 May 14 02:46:02 orion kernel: 2nd 0xd111bcd0 zfs:&db->db_mtx (zfs:&db->db_mtx) @ /usr/src/sys/modules/zfs/../../contrib/opensolaris/uts/common/fs/zfs/dbuf. c:1888 May 14 02:46:02 orion kernel: KDB: stack backtrace: May 14 02:46:02 orion kernel: db_trace_self_wrapper(c0962c19) at db_trace_self_wrapper+0x25 May 14 02:46:02 orion kernel: kdb_backtrace(0,ffffffff,c0a72c70,c0a72400,c0a0de2c,...) at kdb_backtrace+0x29 May 14 02:46:02 orion kernel: witness_checkorder(d111bcd0,9,c85b1f8c,760) at witness_checkorder+0x586 May 14 02:46:02 orion kernel: _sx_xlock(d111bcd0,c85b1f8c,760,c837bd20,637,...) at _sx_xlock+0x52 May 14 02:46:02 orion kernel: dbuf_sync_list(c737da38,d0020e80,0,7,c8515000,...) at dbuf_sync_list+0x15b May 14 02:46:02 orion kernel: dbuf_sync_list(c4c559e0,d0020e80,257,3c,0,...) at dbuf_sync_list+0xde May 14 02:46:02 orion kernel: dnode_sync(c4c55910,d0020e80,c4b32898,c4c55910,30,...) at dnode_sync+0x3a8 May 14 02:46:02 orion kernel: dmu_objset_sync_dnodes(c8515000,d0020e80,637,0,ca1a5ac8,...) at dmu_objset_sync_dnodes+0x29 May 14 02:46:02 orion kernel: dmu_objset_sync(c4b32800,c486a000,d0020e80,c476a800,0,...) at dmu_objset_sync+0x11d May 14 02:46:02 orion kernel: dsl_pool_sync(c43b8200,637,0,c476a800,637,...) at dsl_pool_sync+0x13e May 14 02:46:02 orion kernel: spa_sync(c476a800,637,0,c43b82ac,c85b592d,...) at spa_sync+0x33f May 14 02:46:02 orion kernel: txg_sync_thread(c43b8200,e6a1ed38) at txg_sync_thread+0x183 May 14 02:46:02 orion kernel: fork_exit(c8584924,c43b8200,e6a1ed38) at fork_exit+0xac May 14 02:46:02 orion kernel: fork_trampoline() at fork_trampoline+0x8 May 14 02:46:02 orion kernel: --- trap 0, eip = 0, esp = 0xe6a1ed70, ebp = 0 --- May 14 02:46:02 orion kernel: lock order reversal: May 14 02:46:02 orion kernel: 1st 0xc4908520 zfs:&dr->dt.di.dr_mtx (zfs:&dr->dt.di.dr_mtx) @ /usr/src/sys/modules/zfs/../../contrib/opensolaris/uts/common/ fs/zfs/dbuf.c:1866 May 14 02:46:02 orion kernel: 2nd 0xd18faaa0 zfs:&db->db_mtx (zfs:&db->db_mtx) @ /usr/src/sys/modules/zfs/../../contrib/opensolaris/uts/common/fs/zfs/dbuf. c:1888 May 14 02:46:02 orion kernel: KDB: stack backtrace: May 14 02:46:02 orion kernel: db_trace_self_wrapper(c0962c19) at db_trace_self_wrapper+0x25 May 14 02:46:02 orion kernel: kdb_backtrace(0,ffffffff,c0a72108,c0a72400,c0a0de2c,...) at kdb_backtrace+0x29 May 14 02:46:02 orion kernel: witness_checkorder(d18faaa0,9,c85b1f8c,760) at witness_checkorder+0x586 May 14 02:46:02 orion kernel: _sx_xlock(d18faaa0,c85b1f8c,760,c81c2620,637,...) at _sx_xlock+0x52 May 14 02:46:02 orion kernel: dbuf_sync_list(c4908538,c9513380,c58bb520,637,0,...) at dbuf_sync_list+0x15b May 14 02:46:02 orion kernel: dbuf_sync_list(c793a810,c9513380,257,3c,0,...) at dbuf_sync_list+0xde May 14 02:46:02 orion kernel: dnode_sync(c793a740,c9513380,c4b32898,c793a740,30,...) at dnode_sync+0x3a8 May 14 02:46:02 orion kernel: dmu_objset_sync_dnodes(c8515000,c9513380,637,0,c7e2f678,...) at dmu_objset_sync_dnodes+0x29 May 14 02:46:02 orion kernel: dmu_objset_sync(c4b32800,c4982000,c9513380,c476a800,0,...) at dmu_objset_sync+0x11d May 14 02:46:02 orion kernel: dsl_pool_sync(c43b8200,637,0,c476a800,637,...) at dsl_pool_sync+0x13e May 14 02:46:02 orion kernel: spa_sync(c476a800,637,0,c43b82ac,c85b592d,...) at spa_sync+0x33f May 14 02:46:02 orion kernel: txg_sync_thread(c43b8200,e6a1ed38) at txg_sync_thread+0x183 May 14 02:46:02 orion kernel: fork_exit(c8584924,c43b8200,e6a1ed38) at fork_exit+0xac May 14 02:46:02 orion kernel: fork_trampoline() at fork_trampoline+0x8 May 14 02:46:02 orion kernel: --- trap 0, eip = 0, esp = 0xe6a1ed70, ebp = 0 --- vfs.zfs.dnlc.enable: 0 vfs.zfs.dnlc.max_nentries: 68506 vfs.zfs.dnlc.nentries: 0 vfs.zfs.dnlc.ncsize: 34253 vfs.zfs.arc_min: 16777216 vfs.zfs.arc_max: 167772160 vfs.zfs.mdcomp_disable: 0 vfs.zfs.prefetch_disable: 0 vfs.zfs.zio.taskq_threads: 0 vfs.zfs.recover: 0 vfs.zfs.vdev.cache.size: 10485760 vfs.zfs.vdev.cache.max: 16384 vfs.zfs.cache_flush_disable: 0 vfs.zfs.zil_disable: 0 vfs.zfs.debug: 0 kstat.zfs.misc.dnlcstats.hits: 0 kstat.zfs.misc.dnlcstats.misses: 0 kstat.zfs.misc.dnlcstats.negative_cache_hits: 0 kstat.zfs.misc.dnlcstats.enters: 0 kstat.zfs.misc.dnlcstats.double_enters: 0 kstat.zfs.misc.dnlcstats.purge_total_entries: 0 kstat.zfs.misc.dnlcstats.purge_all: 0 kstat.zfs.misc.dnlcstats.purge_vp: 0 kstat.zfs.misc.dnlcstats.purge_vfs: 0 kstat.zfs.misc.dnlcstats.purge_fs1: 0 kstat.zfs.misc.dnlcstats.pick_free: 0 kstat.zfs.misc.dnlcstats.pick_heuristic: 0 kstat.zfs.misc.dnlcstats.pick_last: 0 kstat.zfs.misc.arcstats.hits: 376113 kstat.zfs.misc.arcstats.misses: 84432 kstat.zfs.misc.arcstats.demand_data_hits: 232938 kstat.zfs.misc.arcstats.demand_data_misses: 3687 kstat.zfs.misc.arcstats.demand_metadata_hits: 123332 kstat.zfs.misc.arcstats.demand_metadata_misses: 75398 kstat.zfs.misc.arcstats.prefetch_data_hits: 296 kstat.zfs.misc.arcstats.prefetch_data_misses: 184 kstat.zfs.misc.arcstats.prefetch_metadata_hits: 19547 kstat.zfs.misc.arcstats.prefetch_metadata_misses: 5163 kstat.zfs.misc.arcstats.mru_hits: 44441 kstat.zfs.misc.arcstats.mru_ghost_hits: 1065 kstat.zfs.misc.arcstats.mfu_hits: 311926 kstat.zfs.misc.arcstats.mfu_ghost_hits: 5535 kstat.zfs.misc.arcstats.deleted: 245988 kstat.zfs.misc.arcstats.recycle_miss: 272594 kstat.zfs.misc.arcstats.mutex_miss: 129 kstat.zfs.misc.arcstats.evict_skip: 4839885 kstat.zfs.misc.arcstats.hash_elements: 6228 kstat.zfs.misc.arcstats.hash_elements_max: 13979 kstat.zfs.misc.arcstats.hash_collisions: 65501 kstat.zfs.misc.arcstats.hash_chains: 979 kstat.zfs.misc.arcstats.hash_chain_max: 6 kstat.zfs.misc.arcstats.p: 36156672 kstat.zfs.misc.arcstats.c: 46258688 kstat.zfs.misc.arcstats.c_min: 16777216 kstat.zfs.misc.arcstats.c_max: 167772160 kstat.zfs.misc.arcstats.size: 43229184 quetzal@orion:~> uname -a FreeBSD orion.zone3000.net 7.0-CURRENT FreeBSD 7.0-CURRENT #0: Fri May 11 01:37:28 EEST 2007 root@orion.zone3000.net:/usr/obj/usr/src/sys/GENERIC i386 -- ====================================================================== - Best regards, Nikolay Pavlov. <<<----------------------------------- ====================================================================== From owner-freebsd-fs@FreeBSD.ORG Mon May 14 07:07:15 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 680) id 0EF1416A406; Mon, 14 May 2007 07:07:15 +0000 (UTC) Date: Mon, 14 May 2007 07:07:15 +0000 From: Darren Reed To: Ali Mashtizadeh Message-ID: <20070514070715.GA82322@hub.freebsd.org> References: <20070409011723.GB74547@garage.freebsd.pl> <20070409094319.GB76673@garage.freebsd.pl> <70e8236f0704090808y5d305175wdc3cee5be1a26a9@mail.gmail.com> <20070409153338.GH76673@garage.freebsd.pl> <440b3e930705131031v5e97db7fq486d8d17aeb9f622@mail.gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <440b3e930705131031v5e97db7fq486d8d17aeb9f622@mail.gmail.com> User-Agent: Mutt/1.4.2.1i Cc: freebsd-fs@freebsd.org, Joao Barros , freebsd-current@freebsd.org, Pawel Jakub Dawidek Subject: Re: ZFS: amd64, devd, root file system. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 May 2007 07:07:15 -0000 > On 4/9/07, Pawel Jakub Dawidek wrote: ... > >The biggest problem I see is lack of motivation on my side. Really. Why > >someone would like to keep kernel on ZFS so much? This would be a huge > >amount of work, I expect, and what we get in turn? On Solaris you can > >only boot from a single-disk pool or from a mirrored pool. Is it really > >worth the effort for us? I much more prefer to spend the time working on > >something more useful than that and keep my small /boot/ file system > >protected by gmirror on UFS - with this approach there are no > >limitations - we can keep our root file system on compressed RAID-Z > >pool. Pawel, to answer your question... It is not a lot of fun having to support the kernel and 'some' filesystems being of a different type of filesystem to other parts (from a system admin perspective.) This is especially true of those filesystems that make up the "root". ZFS also raises another issue: why wouldn't you want the same data reliability for your root/boot volumes as for your data? I understand that you may not have enough drive/desire to do the work, but don't understate the value of having all your filesystems being of one type, especially if it means your root volume is dynamic in size. Darren From owner-freebsd-fs@FreeBSD.ORG Mon May 14 07:22:35 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id E531F16A403; Mon, 14 May 2007 07:22:35 +0000 (UTC) (envelope-from des@des.no) Received: from tim.des.no (tim.des.no [194.63.250.121]) by mx1.freebsd.org (Postfix) with ESMTP id 9CA4913C44C; Mon, 14 May 2007 07:22:35 +0000 (UTC) (envelope-from des@des.no) Received: from tim.des.no (localhost [127.0.0.1]) by spam.des.no (Postfix) with ESMTP id EC1C420A7; Mon, 14 May 2007 09:22:31 +0200 (CEST) X-Spam-Tests: AWL X-Spam-Learn: disabled X-Spam-Score: 0.0/3.0 X-Spam-Checker-Version: SpamAssassin 3.1.7 (2006-10-05) on tim.des.no Received: from dwp.des.no (des.no [80.203.243.180]) by smtp.des.no (Postfix) with ESMTP id DBC8520A6; Mon, 14 May 2007 09:22:31 +0200 (CEST) Received: by dwp.des.no (Postfix, from userid 1001) id C277D5649; Mon, 14 May 2007 09:22:31 +0200 (CEST) From: des@des.no (Dag-Erling =?utf-8?Q?Sm=C3=B8rgrav?=) To: Darren Reed References: <20070409011723.GB74547@garage.freebsd.pl> <20070409094319.GB76673@garage.freebsd.pl> <70e8236f0704090808y5d305175wdc3cee5be1a26a9@mail.gmail.com> <20070409153338.GH76673@garage.freebsd.pl> <440b3e930705131031v5e97db7fq486d8d17aeb9f622@mail.gmail.com> <20070514070715.GA82322@hub.freebsd.org> Date: Mon, 14 May 2007 09:22:31 +0200 In-Reply-To: <20070514070715.GA82322@hub.freebsd.org> (Darren Reed's message of "Mon\, 14 May 2007 07\:07\:15 +0000") Message-ID: <86646vyimg.fsf@dwp.des.no> User-Agent: Gnus/5.110006 (No Gnus v0.6) Emacs/21.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org, Pawel Jakub Dawidek , Joao Barros Subject: Re: ZFS: amd64, devd, root file system. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 May 2007 07:22:36 -0000 Darren Reed writes: > It is not a lot of fun having to support the kernel and 'some' > filesystems being of a different type of filesystem to other parts > (from a system admin perspective.) This is especially true of those > filesystems that make up the "root". I don't see what the problem is. I am perfectly content with having my root file system on UFS2 on a pair of mirror disks. Considering the amount of work which would be required to allow FreeBSD to boot from ZFS (which you apparently do not appreciate), I perfectly understand Pawel's choice. Remember that unlike Sun, we do not make the hardware our OS runs on, nor do we write the firmware for it. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-fs@FreeBSD.ORG Mon May 14 14:54:58 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 333D516A403 for ; Mon, 14 May 2007 14:54:58 +0000 (UTC) (envelope-from freebsd-fs@tychl.net) Received: from rwcrmhc14.comcast.net (rwcrmhc14.comcast.net [204.127.192.84]) by mx1.freebsd.org (Postfix) with ESMTP id 1CA1B13C4C3 for ; Mon, 14 May 2007 14:54:58 +0000 (UTC) (envelope-from freebsd-fs@tychl.net) Received: from masq.tychl.net (tychl.no-ip.org[67.174.137.176]) by comcast.net (rwcrmhc14) with ESMTP id <20070514144302m1400imrfle>; Mon, 14 May 2007 14:43:02 +0000 Received: from localhost (localhost [127.0.0.1]) by masq.tychl.net (Postfix) with ESMTP id 7F0E01CB4B; Mon, 14 May 2007 10:42:51 -0400 (EDT) X-Virus-Scanned: amavisd-new at tychl.net Received: from masq.tychl.net ([127.0.0.1]) by localhost (masq.tychl.net [127.0.0.1]) (amavisd-new, port 10024) with SMTP id tauz6yHZ8zzs; Mon, 14 May 2007 10:42:46 -0400 (EDT) Received: from [127.0.0.1] (67-39-225-23.ded.ameritech.net [67.39.225.23]) by masq.tychl.net (Postfix) with ESMTP id 34DD11CB4A; Mon, 14 May 2007 10:42:46 -0400 (EDT) Message-ID: <46487565.40205@tychl.net> Date: Mon, 14 May 2007 10:42:45 -0400 From: Nick Gustas User-Agent: Thunderbird 1.5.0.10 (Windows/20070221) MIME-Version: 1.0 To: staalebk@ifi.uio.no, freebsd-fs@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: quoted-printable Cc: Subject: RE: ZFS performance X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 May 2007 14:54:58 -0000 Sorry for the formatting, > *St=E5le Kristoffersen* staalebk at ifi.uio.no=20 > > /Wed May 2 01:58:27 UTC 2007/ > > * Previous message: infrequently used filesystem gets VERY slow > <003153.html> > * Next message: ZFS vs UFS2 overhead and may be a bug? <003155.html= > > * *Messages sorted by:* [ date ] [ thread ] > [ subject ] [ author ] > > > -----------------------------------------------------------------------= - > On 2007-04-28 at 02:31, St=E5le Kristoffersen wrote: > >/ cvsup'ed, and buildt a new kernel and now the problem is gone, sorry= about > />/ the noise. > / > Not entirely true. I still experience very strange (to me) issues. > > I'm trying to stream something off the server at about 100 KB/s. It's a > video-file located on zfs. I can see that the data pushed across the > network is about the correct speed (fluctuating around 100 KB/s). Now i= f I > start up 'zpool iostat -v 1' things start to look strange. it first rea= ds > a couple of hundres KB's, then about 1 MB the next second, then 3 MB, t= hen > 10 MB then 20 MB, before the cycle restart at around a 3-400 KB again. = The > numbers differ from run to run but the overall pattern is the same. > > This makes streaming several streams impossible as it is not able to > deliver enough data. (It should easily push 5 streams of 100 KB/s, righ= t?) > > I'm using samba, but the problem is there when transfering using proftp= d, > (limiting the transfer on the receiver side to 100 KB/s) > alltho the "pattern" is not as clear, it still reads _way_ more data th= an > is transferred. > > I'm not sure how to debug this, I'm willing to try anything! > Below is a snip from 'zpool iostat -v 1', notice that the file beeing > streamed is located on the last disc of the pool. (the pool was full, I > added one disc and therefor all new data ends up on that disc). It > completes two "cycles": > > capacity operations bandwidth > pool used avail read write read write > ---------- ----- ----- ----- ----- ----- ----- > stash 1.42T 76.3G 0 0 0 0 > ad14 298G 87.4M 0 0 0 0 > ad15 298G 174M 0 0 0 0 > ad8 298G 497M 0 0 0 0 > ad10s1d 340G 441M 0 0 0 0 > ad16 223G 75.1G 0 0 0 0 > ---------- ----- ----- ----- ----- ----- ----- > > capacity operations bandwidth > pool used avail read write read write > ---------- ----- ----- ----- ----- ----- ----- > stash 1.42T 76.3G 4 0 639K 0 > ad14 298G 87.4M 0 0 0 0 > ad15 298G 174M 0 0 0 0 > ad8 298G 497M 0 0 0 0 > ad10s1d 340G 441M 0 0 0 0 > ad16 223G 75.1G 4 0 639K 0 > ---------- ----- ----- ----- ----- ----- ----- > > capacity operations bandwidth > pool used avail read write read write > ---------- ----- ----- ----- ----- ----- ----- > stash 1.42T 76.3G 2 0 384K 0 > ad14 298G 87.4M 0 0 0 0 > ad15 298G 174M 0 0 0 0 > ad8 298G 497M 0 0 0 0 > ad10s1d 340G 441M 0 0 0 0 > ad16 223G 75.1G 2 0 384K 0 > ---------- ----- ----- ----- ----- ----- ----- > > capacity operations bandwidth > pool used avail read write read write > ---------- ----- ----- ----- ----- ----- ----- > stash 1.42T 76.3G 7 0 1023K 0 > ad14 298G 87.4M 0 0 0 0 > ad15 298G 174M 0 0 0 0 > ad8 298G 497M 0 0 0 0 > ad10s1d 340G 441M 0 0 0 0 > ad16 223G 75.1G 7 0 1023K 0 > ---------- ----- ----- ----- ----- ----- ----- > > capacity operations bandwidth > pool used avail read write read write > ---------- ----- ----- ----- ----- ----- ----- > stash 1.42T 76.3G 27 0 3.50M 0 > ad14 298G 87.4M 0 0 0 0 > ad15 298G 174M 0 0 0 0 > ad8 298G 497M 0 0 0 0 > ad10s1d 340G 441M 0 0 0 0 > ad16 223G 75.1G 27 0 3.50M 0 > ---------- ----- ----- ----- ----- ----- ----- > > capacity operations bandwidth > pool used avail read write read write > ---------- ----- ----- ----- ----- ----- ----- > stash 1.42T 76.3G 101 0 12.6M 0 > ad14 298G 87.4M 0 0 5.99K 0 > ad15 298G 174M 0 0 0 0 > ad8 298G 497M 0 0 0 0 > ad10s1d 340G 441M 0 0 0 0 > ad16 223G 75.1G 100 0 12.6M 0 > ---------- ----- ----- ----- ----- ----- ----- > > capacity operations bandwidth > pool used avail read write read write > ---------- ----- ----- ----- ----- ----- ----- > stash 1.42T 76.3G 127 0 16.0M 0 > ad14 298G 87.4M 0 0 0 0 > ad15 298G 174M 0 0 0 0 > ad8 298G 497M 0 0 0 0 > ad10s1d 340G 441M 0 0 0 0 > ad16 223G 75.1G 127 0 16.0M 0 > ---------- ----- ----- ----- ----- ----- ----- > > capacity operations bandwidth > pool used avail read write read write > ---------- ----- ----- ----- ----- ----- ----- > stash 1.42T 76.3G 2 0 384K 0 > ad14 298G 87.4M 0 0 0 0 > ad15 298G 174M 0 0 0 0 > ad8 298G 497M 0 0 0 0 > ad10s1d 340G 441M 0 0 0 0 > ad16 223G 75.1G 2 0 384K 0 > ---------- ----- ----- ----- ----- ----- ----- > > =20 snip! I see the same behavior that St=E5le is seeing, I can "fix" it by setting= =20 vfs.zfs.prefetch_disable=3D"1" in loader.conf. I'm assuming something in= =20 the prefetch code isn't quite right? I believe I saw similar behavior in=20 solaris 10 when playing with ZFS a few weeks ago, but I need to revisit=20 that machine or install opensolaris on this one before I can be sure. # uname -a FreeBSD 7.0-CURRENT-200705 FreeBSD 7.0-CURRENT-200705 #0: Fri May 11=20 14:41:37 UTC 2007 root@:/usr/src/sys/amd64/compile/ZFS amd64 load_zfs=3D"YES" in loader.conf CPU: AMD Athlon(tm) 64 X2 Dual Core Processor 3800+ (2008.94-MHz=20 K8-class CPU) usable memory =3D 1026355200 (978 MB) I'm doing basic speed testing so I have WITNESS and friends turned off,=20 as well as malloc.conf symlinked to 'aj' zpool is a gpt partition residing on a single sata seagate 400 # zpool status pool: big state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM big ONLINE 0 0 0 ad6p1 ONLINE 0 0 0 Results with ftp to a 6.2 client over 100mbit lan: vfs.zfs.prefetch_disable=3D"0" (default) netstat -w1 input (Total) output packets errs bytes packets errs bytes colls 17 0 1856 3 0 382 0 8 0 689 3 0 382 0 1232 0 81719 2422 0 3657607 0 3994 0 263727 7924 0 11995892 0 4108 0 271701 8130 0 12304874 0 2966 0 196035 5856 0 8861816 0 4101 0 271084 8129 0 12304594 0 3988 0 263471 7894 0 11943838 0 2702 0 179088 5340 0 8083084 0 4105 0 271326 8129 0 12304594 0 4101 0 270825 8129 0 12304594 0 2752 0 182146 5428 0 8213760 0 4102 0 271243 8130 0 12305076 0 4099 0 270849 8129 0 12307622 0 2729 0 180453 5406 0 8181484 0 4096 0 270583 8129 0 12304594 0 4109 0 271980 8130 0 12304594 0 2742 0 181391 5417 0 8192630 0 4106 0 271583 8129 0 12304594 0 4098 0 270617 8129 0 12309136 0 3128 0 207274 6189 0 9354050 0 3718 0 245621 7368 0 11159762 0 4102 0 271383 8129 0 12304594 0 3879 0 256447 7683 0 11620508 0 2977 0 197005 5892 0 8929640 0 4103 0 271242 8129 0 12304808 0 4109 0 271337 8130 0 12303080 0 1596 0 105573 3133 0 4721738 0 15 0 1668 4 0 448 0 15 0 1253 3 0 382 0 zpool iostat 1 capacity operations bandwidth pool used avail read write read write ---------- ----- ----- ----- ----- ----- ----- big 10.5G 276G 0 0 0 0 big 10.5G 276G 0 0 0 0 big 10.5G 276G 0 0 0 0 big 10.5G 276G 285 0 35.5M 0 big 10.5G 276G 106 0 13.3M 0 big 10.5G 276G 89 0 11.2M 0 big 10.5G 276G 225 0 28.1M 0 big 10.5G 276G 0 0 0 0 big 10.5G 276G 0 0 0 0 big 10.5G 276G 257 0 32.1M 0 big 10.5G 276G 0 0 0 0 big 10.5G 276G 0 0 0 0 big 10.5G 276G 257 0 32.1M 0 big 10.5G 276G 0 0 0 0 big 10.5G 276G 0 0 0 0 big 10.5G 276G 257 0 32.1M 0 big 10.5G 276G 0 0 0 0 big 10.5G 276G 0 0 0 0 big 10.5G 276G 235 0 29.3M 0 big 10.5G 276G 21 0 2.75M 0 big 10.5G 276G 0 0 0 0 big 10.5G 276G 133 0 16.7M 0 big 10.5G 276G 123 0 15.4M 0 big 10.5G 276G 0 0 0 0 big 10.5G 276G 35 0 4.50M 0 big 10.5G 276G 221 0 27.6M 0 big 10.5G 276G 0 0 0 0 big 10.5G 276G 0 0 0 0 big 10.5G 276G 257 0 32.1M 0 You can see how the network speeds fluctuate. Now, I reboot with vfs.zfs.prefetch_disable=3D"1" set in loader.conf # netstat -w1 input (Total) output packets errs bytes packets errs bytes colls 10 0 969 2 0 316 0 13 0 983 2 0 316 0 3890 0 257204 7672 0 11589854 0 4058 0 268449 8023 0 12125162 0 4063 0 268600 8029 0 12122918 0 4048 0 267395 8013 0 12095290 0 4062 0 268385 8020 0 12117482 0 4070 0 269165 8030 0 12121856 0 4062 0 268440 8021 0 12133816 0 4062 0 268387 8027 0 12125850 0 4059 0 268379 8028 0 12128878 0 4074 0 269529 8038 0 12151286 0 4064 0 268883 8032 0 12131174 0 4069 0 268763 8042 0 12155236 0 4061 0 268229 8033 0 12137728 0 4057 0 268353 8028 0 12133948 0 3953 0 261159 7814 0 11804028 0 4063 0 268331 8026 0 12122004 0 4072 0 269673 8032 0 12138958 0 4059 0 268214 8036 0 12137860 0 4065 0 268491 8043 0 12156648 0 4069 0 269000 8042 0 12146426 0 4056 0 267963 8033 0 12138546 0 4092 0 270247 8115 0 12282992 0 4101 0 271279 8129 0 12304594 0 758 0 50289 1465 0 2198524 0 8 0 657 3 0 382 0 14 0 1225 3 0 382 0 # zpool iostat 1 capacity operations bandwidth pool used avail read write read write ---------- ----- ----- ----- ----- ----- ----- big 10.7G 275G 0 0 0 0 big 10.7G 275G 0 0 0 0 big 10.7G 275G 88 0 11.1M 0 big 10.7G 275G 87 0 11.0M 0 big 10.7G 275G 88 0 11.1M 0 big 10.7G 275G 87 0 11.0M 0 big 10.7G 275G 88 0 11.1M 0 big 10.7G 275G 87 0 11.0M 0 big 10.7G 275G 88 0 11.1M 0 big 10.7G 275G 88 0 11.1M 0 big 10.7G 275G 87 0 11.0M 0 big 10.7G 275G 88 0 11.1M 0 big 10.7G 275G 88 0 11.1M 0 big 10.7G 275G 87 0 11.0M 0 big 10.7G 275G 88 0 11.1M 0 big 10.7G 275G 88 0 11.1M 0 big 10.7G 275G 85 0 10.7M 0 big 10.7G 275G 87 0 11.0M 0 big 10.7G 275G 88 0 11.1M 0 big 10.7G 275G 88 0 11.1M 0 big 10.7G 275G 87 0 11.0M 0 big 10.7G 275G 88 0 11.1M 0 big 10.7G 275G 88 0 11.1M 0 big 10.7G 275G 7 0 1023K 0 big 10.7G 275G 0 0 0 0 May not be the same total data in each test, I was using 8GB test file=20 and stopping the transfer after a bit, behavior is consistent though. ps: Pawel, thanks for all your work on ZFS for FreeBSD, I really=20 appreciate it! From owner-freebsd-fs@FreeBSD.ORG Mon May 14 19:08:40 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id C20C016A404; Mon, 14 May 2007 19:08:40 +0000 (UTC) (envelope-from peter.schuller@infidyne.com) Received: from mxfep04.bredband.com (mxfep04.bredband.com [195.54.107.79]) by mx1.freebsd.org (Postfix) with ESMTP id B7A7113C448; Mon, 14 May 2007 19:08:39 +0000 (UTC) (envelope-from peter.schuller@infidyne.com) Received: from ironport.bredband.com ([195.54.107.82] [195.54.107.82]) by mxfep04.bredband.com with ESMTP id <20070514190838.EGTN24095.mxfep04.bredband.com@ironport.bredband.com>; Mon, 14 May 2007 21:08:38 +0200 Received: from c-5416e555.03-51-73746f3.cust.bredbandsbolaget.se (HELO scode.mine.nu) ([85.229.22.84]) by ironport.bredband.com with ESMTP; 14 May 2007 21:08:38 +0200 Received: from scode.mine.nu (localhost [127.0.0.1]) by scode.mine.nu (Postfix) with ESMTP id CBB4B1BA17; Mon, 14 May 2007 21:08:37 +0200 (CEST) Message-ID: <4648B3AC.8060205@infidyne.com> Date: Mon, 14 May 2007 21:08:28 +0200 From: Peter Schuller User-Agent: Thunderbird 2.0.0.0 (X11/20070501) MIME-Version: 1.0 To: =?UTF-8?B?RGFnLUVybGluZyBTbcO4cmdyYXY=?= References: <20070409011723.GB74547@garage.freebsd.pl> <20070409094319.GB76673@garage.freebsd.pl> <70e8236f0704090808y5d305175wdc3cee5be1a26a9@mail.gmail.com> <20070409153338.GH76673@garage.freebsd.pl> <440b3e930705131031v5e97db7fq486d8d17aeb9f622@mail.gmail.com> <20070514070715.GA82322@hub.freebsd.org> <86646vyimg.fsf@dwp.des.no> In-Reply-To: <86646vyimg.fsf@dwp.des.no> X-Enigmail-Version: 0.95.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="------------enig4801035CAED92604AB29B499" Cc: Joao Barros , Pawel Jakub Dawidek , Darren Reed , freebsd-fs@freebsd.org, freebsd-current@freebsd.org Subject: grub, install cd and root-on-zfs (was: Re: ZFS: amd64, devd, root file system) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 May 2007 19:08:40 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig4801035CAED92604AB29B499 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable > Considering the amount of work which would be required to allow FreeBSD= > to boot from ZFS (which you apparently do not appreciate), I perfectly > understand Pawel's choice. I have already given up on using the FreeBSD boot loader on any system except in simple cases. I have had too many issues with the automatic re-writing of the boot configuration nuking partition tables, infinite chainload looping and whatnot. Nowadays I just keep a grub floppy in all my machines and boot off that; that way I know I will always be able to boot. But this also means that with grub supporting ZFS, I should be able to have my FreeBSD root on ZFS too. I would suggest that as a workaround for anyone who does want root on ZFS. However, in order for it to be truly seamless one would like to see support for it in the installer. Currently my procedure for setting up "everything but small root on ZFS" is: * install freebsd on disk A * parttion/slice disk B; set up glabel and gmirror for location independence * create ufs fs and zfs pool * move system over, mucking about with fstab * finally boot the new system This does complicate (1) automatic installs and (2) any situation where you don't have spare drives available for use as a temporary install destination. Actually; I have been wondering. Has there ever been any discussions w.r.t. providing a FreeBSD live CD with a fully populated and built /usr/src and fully working system? I know there is FreeSBIE and such, but in terms of an official installation CD/DVD for each release. This would be very useful for at least three reasons: (1) You can do stuff like root-on-ZFS or root-on-gmirror or whatever else you can think up, straight off the bat with no additional hassle and without anyone having to implement support in the installer. (2) There will be no unknown "magic" going on in the background; you will know exactly what you did to get things installed and running (make installworld distribution DESTDIR=3D... and set up boot loader), thus giving you more of a sense of control. (3) A fully populated LiveCD is much more useful for recovery/debugging than a super-stripped floppy-on-a-CD where you are missing basic tools. --=20 / Peter Schuller PGP userID: 0xE9758B7D or 'Peter Schuller ' Key retrieval: Send an E-Mail to getpgpkey@scode.org E-Mail: peter.schuller@infidyne.com Web: http://www.scode.org --------------enig4801035CAED92604AB29B499 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.3 (FreeBSD) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGSLO0DNor2+l1i30RCN7mAJ931Ox+I4Iv6BRAbTsgDJPLmS8WpgCePd/E lzZbjbXznRuch67/61rxkA0= =XkXC -----END PGP SIGNATURE----- --------------enig4801035CAED92604AB29B499-- From owner-freebsd-fs@FreeBSD.ORG Tue May 15 04:45:20 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 2BF7E16A403; Tue, 15 May 2007 04:45:20 +0000 (UTC) (envelope-from peter.schuller@infidyne.com) Received: from mxfep03.bredband.com (mxfep03.bredband.com [195.54.107.76]) by mx1.freebsd.org (Postfix) with ESMTP id 26B2913C455; Tue, 15 May 2007 04:45:18 +0000 (UTC) (envelope-from peter.schuller@infidyne.com) Received: from ironport.bredband.com ([195.54.107.82] [195.54.107.82]) by mxfep03.bredband.com with ESMTP id <20070515044518.GZML23113.mxfep03.bredband.com@ironport.bredband.com>; Tue, 15 May 2007 06:45:18 +0200 Received: from c-5416e555.03-51-73746f3.cust.bredbandsbolaget.se (HELO scode.mine.nu) ([85.229.22.84]) by ironport.bredband.com with ESMTP; 15 May 2007 06:45:18 +0200 Received: from scode.mine.nu (localhost [127.0.0.1]) by scode.mine.nu (Postfix) with ESMTP id 68F8B1BBF0; Tue, 15 May 2007 06:45:17 +0200 (CEST) Message-ID: <46493ADC.9010903@infidyne.com> Date: Tue, 15 May 2007 06:45:16 +0200 From: Peter Schuller User-Agent: Thunderbird 2.0.0.0 (X11/20070501) MIME-Version: 1.0 To: =?ISO-8859-1?Q?Dag-Erling_Sm=F8rgrav?= References: <20070409011723.GB74547@garage.freebsd.pl> <20070409094319.GB76673@garage.freebsd.pl> <70e8236f0704090808y5d305175wdc3cee5be1a26a9@mail.gmail.com> <20070409153338.GH76673@garage.freebsd.pl> <440b3e930705131031v5e97db7fq486d8d17aeb9f622@mail.gmail.com> <20070514070715.GA82322@hub.freebsd.org> <86646vyimg.fsf@dwp.des.no> <4648B3AC.8060205@infidyne.com> In-Reply-To: <4648B3AC.8060205@infidyne.com> X-Enigmail-Version: 0.95.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="------------enig4ADCD1C6FCFCCCDBEA0D4E94" Cc: freebsd-fs@freebsd.org, Joao Barros , Darren Reed , Pawel Jakub Dawidek , freebsd-current@freebsd.org Subject: Re: grub, install cd and root-on-zfs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 May 2007 04:45:20 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig4ADCD1C6FCFCCCDBEA0D4E94 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable > But this also means that with grub supporting ZFS, I should be able to > have my FreeBSD root on ZFS too. I would suggest that as a workaround > for anyone who does want root on ZFS. For the archives and innocent bystanders: This is me being an idiot. grub boots /boot/loader which still would need ZFS support, so forget that paragraph. --=20 / Peter Schuller PGP userID: 0xE9758B7D or 'Peter Schuller ' Key retrieval: Send an E-Mail to getpgpkey@scode.org E-Mail: peter.schuller@infidyne.com Web: http://www.scode.org --------------enig4ADCD1C6FCFCCCDBEA0D4E94 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.3 (FreeBSD) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGSTrdDNor2+l1i30RCHBLAKCWryLkkxRubhGzCI3NguLNrwePFgCgwE/a YNz5S79taIsC3cDmh/kJ0kA= =5xux -----END PGP SIGNATURE----- --------------enig4ADCD1C6FCFCCCDBEA0D4E94-- From owner-freebsd-fs@FreeBSD.ORG Tue May 15 06:15:32 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 7D4A616A406; Tue, 15 May 2007 06:15:32 +0000 (UTC) (envelope-from des@des.no) Received: from tim.des.no (tim.des.no [194.63.250.121]) by mx1.freebsd.org (Postfix) with ESMTP id 3300A13C480; Tue, 15 May 2007 06:15:32 +0000 (UTC) (envelope-from des@des.no) Received: from tim.des.no (localhost [127.0.0.1]) by spam.des.no (Postfix) with ESMTP id 6056F20A7; Tue, 15 May 2007 08:15:28 +0200 (CEST) X-Spam-Tests: AWL X-Spam-Learn: disabled X-Spam-Score: 0.0/3.0 X-Spam-Checker-Version: SpamAssassin 3.1.7 (2006-10-05) on tim.des.no Received: from dwp.des.no (des.no [80.203.243.180]) by smtp.des.no (Postfix) with ESMTP id D7AA320A6; Tue, 15 May 2007 08:15:27 +0200 (CEST) Received: by dwp.des.no (Postfix, from userid 1001) id BFD275156; Tue, 15 May 2007 08:15:27 +0200 (CEST) From: des@des.no (Dag-Erling =?utf-8?Q?Sm=C3=B8rgrav?=) To: Peter Schuller References: <20070409011723.GB74547@garage.freebsd.pl> <20070409094319.GB76673@garage.freebsd.pl> <70e8236f0704090808y5d305175wdc3cee5be1a26a9@mail.gmail.com> <20070409153338.GH76673@garage.freebsd.pl> <440b3e930705131031v5e97db7fq486d8d17aeb9f622@mail.gmail.com> <20070514070715.GA82322@hub.freebsd.org> <86646vyimg.fsf@dwp.des.no> <4648B3AC.8060205@infidyne.com> Date: Tue, 15 May 2007 08:15:27 +0200 In-Reply-To: <4648B3AC.8060205@infidyne.com> (Peter Schuller's message of "Mon\, 14 May 2007 21\:08\:28 +0200") Message-ID: <86wszawr28.fsf@dwp.des.no> User-Agent: Gnus/5.110006 (No Gnus v0.6) Emacs/21.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: Joao Barros , Pawel Jakub Dawidek , Darren Reed , freebsd-fs@freebsd.org, freebsd-current@freebsd.org Subject: Re: grub, install cd and root-on-zfs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 May 2007 06:15:32 -0000 Peter Schuller writes: > Dag-Erling Sm=C3=B8rgrav writes: > > Considering the amount of work which would be required to allow FreeBSD > > to boot from ZFS (which you apparently do not appreciate), I perfectly > > understand Pawel's choice. > I have already given up on using the FreeBSD boot loader on any system > except in simple cases. I have had too many issues with the automatic > re-writing of the boot configuration nuking partition tables, infinite > chainload looping and whatnot. Nowadays I just keep a grub floppy in all > my machines and boot off that; that way I know I will always be able to > boot. You are confusing boot stages. Booting from ZFS has nothing to do with boot0, which is what you are complaining about here. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-fs@FreeBSD.ORG Tue May 15 17:21:45 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 93E7816A403; Tue, 15 May 2007 17:21:45 +0000 (UTC) (envelope-from pj@smo.de) Received: from ilk.de (mx-out12.ilk.de [194.121.104.12]) by mx1.freebsd.org (Postfix) with ESMTP id 1066A13C448; Tue, 15 May 2007 17:21:44 +0000 (UTC) (envelope-from pj@smo.de) Received: from bologna.intern.smo.de (pool45.ka.ilk.net [212.86.194.45]) by ilk.de (8.13.4/8.13.4/ilk-relay) with ESMTP id l4FEIwJa031711; Tue, 15 May 2007 16:18:58 +0200 Received: from [192.168.153.208] (herdubreid.intern.smo.de [192.168.153.208]) by bologna.intern.smo.de (8.13.4+Sun/8.13.4) with ESMTP id l4FEGb50016729; Tue, 15 May 2007 16:16:42 +0200 (CEST) Message-ID: <4649C18D.4020801@smo.de> Date: Tue, 15 May 2007 16:19:57 +0200 From: Philipp Ost User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.13) Gecko/20070323 X-Accept-Language: de, en-us, en MIME-Version: 1.0 To: Peter Schuller References: <20070409011723.GB74547@garage.freebsd.pl> <20070409094319.GB76673@garage.freebsd.pl> <70e8236f0704090808y5d305175wdc3cee5be1a26a9@mail.gmail.com> <20070409153338.GH76673@garage.freebsd.pl> <440b3e930705131031v5e97db7fq486d8d17aeb9f622@mail.gmail.com> <20070514070715.GA82322@hub.freebsd.org> <86646vyimg.fsf@dwp.des.no> <4648B3AC.8060205@infidyne.com> In-Reply-To: <4648B3AC.8060205@infidyne.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Cc: Joao Barros , Pawel Jakub Dawidek , Darren Reed , freebsd-fs@freebsd.org, freebsd-current@freebsd.org, =?UTF-8?B?RGFnLUVybGluZyBTbcO4cmdyYXY=?= Subject: Re: grub, install cd and root-on-zfs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 May 2007 17:21:45 -0000 Peter Schuller wrote: [snipped a lot of text] > (3) A fully populated LiveCD is much more useful for recovery/debugging > than a super-stripped floppy-on-a-CD where you are missing basic tools. You know about the Fixit-mode of the installation CD? There may be some restrictions, but there are more tools available than on that 'super-stripped floppy-on-a-CD' ;-) HTH, Philipp -- www.familie-ost.info/~pj From owner-freebsd-fs@FreeBSD.ORG Wed May 16 10:29:06 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 14B8C16A400 for ; Wed, 16 May 2007 10:29:06 +0000 (UTC) (envelope-from albinootje@gmail.com) Received: from ug-out-1314.google.com (ug-out-1314.google.com [66.249.92.168]) by mx1.freebsd.org (Postfix) with ESMTP id A4DBE13C455 for ; Wed, 16 May 2007 10:29:05 +0000 (UTC) (envelope-from albinootje@gmail.com) Received: by ug-out-1314.google.com with SMTP id 71so62680ugh for ; Wed, 16 May 2007 03:29:04 -0700 (PDT) DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:user-agent:mime-version:to:subject:content-type:content-transfer-encoding; b=ECbgqBCWAwaNYU9LLn7YLW5b10wDzTtZGxUHcy5uURwa0AfYpOhuvq2CluP3RvcjNOuAS8nNXDNbyvOncDDPPeXg1dqH+MWPvGgAg3l1+PqnES8OUQRbKOQpRhkFaap/XRJ9OCmCfNf25oVjVTcAx3yWRS5NC0Ruf0cHJ/M3Xn4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:user-agent:mime-version:to:subject:content-type:content-transfer-encoding; b=l3BB3VJMUfk/lA65UFeJg0la26vn83L0hpIzLIPzvDDlpGxNEDXsE1xg2O2A7pVdvs4CBXqjGmNQevY18ueBxbwMS9ykUBKojnshmfRpQXLrB9ouVG9DabAGJWu6DeQ46eK5oyMl4rOgDdKwxxQTrRfSSkOjp6EP4PrdAbD1B8Y= Received: by 10.66.221.19 with SMTP id t19mr6623471ugg.1179311343926; Wed, 16 May 2007 03:29:03 -0700 (PDT) Received: from amandla2.amsterdam-wireless.nl ( [217.19.30.147]) by mx.google.com with ESMTP id q40sm6305484ugc.2007.05.16.03.29.02; Wed, 16 May 2007 03:29:03 -0700 (PDT) Message-ID: <464ADCE8.5040201@gmail.com> Date: Wed, 16 May 2007 12:28:56 +0200 From: albinootje User-Agent: Thunderbird 2.0.0.0 (X11/20070326) MIME-Version: 1.0 To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Subject: ZFS + snapshots !? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 May 2007 10:29:06 -0000 hi, i read about ZFS, and i am using it now on a machine with a pretty recent 7.0-CURRENT, and i am very happy and excited about having ZFS in FreeBSD, it's awesome ! i have tried some things from the excellent quickstart-list here : http://lists.freebsd.org/pipermail/freebsd-current/2007-April/070616.html everything works great, except i cannot find any .zfs dirs when i try to use snapshots [ root@testbak:~ ] # uname -v FreeBSD 7.0-CURRENT #6: Mon May 14 13:21:36 CEST 2007 [cut] [ root@testbak:~ ] # zfs list NAME USED AVAIL REFER MOUNTPOINT backup1 3.14G 43.9G 3.14G /backup1 backup2 14.0G 32.8G 18K /backup2 backup2/swap 16K 36.8G 16K - backup2/ufs 3.83M 42.8G 3.79M - backup2/ufs@today 41K - 3.79M - [ root@testbak:~ ] # zpool list NAME SIZE USED AVAIL CAP HEALTH ALTROOT backup1 95.5G 6.28G 89.2G 6% ONLINE - backup2 47.5G 3.98M 47.5G 0% ONLINE - zfs create -V 10g backup2/ufs newfs /dev/zvol/backup2/ufs mkdir /ufs mount /dev/zvol/backup2/ufs /ufs zfs snapshot backup2/ufs@today mkdir /ufs-today mount -r /dev/zvol/backup2/ufs@today /ufs-today gives this : mount_nfs: path@server syntax is deprecated, use server:path and later [udp] today:/dev/zvol/backup2/ufs: RPCPROG_NFS: RPC: Port mapper failure - RPC: Timed out what am i doing wrong concerning the snapshots and .zfs dirs which i don't find ? From owner-freebsd-fs@FreeBSD.ORG Wed May 16 12:21:54 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 81C4216A400 for ; Wed, 16 May 2007 12:21:54 +0000 (UTC) (envelope-from albinootje@gmail.com) Received: from hu-out-0506.google.com (hu-out-0506.google.com [72.14.214.226]) by mx1.freebsd.org (Postfix) with ESMTP id 1693F13C45B for ; Wed, 16 May 2007 12:21:53 +0000 (UTC) (envelope-from albinootje@gmail.com) Received: by hu-out-0506.google.com with SMTP id 38so3276huc for ; Wed, 16 May 2007 05:21:52 -0700 (PDT) DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:user-agent:mime-version:to:subject:references:in-reply-to:content-type:content-transfer-encoding; b=thz/cIKgtrHOxLBCPtSXaCyVsUtxt1/Ez13eukIepGvt35BpVisVFuVDu5jFyGFLrrlc7nCPoGb1kM7Mk2Mjk/h7/A67ybuECbEPH4zOscsUtuh5i5LOKrg7ASw5QP0Rch9ia17RTdiMq3sn4/VTN9MtSMUaWrtEnKncnkBKP4c= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:user-agent:mime-version:to:subject:references:in-reply-to:content-type:content-transfer-encoding; b=m7N78ziBHa4Ie+4r/fWubakl5Lkr1sR3rYj+fkfWch9w+1CAnPkTnbQIkbtB+iqhMovMG9vfDifhvEqrqVu7yelwS8JZFRQfdWeDgANqMmVc6OtQEyjbyuSdYNmJe+ysoP1nQFk2Fpd2YjlaqlqYb+B2k9ymjkeiQFoVuGl2Xz4= Received: by 10.66.232.9 with SMTP id e9mr6671933ugh.1179318112289; Wed, 16 May 2007 05:21:52 -0700 (PDT) Received: from amandla2.amsterdam-wireless.nl ( [217.19.30.147]) by mx.google.com with ESMTP id q1sm8696968uge.2007.05.16.05.21.50; Wed, 16 May 2007 05:21:51 -0700 (PDT) Message-ID: <464AF757.10807@gmail.com> Date: Wed, 16 May 2007 14:21:43 +0200 From: albinootje User-Agent: Thunderbird 2.0.0.0 (X11/20070326) MIME-Version: 1.0 To: freebsd-fs@freebsd.org References: <464ADCE8.5040201@gmail.com> In-Reply-To: <464ADCE8.5040201@gmail.com> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Subject: Re: ZFS + snapshots !? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 May 2007 12:21:54 -0000 albinootje wrote: > everything works great, except i cannot find any .zfs dirs when i try to > use snapshots hmm, after : zfs set snapdir=visible backup1 i've found them, i'll do some more RTFM to find out about the portmap-timeout error From owner-freebsd-fs@FreeBSD.ORG Wed May 16 16:32:09 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 4176516A405; Wed, 16 May 2007 16:32:09 +0000 (UTC) (envelope-from anderson@freebsd.org) Received: from mh1.centtech.com (moat3.centtech.com [64.129.166.50]) by mx1.freebsd.org (Postfix) with ESMTP id 12C2713C46C; Wed, 16 May 2007 16:32:08 +0000 (UTC) (envelope-from anderson@freebsd.org) Received: from neutrino.centtech.com (neutrino.centtech.com [10.177.171.220]) by mh1.centtech.com (8.13.8/8.13.8) with ESMTP id l4GGW6VU042903; Wed, 16 May 2007 11:32:06 -0500 (CDT) (envelope-from anderson@freebsd.org) Message-ID: <464B3206.7000007@freebsd.org> Date: Wed, 16 May 2007 11:32:06 -0500 From: Eric Anderson User-Agent: Thunderbird 2.0.0.0 (X11/20070420) MIME-Version: 1.0 To: Andrew Edwards References: <5230D3C40B842D4F9FB3CD368021BEF72F08F1@exchange-2.sandvine.com> In-Reply-To: <5230D3C40B842D4F9FB3CD368021BEF72F08F1@exchange-2.sandvine.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.88.4/3260/Wed May 16 08:17:18 2007 on mh1.centtech.com X-Virus-Status: Clean X-Spam-Status: No, score=-2.6 required=8.0 tests=AWL,BAYES_00 autolearn=ham version=3.1.6 X-Spam-Checker-Version: SpamAssassin 3.1.6 (2006-10-03) on mh1.centtech.com Cc: "freebsd-fs@freebsd.org" Subject: Re: Ufs dead-locks on freebsd 6.2 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 May 2007 16:32:09 -0000 On 05/16/07 11:08, Andrew Edwards wrote: > I have a system running a dual intel zeon 2.8Ghz with 4G of ram and > using an intel raid controller model SRCU42X which uses the amr driver. > I have had this server running 5.4 upgraded to 6.2 and was running fine > for several months and then after a normal reboot I've started having > all sorts of problems with what appears to be dead-locks in the > filesystem. This server is my backup server and I rsync files from > various servers onto this one fairly non-stop. If I stop the rsync's > the system appears to be stable although I did have a kernel core just > last night. > > When I have been able to observe the problem I ususally see one > filesystem become inaccessible, perhaps var but I'm not sure, and then > in a short period of time the whole system is inaccessible. Usually if > I startup just one of the rsync's within a couple of hours the system > will be un-usable. > > I did find this thread which seems to describe similar issues but this > is a different driver. > http://lists.freebsd.org/pipermail/freebsd-questions/2006-August/127835. > html > > Currently I'm running with debug.mpsafevfs=0, debug.mpsafenet=1and > debug.mpsafevm=0 but this doesn't seem to help. > > On perahps a related issue I have two other nearly identical systems > which were going to be upgrading to 6.2 as on 5.4 I am experiencing > deadlocks and when I hit ctrl-t I see the system is either stuck in ufs > or zoneinfo and I have not found very much information about zoneinfo. What's ctrl-t? > Does anyone have any suggestions on what I can look for or other tuning > options or had similar experiences? When you say deadlock, you mean no access at all, but pingable? Does it panic? Can you still run things like 'ps'? You should (if you haven't already) add debugging to your kernel, and make sure you are running 6-STABLE. A ps -auxl of a system would be great if you can get it. Otherwise, you'll need to get a crash dump. You said you have a kernel core, can you send the output of a full back trace to the list? Also - I've cc'ed freebsd-fs@, since it's more appropriate there. Eric From owner-freebsd-fs@FreeBSD.ORG Wed May 16 16:38:19 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 9537616A400 for ; Wed, 16 May 2007 16:38:19 +0000 (UTC) (envelope-from rick@kiwi-computer.com) Received: from kiwi-computer.com (keira.kiwi-computer.com [63.224.10.3]) by mx1.freebsd.org (Postfix) with SMTP id 2466F13C45A for ; Wed, 16 May 2007 16:38:18 +0000 (UTC) (envelope-from rick@kiwi-computer.com) Received: (qmail 42656 invoked by uid 2001); 16 May 2007 16:40:02 -0000 Date: Wed, 16 May 2007 11:40:02 -0500 From: "Rick C. Petty" To: Eric Anderson Message-ID: <20070516164002.GA42615@keira.kiwi-computer.com> References: <5230D3C40B842D4F9FB3CD368021BEF72F08F1@exchange-2.sandvine.com> <464B3206.7000007@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <464B3206.7000007@freebsd.org> User-Agent: Mutt/1.4.2.1i Cc: "freebsd-fs@freebsd.org" , Andrew Edwards Subject: Re: Ufs dead-locks on freebsd 6.2 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: rick-freebsd@kiwi-computer.com List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 May 2007 16:38:19 -0000 On Wed, May 16, 2007 at 11:32:06AM -0500, Eric Anderson wrote: > > > >On perahps a related issue I have two other nearly identical systems > >which were going to be upgrading to 6.2 as on 5.4 I am experiencing > >deadlocks and when I hit ctrl-t I see the system is either stuck in ufs > >or zoneinfo and I have not found very much information about zoneinfo. > > What's ctrl-t? I think he means sending SIGINFO to the running process. /bin/tcsh by default maps control-E to "go to end of line" and control-T to "send SIGINFO to current foreground process". -- Rick C. Petty From owner-freebsd-fs@FreeBSD.ORG Wed May 16 17:19:37 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id C7A7716A400; Wed, 16 May 2007 17:19:37 +0000 (UTC) (envelope-from kris@obsecurity.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id B6E9513C45D; Wed, 16 May 2007 17:19:37 +0000 (UTC) (envelope-from kris@obsecurity.org) Received: from obsecurity.dyndns.org (elvis.mu.org [192.203.228.196]) by elvis.mu.org (Postfix) with ESMTP id 84D401A3C19; Wed, 16 May 2007 10:20:29 -0700 (PDT) Received: by obsecurity.dyndns.org (Postfix, from userid 1000) id CDE4351406; Wed, 16 May 2007 13:19:36 -0400 (EDT) Date: Wed, 16 May 2007 13:19:36 -0400 From: Kris Kennaway To: Andrew Edwards Message-ID: <20070516171936.GA74455@xor.obsecurity.org> References: <20070516163305.GA73495@xor.obsecurity.org> <5230D3C40B842D4F9FB3CD368021BEF72F08F3@exchange-2.sandvine.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5230D3C40B842D4F9FB3CD368021BEF72F08F3@exchange-2.sandvine.com> User-Agent: Mutt/1.4.2.2i Cc: freebsd-fs@freebsd.org, freebsd-performance@freebsd.org Subject: Re: Ufs dead-locks on freebsd 6.2 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 May 2007 17:19:37 -0000 On Wed, May 16, 2007 at 01:16:27PM -0400, Andrew Edwards wrote: > Here's the backtrace from the last crash along with the output from show > alllocks when the system was deadlocked. I have been running > 6.2-release and compliled with makeoptions debug=-g, invariants, > invariant_support and witness. I will update to 6-STABLE add > diagnositc, debug_locks and debug_vfs_locks as per the handbook > recommendation and retry. > > Yes, when the system was un-usable I was still able to ping it. I have > the serial console setup as the default console so I can remotely access > the box and break into the debugger etc. > > (kgdb) bt > #0 doadump () at pcpu.h:165 > #1 0xc059b480 in boot (howto=260) at > /usr/src/sys/kern/kern_shutdown.c:409 > #2 0xc059b795 in panic (fmt=0xc0787b04 "Most recently used by %s\n") > at /usr/src/sys/kern/kern_shutdown.c:565 That's a memory use-after-free. Check the DEBUG_MEMGUARD infrastructure for debugging of this. Kris From owner-freebsd-fs@FreeBSD.ORG Wed May 16 17:28:37 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 108CD16A401 for ; Wed, 16 May 2007 17:28:37 +0000 (UTC) (envelope-from aedwards@sandvine.com) Received: from gw.sandvine.com (gw.sandvine.com [199.243.201.138]) by mx1.freebsd.org (Postfix) with ESMTP id 4D89013C457 for ; Wed, 16 May 2007 17:28:36 +0000 (UTC) (envelope-from aedwards@sandvine.com) Received: from exchange-2.sandvine.com ([192.168.16.12]) by gw.sandvine.com with Microsoft SMTPSVC(6.0.3790.1830); Wed, 16 May 2007 13:16:29 -0400 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: quoted-printable Date: Wed, 16 May 2007 13:16:27 -0400 Message-ID: <5230D3C40B842D4F9FB3CD368021BEF72F08F3@exchange-2.sandvine.com> In-Reply-To: <20070516163305.GA73495@xor.obsecurity.org> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Ufs dead-locks on freebsd 6.2 Thread-Index: AceX1+P2HSPmcMyKSLK15m7jv2eBpAAAVXqQ From: "Andrew Edwards" To: X-OriginalArrivalTime: 16 May 2007 17:16:29.0850 (UTC) FILETIME=[F17E47A0:01C797DD] Cc: freebsd-fs@freebsd.org Subject: RE: Ufs dead-locks on freebsd 6.2 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 May 2007 17:28:37 -0000 Here's the backtrace from the last crash along with the output from show alllocks when the system was deadlocked. I have been running 6.2-release and compliled with makeoptions debug=3D-g, invariants, invariant_support and witness. I will update to 6-STABLE add diagnositc, debug_locks and debug_vfs_locks as per the handbook recommendation and retry. Yes, when the system was un-usable I was still able to ping it. I have the serial console setup as the default console so I can remotely access the box and break into the debugger etc. (kgdb) bt #0 doadump () at pcpu.h:165 #1 0xc059b480 in boot (howto=3D260) at /usr/src/sys/kern/kern_shutdown.c:409 #2 0xc059b795 in panic (fmt=3D0xc0787b04 "Most recently used by %s\n") at /usr/src/sys/kern/kern_shutdown.c:565 #3 0xc06c4a4d in mtrash_ctor (mem=3D0xce74fa00, size=3D0, arg=3D0x0, flags=3D258) at /usr/src/sys/vm/uma_dbg.c:137 #4 0xc06c2c07 in uma_zalloc_arg (zone=3D0xc10615a0, udata=3D0x0, = flags=3D258) at /usr/src/sys/vm/uma_core.c:1850 #5 0xc0591416 in malloc (size=3D272, mtp=3D0xc07c32c0, flags=3D258) at uma.h:275 #6 0xc05edfab in __mnt_vnode_first (mvp=3D0xf3741c48, mp=3D0xcaa14cf8) at /usr/src/sys/kern/vfs_mount.c:1813 #7 0xc05f2467 in vfs_msync (mp=3D0xcaa14cf8, flags=3D2) at /usr/src/sys/kern/vfs_subr.c:2874 #8 0xc05f2bbd in sync_fsync (ap=3D0x0) at /usr/src/sys/kern/vfs_subr.c:3119 #9 0xc072f4ee in VOP_FSYNC_APV (vop=3D0x0, a=3D0xf3741cbc) at vnode_if.c:1020 #10 0xc05f097c in sync_vnode (bo=3D0xca854e90, td=3D0xca435000) at vnode_if.h:537 #11 0xc05f0bf1 in sched_sync () at /usr/src/sys/kern/vfs_subr.c:1698 #12 0xc0587248 in fork_exit (callout=3D0xc05f0a04 , = arg=3D0x0, frame=3D0xf3741d38) at /usr/src/sys/kern/kern_fork.c:821 #13 0xc070712c in fork_trampoline () at /usr/src/sys/i386/i386/exception.s:208 db> show alllocks Process 36596 (sshd) thread 0xd1238c00 (102406) exclusive sleep mutex vm object (standard object) r =3D 0 (0xce2c87bc) locked @ /usr/src/sys/vm/vm_object.c:446 exclusive sx user map r =3D 0 (0xd128060c) locked @ /usr/src/sys/vm/vm_map.c:307 Process 887 (sshd) thread 0xca7d2000 (100056) exclusive sleep mutex vm object (standard object) r =3D 0 (0xcb713ad4) locked @ /usr/src/sys/vm/vm_fault.c:297 exclusive sx user map r =3D 0 (0xcaae4734) locked @ /usr/src/sys/vm/vm_map.c:3074 db> show lockedvnods Locked vnodes 0xcaa78660: tag ufs, type VREG usecount 2, writecount 1, refcount 3 mountedhere 0 flags () v_object 0xc1046738 ref 0 pages 1596 lock type ufs: EXCL (count 1) by thread 0xca689c00 (pid 536) with 1 pending ino 494620, on dev amrd0s1d 0xcaa86110: tag ufs, type VREG usecount 1, writecount 1, refcount 3 mountedhere 0 flags () v_object 0xca85f738 ref 0 pages 44 lock type ufs: EXCL (count 1) by thread 0xca7d2780 (pid 715) ino 494633, on dev amrd0s1d 0xcabe4110: tag ufs, type VDIR usecount 12, writecount 0, refcount 14 mountedhere 0 flags () v_object 0xcab28840 ref 0 pages 1 lock type ufs: EXCL (count 1) by thread 0xca9bbd80 (pid 14253) with 3 pending ino 423947, on dev amrd0s1d 0xcb437990: tag ufs, type VREG usecount 1, writecount 1, refcount 2 mountedhere 0 flags () v_object 0xcb3c3dec ref 0 pages 4100 lock type ufs: EXCL (count 1) by thread 0xcaffac00 (pid 20868) ino 282640, on dev amrd0s1d 0xcb99e550: tag ufs, type VDIR usecount 2, writecount 0, refcount 4 mountedhere 0 flags () v_object 0xcef979cc ref 0 pages 1 lock type ufs: EXCL (count 1) by thread 0xca9bb600 (pid 881) with 1 pending ino 423987, on dev amrd0s1d 0xcfc97dd0: tag ufs, type VREG usecount 1, writecount 1, refcount 2 mountedhere 0 flags () v_object 0xcd4975ac ref 0 pages 1 lock type ufs: EXCL (count 1) by thread 0xcb748900 (pid 2518) ino 424275, on dev amrd0s1d 0xccad9aa0: tag ufs, type VREG usecount 1, writecount 1, refcount 3 mountedhere 0 flags () v_object 0xcf0c539c ref 0 pages 5 lock type ufs: EXCL (count 1) by thread 0xca7d1c00 (pid 600) ino 188446, on dev amrd0s1d 0xccb0f110: tag ufs, type VREG usecount 1, writecount 1, refcount 2 mountedhere 0 flags () v_object 0xcb4609cc ref 0 pages 1 lock type ufs: EXCL (count 1) by thread 0xcafbf480 (pid 11054) ino 424100, on dev amrd0s1d 0xcc501bb0: tag ufs, type VREG usecount 1, writecount 1, refcount 3 mountedhere 0 flags () v_object 0xcc7d19cc ref 0 pages 1 lock type ufs: EXCL (count 1) by thread 0xcb76d600 (pid 13743) with 1 pending ino 424279, on dev amrd0s1d 0xcf96b220: tag ufs, type VREG usecount 1, writecount 1, refcount 2 mountedhere 0 flags () v_object 0xcf135c60 ref 0 pages 1 lock type ufs: EXCL (count 1) by thread 0xcafbf900 (pid 29458) ino 424374, on dev amrd0s1d 0xcc5bbbb0: tag ufs, type VREG usecount 1, writecount 1, refcount 2 mountedhere 0 flags () v_object 0xcdd5b318 ref 0 pages 1 lock type ufs: EXCL (count 1) by thread 0xcaad9900 (pid 50782) ino 424276, on dev amrd0s1d 0xcec1d000: tag ufs, type VREG usecount 1, writecount 1, refcount 2 mountedhere 0 flags () v_object 0xcd3d7108 ref 0 pages 1 lock type ufs: EXCL (count 1) by thread 0xcb76dc00 (pid 59514) ino 424500, on dev amrd0s1d 0xcebe5110: tag ufs, type VREG usecount 1, writecount 1, refcount 2 mountedhere 0 flags () v_object 0xccee95ac ref 0 pages 1 lock type ufs: EXCL (count 1) by thread 0xcb650780 (pid 59975) ino 424509, on dev amrd0s1d 0xce0c1880: tag ufs, type VREG usecount 1, writecount 1, refcount 2 mountedhere 0 flags () v_object 0xca8b64a4 ref 0 pages 1 lock type ufs: EXCL (count 1) by thread 0xcb768a80 (pid 69466) ino 424555, on dev amrd0s1d 0xcf652110: tag ufs, type VREG usecount 1, writecount 1, refcount 2 mountedhere 0 flags () v_object 0xcf4a318c ref 0 pages 1 lock type ufs: EXCL (count 1) by thread 0xcaff8600 (pid 75577) ino 424579, on dev amrd0s1d 0xce282550: tag ufs, type VREG usecount 1, writecount 1, refcount 2 mountedhere 0 flags () v_object 0xcf261318 ref 0 pages 1 lock type ufs: EXCL (count 1) by thread 0xd0235a80 (pid 81734) ino 424927, on dev amrd0s1d 0xcc1d4dd0: tag ufs, type VREG usecount 1, writecount 1, refcount 2 mountedhere 0 flags () v_object 0xcd4a6630 ref 0 pages 1 lock type ufs: EXCL (count 1) by thread 0xcaccb900 (pid 81772) ino 424928, on dev amrd0s1d 0xcb820bb0: tag ufs, type VREG usecount 1, writecount 1, refcount 2 mountedhere 0 flags () v_object 0xcb251ad4 ref 0 pages 1 lock type ufs: EXCL (count 1) by thread 0xcadaf480 (pid 84037) ino 424935, on dev amrd0s1d 0xced5aaa0: tag ufs, type VREG usecount 1, writecount 1, refcount 2 mountedhere 0 flags () v_object 0xcb784210 ref 0 pages 1 lock type ufs: EXCL (count 1) by thread 0xd0236000 (pid 202) ino 425039, on dev amrd0s1d 0xcbe45220: tag ufs, type VREG usecount 1, writecount 1, refcount 2 mountedhere 0 flags () v_object 0xce55de70 ref 0 pages 1 lock type ufs: EXCL (count 1) by thread 0xcaad9a80 (pid 230) ino 425043, on dev amrd0s1d 0xcc098220: tag ufs, type VREG usecount 1, writecount 1, refcount 2 mountedhere 0 flags () v_object 0xce4a9dec ref 0 pages 1 lock type ufs: EXCL (count 1) by thread 0xcafbfd80 (pid 9902) ino 425093, on dev amrd0s1d 0xcd585110: tag ufs, type VREG usecount 1, writecount 1, refcount 2 mountedhere 0 flags () v_object 0xcf8c1e70 ref 0 pages 1 lock type ufs: EXCL (count 1) by thread 0xcaccba80 (pid 24017) ino 425144, on dev amrd0s1d 0xceeac000: tag ufs, type VREG usecount 1, writecount 1, refcount 2 mountedhere 0 flags () v_object 0xcb1225ac ref 0 pages 1 lock type ufs: EXCL (count 1) by thread 0xcaff5a80 (pid 24775) ino 425149, on dev amrd0s1d 0xcc549aa0: tag ufs, type VREG usecount 1, writecount 1, refcount 2 mountedhere 0 flags () v_object 0xcab2d318 ref 0 pages 1 lock type ufs: EXCL (count 1) by thread 0xcaff8a80 (pid 42358) ino 425227, on dev amrd0s1d 0xcc6f7000: tag ufs, type VREG usecount 1, writecount 1, refcount 2 mountedhere 0 flags () v_object 0xcfd4139c ref 0 pages 1 lock type ufs: EXCL (count 1) by thread 0xcafbf600 (pid 43117) ino 425230, on dev amrd0s1d 0xccc44bb0: tag ufs, type VREG usecount 1, writecount 1, refcount 2 mountedhere 0 flags () v_object 0xcfd18d68 ref 0 pages 1 lock type ufs: EXCL (count 1) by thread 0xcadafd80 (pid 42859) ino 425234, on dev amrd0s1d 0xcc7a7220: tag ufs, type VREG usecount 1, writecount 1, refcount 2 mountedhere 0 flags () v_object 0xcfedf420 ref 0 pages 1 lock type ufs: EXCL (count 1) by thread 0xcb76c600 (pid 48968) ino 425264, on dev amrd0s1d 0xcc693aa0: tag ufs, type VREG usecount 1, writecount 1, refcount 2 mountedhere 0 flags () v_object 0xcf92f738 ref 0 pages 1 lock type ufs: EXCL (count 1) by thread 0xcb655300 (pid 55381) ino 425286, on dev amrd0s1d 0xcbabf220: tag ufs, type VREG usecount 1, writecount 1, refcount 2 mountedhere 0 flags () v_object 0xcfe297bc ref 0 pages 1 lock type ufs: EXCL (count 1) by thread 0xd0234a80 (pid 63802) ino 425322, on dev amrd0s1d 0xcd760220: tag ufs, type VREG usecount 1, writecount 1, refcount 2 mountedhere 0 flags () v_object 0xcb11ac60 ref 0 pages 1 lock type ufs: EXCL (count 1) by thread 0xcadb0480 (pid 69938) ino 425348, on dev amrd0s1d 0xcc044990: tag ufs, type VREG usecount 1, writecount 1, refcount 2 mountedhere 0 flags () v_object 0xcaaff084 ref 0 pages 1 lock type ufs: EXCL (count 1) by thread 0xcadeda80 (pid 70418) ino 425360, on dev amrd0s1d 0xcc190660: tag ufs, type VREG usecount 1, writecount 1, refcount 2 mountedhere 0 flags () v_object 0xcfed9108 ref 0 pages 1 lock type ufs: EXCL (count 1) by thread 0xca7d2900 (pid 76803) ino 425378, on dev amrd0s1d 0xcc676330: tag ufs, type VREG usecount 1, writecount 1, refcount 2 mountedhere 0 flags () v_object 0xcf8c14a4 ref 0 pages 1 lock type ufs: EXCL (count 1) by thread 0xcaff8900 (pid 76841) ino 425384, on dev amrd0s1d 0xcf0ad110: tag ufs, type VREG usecount 1, writecount 1, refcount 2 mountedhere 0 flags () v_object 0xce53b5ac ref 0 pages 1 lock type ufs: EXCL (count 1) by thread 0xcadb0900 (pid 79849) ino 425394, on dev amrd0s1d 0xce4f6aa0: tag ufs, type VREG usecount 1, writecount 1, refcount 2 mountedhere 0 flags () v_object 0xce3cc4a4 ref 0 pages 1 lock type ufs: EXCL (count 1) by thread 0xcb767300 (pid 79620) ino 425402, on dev amrd0s1d 0xce80d110: tag ufs, type VREG usecount 1, writecount 1, refcount 2 mountedhere 0 flags () v_object 0xcf502108 ref 0 pages 1 lock type ufs: EXCL (count 1) by thread 0xcaff5d80 (pid 98225) ino 425478, on dev amrd0s1d 0xcd218990: tag ufs, type VREG usecount 1, writecount 1, refcount 2 mountedhere 0 flags () v_object 0xcf08e18c ref 0 pages 1 lock type ufs: EXCL (count 1) by thread 0xcaff5000 (pid 98241) ino 425482, on dev amrd0s1d 0xcbcb3440: tag ufs, type VREG usecount 1, writecount 1, refcount 2 mountedhere 0 flags () v_object 0xcf8a8738 ref 0 pages 1 lock type ufs: EXCL (count 1) by thread 0xcaff8180 (pid 1341) ino 425505, on dev amrd0s1d 0xcf6fe440: tag ufs, type VREG usecount 1, writecount 1, refcount 2 mountedhere 0 flags () v_object 0xcf88f39c ref 0 pages 1 lock type ufs: EXCL (count 1) by thread 0xcaad7900 (pid 4512) ino 425512, on dev amrd0s1d 0xcdd07aa0: tag ufs, type VREG usecount 1, writecount 1, refcount 2 mountedhere 0 flags () v_object 0xcf7e9a50 ref 0 pages 1 lock type ufs: EXCL (count 1) by thread 0xcadaf600 (pid 4464) ino 425513, on dev amrd0s1d 0xcc18eaa0: tag ufs, type VREG usecount 1, writecount 1, refcount 2 mountedhere 0 flags () v_object 0xce2fa108 ref 0 pages 1 lock type ufs: EXCL (count 1) by thread 0xcafbf300 (pid 13669) ino 425549, on dev amrd0s1d 0xcb9e1440: tag ufs, type VREG usecount 1, writecount 1, refcount 2 mountedhere 0 flags () v_object 0xcfbea39c ref 0 pages 1 lock type ufs: EXCL (count 1) by thread 0xcb64e480 (pid 13656) ino 425555, on dev amrd0s1d 0xcb8a8bb0: tag ufs, type VREG usecount 1, writecount 1, refcount 2 mountedhere 0 flags () v_object 0xcf6565ac ref 0 pages 1 lock type ufs: EXCL (count 1) by thread 0xcadb0180 (pid 22845) ino 425596, on dev amrd0s1d 0xccc47660: tag ufs, type VREG usecount 1, writecount 1, refcount 2 mountedhere 0 flags () v_object 0xcf551b58 ref 0 pages 1 lock type ufs: EXCL (count 1) by thread 0xcb64e180 (pid 22870) ino 425597, on dev amrd0s1d 0xcec8ecc0: tag ufs, type VREG usecount 1, writecount 1, refcount 2 mountedhere 0 flags () v_object 0xcab61948 ref 0 pages 1 lock type ufs: EXCL (count 1) by thread 0xd0234600 (pid 32036) ino 425633, on dev amrd0s1d 0xcdca1bb0: tag ufs, type VREG usecount 2, writecount 2, refcount 888 mountedhere 0 flags () v_object 0xcaf23294 ref 0 pages 41308 lock type ufs: EXCL (count 1) by thread 0xcadb0a80 (pid 5541) ino 130855246, on dev amrd1s1d=20 > -----Original Message----- > From: Kris Kennaway [mailto:kris@obsecurity.org]=20 > Sent: Wednesday, May 16, 2007 12:33 PM > To: Andrew Edwards > Cc: freebsd-performance@freebsd.org > Subject: Re: Ufs dead-locks on freebsd 6.2 >=20 > On Wed, May 16, 2007 at 12:08:24PM -0400, Andrew Edwards wrote: > > I have a system running a dual intel zeon 2.8Ghz with 4G of ram and=20 > > using an intel raid controller model SRCU42X which uses the=20 > amr driver. > > I have had this server running 5.4 upgraded to 6.2 and was running=20 > > fine for several months and then after a normal reboot I've started=20 > > having all sorts of problems with what appears to be=20 > dead-locks in the=20 > > filesystem. This server is my backup server and I rsync files from=20 > > various servers onto this one fairly non-stop. If I stop=20 > the rsync's=20 > > the system appears to be stable although I did have a=20 > kernel core just=20 > > last night. > >=20 > > When I have been able to observe the problem I ususally see one=20 > > filesystem become inaccessible, perhaps var but I'm not=20 > sure, and then=20 > > in a short period of time the whole system is inaccessible.=20 > Usually=20 > > if I startup just one of the rsync's within a couple of hours the=20 > > system will be un-usable. > >=20 > > I did find this thread which seems to describe similar=20 > issues but this=20 > > is a different driver. > >=20 > http://lists.freebsd.org/pipermail/freebsd-questions/2006-Augu st/127835. > > html >=20 > Probably not relevant then. Deadlocks come in many=20 > varieties, all different. >=20 > > Currently I'm running with debug.mpsafevfs=3D0, = debug.mpsafenet=3D1and=20 > > debug.mpsafevm=3D0 but this doesn't seem to help. > >=20 > > On perahps a related issue I have two other nearly=20 > identical systems=20 > > which were going to be upgrading to 6.2 as on 5.4 I am experiencing=20 > > deadlocks and when I hit ctrl-t I see the system is either stuck in=20 > > ufs or zoneinfo and I have not found very much information=20 > about zoneinfo. > >=20 > > Does anyone have any suggestions on what I can look for or other=20 > > tuning options or had similar experiences? >=20 > See the chapter on kernel debugging in the developers=20 > handbook for the information you need to provide before we=20 > can begin to debug your problem. >=20 > Kris >=20 >=20 From owner-freebsd-fs@FreeBSD.ORG Wed May 16 17:42:19 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 6C5A316A400; Wed, 16 May 2007 17:42:19 +0000 (UTC) (envelope-from cswiger@mac.com) Received: from mail-out4.apple.com (mail-out4.apple.com [17.254.13.23]) by mx1.freebsd.org (Postfix) with ESMTP id 53DB813C458; Wed, 16 May 2007 17:42:19 +0000 (UTC) (envelope-from cswiger@mac.com) Received: from relay7.apple.com (a17-128-113-37.apple.com [17.128.113.37]) by mail-out4.apple.com (Postfix) with ESMTP id 5732B15E31F; Wed, 16 May 2007 10:26:34 -0700 (PDT) Received: from relay7.apple.com (unknown [127.0.0.1]) by relay7.apple.com (Symantec Mail Security) with ESMTP id 47689304D7; Wed, 16 May 2007 10:26:34 -0700 (PDT) X-AuditID: 11807125-a1a73bb00000318d-e4-464b3eca5c67 Received: from [17.214.13.96] (cswiger1.apple.com [17.214.13.96]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by relay7.apple.com (Apple SCV relay) with ESMTP id 33EB1304C7; Wed, 16 May 2007 10:26:34 -0700 (PDT) In-Reply-To: <464B3206.7000007@freebsd.org> References: <5230D3C40B842D4F9FB3CD368021BEF72F08F1@exchange-2.sandvine.com> <464B3206.7000007@freebsd.org> Mime-Version: 1.0 (Apple Message framework v752.2) Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: Content-Transfer-Encoding: 7bit From: Chuck Swiger Date: Wed, 16 May 2007 10:26:33 -0700 To: Eric Anderson X-Mailer: Apple Mail (2.752.2) X-Brightmail-Tracker: AAAAAA== Cc: freebsd-fs@freebsd.org Subject: Re: Ufs dead-locks on freebsd 6.2 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 May 2007 17:42:19 -0000 On May 16, 2007, at 9:32 AM, Eric Anderson wrote: >> On perahps a related issue I have two other nearly identical systems >> which were going to be upgrading to 6.2 as on 5.4 I am experiencing >> deadlocks and when I hit ctrl-t I see the system is either stuck >> in ufs >> or zoneinfo and I have not found very much information about >> zoneinfo. > > What's ctrl-t? It's normally bound to SIGINFO (via stty), which causes programs to display their process status. -- -Chuck From owner-freebsd-fs@FreeBSD.ORG Thu May 17 09:36:10 2007 Return-Path: X-Original-To: fs@freebsd.org Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 2C45916A401 for ; Thu, 17 May 2007 09:36:10 +0000 (UTC) (envelope-from howard0su@gmail.com) Received: from py-out-1112.google.com (py-out-1112.google.com [64.233.166.180]) by mx1.freebsd.org (Postfix) with ESMTP id E4DF013C44C for ; Thu, 17 May 2007 09:36:09 +0000 (UTC) (envelope-from howard0su@gmail.com) Received: by py-out-1112.google.com with SMTP id f31so771984pyh for ; Thu, 17 May 2007 02:36:09 -0700 (PDT) DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; b=GKDBeE3QiWMtZmWo8i2vfhkcNDpSBZXlMTCzbFOBQAG3RMNKa3QPL9lz61ytTDYk6h16GJ1pI/wx0Mi3xjbMniLmP8GL/gp8oEFUTd1TA+BCfz6v6qnDzKWwFJBvgaS/mnljP4TuQYNcb5rPHRcq2zGr62PG6E1ulnsPdWcEYh8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; b=b8pVW9IrUiOEqr0o/6QXIKS+4A2DWqKTX/PgW518k1xQSEs5G/bQd+4A6jpCJkJaeZSNxxgYqSBJDeyCLnweRKfwbrMxfGqGBGpeZHqzz3bS/ZdbdIsSeG5I9Iito1Km0m/AWNI/RbgxK4H0aLZFbgewdlSLs+poCePaXmp27F8= Received: by 10.35.103.12 with SMTP id f12mr449426pym.1179394569388; Thu, 17 May 2007 02:36:09 -0700 (PDT) Received: by 10.35.78.11 with HTTP; Thu, 17 May 2007 02:36:09 -0700 (PDT) Message-ID: Date: Thu, 17 May 2007 17:36:09 +0800 From: "Howard Su" To: fs@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline Cc: Subject: size limit for TMPFS X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 May 2007 09:36:10 -0000 In current implementation of tmpfs, the size limit of a tmpfs (which is passed when mounting) only count the file size. The memory used for directories is not limit. This due to the fact that there is not easy and fast way to track a uma_zone's memory usage. there are several approach: 1. hack vm to support a hook when uma_zone allocate a new page. this approach comes from Rohit Jalan. This also need some new API in uma(9) uma_set_allocate_hook uma_set_deallocate_hook 2. Add a new API to uma(9) to report its memory usage. This will require traveling a link list inside uma_zone. uma_get_paged 3. Guess a file number limit based on size if use doesn't specific explict, or get the number user sepcified. Use the number limit instead of size limit to limit the memory usage. This doesn't need any changes to uma. Also the implementation is clean. I personally perfer way 3. Any suggestions? -- -Howard From owner-freebsd-fs@FreeBSD.ORG Thu May 17 11:34:01 2007 Return-Path: X-Original-To: fs@freebsd.org Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id C86F116A479 for ; Thu, 17 May 2007 11:34:01 +0000 (UTC) (envelope-from anderson@freebsd.org) Received: from mh2.centtech.com (moat3.centtech.com [64.129.166.50]) by mx1.freebsd.org (Postfix) with ESMTP id 93E9F13C465 for ; Thu, 17 May 2007 11:34:01 +0000 (UTC) (envelope-from anderson@freebsd.org) Received: from neutrino.centtech.com (neutrino.centtech.com [10.177.171.220]) by mh2.centtech.com (8.13.8/8.13.8) with ESMTP id l4HBXx76038418; Thu, 17 May 2007 06:33:59 -0500 (CDT) (envelope-from anderson@freebsd.org) Message-ID: <464C3DA7.3020003@freebsd.org> Date: Thu, 17 May 2007 06:33:59 -0500 From: Eric Anderson User-Agent: Thunderbird 2.0.0.0 (X11/20070420) MIME-Version: 1.0 To: Howard Su References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.88.4/3264/Wed May 16 15:43:07 2007 on mh2.centtech.com X-Virus-Status: Clean X-Spam-Status: No, score=-2.5 required=8.0 tests=AWL,BAYES_00 autolearn=ham version=3.1.6 X-Spam-Checker-Version: SpamAssassin 3.1.6 (2006-10-03) on mh2.centtech.com Cc: fs@freebsd.org Subject: Re: size limit for TMPFS X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 May 2007 11:34:01 -0000 On 05/17/07 04:36, Howard Su wrote: > In current implementation of tmpfs, the size limit of a tmpfs (which > is passed when mounting) only count the file size. The memory used for > directories is not limit. This due to the fact that there is not easy > and fast way to track a uma_zone's memory usage. > > there are several approach: > 1. hack vm to support a hook when uma_zone allocate a new page. this > approach comes from Rohit Jalan. This also need some new API in uma(9) > uma_set_allocate_hook > uma_set_deallocate_hook > > > 2. Add a new API to uma(9) to report its memory usage. This will > require traveling a link list inside uma_zone. > uma_get_paged > > 3. Guess a file number limit based on size if use doesn't specific > explict, or get the number user sepcified. Use the number limit > instead of size limit to limit the memory usage. This doesn't need any > changes to uma. Also the implementation is clean. > > I personally perfer way 3. Any suggestions? > Track the memory usage on your own? As you allocate, keep a counter and total up the usage outside of uma. (3) sounds good, except it may not be accurate, and that could lead to confusion for someone. Eric From owner-freebsd-fs@FreeBSD.ORG Thu May 17 12:46:48 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 680) id C412B16A406; Thu, 17 May 2007 12:46:48 +0000 (UTC) Date: Thu, 17 May 2007 12:46:48 +0000 From: Darren Reed To: Dag-Erling Sm??rgrav Message-ID: <20070517124648.GA98052@hub.freebsd.org> References: <20070409011723.GB74547@garage.freebsd.pl> <20070409094319.GB76673@garage.freebsd.pl> <70e8236f0704090808y5d305175wdc3cee5be1a26a9@mail.gmail.com> <20070409153338.GH76673@garage.freebsd.pl> <440b3e930705131031v5e97db7fq486d8d17aeb9f622@mail.gmail.com> <20070514070715.GA82322@hub.freebsd.org> <86646vyimg.fsf@dwp.des.no> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <86646vyimg.fsf@dwp.des.no> User-Agent: Mutt/1.4.2.1i Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org, Pawel Jakub Dawidek , Joao Barros Subject: Re: ZFS: amd64, devd, root file system. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 May 2007 12:46:48 -0000 On Mon, May 14, 2007 at 09:22:31AM +0200, Dag-Erling Sm??rgrav wrote: > Darren Reed writes: > > It is not a lot of fun having to support the kernel and 'some' > > filesystems being of a different type of filesystem to other parts > > (from a system admin perspective.) This is especially true of those > > filesystems that make up the "root". > > I don't see what the problem is. I am perfectly content with having my > root file system on UFS2 on a pair of mirror disks. Then your experience doing systems administration has been a lot... cleaner and easier than what mine has. These days it is uncommon to run out of space on / because disks are generally so damn big. But if it does happen, the only choice you have with a filesystem such as UFS is to clean up files or repartition your disk. But your UFS2 mirror won't provide you with CRC corrected data, so if your PCI card decided to flip a bit or two while sending data out to disk, you've got no way of knowing about it. The data reliability and assurance of correctness delivered via ZFS simply isn't available with UFS2. Now maybe *you* don't care about that now, but someday, someone might when part of their kernel (or LKM) gets corrupted and they have no way of knowing. > Considering the amount of work which would be required to allow FreeBSD > to boot from ZFS (which you apparently do not appreciate), I perfectly > understand Pawel's choice. If you think I don't appreciate the work involved then I'll say that you apparently have no appreciation of how using a single filesystem and being able to use the likes of ZFS for root simplifies system admin. > Remember that unlike Sun, we do not make the hardware our OS runs on, > nor do we write the firmware for it. Do you see my email address as being "@sun.com"? No. My comments have got *nothing* to do with what Sun does and have got everything to do with having worked on multiple different Unix platforms and having worked on systems where there were different constraints on what could be done with managing root because it was a different file- system type. I really don't understand why you needed to be so inflamatory with your email, but then again your response was all about you and what you do, not about what others do or would benefit them. I personally don't want or need ZFS as a root right now, but I can see that some might and that it could be of benefit to me in the future. Darren From owner-freebsd-fs@FreeBSD.ORG Thu May 17 13:01:04 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 01E4B16A401; Thu, 17 May 2007 13:01:04 +0000 (UTC) (envelope-from peter.schuller@infidyne.com) Received: from mxfep03.bredband.com (mxfep03.bredband.com [195.54.107.76]) by mx1.freebsd.org (Postfix) with ESMTP id 1ACB913C44B; Thu, 17 May 2007 13:01:02 +0000 (UTC) (envelope-from peter.schuller@infidyne.com) Received: from ironport2.bredband.com ([195.54.107.84] [195.54.107.84]) by mxfep03.bredband.com with ESMTP id <20070517130051.URML23113.mxfep03.bredband.com@ironport2.bredband.com>; Thu, 17 May 2007 15:00:51 +0200 Received: from c-5416e555.03-51-73746f3.cust.bredbandsbolaget.se (HELO scode.mine.nu) ([85.229.22.84]) by ironport2.bredband.com with ESMTP; 17 May 2007 15:00:46 +0200 Received: from scode.mine.nu (localhost [127.0.0.1]) by scode.mine.nu (Postfix) with ESMTP id 1863A1595A; Thu, 17 May 2007 15:00:38 +0200 (CEST) Message-ID: <464C51EB.4050609@infidyne.com> Date: Thu, 17 May 2007 15:00:27 +0200 From: Peter Schuller User-Agent: Thunderbird 2.0.0.0 (X11/20070501) MIME-Version: 1.0 To: freebsd-fs@FreeBSD.org X-Enigmail-Version: 0.95.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="------------enig3B5F26B846054F8EFC18E4CA" Cc: Pawel Jakub Dawidek Subject: ZFS panic: solaris assert: arc_buf_remove_buf(db->db_buf, db) == 0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 May 2007 13:01:04 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig3B5F26B846054F8EFC18E4CA Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Hello, I am experiencing a panic that I do not believe to have been discussed already. It is being triggered while rsync:ing a directory hierarchy from another machine over ssh onto a 6 drive raidz2 pool. The transfer consists mostly of large files (rather than a huge amount of small files). I seem to be getting this panic within a few hours of starting or resuming said transfer. The machine only has 512+256 minus some megs stolen by built-in VGA of memory. The following is manually transcribed because crashdumps do not work[1], so there may be typos: panic: solaris assert: arc_buf_remove_buf(db->db_buf, db) =3D=3D 0,file: /usr/src/sys/modules/zfs/../../contrib/opensolaris/uts/common/fs/zfs/dbuf= =2Ec, line: 1718 cpuid =3D 0 KBD: enter: panic [thread pid 132 tid 100072] Stopped at kbd_enter+0x2b: nop db> bt Tracing pid 132 tid 100072 td 0xc3899d80 kbd_enter(c095f2c7) at kbd_enter+0x2b panic(c0c5ab69,c0c5b020,c0c5af8c,6b6,a99a,...) at pnic+0x11c dbuf_rele(cb874578,a99a,cb874578,a99a,0,...) at dbuf_rele+0x18c arc_write_done(c940b000,c940b200,c39aa000,c7663000,cbe93000,...) at arc_write_done+0x13a zio_done(c940b000,c0c3380f,c45fec00,0,1,...) at zio_done+0x1ee zio_vdev_io_assess(c940b000,c37bd104,c37bd124,c37bd148,929a53f1,...) at zio_vdev_io_assess+0x116 taskq_thread(c37bd0cc,ddb85d38) at taskq_thread+0x101 fork_exit(c0bfa2c4,c37bd0cc,ddb85d38) at fork_exit+0xac fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip =3D 0, esp =3D 0xddb85db70, ebp =3D 0 --- db> dmesg for the machine: http://distfiles.scode.org/mlref/freebsd-dmesg-zfs-panic-arcbufremove-7cu= rrent.txt It's a version of current compiled somewhere leading up to May 8. Contents of /boot/loader.conf: linux_load=3D"YES" geom_label_load=3D"YES" zfs_load=3D"YES" vm.kmem_size=3D272629760 Contents of /etc/sysctl.conf: kern.ps_arg_cache_limit=3D4096 kern.maxvnodes=3D15000 I have gotten this panic twice confirmed so far. The stacktrace is from the second instance. I did not grab the trace the first time because I was hoping for the crashdump. [1] Even after manually running "dumpon" and making sure it suceeded. Are crashdumps supposed to work on glabel devices? That is the only "special" circumstance I am aware of. --=20 / Peter Schuller PGP userID: 0xE9758B7D or 'Peter Schuller ' Key retrieval: Send an E-Mail to getpgpkey@scode.org E-Mail: peter.schuller@infidyne.com Web: http://www.scode.org --------------enig3B5F26B846054F8EFC18E4CA Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.3 (FreeBSD) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGTFH1DNor2+l1i30RCOMyAJ4lMi3dlHfSmscYWM4wAAIdN2WXuwCg1tNC AkZPb3/Itlp5lDHQgEFnXvc= =LoVD -----END PGP SIGNATURE----- --------------enig3B5F26B846054F8EFC18E4CA-- From owner-freebsd-fs@FreeBSD.ORG Thu May 17 15:44:20 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 1C54B16A401; Thu, 17 May 2007 15:44:20 +0000 (UTC) (envelope-from aedwards@sandvine.com) Received: from gw.sandvine.com (gw.sandvine.com [199.243.201.138]) by mx1.freebsd.org (Postfix) with ESMTP id 7B2C213C489; Thu, 17 May 2007 15:44:17 +0000 (UTC) (envelope-from aedwards@sandvine.com) Received: from exchange-2.sandvine.com ([192.168.16.12]) by gw.sandvine.com with Microsoft SMTPSVC(6.0.3790.1830); Thu, 17 May 2007 11:44:16 -0400 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: quoted-printable Date: Thu, 17 May 2007 11:44:15 -0400 Message-ID: <5230D3C40B842D4F9FB3CD368021BEF72F0908@exchange-2.sandvine.com> In-Reply-To: <5230D3C40B842D4F9FB3CD368021BEF72F08F3@exchange-2.sandvine.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Ufs dead-locks on freebsd 6.2 Thread-Index: AceX1+P2HSPmcMyKSLK15m7jv2eBpAAAVXqQAC/+rgA= From: "Andrew Edwards" To: X-OriginalArrivalTime: 17 May 2007 15:44:16.0473 (UTC) FILETIME=[39C1B490:01C7989A] Cc: freebsd-fs@freebsd.org Subject: RE: Ufs dead-locks on freebsd 6.2 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 May 2007 15:44:20 -0000 I've upgraded to 6-stable, added the kernel options as per the kernel handbook. After about 5 hours of they system in a deadlock it panic'd. Here's the backtrace, and show pcpu, show allpcpu, show locks, show alllocks, show lockedvnods and alltrace. I will have the system down for approx another 15-20mins if there's anything else someone would like while I'm in the debugger. db> bt Tracing pid 46784 tid 105112 td 0xd44a8000 kdb_enter(c0785f13) at kdb_enter+0x2b vfs_badlock(c0785f2c,c0786051,ccd47984) at vfs_badlock+0x47 assert_vop_locked(ccd47984,c0786051) at assert_vop_locked+0x4a vop_lock_post(f9f709dc,0,1002,ccd47984,f9f709f8,...) at vop_lock_post+0x2a VOP_LOCK_APV(c07dc2e0,f9f709dc) at VOP_LOCK_APV+0xa0 vn_lock(ccd47984,1002,d44a8000) at vn_lock+0xac lookup(f9f70c08) at lookup+0xde namei(f9f70c08) at namei+0x39a unp_connect(d44b2de8,d44dc380,d44a8000,d44b2de8,25,...) at unp_connect+0xf0 uipc_connect(d44b2de8,d44dc380,d44a8000) at uipc_connect+0x66 soconnect(d44b2de8,d44dc380,d44a8000) at soconnect+0x4e kern_connect(d44a8000,7,d44dc380,d44dc380,0,...) at kern_connect+0x74 connect(d44a8000,f9f70d04) at connect+0x2f syscall(3b,805003b,bfbf003b,bfbfd920,bfbfd922,...) at syscall+0x25b Xint0x80_syscall() at Xint0x80_syscall+0x1f --- syscall (98, FreeBSD ELF32, connect), eip =3D 0x6812d553, esp =3D 0xbfbfd90c, ebp =3D 0xbfbfd9a8 --- db> show pcpu cpuid =3D 1 curthread =3D 0xd44a8000: pid 46784 "cron" curpcb =3D 0xf9f70d90 fpcurthread =3D none idlethread =3D 0xccb5c600: pid 10 "idle: cpu1" APIC ID =3D 6 currentldt =3D 0x50 spin locks held: db> show allpcpu Current CPU: 1 cpuid =3D 0 curthread =3D 0xd4490d80: pid 46779 "cron" curpcb =3D 0xf9f13d90 fpcurthread =3D none idlethread =3D 0xccb5c780: pid 11 "idle: cpu0" APIC ID =3D 0 currentldt =3D 0x50 spin locks held: cpuid =3D 1 curthread =3D 0xd44a8000: pid 46784 "cron" curpcb =3D 0xf9f70d90 fpcurthread =3D none idlethread =3D 0xccb5c600: pid 10 "idle: cpu1" APIC ID =3D 6 currentldt =3D 0x50 spin locks held: db> show locks db> show alllocks db> show lockedvnods Locked vnodes 0xd12d16cc: tag ufs, type VREG usecount 1, writecount 1, refcount 4 mountedhere 0 flags () v_object 0xcd64ac60 ref 0 pages 48 lock type ufs: EXCL (count 1) by thread 0xcd6a6180 (pid 22025)#0 0xc0593f0d at lockmgr+0x4ed #1 0xc06b8e0e at ffs_lock+0x76 #2 0xc0739787 at VOP_LOCK_APV+0x87 #3 0xc0601c28 at vn_lock+0xac #4 0xc06015c4 at vn_write+0x138 #5 0xc05c4544 at dofilewrite+0x7c #6 0xc05c43e3 at kern_writev+0x3b #7 0xc05c4309 at write+0x45 #8 0xc0723e2b at syscall+0x25b #9 0xc070ee0f at Xint0x80_syscall+0x1f ino 494646, on dev amrd0s1d 0xccf9f2b8: tag ufs, type VDIR usecount 1, writecount 0, refcount 2495 mountedhere 0 flags (VV_ROOT) v_object 0xd23c39cc ref 0 pages 0 lock type ufs: EXCL (count 1) by thread 0xcd23fd80 (pid 40153) with 2493 pending#0 0xc0593f0d at lockmgr+0x4ed #1 0xc06b8e0e at ffs_lock+0x76 #2 0xc0739787 at VOP_LOCK_APV+0x87 #3 0xc0601c28 at vn_lock+0xac #4 0xc05f5d12 at vget+0xbe #5 0xc05ed9f9 at vfs_hash_get+0x8d #6 0xc06b7b8f at ffs_vget+0x27 #7 0xc06c1435 at ufs_root+0x19 #8 0xc05eef1c at lookup+0x7c8 #9 0xc05ee4b2 at namei+0x39a #10 0xc0600a13 at vn_open_cred+0x5b #11 0xc06009b6 at vn_open+0x1e #12 0xc05fa126 at kern_open+0xb6 #13 0xc05fa03a at open+0x1a #14 0xc0723e2b at syscall+0x25b #15 0xc070ee0f at Xint0x80_syscall+0x1f ino 2, on dev amrd0s1d 0xcd030984: tag ufs, type VDIR usecount 7, writecount 0, refcount 9 mountedhere 0 flags () v_object 0xcffb2108 ref 0 pages 0 lock type ufs: EXCL (count 1) by thread 0xcd9bc780 (pid 29680) with 1 pending#0 0xc0593f0d at lockmgr+0x4ed #1 0xc06b8e0e at ffs_lock+0x76 #2 0xc0739787 at VOP_LOCK_APV+0x87 #3 0xc0601c28 at vn_lock+0xac #4 0xc05f5d12 at vget+0xbe #5 0xc05ea48e at cache_lookup+0x34a #6 0xc05ea9c2 at vfs_cache_lookup+0x92 #7 0xc0737847 at VOP_LOOKUP_APV+0x87 #8 0xc05eebf8 at lookup+0x4a4 #9 0xc05ee4b2 at namei+0x39a #10 0xc0600a13 at vn_open_cred+0x5b #11 0xc06009b6 at vn_open+0x1e #12 0xc05fa126 at kern_open+0xb6 #13 0xc05fa03a at open+0x1a #14 0xc0723e2b at syscall+0x25b #15 0xc070ee0f at Xint0x80_syscall+0x1f ino 494592, on dev amrd0s1d 0xd1bad828: tag ufs, type VREG usecount 2, writecount 2, refcount 6 mountedhere 0 flags () v_object 0xd08b28c4 ref 0 pages 82 lock type ufs: EXCL (count 1) by thread 0xcd5b0780 (pid 36375) with 2 pending#0 0xc0593f0d at lockmgr+0x4ed #1 0xc06b8e0e at ffs_lock+0x76 #2 0xc0739787 at VOP_LOCK_APV+0x87 #3 0xc0601c28 at vn_lock+0xac #4 0xc06015c4 at vn_write+0x138 #5 0xc05c4544 at dofilewrite+0x7c #6 0xc05c43e3 at kern_writev+0x3b #7 0xc05c4309 at write+0x45 #8 0xc0723e2b at syscall+0x25b #9 0xc070ee0f at Xint0x80_syscall+0x1f ino 494775, on dev amrd0s1d 0xcfed0984: tag ufs, type VREG usecount 1, writecount 1, refcount 4 mountedhere 0 flags () v_object 0xd0dcf39c ref 0 pages 7 lock type ufs: EXCL (count 1) by thread 0xccfa9000 (pid 653)#0 0xc0593f0d at lockmgr+0x4ed #1 0xc06b8e0e at ffs_lock+0x76 #2 0xc0739787 at VOP_LOCK_APV+0x87 #3 0xc0601c28 at vn_lock+0xac #4 0xc05fd950 at fsync+0x9c #5 0xc0723e2b at syscall+0x25b #6 0xc070ee0f at Xint0x80_syscall+0x1f ino 188447, on dev amrd0s1d 0xd2e3b828: tag ufs, type VREG usecount 2, writecount 2, refcount 1612 mountedhere 0 flags () v_object 0xd2dee630 ref 0 pages 6436 lock type ufs: SHARED (count 1) with 1 pending#0 0xc0593b80 at lockmgr+0x160 #1 0xc06b8e0e at ffs_lock+0x76 #2 0xc0739787 at VOP_LOCK_APV+0x87 #3 0xc0601c28 at vn_lock+0xac #4 0xc06012fe at vn_read+0x132 #5 0xc05c4269 at dofileread+0x85 #6 0xc05c4102 at kern_readv+0x36 #7 0xc05c402d at read+0x45 #8 0xc0723e2b at syscall+0x25b #9 0xc070ee0f at Xint0x80_syscall+0x1f ino 130807917, on dev amrd1s1d 0xd157f570: tag ufs, type VREG usecount 1, writecount 1, refcount 2713 mountedhere 0 flags () v_object 0xd2d5118c ref 0 pages 245644 lock type ufs: EXCL (count 1) by thread 0xcd6a6300 (pid 26383)#0 0xc0593f0d at lockmgr+0x4ed #1 0xc06b8e0e at ffs_lock+0x76 #2 0xc0739787 at VOP_LOCK_APV+0x87 #3 0xc0601c28 at vn_lock+0xac #4 0xc06015c4 at vn_write+0x138 #5 0xc05c4544 at dofilewrite+0x7c #6 0xc05c43e3 at kern_writev+0x3b #7 0xc05c4309 at write+0x45 #8 0xc0723e2b at syscall+0x25b #9 0xc070ee0f at Xint0x80_syscall+0x1f ino 116230408, on dev amrd1s1d db> alltrace Tracing command pid 46787 tid 105110 td 0xd44a8300 *** error reading from address 72d76109 *** db> > -----Original Message----- > From: owner-freebsd-performance@freebsd.org=20 > [mailto:owner-freebsd-performance@freebsd.org] On Behalf Of=20 > Andrew Edwards > Sent: Wednesday, May 16, 2007 1:16 PM > To: freebsd-performance@freebsd.org > Cc: freebsd-fs@freebsd.org > Subject: RE: Ufs dead-locks on freebsd 6.2 >=20 > Here's the backtrace from the last crash along with the=20 > output from show alllocks when the system was deadlocked. I=20 > have been running 6.2-release and compliled with makeoptions=20 > debug=3D-g, invariants, invariant_support and witness. I will=20 > update to 6-STABLE add diagnositc, debug_locks and=20 > debug_vfs_locks as per the handbook recommendation and retry. >=20 > Yes, when the system was un-usable I was still able to ping=20 > it. I have the serial console setup as the default console=20 > so I can remotely access the box and break into the debugger etc. >=20 > (kgdb) bt > #0 doadump () at pcpu.h:165 > #1 0xc059b480 in boot (howto=3D260) at > /usr/src/sys/kern/kern_shutdown.c:409 > #2 0xc059b795 in panic (fmt=3D0xc0787b04 "Most recently used by = %s\n") > at /usr/src/sys/kern/kern_shutdown.c:565 > #3 0xc06c4a4d in mtrash_ctor (mem=3D0xce74fa00, size=3D0, arg=3D0x0, > flags=3D258) > at /usr/src/sys/vm/uma_dbg.c:137 > #4 0xc06c2c07 in uma_zalloc_arg (zone=3D0xc10615a0, udata=3D0x0,=20 > flags=3D258) > at /usr/src/sys/vm/uma_core.c:1850 > #5 0xc0591416 in malloc (size=3D272, mtp=3D0xc07c32c0, flags=3D258) = at > uma.h:275 > #6 0xc05edfab in __mnt_vnode_first (mvp=3D0xf3741c48, = mp=3D0xcaa14cf8) > at /usr/src/sys/kern/vfs_mount.c:1813 > #7 0xc05f2467 in vfs_msync (mp=3D0xcaa14cf8, flags=3D2) > at /usr/src/sys/kern/vfs_subr.c:2874 > #8 0xc05f2bbd in sync_fsync (ap=3D0x0) at > /usr/src/sys/kern/vfs_subr.c:3119 > #9 0xc072f4ee in VOP_FSYNC_APV (vop=3D0x0, a=3D0xf3741cbc) at=20 > vnode_if.c:1020 #10 0xc05f097c in sync_vnode (bo=3D0xca854e90,=20 > td=3D0xca435000) at > vnode_if.h:537 > #11 0xc05f0bf1 in sched_sync () at /usr/src/sys/kern/vfs_subr.c:1698 > #12 0xc0587248 in fork_exit (callout=3D0xc05f0a04 , = arg=3D0x0, > frame=3D0xf3741d38) at /usr/src/sys/kern/kern_fork.c:821 > #13 0xc070712c in fork_trampoline () at > /usr/src/sys/i386/i386/exception.s:208 >=20 >=20 > db> show alllocks > Process 36596 (sshd) thread 0xd1238c00 (102406) exclusive=20 > sleep mutex vm object (standard object) r =3D 0 (0xce2c87bc)=20 > locked @ /usr/src/sys/vm/vm_object.c:446 exclusive sx user=20 > map r =3D 0 (0xd128060c) locked @ > /usr/src/sys/vm/vm_map.c:307 > Process 887 (sshd) thread 0xca7d2000 (100056) exclusive sleep=20 > mutex vm object (standard object) r =3D 0 (0xcb713ad4) locked @=20 > /usr/src/sys/vm/vm_fault.c:297 exclusive sx user map r =3D 0=20 > (0xcaae4734) locked @ > /usr/src/sys/vm/vm_map.c:3074 > db> show lockedvnods > Locked vnodes >=20 > 0xcaa78660: tag ufs, type VREG > usecount 2, writecount 1, refcount 3 mountedhere 0 > flags () > v_object 0xc1046738 ref 0 pages 1596 > lock type ufs: EXCL (count 1) by thread 0xca689c00 (pid=20 > 536) with 1 pending > ino 494620, on dev amrd0s1d >=20 > 0xcaa86110: tag ufs, type VREG > usecount 1, writecount 1, refcount 3 mountedhere 0 > flags () > v_object 0xca85f738 ref 0 pages 44 > lock type ufs: EXCL (count 1) by thread 0xca7d2780 (pid 715) > ino 494633, on dev amrd0s1d >=20 > 0xcabe4110: tag ufs, type VDIR > usecount 12, writecount 0, refcount 14 mountedhere 0 > flags () > v_object 0xcab28840 ref 0 pages 1 > lock type ufs: EXCL (count 1) by thread 0xca9bbd80 (pid=20 > 14253) with > 3 pending > ino 423947, on dev amrd0s1d >=20 > 0xcb437990: tag ufs, type VREG > usecount 1, writecount 1, refcount 2 mountedhere 0 > flags () > v_object 0xcb3c3dec ref 0 pages 4100 > lock type ufs: EXCL (count 1) by thread 0xcaffac00 (pid 20868) > ino 282640, on dev amrd0s1d >=20 > 0xcb99e550: tag ufs, type VDIR > usecount 2, writecount 0, refcount 4 mountedhere 0 > flags () > v_object 0xcef979cc ref 0 pages 1 > lock type ufs: EXCL (count 1) by thread 0xca9bb600 (pid=20 > 881) with 1 pending > ino 423987, on dev amrd0s1d >=20 > 0xcfc97dd0: tag ufs, type VREG > usecount 1, writecount 1, refcount 2 mountedhere 0 > flags () > v_object 0xcd4975ac ref 0 pages 1 > lock type ufs: EXCL (count 1) by thread 0xcb748900 (pid 2518) > ino 424275, on dev amrd0s1d >=20 > 0xccad9aa0: tag ufs, type VREG > usecount 1, writecount 1, refcount 3 mountedhere 0 > flags () > v_object 0xcf0c539c ref 0 pages 5 > lock type ufs: EXCL (count 1) by thread 0xca7d1c00 (pid 600) > ino 188446, on dev amrd0s1d >=20 > 0xccb0f110: tag ufs, type VREG > usecount 1, writecount 1, refcount 2 mountedhere 0 > flags () > v_object 0xcb4609cc ref 0 pages 1 > lock type ufs: EXCL (count 1) by thread 0xcafbf480 (pid 11054) > ino 424100, on dev amrd0s1d >=20 > 0xcc501bb0: tag ufs, type VREG > usecount 1, writecount 1, refcount 3 mountedhere 0 > flags () > v_object 0xcc7d19cc ref 0 pages 1 > lock type ufs: EXCL (count 1) by thread 0xcb76d600 (pid=20 > 13743) with > 1 pending > ino 424279, on dev amrd0s1d >=20 > 0xcf96b220: tag ufs, type VREG > usecount 1, writecount 1, refcount 2 mountedhere 0 > flags () > v_object 0xcf135c60 ref 0 pages 1 > lock type ufs: EXCL (count 1) by thread 0xcafbf900 (pid 29458) > ino 424374, on dev amrd0s1d >=20 > 0xcc5bbbb0: tag ufs, type VREG > usecount 1, writecount 1, refcount 2 mountedhere 0 > flags () > v_object 0xcdd5b318 ref 0 pages 1 > lock type ufs: EXCL (count 1) by thread 0xcaad9900 (pid 50782) > ino 424276, on dev amrd0s1d >=20 > 0xcec1d000: tag ufs, type VREG > usecount 1, writecount 1, refcount 2 mountedhere 0 > flags () > v_object 0xcd3d7108 ref 0 pages 1 > lock type ufs: EXCL (count 1) by thread 0xcb76dc00 (pid 59514) > ino 424500, on dev amrd0s1d >=20 > 0xcebe5110: tag ufs, type VREG > usecount 1, writecount 1, refcount 2 mountedhere 0 > flags () > v_object 0xccee95ac ref 0 pages 1 > lock type ufs: EXCL (count 1) by thread 0xcb650780 (pid 59975) > ino 424509, on dev amrd0s1d >=20 > 0xce0c1880: tag ufs, type VREG > usecount 1, writecount 1, refcount 2 mountedhere 0 > flags () > v_object 0xca8b64a4 ref 0 pages 1 > lock type ufs: EXCL (count 1) by thread 0xcb768a80 (pid 69466) > ino 424555, on dev amrd0s1d >=20 > 0xcf652110: tag ufs, type VREG > usecount 1, writecount 1, refcount 2 mountedhere 0 > flags () > v_object 0xcf4a318c ref 0 pages 1 > lock type ufs: EXCL (count 1) by thread 0xcaff8600 (pid 75577) > ino 424579, on dev amrd0s1d >=20 > 0xce282550: tag ufs, type VREG > usecount 1, writecount 1, refcount 2 mountedhere 0 > flags () > v_object 0xcf261318 ref 0 pages 1 > lock type ufs: EXCL (count 1) by thread 0xd0235a80 (pid 81734) > ino 424927, on dev amrd0s1d >=20 > 0xcc1d4dd0: tag ufs, type VREG > usecount 1, writecount 1, refcount 2 mountedhere 0 > flags () > v_object 0xcd4a6630 ref 0 pages 1 > lock type ufs: EXCL (count 1) by thread 0xcaccb900 (pid 81772) > ino 424928, on dev amrd0s1d >=20 > 0xcb820bb0: tag ufs, type VREG > usecount 1, writecount 1, refcount 2 mountedhere 0 > flags () > v_object 0xcb251ad4 ref 0 pages 1 > lock type ufs: EXCL (count 1) by thread 0xcadaf480 (pid 84037) > ino 424935, on dev amrd0s1d >=20 > 0xced5aaa0: tag ufs, type VREG > usecount 1, writecount 1, refcount 2 mountedhere 0 > flags () > v_object 0xcb784210 ref 0 pages 1 > lock type ufs: EXCL (count 1) by thread 0xd0236000 (pid 202) > ino 425039, on dev amrd0s1d >=20 > 0xcbe45220: tag ufs, type VREG > usecount 1, writecount 1, refcount 2 mountedhere 0 > flags () > v_object 0xce55de70 ref 0 pages 1 > lock type ufs: EXCL (count 1) by thread 0xcaad9a80 (pid 230) > ino 425043, on dev amrd0s1d >=20 > 0xcc098220: tag ufs, type VREG > usecount 1, writecount 1, refcount 2 mountedhere 0 > flags () > v_object 0xce4a9dec ref 0 pages 1 > lock type ufs: EXCL (count 1) by thread 0xcafbfd80 (pid 9902) > ino 425093, on dev amrd0s1d >=20 > 0xcd585110: tag ufs, type VREG > usecount 1, writecount 1, refcount 2 mountedhere 0 > flags () > v_object 0xcf8c1e70 ref 0 pages 1 > lock type ufs: EXCL (count 1) by thread 0xcaccba80 (pid 24017) > ino 425144, on dev amrd0s1d >=20 > 0xceeac000: tag ufs, type VREG > usecount 1, writecount 1, refcount 2 mountedhere 0 > flags () > v_object 0xcb1225ac ref 0 pages 1 > lock type ufs: EXCL (count 1) by thread 0xcaff5a80 (pid 24775) > ino 425149, on dev amrd0s1d >=20 > 0xcc549aa0: tag ufs, type VREG > usecount 1, writecount 1, refcount 2 mountedhere 0 > flags () > v_object 0xcab2d318 ref 0 pages 1 > lock type ufs: EXCL (count 1) by thread 0xcaff8a80 (pid 42358) > ino 425227, on dev amrd0s1d >=20 > 0xcc6f7000: tag ufs, type VREG > usecount 1, writecount 1, refcount 2 mountedhere 0 > flags () > v_object 0xcfd4139c ref 0 pages 1 > lock type ufs: EXCL (count 1) by thread 0xcafbf600 (pid 43117) > ino 425230, on dev amrd0s1d >=20 > 0xccc44bb0: tag ufs, type VREG > usecount 1, writecount 1, refcount 2 mountedhere 0 > flags () > v_object 0xcfd18d68 ref 0 pages 1 > lock type ufs: EXCL (count 1) by thread 0xcadafd80 (pid 42859) > ino 425234, on dev amrd0s1d >=20 > 0xcc7a7220: tag ufs, type VREG > usecount 1, writecount 1, refcount 2 mountedhere 0 > flags () > v_object 0xcfedf420 ref 0 pages 1 > lock type ufs: EXCL (count 1) by thread 0xcb76c600 (pid 48968) > ino 425264, on dev amrd0s1d >=20 > 0xcc693aa0: tag ufs, type VREG > usecount 1, writecount 1, refcount 2 mountedhere 0 > flags () > v_object 0xcf92f738 ref 0 pages 1 > lock type ufs: EXCL (count 1) by thread 0xcb655300 (pid 55381) > ino 425286, on dev amrd0s1d >=20 > 0xcbabf220: tag ufs, type VREG > usecount 1, writecount 1, refcount 2 mountedhere 0 > flags () > v_object 0xcfe297bc ref 0 pages 1 > lock type ufs: EXCL (count 1) by thread 0xd0234a80 (pid 63802) > ino 425322, on dev amrd0s1d >=20 > 0xcd760220: tag ufs, type VREG > usecount 1, writecount 1, refcount 2 mountedhere 0 > flags () > v_object 0xcb11ac60 ref 0 pages 1 > lock type ufs: EXCL (count 1) by thread 0xcadb0480 (pid 69938) > ino 425348, on dev amrd0s1d >=20 > 0xcc044990: tag ufs, type VREG > usecount 1, writecount 1, refcount 2 mountedhere 0 > flags () > v_object 0xcaaff084 ref 0 pages 1 > lock type ufs: EXCL (count 1) by thread 0xcadeda80 (pid 70418) > ino 425360, on dev amrd0s1d >=20 > 0xcc190660: tag ufs, type VREG > usecount 1, writecount 1, refcount 2 mountedhere 0 > flags () > v_object 0xcfed9108 ref 0 pages 1 > lock type ufs: EXCL (count 1) by thread 0xca7d2900 (pid 76803) > ino 425378, on dev amrd0s1d >=20 > 0xcc676330: tag ufs, type VREG > usecount 1, writecount 1, refcount 2 mountedhere 0 > flags () > v_object 0xcf8c14a4 ref 0 pages 1 > lock type ufs: EXCL (count 1) by thread 0xcaff8900 (pid 76841) > ino 425384, on dev amrd0s1d >=20 > 0xcf0ad110: tag ufs, type VREG > usecount 1, writecount 1, refcount 2 mountedhere 0 > flags () > v_object 0xce53b5ac ref 0 pages 1 > lock type ufs: EXCL (count 1) by thread 0xcadb0900 (pid 79849) > ino 425394, on dev amrd0s1d >=20 > 0xce4f6aa0: tag ufs, type VREG > usecount 1, writecount 1, refcount 2 mountedhere 0 > flags () > v_object 0xce3cc4a4 ref 0 pages 1 > lock type ufs: EXCL (count 1) by thread 0xcb767300 (pid 79620) > ino 425402, on dev amrd0s1d >=20 > 0xce80d110: tag ufs, type VREG > usecount 1, writecount 1, refcount 2 mountedhere 0 > flags () > v_object 0xcf502108 ref 0 pages 1 > lock type ufs: EXCL (count 1) by thread 0xcaff5d80 (pid 98225) > ino 425478, on dev amrd0s1d >=20 > 0xcd218990: tag ufs, type VREG > usecount 1, writecount 1, refcount 2 mountedhere 0 > flags () > v_object 0xcf08e18c ref 0 pages 1 > lock type ufs: EXCL (count 1) by thread 0xcaff5000 (pid 98241) > ino 425482, on dev amrd0s1d >=20 > 0xcbcb3440: tag ufs, type VREG > usecount 1, writecount 1, refcount 2 mountedhere 0 > flags () > v_object 0xcf8a8738 ref 0 pages 1 > lock type ufs: EXCL (count 1) by thread 0xcaff8180 (pid 1341) > ino 425505, on dev amrd0s1d >=20 > 0xcf6fe440: tag ufs, type VREG > usecount 1, writecount 1, refcount 2 mountedhere 0 > flags () > v_object 0xcf88f39c ref 0 pages 1 > lock type ufs: EXCL (count 1) by thread 0xcaad7900 (pid 4512) > ino 425512, on dev amrd0s1d >=20 > 0xcdd07aa0: tag ufs, type VREG > usecount 1, writecount 1, refcount 2 mountedhere 0 > flags () > v_object 0xcf7e9a50 ref 0 pages 1 > lock type ufs: EXCL (count 1) by thread 0xcadaf600 (pid 4464) > ino 425513, on dev amrd0s1d >=20 > 0xcc18eaa0: tag ufs, type VREG > usecount 1, writecount 1, refcount 2 mountedhere 0 > flags () > v_object 0xce2fa108 ref 0 pages 1 > lock type ufs: EXCL (count 1) by thread 0xcafbf300 (pid 13669) > ino 425549, on dev amrd0s1d >=20 > 0xcb9e1440: tag ufs, type VREG > usecount 1, writecount 1, refcount 2 mountedhere 0 > flags () > v_object 0xcfbea39c ref 0 pages 1 > lock type ufs: EXCL (count 1) by thread 0xcb64e480 (pid 13656) > ino 425555, on dev amrd0s1d >=20 > 0xcb8a8bb0: tag ufs, type VREG > usecount 1, writecount 1, refcount 2 mountedhere 0 > flags () > v_object 0xcf6565ac ref 0 pages 1 > lock type ufs: EXCL (count 1) by thread 0xcadb0180 (pid 22845) > ino 425596, on dev amrd0s1d >=20 > 0xccc47660: tag ufs, type VREG > usecount 1, writecount 1, refcount 2 mountedhere 0 > flags () > v_object 0xcf551b58 ref 0 pages 1 > lock type ufs: EXCL (count 1) by thread 0xcb64e180 (pid 22870) > ino 425597, on dev amrd0s1d >=20 > 0xcec8ecc0: tag ufs, type VREG > usecount 1, writecount 1, refcount 2 mountedhere 0 > flags () > v_object 0xcab61948 ref 0 pages 1 > lock type ufs: EXCL (count 1) by thread 0xd0234600 (pid 32036) > ino 425633, on dev amrd0s1d >=20 > 0xcdca1bb0: tag ufs, type VREG > usecount 2, writecount 2, refcount 888 mountedhere 0 > flags () > v_object 0xcaf23294 ref 0 pages 41308 > lock type ufs: EXCL (count 1) by thread 0xcadb0a80 (pid 5541) > ino 130855246, on dev amrd1s1d=20 >=20 > > -----Original Message----- > > From: Kris Kennaway [mailto:kris@obsecurity.org] > > Sent: Wednesday, May 16, 2007 12:33 PM > > To: Andrew Edwards > > Cc: freebsd-performance@freebsd.org > > Subject: Re: Ufs dead-locks on freebsd 6.2 > >=20 > > On Wed, May 16, 2007 at 12:08:24PM -0400, Andrew Edwards wrote: > > > I have a system running a dual intel zeon 2.8Ghz with 4G=20 > of ram and=20 > > > using an intel raid controller model SRCU42X which uses the > > amr driver. > > > I have had this server running 5.4 upgraded to 6.2 and=20 > was running=20 > > > fine for several months and then after a normal reboot=20 > I've started=20 > > > having all sorts of problems with what appears to be > > dead-locks in the > > > filesystem. This server is my backup server and I rsync=20 > files from=20 > > > various servers onto this one fairly non-stop. If I stop > > the rsync's > > > the system appears to be stable although I did have a > > kernel core just > > > last night. > > >=20 > > > When I have been able to observe the problem I ususally see one=20 > > > filesystem become inaccessible, perhaps var but I'm not > > sure, and then > > > in a short period of time the whole system is inaccessible.=20 > > Usually > > > if I startup just one of the rsync's within a couple of hours the=20 > > > system will be un-usable. > > >=20 > > > I did find this thread which seems to describe similar > > issues but this > > > is a different driver. > > >=20 > > http://lists.freebsd.org/pipermail/freebsd-questions/2006-Augu > st/127835. > > > html > >=20 > > Probably not relevant then. Deadlocks come in many varieties, all=20 > > different. > >=20 > > > Currently I'm running with debug.mpsafevfs=3D0,=20 > debug.mpsafenet=3D1and=20 > > > debug.mpsafevm=3D0 but this doesn't seem to help. > > >=20 > > > On perahps a related issue I have two other nearly > > identical systems > > > which were going to be upgrading to 6.2 as on 5.4 I am=20 > experiencing=20 > > > deadlocks and when I hit ctrl-t I see the system is=20 > either stuck in=20 > > > ufs or zoneinfo and I have not found very much information > > about zoneinfo. > > >=20 > > > Does anyone have any suggestions on what I can look for or other=20 > > > tuning options or had similar experiences? > >=20 > > See the chapter on kernel debugging in the developers=20 > handbook for the=20 > > information you need to provide before we can begin to debug your=20 > > problem. > >=20 > > Kris > >=20 > >=20 > _______________________________________________ > freebsd-performance@freebsd.org mailing list=20 > http://lists.freebsd.org/mailman/listinfo/freebsd-performance > To unsubscribe, send any mail to=20 > "freebsd-performance-unsubscribe@freebsd.org" >=20 From owner-freebsd-fs@FreeBSD.ORG Thu May 17 16:30:45 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3E1D116A400 for ; Thu, 17 May 2007 16:30:45 +0000 (UTC) (envelope-from hv@tuebingen.mpg.de) Received: from tuebingen.mpg.de (smtp-out.tuebingen.mpg.de [192.124.26.249]) by mx1.freebsd.org (Postfix) with ESMTP id AC31013C447 for ; Thu, 17 May 2007 16:30:44 +0000 (UTC) (envelope-from hv@tuebingen.mpg.de) Received: from [87.179.101.162] (HELO [192.168.2.32]) by tuebingen.mpg.de (CommuniGate Pro SMTP 4.2.10) with ESMTP id 33942741 for freebsd-fs@freebsd.org; Thu, 17 May 2007 18:30:43 +0200 Mime-Version: 1.0 (Apple Message framework v752.3) To: freebsd-fs@freebsd.org Message-Id: <7DE7C08F-8491-4503-AB77-E5D6A99308DE@tuebingen.mpg.de> Content-Type: multipart/signed; protocol="application/pgp-signature"; micalg=pgp-sha1; boundary="Apple-Mail-20--751215928" From: Henry Vogt Date: Thu, 17 May 2007 18:30:46 +0200 Content-Transfer-Encoding: 7bit X-Pgp-Agent: GPGMail 1.1.2 (Tiger) X-Mailer: Apple Mail (2.752.3) X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: growfs filesystem size limits ? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 May 2007 16:30:45 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --Apple-Mail-20--751215928 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=ISO-8859-1; delsp=yes; format=flowed While growing filesystems < 2TB (on FreeBSD 6.2-STABLE) works (even =20 with negative superblocknumbers in the output) it seems impossible to grow larger filesystems: Trying to grow a filesystem from ~4TB to ~6TB on a disk with a GPT =20 (single partition) start size index contents 0 1 PMBR 1 1 Pri GPT header 2 32 Pri GPT table 34 14648432573 1 GPT part - FreeBSD UFS/UFS2 14648432607 32 Sec GPT table 14648432639 1 Sec GPT header growfs fails: growfs: we are not growing (2197264879->440882671) What are the limits, are they bound to the largest possible bsdlabel =20 (i.e unsupported beyond 2^31-1 Sectors) ? Is it possible to grow such a filesystem ? What am i doing wrong ? Hints appreciated. Henry -- Netzwerk- und System Administration am MPI Campus T=FCbingen. Email: hv@tuebingen.mpg.de Tel. 07071/601-511 --Apple-Mail-20--751215928 content-type: application/pgp-signature; x-mac-type=70674453; name=PGP.sig content-description: Signierter Teil der Nachricht content-disposition: inline; filename=PGP.sig content-transfer-encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (Darwin) iD8DBQFGTIM2iF3PvXvQ0FARAolTAJ4z/GwDUkAuVc56ncGybMp8ihWnsgCfc2JZ Sv5HNiANF6Z6zHt+zIuVX6c= =F7uy -----END PGP SIGNATURE----- --Apple-Mail-20--751215928-- From owner-freebsd-fs@FreeBSD.ORG Thu May 17 16:40:06 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 187C416A400; Thu, 17 May 2007 16:40:06 +0000 (UTC) (envelope-from des@des.no) Received: from tim.des.no (tim.des.no [194.63.250.121]) by mx1.freebsd.org (Postfix) with ESMTP id BDD9713C44B; Thu, 17 May 2007 16:40:05 +0000 (UTC) (envelope-from des@des.no) Received: from tim.des.no (localhost [127.0.0.1]) by spam.des.no (Postfix) with ESMTP id AEC2E2086; Thu, 17 May 2007 18:39:58 +0200 (CEST) X-Spam-Tests: AWL X-Spam-Learn: disabled X-Spam-Score: 0.0/3.0 X-Spam-Checker-Version: SpamAssassin 3.1.7 (2006-10-05) on tim.des.no Received: from dwp.des.no (des.no [80.203.243.180]) by smtp.des.no (Postfix) with ESMTP id 91BF22084; Thu, 17 May 2007 18:39:58 +0200 (CEST) Received: by dwp.des.no (Postfix, from userid 1001) id 693D650AD; Thu, 17 May 2007 18:39:58 +0200 (CEST) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Darren Reed References: <20070409011723.GB74547@garage.freebsd.pl> <20070409094319.GB76673@garage.freebsd.pl> <70e8236f0704090808y5d305175wdc3cee5be1a26a9@mail.gmail.com> <20070409153338.GH76673@garage.freebsd.pl> <440b3e930705131031v5e97db7fq486d8d17aeb9f622@mail.gmail.com> <20070514070715.GA82322@hub.freebsd.org> <86646vyimg.fsf@dwp.des.no> <20070517124648.GA98052@hub.freebsd.org> Date: Thu, 17 May 2007 18:39:58 +0200 In-Reply-To: <20070517124648.GA98052@hub.freebsd.org> (Darren Reed's message of "Thu\, 17 May 2007 12\:46\:48 +0000") Message-ID: <86lkfntndt.fsf@dwp.des.no> User-Agent: Gnus/5.110006 (No Gnus v0.6) Emacs/21.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org, Pawel Jakub Dawidek , Joao Barros Subject: Re: ZFS: amd64, devd, root file system. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 May 2007 16:40:06 -0000 Darren Reed writes: > Dag-Erling Sm=C3=B8rgrav writes: > > Remember that unlike Sun, we do not make the hardware our OS runs on, > > nor do we write the firmware for it. > Do you see my email address as being "@sun.com"? No, but ZFS comes from Sun, and Solaris is the only OS that can currently boot from ZFS; therefore the comparison is germane. My point is that it is a lot more difficult for us to implement ZFS support in an OS that is intended to run on everything and the kitchen sink, from the crappiest 486 to the latest quad-core Xeon, with all kinds of crappy disk controllers, than it is for Sun to do the same in Solaris, which mostly runs on hardware designed and manufactured by Sun for the specific purpose of running Solaris. > I really don't understand why you needed to be so inflamatory with > your email, I'm not being inflammatory. I am just pointing out that we have limited resources and what you ask for is a lot of work - more than you probably realize - for little real benefit. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-fs@FreeBSD.ORG Thu May 17 16:46:37 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id B98D916A401 for ; Thu, 17 May 2007 16:46:37 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: from web30301.mail.mud.yahoo.com (web30301.mail.mud.yahoo.com [209.191.69.63]) by mx1.freebsd.org (Postfix) with SMTP id 69D4A13C45A for ; Thu, 17 May 2007 16:46:37 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: (qmail 98058 invoked by uid 60001); 17 May 2007 16:46:36 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding:Message-ID; b=CDbqFgOJawnSsIetDDRGsgmvXvRORmyeqR7bzgsN3xlzmBAzIE2F1qEzxdSGvVOGjZEN/de+DpVvNZmRnzRM5tZbZZNjldsVuaFHyBUrdVED9ckJ4sBOMdcEs8JnOutBpZ8DfQNgKMlRovG8jc63Jjm1OyfiNCCyluc+mI+EDE8=; X-YMail-OSG: 9Z8Uw1MVM1nzvCt7mh4EA1xvvXM1GlzuZ1I1EwLf9k8s6gJERwX1MreG4hWV.jyip017sIwdI3UFbmx3RVMP5AbdiAKKLQr8SEu8yqiArNBX3WgXVw9QUsHMqrIOWw-- Received: from [85.212.41.198] by web30301.mail.mud.yahoo.com via HTTP; Thu, 17 May 2007 09:46:36 PDT Date: Thu, 17 May 2007 09:46:36 -0700 (PDT) From: Arne "Wörner" To: Henry Vogt , freebsd-fs@freebsd.org In-Reply-To: <7DE7C08F-8491-4503-AB77-E5D6A99308DE@tuebingen.mpg.de> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Message-ID: <560126.97982.qm@web30301.mail.mud.yahoo.com> Cc: Subject: Re: growfs filesystem size limits ? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 May 2007 16:46:37 -0000 Hi Henry! --- Henry Vogt wrote: > What are the limits, are they bound to the largest possible bsdlabel > (i.e unsupported beyond 2^31-1 Sectors) ? > Is it possible to grow such a filesystem ? > What am i doing wrong ? Hints appreciated. > Yup... In line 1981 of the version 1.5 of src/sbin/growfs/growfs.c i c u_int32_t p_size; U might want to change it into u_int64_t p_size; or off_t p_size; and then compile it (maybe that helps)... :-)) *whoop*whoop* Use it on a test file system... :-)) I dont guarantee, that it will work... :-)) Do a fsck before and after... :-) Bye Arne ____________________________________________________________________________________Boardwalk for $500? In 2007? Ha! Play Monopoly Here and Now (it's updated for today's economy) at Yahoo! Games. http://get.games.yahoo.com/proddesc?gamekey=monopolyherenow From owner-freebsd-fs@FreeBSD.ORG Thu May 17 16:56:54 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D71A116A401 for ; Thu, 17 May 2007 16:56:54 +0000 (UTC) (envelope-from hv@tuebingen.mpg.de) Received: from tuebingen.mpg.de (smtp-out.tuebingen.mpg.de [192.124.26.249]) by mx1.freebsd.org (Postfix) with ESMTP id 4C7EC13C457 for ; Thu, 17 May 2007 16:56:53 +0000 (UTC) (envelope-from hv@tuebingen.mpg.de) Received: from [87.179.101.162] (HELO [192.168.2.32]) by tuebingen.mpg.de (CommuniGate Pro SMTP 4.2.10) with ESMTP id 33941106 for freebsd-fs@freebsd.org; Thu, 17 May 2007 17:56:52 +0200 Mime-Version: 1.0 (Apple Message framework v752.3) To: freebsd-fs@freebsd.org Message-Id: Content-Type: multipart/signed; protocol="application/pgp-signature"; micalg=pgp-sha1; boundary="Apple-Mail-14--753252725" From: Henry Vogt Date: Thu, 17 May 2007 17:56:50 +0200 Content-Transfer-Encoding: 7bit X-Pgp-Agent: GPGMail 1.1.2 (Tiger) X-Mailer: Apple Mail (2.752.3) X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: growfs filesystem size limits ? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 May 2007 16:56:54 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --Apple-Mail-14--753252725 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=ISO-8859-1; delsp=yes; format=flowed While growing filesystems < 2TB (on FreeBSD 6.2-STABLE) works (even =20 with negative superblocknumbers in the output) it seems impossible to grow larger filesystems: Trying to grow a filesystem from ~4TB to ~6TB on a disk with a GPT =20 (single partition) start size index contents 0 1 PMBR 1 1 Pri GPT header 2 32 Pri GPT table 34 14648432573 1 GPT part - FreeBSD UFS/UFS2 14648432607 32 Sec GPT table 14648432639 1 Sec GPT header growfs fails: growfs: we are not growing (2197264879->440882671) What are the limits, are they bound to the largest possible bsdlabel =20 (i.e unsupported beyond 2^31-1 Sectors) ? Is it possible to grow such a filesystem ? What am i doing wrong ? Hints appreciated. Henry -- Netzwerk- und System Administration am MPI Campus T=FCbingen. Email: hv@tuebingen.mpg.de Tel. 07071/601-511 --Apple-Mail-14--753252725 content-type: application/pgp-signature; x-mac-type=70674453; name=PGP.sig content-description: Signierter Teil der Nachricht content-disposition: inline; filename=PGP.sig content-transfer-encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (Darwin) iD8DBQFGTHtCiF3PvXvQ0FARAtGMAKCnIWRSrmuaQ9nuQg6Q3BK/E513UACgmAlC XkvU3yh6eyuU2DuF/IxKYH0= =uwQF -----END PGP SIGNATURE----- --Apple-Mail-14--753252725-- From owner-freebsd-fs@FreeBSD.ORG Thu May 17 17:01:11 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 90D7616A409 for ; Thu, 17 May 2007 17:01:11 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from relay01.kiev.sovam.com (relay01.kiev.sovam.com [62.64.120.200]) by mx1.freebsd.org (Postfix) with ESMTP id 1D21213C4D3 for ; Thu, 17 May 2007 17:01:11 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from [212.82.216.227] (helo=fw.zoral.com.ua) by relay01.kiev.sovam.com with esmtps (TLSv1:AES256-SHA:256) (Exim 4.60) (envelope-from ) id 1HojLh-0009bW-1o; Thu, 17 May 2007 20:01:09 +0300 Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by fw.zoral.com.ua (8.13.4/8.13.4) with ESMTP id l4HH119P086194 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 17 May 2007 20:01:01 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.1/8.14.1) with ESMTP id l4HH11tj043395; Thu, 17 May 2007 20:01:01 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.1/8.14.1/Submit) id l4HH10BM043394; Thu, 17 May 2007 20:01:00 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 17 May 2007 20:01:00 +0300 From: Kostik Belousov To: Andrew Edwards Message-ID: <20070517170100.GA41395@deviant.kiev.zoral.com.ua> References: <5230D3C40B842D4F9FB3CD368021BEF72F08F3@exchange-2.sandvine.com> <5230D3C40B842D4F9FB3CD368021BEF72F0908@exchange-2.sandvine.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="oyUTqETQ0mS9luUI" Content-Disposition: inline In-Reply-To: <5230D3C40B842D4F9FB3CD368021BEF72F0908@exchange-2.sandvine.com> User-Agent: Mutt/1.4.2.2i X-Virus-Scanned: ClamAV version 0.88.7, clamav-milter version 0.88.7 on fw.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-0.1 required=5.0 tests=ALL_TRUSTED,SPF_NEUTRAL autolearn=failed version=3.1.8 X-Spam-Checker-Version: SpamAssassin 3.1.8 (2007-02-13) on fw.zoral.com.ua X-Scanner-Signature: a96565856d7d777f03ae631b61e05b17 X-DrWeb-checked: yes X-SpamTest-Envelope-From: kostikbel@gmail.com X-SpamTest-Group-ID: 00000000 X-SpamTest-Info: Profiles 1053 [May 16 2007] X-SpamTest-Info: helo_type=3 X-SpamTest-Info: {received from trusted relay: not dialup} X-SpamTest-Method: none X-SpamTest-Method: Local Lists X-SpamTest-Rate: 0 X-SpamTest-Status: Not detected X-SpamTest-Status-Extended: not_detected X-SpamTest-Version: SMTP-Filter Version 3.0.0 [0255], KAS30/Release Cc: freebsd-fs@freebsd.org, freebsd-performance@freebsd.org Subject: Re: Ufs dead-locks on freebsd 6.2 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 May 2007 17:01:11 -0000 --oyUTqETQ0mS9luUI Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, May 17, 2007 at 11:44:15AM -0400, Andrew Edwards wrote: > I've upgraded to 6-stable, added the kernel options as per the kernel > handbook. After about 5 hours of they system in a deadlock it panic'd. > Here's the backtrace, and show pcpu, show allpcpu, show locks, show > alllocks, show lockedvnods and alltrace. >=20 > I will have the system down for approx another 15-20mins if there's > anything else someone would like while I'm in the debugger. >=20 >=20 > db> bt > Tracing pid 46784 tid 105112 td 0xd44a8000 > kdb_enter(c0785f13) at kdb_enter+0x2b > vfs_badlock(c0785f2c,c0786051,ccd47984) at vfs_badlock+0x47 > assert_vop_locked(ccd47984,c0786051) at assert_vop_locked+0x4a > vop_lock_post(f9f709dc,0,1002,ccd47984,f9f709f8,...) at > vop_lock_post+0x2a > VOP_LOCK_APV(c07dc2e0,f9f709dc) at VOP_LOCK_APV+0xa0 > vn_lock(ccd47984,1002,d44a8000) at vn_lock+0xac > lookup(f9f70c08) at lookup+0xde > namei(f9f70c08) at namei+0x39a > unp_connect(d44b2de8,d44dc380,d44a8000,d44b2de8,25,...) at > unp_connect+0xf0 > uipc_connect(d44b2de8,d44dc380,d44a8000) at uipc_connect+0x66 > soconnect(d44b2de8,d44dc380,d44a8000) at soconnect+0x4e > kern_connect(d44a8000,7,d44dc380,d44dc380,0,...) at kern_connect+0x74 > connect(d44a8000,f9f70d04) at connect+0x2f > syscall(3b,805003b,bfbf003b,bfbfd920,bfbfd922,...) at syscall+0x25b > Xint0x80_syscall() at Xint0x80_syscall+0x1f Could you, please, do the "show vnode 0xccd47984" from ddb prompt or "p/x *(struct vnode *)0xccd47984" from kgdb using dump for this panic ? Side note: it seems that on HEAD, vnode_if.awk does not generate call to vop_lock_{pre,post} due to mismatch in the name of vop and pre/post names. --oyUTqETQ0mS9luUI Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (FreeBSD) iD8DBQFGTIpLC3+MBN1Mb4gRAtdjAJ4pBRLR0W6+tKEzFLZBQaO/8TiSlgCZAa1B npATY/aNbd2+iRhJy8Mdjn8= =WWY8 -----END PGP SIGNATURE----- --oyUTqETQ0mS9luUI-- From owner-freebsd-fs@FreeBSD.ORG Thu May 17 17:04:40 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id CABFE16A411; Thu, 17 May 2007 17:04:40 +0000 (UTC) (envelope-from aedwards@sandvine.com) Received: from gw.sandvine.com (gw.sandvine.com [199.243.201.138]) by mx1.freebsd.org (Postfix) with ESMTP id 3E23413C501; Thu, 17 May 2007 17:04:39 +0000 (UTC) (envelope-from aedwards@sandvine.com) Received: from exchange-2.sandvine.com ([192.168.16.12]) by gw.sandvine.com with Microsoft SMTPSVC(6.0.3790.1830); Thu, 17 May 2007 13:03:38 -0400 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: quoted-printable Date: Thu, 17 May 2007 13:03:37 -0400 Message-ID: <5230D3C40B842D4F9FB3CD368021BEF72F090C@exchange-2.sandvine.com> In-Reply-To: <20070517170100.GA41395@deviant.kiev.zoral.com.ua> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Ufs dead-locks on freebsd 6.2 Thread-Index: AceYpPfEkW01y9yNRIWuy5UIKL+SuwAADblQ From: "Andrew Edwards" To: , X-OriginalArrivalTime: 17 May 2007 17:03:38.0127 (UTC) FILETIME=[4FEC71F0:01C798A5] Cc: Subject: RE: Ufs dead-locks on freebsd 6.2 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 May 2007 17:04:40 -0000 Here it is. db> show vnode 0xccd47984 vnode 0xccd47984: tag ufs, type VDIR usecount 5135, writecount 0, refcount 5137 mountedhere 0 flags (VV_ROOT) v_object 0xcd02518c ref 0 pages 1 #0 0xc0593f0d at lockmgr+0x4ed #1 0xc06b8e0e at ffs_lock+0x76 #2 0xc0739787 at VOP_LOCK_APV+0x87 #3 0xc0601c28 at vn_lock+0xac #4 0xc05ee832 at lookup+0xde #5 0xc05ee4b2 at namei+0x39a #6 0xc05e2ab0 at unp_connect+0xf0 #7 0xc05e1a6a at uipc_connect+0x66 #8 0xc05d9992 at soconnect+0x4e #9 0xc05dec60 at kern_connect+0x74 #10 0xc05debdf at connect+0x2f #11 0xc0723e2b at syscall+0x25b #12 0xc070ee0f at Xint0x80_syscall+0x1f ino 2, on dev amrd0s1a=20 > -----Original Message----- > From: Kostik Belousov [mailto:kostikbel@gmail.com]=20 > Sent: Thursday, May 17, 2007 1:01 PM > To: Andrew Edwards > Cc: freebsd-performance@freebsd.org; freebsd-fs@freebsd.org > Subject: Re: Ufs dead-locks on freebsd 6.2 >=20 > On Thu, May 17, 2007 at 11:44:15AM -0400, Andrew Edwards wrote: > > I've upgraded to 6-stable, added the kernel options as per=20 > the kernel=20 > > handbook. After about 5 hours of they system in a deadlock=20 > it panic'd. > > Here's the backtrace, and show pcpu, show allpcpu, show locks, show=20 > > alllocks, show lockedvnods and alltrace. > >=20 > > I will have the system down for approx another 15-20mins if there's=20 > > anything else someone would like while I'm in the debugger. > >=20 > >=20 > > db> bt > > Tracing pid 46784 tid 105112 td 0xd44a8000 > > kdb_enter(c0785f13) at kdb_enter+0x2b > > vfs_badlock(c0785f2c,c0786051,ccd47984) at vfs_badlock+0x47 > > assert_vop_locked(ccd47984,c0786051) at assert_vop_locked+0x4a > > vop_lock_post(f9f709dc,0,1002,ccd47984,f9f709f8,...) at=20 > > vop_lock_post+0x2a > > VOP_LOCK_APV(c07dc2e0,f9f709dc) at VOP_LOCK_APV+0xa0 > > vn_lock(ccd47984,1002,d44a8000) at vn_lock+0xac > > lookup(f9f70c08) at lookup+0xde > > namei(f9f70c08) at namei+0x39a > > unp_connect(d44b2de8,d44dc380,d44a8000,d44b2de8,25,...) at=20 > > unp_connect+0xf0 > > uipc_connect(d44b2de8,d44dc380,d44a8000) at uipc_connect+0x66 > > soconnect(d44b2de8,d44dc380,d44a8000) at soconnect+0x4e > > kern_connect(d44a8000,7,d44dc380,d44dc380,0,...) at=20 > kern_connect+0x74 > > connect(d44a8000,f9f70d04) at connect+0x2f > > syscall(3b,805003b,bfbf003b,bfbfd920,bfbfd922,...) at syscall+0x25b > > Xint0x80_syscall() at Xint0x80_syscall+0x1f >=20 > Could you, please, do the "show vnode 0xccd47984" from ddb=20 > prompt or "p/x *(struct vnode *)0xccd47984" from kgdb using=20 > dump for this panic ? >=20 > Side note: it seems that on HEAD, vnode_if.awk does not=20 > generate call to vop_lock_{pre,post} due to mismatch in the=20 > name of vop and pre/post names. >=20 >=20 >=20 From owner-freebsd-fs@FreeBSD.ORG Thu May 17 17:47:44 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 9178616A403; Thu, 17 May 2007 17:47:44 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from relay01.kiev.sovam.com (relay01.kiev.sovam.com [62.64.120.200]) by mx1.freebsd.org (Postfix) with ESMTP id 22D3313C455; Thu, 17 May 2007 17:47:44 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from [212.82.216.227] (helo=fw.zoral.com.ua) by relay01.kiev.sovam.com with esmtps (TLSv1:AES256-SHA:256) (Exim 4.60) (envelope-from ) id 1Hok4k-0000Sr-F3; Thu, 17 May 2007 20:47:43 +0300 Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by fw.zoral.com.ua (8.13.4/8.13.4) with ESMTP id l4HHlanA087265 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 17 May 2007 20:47:36 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.1/8.14.1) with ESMTP id l4HHlZpq044397; Thu, 17 May 2007 20:47:35 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.1/8.14.1/Submit) id l4HHlZSt044396; Thu, 17 May 2007 20:47:35 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 17 May 2007 20:47:35 +0300 From: Kostik Belousov To: Andrew Edwards Message-ID: <20070517174735.GB41395@deviant.kiev.zoral.com.ua> References: <20070517170100.GA41395@deviant.kiev.zoral.com.ua> <5230D3C40B842D4F9FB3CD368021BEF72F090C@exchange-2.sandvine.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="9zSXsLTf0vkW971A" Content-Disposition: inline In-Reply-To: <5230D3C40B842D4F9FB3CD368021BEF72F090C@exchange-2.sandvine.com> User-Agent: Mutt/1.4.2.2i X-Virus-Scanned: ClamAV version 0.88.7, clamav-milter version 0.88.7 on fw.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-0.1 required=5.0 tests=ALL_TRUSTED,SPF_NEUTRAL autolearn=failed version=3.1.8 X-Spam-Checker-Version: SpamAssassin 3.1.8 (2007-02-13) on fw.zoral.com.ua X-Scanner-Signature: 811d4bb63238b2e39a625e79cff8aaca X-DrWeb-checked: yes X-SpamTest-Envelope-From: kostikbel@gmail.com X-SpamTest-Group-ID: 00000000 X-SpamTest-Info: Profiles 1053 [May 16 2007] X-SpamTest-Info: helo_type=3 X-SpamTest-Info: {received from trusted relay: not dialup} X-SpamTest-Method: none X-SpamTest-Method: Local Lists X-SpamTest-Rate: 0 X-SpamTest-Status: Not detected X-SpamTest-Status-Extended: not_detected X-SpamTest-Version: SMTP-Filter Version 3.0.0 [0255], KAS30/Release Cc: freebsd-fs@freebsd.org, freebsd-performance@freebsd.org Subject: Re: Ufs dead-locks on freebsd 6.2 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 May 2007 17:47:44 -0000 --9zSXsLTf0vkW971A Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, May 17, 2007 at 01:03:37PM -0400, Andrew Edwards wrote: > Here it is. >=20 > db> show vnode 0xccd47984 > vnode 0xccd47984: tag ufs, type VDIR > usecount 5135, writecount 0, refcount 5137 mountedhere 0 > flags (VV_ROOT) > v_object 0xcd02518c ref 0 pages 1 > #0 0xc0593f0d at lockmgr+0x4ed > #1 0xc06b8e0e at ffs_lock+0x76 > #2 0xc0739787 at VOP_LOCK_APV+0x87 > #3 0xc0601c28 at vn_lock+0xac > #4 0xc05ee832 at lookup+0xde > #5 0xc05ee4b2 at namei+0x39a > #6 0xc05e2ab0 at unp_connect+0xf0 > #7 0xc05e1a6a at uipc_connect+0x66 > #8 0xc05d9992 at soconnect+0x4e > #9 0xc05dec60 at kern_connect+0x74 > #10 0xc05debdf at connect+0x2f > #11 0xc0723e2b at syscall+0x25b > #12 0xc070ee0f at Xint0x80_syscall+0x1f >=20 > ino 2, on dev amrd0s1a=20 It seems to be the sort of things that cannot happen. VOP_LOCK() returned 0, but vnode was not really locked. Although claiming that kernel code cannot have such bug is too optimistic, I would first make sure that: 1. You checked the memory of the machine. 2. Your kernel is built from pristine sources. >=20 > > -----Original Message----- > > From: Kostik Belousov [mailto:kostikbel@gmail.com]=20 > > Sent: Thursday, May 17, 2007 1:01 PM > > To: Andrew Edwards > > Cc: freebsd-performance@freebsd.org; freebsd-fs@freebsd.org > > Subject: Re: Ufs dead-locks on freebsd 6.2 > >=20 > > On Thu, May 17, 2007 at 11:44:15AM -0400, Andrew Edwards wrote: > > > I've upgraded to 6-stable, added the kernel options as per=20 > > the kernel=20 > > > handbook. After about 5 hours of they system in a deadlock=20 > > it panic'd. > > > Here's the backtrace, and show pcpu, show allpcpu, show locks, show= =20 > > > alllocks, show lockedvnods and alltrace. > > >=20 > > > I will have the system down for approx another 15-20mins if there's= =20 > > > anything else someone would like while I'm in the debugger. > > >=20 > > >=20 > > > db> bt > > > Tracing pid 46784 tid 105112 td 0xd44a8000 > > > kdb_enter(c0785f13) at kdb_enter+0x2b > > > vfs_badlock(c0785f2c,c0786051,ccd47984) at vfs_badlock+0x47 > > > assert_vop_locked(ccd47984,c0786051) at assert_vop_locked+0x4a > > > vop_lock_post(f9f709dc,0,1002,ccd47984,f9f709f8,...) at=20 > > > vop_lock_post+0x2a > > > VOP_LOCK_APV(c07dc2e0,f9f709dc) at VOP_LOCK_APV+0xa0 > > > vn_lock(ccd47984,1002,d44a8000) at vn_lock+0xac > > > lookup(f9f70c08) at lookup+0xde > > > namei(f9f70c08) at namei+0x39a > > > unp_connect(d44b2de8,d44dc380,d44a8000,d44b2de8,25,...) at=20 > > > unp_connect+0xf0 > > > uipc_connect(d44b2de8,d44dc380,d44a8000) at uipc_connect+0x66 > > > soconnect(d44b2de8,d44dc380,d44a8000) at soconnect+0x4e > > > kern_connect(d44a8000,7,d44dc380,d44dc380,0,...) at=20 > > kern_connect+0x74 > > > connect(d44a8000,f9f70d04) at connect+0x2f > > > syscall(3b,805003b,bfbf003b,bfbfd920,bfbfd922,...) at syscall+0x25b > > > Xint0x80_syscall() at Xint0x80_syscall+0x1f > >=20 > > Could you, please, do the "show vnode 0xccd47984" from ddb=20 > > prompt or "p/x *(struct vnode *)0xccd47984" from kgdb using=20 > > dump for this panic ? > >=20 > > Side note: it seems that on HEAD, vnode_if.awk does not=20 > > generate call to vop_lock_{pre,post} due to mismatch in the=20 > > name of vop and pre/post names. > >=20 > >=20 > >=20 > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" --9zSXsLTf0vkW971A Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (FreeBSD) iD8DBQFGTJU2C3+MBN1Mb4gRAi/sAKC7GVHcLs8Lq1rUKs/VL2JIk01atACfemRU vELK27Wc9MkcDj6Mn2mJRHQ= =h+0C -----END PGP SIGNATURE----- --9zSXsLTf0vkW971A-- From owner-freebsd-fs@FreeBSD.ORG Thu May 17 18:09:09 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id E36D416A406; Thu, 17 May 2007 18:09:09 +0000 (UTC) (envelope-from aedwards@sandvine.com) Received: from gw.sandvine.com (gw.sandvine.com [199.243.201.138]) by mx1.freebsd.org (Postfix) with ESMTP id 962FF13C468; Thu, 17 May 2007 18:09:09 +0000 (UTC) (envelope-from aedwards@sandvine.com) Received: from exchange-2.sandvine.com ([192.168.16.12]) by gw.sandvine.com with Microsoft SMTPSVC(6.0.3790.1830); Thu, 17 May 2007 14:08:08 -0400 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: quoted-printable Date: Thu, 17 May 2007 14:08:07 -0400 Message-ID: <5230D3C40B842D4F9FB3CD368021BEF72F090E@exchange-2.sandvine.com> In-Reply-To: <20070517174735.GB41395@deviant.kiev.zoral.com.ua> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Ufs dead-locks on freebsd 6.2 Thread-Index: AceYq3iW9WrZe9tvSra5wo/FsSspEgAAXReA From: "Andrew Edwards" To: , X-OriginalArrivalTime: 17 May 2007 18:08:08.0919 (UTC) FILETIME=[53186E70:01C798AE] Cc: Subject: RE: Ufs dead-locks on freebsd 6.2 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 May 2007 18:09:10 -0000 When I initially built my kernel it was from a fresh copy of 6.2-release which I then sync'd to 6-stable and I used cvsup to do it so I believe it should be clean. I will look into running memtest86 on it to see if anything had happened to the memory. > -----Original Message----- > From: Kostik Belousov [mailto:kostikbel@gmail.com]=20 > Sent: Thursday, May 17, 2007 1:48 PM > To: Andrew Edwards > Cc: freebsd-performance@freebsd.org; freebsd-fs@freebsd.org > Subject: Re: Ufs dead-locks on freebsd 6.2 >=20 > On Thu, May 17, 2007 at 01:03:37PM -0400, Andrew Edwards wrote: > > Here it is. > >=20 > > db> show vnode 0xccd47984 > > vnode 0xccd47984: tag ufs, type VDIR > > usecount 5135, writecount 0, refcount 5137 mountedhere 0 > > flags (VV_ROOT) > > v_object 0xcd02518c ref 0 pages 1 > > #0 0xc0593f0d at lockmgr+0x4ed > > #1 0xc06b8e0e at ffs_lock+0x76 > > #2 0xc0739787 at VOP_LOCK_APV+0x87 > > #3 0xc0601c28 at vn_lock+0xac > > #4 0xc05ee832 at lookup+0xde > > #5 0xc05ee4b2 at namei+0x39a > > #6 0xc05e2ab0 at unp_connect+0xf0 > > #7 0xc05e1a6a at uipc_connect+0x66 > > #8 0xc05d9992 at soconnect+0x4e > > #9 0xc05dec60 at kern_connect+0x74 > > #10 0xc05debdf at connect+0x2f > > #11 0xc0723e2b at syscall+0x25b > > #12 0xc070ee0f at Xint0x80_syscall+0x1f > >=20 > > ino 2, on dev amrd0s1a > It seems to be the sort of things that cannot happen.=20 > VOP_LOCK() returned 0, but vnode was not really locked. >=20 > Although claiming that kernel code cannot have such bug is=20 > too optimistic, I would first make sure that: > 1. You checked the memory of the machine. > 2. Your kernel is built from pristine sources. >=20 > >=20 > > > -----Original Message----- > > > From: Kostik Belousov [mailto:kostikbel@gmail.com] > > > Sent: Thursday, May 17, 2007 1:01 PM > > > To: Andrew Edwards > > > Cc: freebsd-performance@freebsd.org; freebsd-fs@freebsd.org > > > Subject: Re: Ufs dead-locks on freebsd 6.2 > > >=20 > > > On Thu, May 17, 2007 at 11:44:15AM -0400, Andrew Edwards wrote: > > > > I've upgraded to 6-stable, added the kernel options as per > > > the kernel > > > > handbook. After about 5 hours of they system in a deadlock > > > it panic'd. > > > > Here's the backtrace, and show pcpu, show allpcpu, show locks,=20 > > > > show alllocks, show lockedvnods and alltrace. > > > >=20 > > > > I will have the system down for approx another 15-20mins if=20 > > > > there's anything else someone would like while I'm in=20 > the debugger. > > > >=20 > > > >=20 > > > > db> bt > > > > Tracing pid 46784 tid 105112 td 0xd44a8000 > > > > kdb_enter(c0785f13) at kdb_enter+0x2b > > > > vfs_badlock(c0785f2c,c0786051,ccd47984) at vfs_badlock+0x47 > > > > assert_vop_locked(ccd47984,c0786051) at assert_vop_locked+0x4a > > > > vop_lock_post(f9f709dc,0,1002,ccd47984,f9f709f8,...) at=20 > > > > vop_lock_post+0x2a > > > > VOP_LOCK_APV(c07dc2e0,f9f709dc) at VOP_LOCK_APV+0xa0 > > > > vn_lock(ccd47984,1002,d44a8000) at vn_lock+0xac > > > > lookup(f9f70c08) at lookup+0xde > > > > namei(f9f70c08) at namei+0x39a > > > > unp_connect(d44b2de8,d44dc380,d44a8000,d44b2de8,25,...) at=20 > > > > unp_connect+0xf0 > > > > uipc_connect(d44b2de8,d44dc380,d44a8000) at uipc_connect+0x66 > > > > soconnect(d44b2de8,d44dc380,d44a8000) at soconnect+0x4e > > > > kern_connect(d44a8000,7,d44dc380,d44dc380,0,...) at > > > kern_connect+0x74 > > > > connect(d44a8000,f9f70d04) at connect+0x2f > > > > syscall(3b,805003b,bfbf003b,bfbfd920,bfbfd922,...) at=20 > > > > syscall+0x25b > > > > Xint0x80_syscall() at Xint0x80_syscall+0x1f > > >=20 > > > Could you, please, do the "show vnode 0xccd47984" from=20 > ddb prompt or=20 > > > "p/x *(struct vnode *)0xccd47984" from kgdb using dump for this=20 > > > panic ? > > >=20 > > > Side note: it seems that on HEAD, vnode_if.awk does not generate=20 > > > call to vop_lock_{pre,post} due to mismatch in the name=20 > of vop and=20 > > > pre/post names. > > >=20 > > >=20 > > >=20 > > _______________________________________________ > > freebsd-fs@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > > To unsubscribe, send any mail to=20 > "freebsd-fs-unsubscribe@freebsd.org" >=20 From owner-freebsd-fs@FreeBSD.ORG Thu May 17 19:28:45 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 80F2A16A402 for ; Thu, 17 May 2007 19:28:45 +0000 (UTC) (envelope-from hv@tuebingen.mpg.de) Received: from tuebingen.mpg.de (smtp-out.tuebingen.mpg.de [192.124.26.249]) by mx1.freebsd.org (Postfix) with ESMTP id 1AB3913C48C for ; Thu, 17 May 2007 19:28:44 +0000 (UTC) (envelope-from hv@tuebingen.mpg.de) Received: from [87.179.113.215] (HELO [192.168.2.32]) by tuebingen.mpg.de (CommuniGate Pro SMTP 4.2.10) with ESMTP id 33949198; Thu, 17 May 2007 21:28:43 +0200 In-Reply-To: <560126.97982.qm@web30301.mail.mud.yahoo.com> References: <560126.97982.qm@web30301.mail.mud.yahoo.com> Mime-Version: 1.0 (Apple Message framework v752.3) Content-Type: text/plain; charset=ISO-8859-1; delsp=yes; format=flowed Message-Id: <5A599F41-5F08-49BF-9902-1365629D717C@tuebingen.mpg.de> Content-Transfer-Encoding: quoted-printable From: Henry Vogt Date: Thu, 17 May 2007 21:28:40 +0200 To: =?ISO-8859-1?Q?Arne_W=F6rner?= X-Pgp-Agent: GPGMail 1.1.2 (Tiger) X-Mailer: Apple Mail (2.752.3) Cc: freebsd-fs@freebsd.org Subject: Re: growfs filesystem size limits ? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 May 2007 19:28:45 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi Arne, Am 17.05.2007 um 18:46 schrieb Arne W=F6rner: > ... > Yup... In line 1981 of the version 1.5 of > src/sbin/growfs/growfs.c > i c > u_int32_t p_size; > > U might want to change it into > u_int64_t p_size; > or > off_t p_size; > and then compile it > (maybe that helps)... :-)) > > *whoop*whoop* > Use it on a test file system... :-)) > I dont guarantee, that it will work... :-)) > Do a fsck before and after... Thanks for the tip. I've changed the appropriate lines (including =20 where referenced as u_int64_t) and compiled. Unfortunately it still doesn't succeed: growfs da1p1 ... new file systemsize is: 3662108143 frags Warning: 60028 sector(s) cannot be allocated. growfs: not enough new space Any further ideas ? btw. Sorry for twice posting the same mail, i had the impression the =20 first attempt didn't work.. Henry - -- Netzwerk- und System Administration am MPI Campus T=FCbingen. Email: hv@tuebingen.mpg.de Tel. 07071/601-511 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (Darwin) iD8DBQFGTKzoiF3PvXvQ0FARAg2oAJ4x1ymqpEPujLtlcv9qvTJQx2ynaQCglOE5 WdXosuTQrVVzOsleDR5gzmI=3D =3DoDEw -----END PGP SIGNATURE----- From owner-freebsd-fs@FreeBSD.ORG Thu May 17 20:21:50 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5D0DA16A405 for ; Thu, 17 May 2007 20:21:50 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: from web30312.mail.mud.yahoo.com (web30312.mail.mud.yahoo.com [209.191.69.74]) by mx1.freebsd.org (Postfix) with SMTP id 07FBE13C448 for ; Thu, 17 May 2007 20:21:49 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: (qmail 97705 invoked by uid 60001); 17 May 2007 20:21:49 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding:Message-ID; b=ORitLLeEeWm586YbKj+X5ia8DuEfZivX8PrvPauC4csngjCamudHkCVEqCAbu2odqmNyTEbyTVRriRHDWKRhrEUZlcAyddDuVeBOuVZJPZ5A0y/aQddDHd0aiBx+s+PnLQ9Ir9rwWOHkHizf0tLPx2hmjE2d1WVWqjVXO51XL98=; X-YMail-OSG: fUZkSdAVM1nMWX3uHo3luOxC2qEix05U1w2C7lkdWYXNJbhituobbbPns9sMb_29FQ-- Received: from [85.212.41.198] by web30312.mail.mud.yahoo.com via HTTP; Thu, 17 May 2007 13:21:49 PDT Date: Thu, 17 May 2007 13:21:49 -0700 (PDT) From: Arne "Wörner" To: Henry Vogt In-Reply-To: <5A599F41-5F08-49BF-9902-1365629D717C@tuebingen.mpg.de> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Message-ID: <188842.96824.qm@web30312.mail.mud.yahoo.com> Cc: freebsd-fs@freebsd.org Subject: Re: growfs filesystem size limits ? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 May 2007 20:21:50 -0000 --- Henry Vogt wrote: > Thanks for the tip. I've changed the appropriate lines (including > where referenced as u_int64_t) and compiled. > What do u mean with "referenced as u_int64_t"? There was just one line to change, I think... > growfs da1p1 > ... > new file systemsize is: 3662108143 frags > Warning: 60028 sector(s) cannot be allocated. > growfs: not enough new space > > Any further ideas ? > 1. backup & newfs & restore :-) 2. Change line 2231 from errx(1, "not enough new space"); to errx(1, "not enough new space (II) (%jd->%jd)", (intmax_t)osblock.fs_size, (intmax_t)sblock.fs_size); -Arne ____________________________________________________________________________________Got a little couch potato? Check out fun summer activities for kids. http://search.yahoo.com/search?fr=oni_on_mail&p=summer+activities+for+kids&cs=bz From owner-freebsd-fs@FreeBSD.ORG Fri May 18 04:38:22 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 70D2F16A407 for ; Fri, 18 May 2007 04:38:22 +0000 (UTC) (envelope-from anderson@freebsd.org) Received: from mh1.centtech.com (moat3.centtech.com [64.129.166.50]) by mx1.freebsd.org (Postfix) with ESMTP id 43E4813C458 for ; Fri, 18 May 2007 04:38:22 +0000 (UTC) (envelope-from anderson@freebsd.org) Received: from neutrino.centtech.com (andersonbox1.centtech.com [192.168.42.21]) by mh1.centtech.com (8.13.8/8.13.8) with ESMTP id l4I4cKCa028009; Thu, 17 May 2007 23:38:21 -0500 (CDT) (envelope-from anderson@freebsd.org) Message-ID: <464D2DBC.4040303@freebsd.org> Date: Thu, 17 May 2007 23:38:20 -0500 From: Eric Anderson User-Agent: Thunderbird 2.0.0.0 (X11/20070420) MIME-Version: 1.0 To: Kostik Belousov References: <20070517170100.GA41395@deviant.kiev.zoral.com.ua> <5230D3C40B842D4F9FB3CD368021BEF72F090C@exchange-2.sandvine.com> <20070517174735.GB41395@deviant.kiev.zoral.com.ua> In-Reply-To: <20070517174735.GB41395@deviant.kiev.zoral.com.ua> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.88.4/3267/Thu May 17 15:40:58 2007 on mh1.centtech.com X-Virus-Status: Clean X-Spam-Status: No, score=-2.6 required=8.0 tests=AWL,BAYES_00 autolearn=ham version=3.1.6 X-Spam-Checker-Version: SpamAssassin 3.1.6 (2006-10-03) on mh1.centtech.com Cc: freebsd-fs@freebsd.org Subject: Re: Ufs dead-locks on freebsd 6.2 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 May 2007 04:38:22 -0000 On 05/17/07 12:47, Kostik Belousov wrote: > On Thu, May 17, 2007 at 01:03:37PM -0400, Andrew Edwards wrote: >> Here it is. >> >> db> show vnode 0xccd47984 >> vnode 0xccd47984: tag ufs, type VDIR >> usecount 5135, writecount 0, refcount 5137 mountedhere 0 >> flags (VV_ROOT) >> v_object 0xcd02518c ref 0 pages 1 >> #0 0xc0593f0d at lockmgr+0x4ed >> #1 0xc06b8e0e at ffs_lock+0x76 >> #2 0xc0739787 at VOP_LOCK_APV+0x87 >> #3 0xc0601c28 at vn_lock+0xac >> #4 0xc05ee832 at lookup+0xde >> #5 0xc05ee4b2 at namei+0x39a >> #6 0xc05e2ab0 at unp_connect+0xf0 >> #7 0xc05e1a6a at uipc_connect+0x66 >> #8 0xc05d9992 at soconnect+0x4e >> #9 0xc05dec60 at kern_connect+0x74 >> #10 0xc05debdf at connect+0x2f >> #11 0xc0723e2b at syscall+0x25b >> #12 0xc070ee0f at Xint0x80_syscall+0x1f >> >> ino 2, on dev amrd0s1a > It seems to be the sort of things that cannot happen. VOP_LOCK() > returned 0, but vnode was not really locked. > > Although claiming that kernel code cannot have such bug is too optimistic, > I would first make sure that: > 1. You checked the memory of the machine. > 2. Your kernel is built from pristine sources. This looks precisely like a lock I was seeing on one of my NFS servers. Only one of the filesystems would cause it, but it was the same one each time, not necessarily under any kind of load. Things like mountd would get wedged in state 'ufs', and other things would get stuck in one of the lock states (I can't recall). After lots of hair pulling, I unmounted the file system, forcefully fsck'ed that one filesystem, and rebooted the box. Haven't had the issue since, and it's been up for about 100days. Eric From owner-freebsd-fs@FreeBSD.ORG Fri May 18 06:28:43 2007 Return-Path: X-Original-To: fs@freebsd.org Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id C272616A400 for ; Fri, 18 May 2007 06:28:43 +0000 (UTC) (envelope-from howard0su@gmail.com) Received: from qb-out-0506.google.com (qb-out-0506.google.com [72.14.204.231]) by mx1.freebsd.org (Postfix) with ESMTP id 7B45813C448 for ; Fri, 18 May 2007 06:28:41 +0000 (UTC) (envelope-from howard0su@gmail.com) Received: by qb-out-0506.google.com with SMTP id q12so430209qba for ; Thu, 17 May 2007 23:28:40 -0700 (PDT) DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=hic7BPQ0tOF2jODWpUJU6xAanKZciH7JLEl+QjsL3vhYw7zn36NiGDSIznIgQe4byI4mHQgfvKIVhJSIFwqMeGMrcGnrnwwU6AsP1g/Ox7KBygN4fPwd1XpnchY4tyciZV+r+CwcoYBNYGX3/Z3i5mmuWhHs3FbeWw/dWCxEvn8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=cqHHho2+hunHRB3c7XzQoEy7kenWzCirAFQZl5qwS1PnMpyhPNiHbrXr2fOhMOZUt1AWc4uYpQBk1+q1S4lSdb2c2cQxG5WqCCLdI7adtAJs6WfoMWqjJ9+M4ofZSHuF0nfW8aafPtYWr+ahyl/YBoO5HKngEr5fAtOTtklgGqs= Received: by 10.35.89.10 with SMTP id r10mr2214532pyl.1179469720779; Thu, 17 May 2007 23:28:40 -0700 (PDT) Received: by 10.35.78.11 with HTTP; Thu, 17 May 2007 23:28:40 -0700 (PDT) Message-ID: Date: Fri, 18 May 2007 14:28:40 +0800 From: "Howard Su" To: "Eric Anderson" In-Reply-To: <464C3DA7.3020003@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <464C3DA7.3020003@freebsd.org> Cc: fs@freebsd.org Subject: Re: size limit for TMPFS X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 May 2007 06:28:43 -0000 On 5/17/07, Eric Anderson wrote: > > Track the memory usage on your own? As you allocate, keep a counter and This is doable. The down site for this is we need another mutex to protect this resource usage. This mutex will result performance issue. because this mutex is per mount point. and when alloc/read a node, a directory. > total up the usage outside of uma. (3) sounds good, except it may not > be accurate, and that could lead to confusion for someone. Can we state, the tmpfs size only limit to the file size. Meta data will not be counted. Meta data limit will be adjusted by the kernel resource or explicitly specified by the user. -- -Howard From owner-freebsd-fs@FreeBSD.ORG Fri May 18 08:53:39 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3E90C16A403 for ; Fri, 18 May 2007 08:53:39 +0000 (UTC) (envelope-from hv@tuebingen.mpg.de) Received: from tuebingen.mpg.de (smtp-out.tuebingen.mpg.de [192.124.26.249]) by mx1.freebsd.org (Postfix) with ESMTP id D0FEC13C45A for ; Fri, 18 May 2007 08:53:38 +0000 (UTC) (envelope-from hv@tuebingen.mpg.de) Received: from [87.179.113.215] (HELO [192.168.2.32]) by tuebingen.mpg.de (CommuniGate Pro SMTP 4.2.10) with ESMTP id 33965362; Fri, 18 May 2007 10:53:32 +0200 In-Reply-To: <188842.96824.qm@web30312.mail.mud.yahoo.com> References: <188842.96824.qm@web30312.mail.mud.yahoo.com> Mime-Version: 1.0 (Apple Message framework v752.3) Content-Type: text/plain; charset=ISO-8859-1; format=flowed Message-Id: <334A2A05-4429-40DB-80D9-FAAFD5208DFA@tuebingen.mpg.de> Content-Transfer-Encoding: quoted-printable From: Henry Vogt Date: Fri, 18 May 2007 10:53:21 +0200 To: =?ISO-8859-1?Q?Arne_W=F6rner?= X-Pgp-Agent: GPGMail 1.1.2 (Tiger) X-Mailer: Apple Mail (2.752.3) Cc: freebsd-fs@freebsd.org Subject: Re: growfs filesystem size limits ? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 May 2007 08:53:39 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi Arne, Am 17.05.2007 um 22:21 schrieb Arne W=F6rner: > errx(1, "not enough new space (II) (%jd->%jd)", > (intmax_t)osblock.fs_size, (intmax_t)sblock.fs_size); # growfs -N da1p1 new file systemsize is: 3662108143 frags Warning: 60028 sector(s) cannot be allocated. growfs: not enough new space (II) (2197264879->-632874160) So, sblocks.fs_size is the problem ? Regards Henry - -- Netzwerk- und System Administration am MPI Campus T=FCbingen. Email: hv@tuebingen.mpg.de Tel. 07071/601-511 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (Darwin) iD8DBQFGTWmBiF3PvXvQ0FARAqwzAJ9n2aa4KebuD0Y/ar2MBUhrPT5iAACgvBgV qM7t9GaG9SdD+21UfNQadMU=3D =3Dgqku -----END PGP SIGNATURE----- From owner-freebsd-fs@FreeBSD.ORG Fri May 18 09:21:31 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id CB8F916A403 for ; Fri, 18 May 2007 09:21:31 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: from web30305.mail.mud.yahoo.com (web30305.mail.mud.yahoo.com [209.191.69.67]) by mx1.freebsd.org (Postfix) with SMTP id 7D88813C448 for ; Fri, 18 May 2007 09:21:31 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: (qmail 62418 invoked by uid 60001); 18 May 2007 09:21:31 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding:Message-ID; b=sTQpGJe2HVCUcgsWLAMw7ByHMUkzItcn04jbtF3byhKSIgT77JodA//DsHCaYXBqLxbs82U45s/h1R5jm9heKv46bIkopoxEA1ISGo8aD5hD3+5aGsRn5oCgzThgMM5jUnGrvwaoLZy6MK7NI9LxIBXEknujEdUAgR4dhNPiqOc=; X-YMail-OSG: LU02Kz8VM1mR.09FIoU48hoLZlBZu_JNXHzE0PdfeVZHDc299ZV_ymghss4rJJu09Q-- Received: from [85.212.48.180] by web30305.mail.mud.yahoo.com via HTTP; Fri, 18 May 2007 02:21:30 PDT Date: Fri, 18 May 2007 02:21:30 -0700 (PDT) From: Arne "Wörner" To: Henry Vogt In-Reply-To: <334A2A05-4429-40DB-80D9-FAAFD5208DFA@tuebingen.mpg.de> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Message-ID: <909641.62410.qm@web30305.mail.mud.yahoo.com> Cc: freebsd-fs@freebsd.org Subject: Re: growfs filesystem size limits ? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 May 2007 09:21:32 -0000 --- Henry Vogt wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Hi Arne, > Am 17.05.2007 um 22:21 schrieb Arne Wörner: > > > errx(1, "not enough new space (II) (%jd->%jd)", > > (intmax_t)osblock.fs_size, (intmax_t)sblock.fs_size); > > # growfs -N da1p1 > new file systemsize is: 3662108143 frags > Warning: 60028 sector(s) cannot be allocated. > growfs: not enough new space (II) (2197264879->-632874160) > > So, sblocks.fs_size is the problem ? > Nope... fs_size is 64bit... But it just came to me by looking at the code, that in line 2220 we build the product of two 32bit integers, which results in another 32bit integer, which is not right: sblock.fs_size = sblock.fs_ncg * sblock.fs_fpg; Better is this: sblock.fs_size = ((int64_t)sblock.fs_ncg) * sblock.fs_fpg; Or u could shape ur partition, so that this computation will not happen... Maybe there r further such mistakes... Integer arithmetric can be quite nasty (2+3=1 with 2bit unsigned integers)... But fast... :) -Arne ____________________________________________________________________________________ Park yourself in front of a world of choices in alternative vehicles. Visit the Yahoo! Auto Green Center. http://autos.yahoo.com/green_center/ From owner-freebsd-fs@FreeBSD.ORG Fri May 18 10:32:30 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 680) id 904E916A404; Fri, 18 May 2007 10:32:30 +0000 (UTC) Date: Fri, 18 May 2007 10:32:30 +0000 From: Darren Reed To: Dag-Erling Sm??rgrav Message-ID: <20070518103230.GA50946@hub.freebsd.org> References: <20070409011723.GB74547@garage.freebsd.pl> <20070409094319.GB76673@garage.freebsd.pl> <70e8236f0704090808y5d305175wdc3cee5be1a26a9@mail.gmail.com> <20070409153338.GH76673@garage.freebsd.pl> <440b3e930705131031v5e97db7fq486d8d17aeb9f622@mail.gmail.com> <20070514070715.GA82322@hub.freebsd.org> <86646vyimg.fsf@dwp.des.no> <20070517124648.GA98052@hub.freebsd.org> <86lkfntndt.fsf@dwp.des.no> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <86lkfntndt.fsf@dwp.des.no> User-Agent: Mutt/1.4.2.1i Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org, Pawel Jakub Dawidek , Joao Barros Subject: Re: ZFS: amd64, devd, root file system. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 May 2007 10:32:30 -0000 On Thu, May 17, 2007 at 06:39:58PM +0200, Dag-Erling Sm??rgrav wrote: > Darren Reed writes: > > Dag-Erling Sm??rgrav writes: > > > Remember that unlike Sun, we do not make the hardware our OS runs on, > > > nor do we write the firmware for it. > > Do you see my email address as being "@sun.com"? > > No, but ZFS comes from Sun, and Solaris is the only OS that can > currently boot from ZFS; therefore the comparison is germane. Solaris uses grub to boot from ZFS...I don't know how difficult that work was, but ZFS root can't be done with the old-style boot+loader on either sparc/x86. There may be implications in that for FreeBSD too. > My point is that it is a lot more difficult for us to implement ZFS > support in an OS that is intended to run on everything and the kitchen > sink, from the crappiest 486 to the latest quad-core Xeon, with all > kinds of crappy disk controllers, than it is for Sun to do the same in > Solaris, which mostly runs on hardware designed and manufactured by Sun > for the specific purpose of running Solaris. Which is why I'm using FreeBSD+ZFS rather than Solaris+ZFS :-) Maybe more layers are required in FreeBSD disk I/O subsystem to shield things like ZFS from crappy controllers? Darren From owner-freebsd-fs@FreeBSD.ORG Fri May 18 11:19:31 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 4A56316A401; Fri, 18 May 2007 11:19:31 +0000 (UTC) (envelope-from des@des.no) Received: from tim.des.no (tim.des.no [194.63.250.121]) by mx1.freebsd.org (Postfix) with ESMTP id CA8B713C43E; Fri, 18 May 2007 11:19:25 +0000 (UTC) (envelope-from des@des.no) Received: from tim.des.no (localhost [127.0.0.1]) by spam.des.no (Postfix) with ESMTP id D9D722086; Fri, 18 May 2007 13:19:21 +0200 (CEST) X-Spam-Tests: AWL X-Spam-Learn: disabled X-Spam-Score: 0.0/3.0 X-Spam-Checker-Version: SpamAssassin 3.1.7 (2006-10-05) on tim.des.no Received: from dwp.des.no (des.no [80.203.243.180]) by smtp.des.no (Postfix) with ESMTP id 56A702083; Fri, 18 May 2007 13:19:21 +0200 (CEST) Received: by dwp.des.no (Postfix, from userid 1001) id 3E05456C1; Fri, 18 May 2007 13:19:21 +0200 (CEST) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Darren Reed References: <20070409011723.GB74547@garage.freebsd.pl> <20070409094319.GB76673@garage.freebsd.pl> <70e8236f0704090808y5d305175wdc3cee5be1a26a9@mail.gmail.com> <20070409153338.GH76673@garage.freebsd.pl> <440b3e930705131031v5e97db7fq486d8d17aeb9f622@mail.gmail.com> <20070514070715.GA82322@hub.freebsd.org> <86646vyimg.fsf@dwp.des.no> <20070517124648.GA98052@hub.freebsd.org> <86lkfntndt.fsf@dwp.des.no> <20070518103230.GA50946@hub.freebsd.org> Date: Fri, 18 May 2007 13:19:21 +0200 In-Reply-To: <20070518103230.GA50946@hub.freebsd.org> (Darren Reed's message of "Fri\, 18 May 2007 10\:32\:30 +0000") Message-ID: <86tzuav0p2.fsf@dwp.des.no> User-Agent: Gnus/5.110006 (No Gnus v0.6) Emacs/21.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org, Pawel Jakub Dawidek , Joao Barros Subject: Re: ZFS: amd64, devd, root file system. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 May 2007 11:19:31 -0000 Darren Reed writes: > Dag-Erling Sm=C3=B8rgrav writes: > > My point is that it is a lot more difficult for us to implement ZFS > > support in an OS that is intended to run on everything and the kitchen > > sink, from the crappiest 486 to the latest quad-core Xeon, with all > > kinds of crappy disk controllers, than it is for Sun to do the same in > > Solaris, which mostly runs on hardware designed and manufactured by Sun > > for the specific purpose of running Solaris. > > Which is why I'm using FreeBSD+ZFS rather than Solaris+ZFS :-) > > Maybe more layers are required in FreeBSD disk I/O subsystem to > shield things like ZFS from crappy controllers? You completely misunderstand - the issue is not shielding ZFS from the crappy controllers - that is already taken care of - but having to reimplement the entire stack in the BTX loader, and in boot2 (which needs to be able to locate and load the BTX loader). DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-fs@FreeBSD.ORG Fri May 18 13:10:18 2007 Return-Path: X-Original-To: fs@freebsd.org Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3F4D816A401 for ; Fri, 18 May 2007 13:10:18 +0000 (UTC) (envelope-from anderson@freebsd.org) Received: from mh2.centtech.com (moat3.centtech.com [64.129.166.50]) by mx1.freebsd.org (Postfix) with ESMTP id 13BBF13C46A for ; Fri, 18 May 2007 13:10:17 +0000 (UTC) (envelope-from anderson@freebsd.org) Received: from neutrino.centtech.com (neutrino.centtech.com [10.177.171.220]) by mh2.centtech.com (8.13.8/8.13.8) with ESMTP id l4IDAFWH011030; Fri, 18 May 2007 08:10:16 -0500 (CDT) (envelope-from anderson@freebsd.org) Message-ID: <464DA5B7.1000908@freebsd.org> Date: Fri, 18 May 2007 08:10:15 -0500 From: Eric Anderson User-Agent: Thunderbird 2.0.0.0 (X11/20070420) MIME-Version: 1.0 To: Howard Su References: <464C3DA7.3020003@freebsd.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.88.4/3267/Thu May 17 15:40:58 2007 on mh2.centtech.com X-Virus-Status: Clean X-Spam-Status: No, score=-2.5 required=8.0 tests=AWL,BAYES_00 autolearn=ham version=3.1.6 X-Spam-Checker-Version: SpamAssassin 3.1.6 (2006-10-03) on mh2.centtech.com Cc: fs@freebsd.org Subject: Re: size limit for TMPFS X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 May 2007 13:10:18 -0000 On 05/18/07 01:28, Howard Su wrote: > On 5/17/07, Eric Anderson wrote: >> Track the memory usage on your own? As you allocate, keep a counter and > This is doable. The down site for this is we need another mutex to > protect this resource usage. This mutex will result performance issue. > because this mutex is per mount point. and when alloc/read a node, a > directory. From a quick glance, you are storing your inode count in the mount structure of the tmpfs. So, anytime you do anything with the fs, you have to lock the mount struct anyway, so there's no additional mutex when updating the block count vs inode count, right? Please correct me if I'm wrong here, since I'm still learning much about locking/vfs/etc. >> total up the usage outside of uma. (3) sounds good, except it may not >> be accurate, and that could lead to confusion for someone. > Can we state, the tmpfs size only limit to the file size. Meta data > will not be counted. Meta data limit will be adjusted by the kernel > resource or explicitly specified by the user. Seems reasonable, and if it's documented in the man page, it should reduce any confusion. Eric From owner-freebsd-fs@FreeBSD.ORG Fri May 18 15:02:49 2007 Return-Path: X-Original-To: fs@freebsd.org Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id C7FDE16A409 for ; Fri, 18 May 2007 15:02:49 +0000 (UTC) (envelope-from howard0su@gmail.com) Received: from py-out-1112.google.com (py-out-1112.google.com [64.233.166.178]) by mx1.freebsd.org (Postfix) with ESMTP id 8596413C448 for ; Fri, 18 May 2007 15:02:49 +0000 (UTC) (envelope-from howard0su@gmail.com) Received: by py-out-1112.google.com with SMTP id f31so1157657pyh for ; Fri, 18 May 2007 08:02:49 -0700 (PDT) DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=j+YdzjHsPRex3MYwCaDcaEmnj0v4EbP0duiGextUiKHqSan7FUXhHQOVONqJSbGph2ce8NnFsghXp6RJE7o3aKJJG5way7eWfWgXi7vu5V6S/nt7sXU4bnA8CEAXPztkakt13jn7h6KG5TkVzEoAeXno3cYAdEh+O2WGsXj2wvU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=Yj3I5sq4io943mRI81A3ELvtu+uOdP7Uj0AotITDfg8MJWTRYzqAdVBSFqch9pRirtEU5+0vOZiK+DqaZUfKNMmk9123Ficlfh8Ao7DM1+xEJyAW/bA4haptvjCPnrFJeRAgXwLtIMWIJgChGYqp1chMq0KvX79bUCignWl+fy8= Received: by 10.35.99.14 with SMTP id b14mr2910663pym.1179500568993; Fri, 18 May 2007 08:02:48 -0700 (PDT) Received: by 10.35.78.11 with HTTP; Fri, 18 May 2007 08:02:48 -0700 (PDT) Message-ID: Date: Fri, 18 May 2007 23:02:48 +0800 From: "Howard Su" To: "Eric Anderson" In-Reply-To: <464DA5B7.1000908@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <464C3DA7.3020003@freebsd.org> <464DA5B7.1000908@freebsd.org> Cc: fs@freebsd.org Subject: Re: size limit for TMPFS X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 May 2007 15:02:49 -0000 On 5/18/07, Eric Anderson wrote: > On 05/18/07 01:28, Howard Su wrote: > From a quick glance, you are storing your inode count in the mount > structure of the tmpfs. So, anytime you do anything with the fs, you > have to lock the mount struct anyway, so there's no additional mutex > when updating the block count vs inode count, right? Please correct me > if I'm wrong here, since I'm still learning much about locking/vfs/etc. You are right in one persipective. the memory consumed by inode can be counted in this way. However when we allocate the filename, dirent, you need acquire the mutex which is not needed currently. -- -Howard From owner-freebsd-fs@FreeBSD.ORG Fri May 18 15:52:11 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 44D2616A402 for ; Fri, 18 May 2007 15:52:11 +0000 (UTC) (envelope-from hv@tuebingen.mpg.de) Received: from tuebingen.mpg.de (smtp-out.tuebingen.mpg.de [192.124.26.249]) by mx1.freebsd.org (Postfix) with ESMTP id A22DB13C46C for ; Fri, 18 May 2007 15:52:09 +0000 (UTC) (envelope-from hv@tuebingen.mpg.de) Received: from [87.179.113.215] (HELO [192.168.2.32]) by tuebingen.mpg.de (CommuniGate Pro SMTP 4.2.10) with ESMTP id 33979445; Fri, 18 May 2007 17:52:08 +0200 In-Reply-To: <909641.62410.qm@web30305.mail.mud.yahoo.com> References: <909641.62410.qm@web30305.mail.mud.yahoo.com> Mime-Version: 1.0 (Apple Message framework v752.3) Content-Type: text/plain; charset=ISO-8859-1; delsp=yes; format=flowed Message-Id: <26D278C6-C000-48D0-AED4-0E6F703706FA@tuebingen.mpg.de> Content-Transfer-Encoding: quoted-printable From: Henry Vogt Date: Fri, 18 May 2007 17:52:05 +0200 To: =?ISO-8859-1?Q?Arne_W=F6rner?= X-Pgp-Agent: GPGMail 1.1.2 (Tiger) X-Mailer: Apple Mail (2.752.3) Cc: freebsd-fs@freebsd.org Subject: Re: growfs filesystem size limits ? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 May 2007 15:52:11 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Am 18.05.2007 um 11:21 schrieb Arne W=F6rner: > ... > Better is this: > sblock.fs_size =3D ((int64_t)sblock.fs_ncg) * sblock.fs_fpg; > > Or u could shape ur partition, so that this computation will not =20 > happen... > > Maybe there r further such mistakes... > Integer arithmetric can be quite nasty (2+3=3D1 with 2bit unsigned =20 > integers)... > But fast... :) > Now it worked! Many Thanks, the fs is now 6.6 TB (see below). The output is still somewhat strange, superblock numbers changing =20 from positive to negative and back.. but everything else looks ok. Many thanks for your tips. Here is my diff , build from you =20 suggestions, against: growfs.c,v 1.25 2006/07/17 20:48:36 - ------- C U T --------- 159c159 < static void get_dev_size(int, u_int64_t *); - --- > static void get_dev_size(int, int *); 1927c1927 < get_dev_size(int fd, u_int64_t *size) - --- > get_dev_size(int fd, int *size) 1980c1980 < u_int64_t p_size; - --- > u_int32_t p_size; 2131c2131 < errx(1, "there is not enough space (%lX < %d)", - --- > errx(1, "there is not enough space (%d < %d)", 2219c2219 < sblock.fs_size =3D ((int64_t)sblock.fs_ncg) * =20 sblock.fs_fpg; - --- > sblock.fs_size =3D sblock.fs_ncg * sblock.fs_fpg; 2230,2231c2230 < errx(1, "not enough new space (II) (%jd->%jd)", < (intmax_t)osblock.fs_size, (intmax_t)sblock.fs_size); - --- > errx(1, "not enough new space"); - ------- C U T --------- Perhaps it would be worth to find out how to correct the output ? =20 (see below) In the menatime i'm happy:-) Thanks again. # growfs da1p1 ... new file systemsize is: 3662108143 frags Warning: 60028 sector(s) cannot be allocated. growfs: 7152525.5MB (14648372544 sectors) block size 16384, fragment =20 size 2048 using 38922 cylinder groups of 183.77MB, 11761 blks, 23552 =20 inodes. with soft updates super-block backups (for fsck -b #) at: 199390176, 199766528, 200142880, 200519232, 200895584, 201271936, =20 201648288, 202024640, 202400992, 202777344, 203153696, 203530048, 203906400, 204282752, 204659104, =20 205035456, 205411808, 205788160, 206164512, 206540864, 206917216, 207293568, 207669920, 208046272, =20 208422624, 208798976, 209175328, ... 2140237440, 2140613792, 2140990144, 2141366496, 2141742848, =20 2142119200, 2142495552, 2142871904, 2143248256, 2143624608, 2144000960, 2144377312, 2144753664, 2145130016, =20 2145506368, 2145882720, 2146259072, 2146635424, 2147011776, 2147388128, -2147202816, -2146826464, -2146450112, =20 - -2146073760, -2145697408, -2145321056, - -2144944704, -2144568352, -2144192000, -2143815648, -2143439296, =20 - -2143062944, -2142686592, -2142310240, - -2141933888, -2141557536, -2141181184, -2140804832, -2140428480, =20 - -2140052128, -2139675776, -2139299424, ... - -8394400, -8018048, -7641696, -72653442, -6512640, -6136288, =20 - -5759936, -5383584, -5007232, - -4630880, -4254528, -3878176, -3501824, -3125472, -2749120, -2372768, =20= - -1996416, -1620064, -1243712, -867360, - -491008, -114656, 261696, 638048, 1014400, 1390752, 1767104, 2143456, =20= 2519808, 2896160, 3272, 3648864, 4025216, 4401568, 4777920, 5154272, 5530624, 5906976, 6283328, =20 6659680, 7036032, 7412384, 7788736, 8165088, 8541440, 8917792, 9294144, 9670496, 10046848, 10423200, 10799552, =20 11175904, 11552256, 11928608, 12304960, 12681312, 13057664, 13434016, 13810368, 14186720, 14563072, 14939424, =20= 15315776, 15692128, 16068480, 16444832, 16821184, 17197536, 17573888, 17950240, 18326592, 18702944, =20= 19079296, 19455648, 19832000, ... # df -h shows and it looks like everything is still there.. /dev/da1p1 6.6T 3.5T 2.5T 58% /mnt Regards Henry - -- Netzwerk- und System Administration am MPI Campus T=FCbingen. Email: hv@tuebingen.mpg.de Tel. 07071/601-511 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (Darwin) iD8DBQFGTculiF3PvXvQ0FARAiZ/AKCTamdDae+53LofsFH5wX1grFrT0ACfYvw9 W/N9mb4JtlrW5I3rRp5BKr4=3D =3DC/Os -----END PGP SIGNATURE----- From owner-freebsd-fs@FreeBSD.ORG Fri May 18 18:48:00 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 02D5216A400 for ; Fri, 18 May 2007 18:48:00 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: from web30308.mail.mud.yahoo.com (web30308.mail.mud.yahoo.com [209.191.69.70]) by mx1.freebsd.org (Postfix) with SMTP id C0CD213C455 for ; Fri, 18 May 2007 18:47:59 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: (qmail 23607 invoked by uid 60001); 18 May 2007 18:47:59 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding:Message-ID; b=toqzFU6AZJ4M2EXWbdzu1eaeEdtJXAxLtwTHwoVwo9fBnCG+4nhJpocP3O4OTkDPqF7dHyjinjDS6UTOzBoVcYJaSXmmjUPPrnXk0tBRM1ZScrltt8PF51tnIYWjpg8bOTt5eLXdvAUrJKBIzfM86lmOvdQ8cdLdx5vZIhCb834=; X-YMail-OSG: Ihxxk58VM1keKKeV3HEHwd3Cfq2Ri9mmq.G2AS34Mv4qBdxIrsudauXRTdfsrLHI9DOuaL94xxbk_THMFo.4jEgyg42c8GrPt8A72kYZXLrcdZ0bvfXMxmO2vA-- Received: from [85.212.48.180] by web30308.mail.mud.yahoo.com via HTTP; Fri, 18 May 2007 11:47:58 PDT Date: Fri, 18 May 2007 11:47:58 -0700 (PDT) From: Arne "Wörner" To: Henry Vogt In-Reply-To: <26D278C6-C000-48D0-AED4-0E6F703706FA@tuebingen.mpg.de> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Message-ID: <125170.23599.qm@web30308.mail.mud.yahoo.com> Cc: freebsd-fs@freebsd.org Subject: Re: growfs filesystem size limits ? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 May 2007 18:48:00 -0000 --- Henry Vogt wrote: > Am 18.05.2007 um 11:21 schrieb Arne Wörner: > Now it worked! Many Thanks, the fs is now 6.6 TB (see below). > The output is still somewhat strange, superblock numbers changing > from positive to negative and back.. but everything else looks ok. > Ohoh... I am afraid, that it damaged some data blocks now... :) Due to wrong offset... Although it looks like cylno maybe 32bit... In line 262-263 it says: j = sprintf(tmpbuf, " %d%s", (int)fsbtodb(&sblock, cgsblock(&sblock, cylno)), There we can see, that it casts the 64bit value to 32bit... Maybe u r right and it is just the output... It should be "%jd"... :-) Efficiency increased from 20% to 30%... ;-) <--- taken from "1984 (Orwell)"... > < static void get_dev_size(int, u_int64_t *); > - --- > > static void get_dev_size(int, int *); > 1927c1927 > < get_dev_size(int fd, u_int64_t *size) > - --- > > get_dev_size(int fd, int *size) > These r important, too... :-) > 1980c1980 > < u_int64_t p_size; > - --- > > u_int32_t p_size; > There is somewhere an "unsigned int size;" or so... It should be "u_int64_t", too... > # df -h shows and it looks like everything is still there.. > > /dev/da1p1 6.6T 3.5T 2.5T 58% /mnt > Have u done a "fsck /dev/da1p1", too? -Arne ____________________________________________________________________________________Pinpoint customers who are looking for what you sell. http://searchmarketing.yahoo.com/ From owner-freebsd-fs@FreeBSD.ORG Fri May 18 19:00:01 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id CC81316A405; Fri, 18 May 2007 19:00:01 +0000 (UTC) (envelope-from kris@obsecurity.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id B3AD913C45A; Fri, 18 May 2007 19:00:01 +0000 (UTC) (envelope-from kris@obsecurity.org) Received: from obsecurity.dyndns.org (elvis.mu.org [192.203.228.196]) by elvis.mu.org (Postfix) with ESMTP id 298041A4D86; Fri, 18 May 2007 12:00:56 -0700 (PDT) Received: by obsecurity.dyndns.org (Postfix, from userid 1000) id 5DE6D51453; Fri, 18 May 2007 15:00:00 -0400 (EDT) Date: Fri, 18 May 2007 15:00:00 -0400 From: Kris Kennaway To: Eric Anderson Message-ID: <20070518185959.GA42884@xor.obsecurity.org> References: <20070517170100.GA41395@deviant.kiev.zoral.com.ua> <5230D3C40B842D4F9FB3CD368021BEF72F090C@exchange-2.sandvine.com> <20070517174735.GB41395@deviant.kiev.zoral.com.ua> <464D2DBC.4040303@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <464D2DBC.4040303@freebsd.org> User-Agent: Mutt/1.4.2.2i Cc: freebsd-fs@freebsd.org Subject: Re: Ufs dead-locks on freebsd 6.2 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 May 2007 19:00:01 -0000 On Thu, May 17, 2007 at 11:38:20PM -0500, Eric Anderson wrote: > On 05/17/07 12:47, Kostik Belousov wrote: > >On Thu, May 17, 2007 at 01:03:37PM -0400, Andrew Edwards wrote: > >>Here it is. > >> > >>db> show vnode 0xccd47984 > >>vnode 0xccd47984: tag ufs, type VDIR > >> usecount 5135, writecount 0, refcount 5137 mountedhere 0 > >> flags (VV_ROOT) > >> v_object 0xcd02518c ref 0 pages 1 > >> #0 0xc0593f0d at lockmgr+0x4ed > >>#1 0xc06b8e0e at ffs_lock+0x76 > >>#2 0xc0739787 at VOP_LOCK_APV+0x87 > >>#3 0xc0601c28 at vn_lock+0xac > >>#4 0xc05ee832 at lookup+0xde > >>#5 0xc05ee4b2 at namei+0x39a > >>#6 0xc05e2ab0 at unp_connect+0xf0 > >>#7 0xc05e1a6a at uipc_connect+0x66 > >>#8 0xc05d9992 at soconnect+0x4e > >>#9 0xc05dec60 at kern_connect+0x74 > >>#10 0xc05debdf at connect+0x2f > >>#11 0xc0723e2b at syscall+0x25b > >>#12 0xc070ee0f at Xint0x80_syscall+0x1f > >> > >> ino 2, on dev amrd0s1a > >It seems to be the sort of things that cannot happen. VOP_LOCK() > >returned 0, but vnode was not really locked. > > > >Although claiming that kernel code cannot have such bug is too optimistic, > >I would first make sure that: > >1. You checked the memory of the machine. > >2. Your kernel is built from pristine sources. > > > This looks precisely like a lock I was seeing on one of my NFS servers. > Only one of the filesystems would cause it, but it was the same one > each time, not necessarily under any kind of load. Things like mountd > would get wedged in state 'ufs', and other things would get stuck in one > of the lock states (I can't recall). ...so you cannot conclude that it looks "precisely like" this case. Please, don't confuse bug reports by this kind of claim unless you have made a detailed comparison of the debugging traces to yours. Kris From owner-freebsd-fs@FreeBSD.ORG Fri May 18 19:09:00 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3C7CA16A404 for ; Fri, 18 May 2007 19:09:00 +0000 (UTC) (envelope-from anderson@freebsd.org) Received: from mh1.centtech.com (moat3.centtech.com [64.129.166.50]) by mx1.freebsd.org (Postfix) with ESMTP id 0EECE13C455 for ; Fri, 18 May 2007 19:08:59 +0000 (UTC) (envelope-from anderson@freebsd.org) Received: from neutrino.centtech.com (neutrino.centtech.com [10.177.171.220]) by mh1.centtech.com (8.13.8/8.13.8) with ESMTP id l4IJ8wwi082704; Fri, 18 May 2007 14:08:59 -0500 (CDT) (envelope-from anderson@freebsd.org) Message-ID: <464DF9CA.9090900@freebsd.org> Date: Fri, 18 May 2007 14:08:58 -0500 From: Eric Anderson User-Agent: Thunderbird 2.0.0.0 (X11/20070420) MIME-Version: 1.0 To: Kris Kennaway References: <20070517170100.GA41395@deviant.kiev.zoral.com.ua> <5230D3C40B842D4F9FB3CD368021BEF72F090C@exchange-2.sandvine.com> <20070517174735.GB41395@deviant.kiev.zoral.com.ua> <464D2DBC.4040303@freebsd.org> <20070518185959.GA42884@xor.obsecurity.org> In-Reply-To: <20070518185959.GA42884@xor.obsecurity.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.88.4/3267/Thu May 17 15:40:58 2007 on mh1.centtech.com X-Virus-Status: Clean X-Spam-Status: No, score=-2.6 required=8.0 tests=AWL,BAYES_00 autolearn=ham version=3.1.6 X-Spam-Checker-Version: SpamAssassin 3.1.6 (2006-10-03) on mh1.centtech.com Cc: freebsd-fs@freebsd.org Subject: Re: Ufs dead-locks on freebsd 6.2 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 May 2007 19:09:00 -0000 On 05/18/07 14:00, Kris Kennaway wrote: > On Thu, May 17, 2007 at 11:38:20PM -0500, Eric Anderson wrote: >> On 05/17/07 12:47, Kostik Belousov wrote: >>> On Thu, May 17, 2007 at 01:03:37PM -0400, Andrew Edwards wrote: >>>> Here it is. >>>> >>>> db> show vnode 0xccd47984 >>>> vnode 0xccd47984: tag ufs, type VDIR >>>> usecount 5135, writecount 0, refcount 5137 mountedhere 0 >>>> flags (VV_ROOT) >>>> v_object 0xcd02518c ref 0 pages 1 >>>> #0 0xc0593f0d at lockmgr+0x4ed >>>> #1 0xc06b8e0e at ffs_lock+0x76 >>>> #2 0xc0739787 at VOP_LOCK_APV+0x87 >>>> #3 0xc0601c28 at vn_lock+0xac >>>> #4 0xc05ee832 at lookup+0xde >>>> #5 0xc05ee4b2 at namei+0x39a >>>> #6 0xc05e2ab0 at unp_connect+0xf0 >>>> #7 0xc05e1a6a at uipc_connect+0x66 >>>> #8 0xc05d9992 at soconnect+0x4e >>>> #9 0xc05dec60 at kern_connect+0x74 >>>> #10 0xc05debdf at connect+0x2f >>>> #11 0xc0723e2b at syscall+0x25b >>>> #12 0xc070ee0f at Xint0x80_syscall+0x1f >>>> >>>> ino 2, on dev amrd0s1a >>> It seems to be the sort of things that cannot happen. VOP_LOCK() >>> returned 0, but vnode was not really locked. >>> >>> Although claiming that kernel code cannot have such bug is too optimistic, >>> I would first make sure that: >>> 1. You checked the memory of the machine. >>> 2. Your kernel is built from pristine sources. >> >> This looks precisely like a lock I was seeing on one of my NFS servers. >> Only one of the filesystems would cause it, but it was the same one >> each time, not necessarily under any kind of load. Things like mountd >> would get wedged in state 'ufs', and other things would get stuck in one >> of the lock states (I can't recall). > > ...so you cannot conclude that it looks "precisely like" this case. > > Please, don't confuse bug reports by this kind of claim unless you > have made a detailed comparison of the debugging traces to yours. Understood - my mistake. Eric From owner-freebsd-fs@FreeBSD.ORG Fri May 18 22:43:36 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id AC5AB16A400; Fri, 18 May 2007 22:43:36 +0000 (UTC) (envelope-from aedwards@sandvine.com) Received: from gw.sandvine.com (gw.sandvine.com [199.243.201.138]) by mx1.freebsd.org (Postfix) with ESMTP id 6746713C447; Fri, 18 May 2007 22:43:36 +0000 (UTC) (envelope-from aedwards@sandvine.com) Received: from exchange-2.sandvine.com ([192.168.16.12]) by gw.sandvine.com with Microsoft SMTPSVC(6.0.3790.1830); Fri, 18 May 2007 18:43:35 -0400 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: quoted-printable Date: Fri, 18 May 2007 18:43:35 -0400 Message-ID: <5230D3C40B842D4F9FB3CD368021BEF72F0926@exchange-2.sandvine.com> In-Reply-To: <464DF9CA.9090900@freebsd.org> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Ufs dead-locks on freebsd 6.2 Thread-Index: AceZgA8XJQM9a6noQX+h86ioyzHh9wAHOkwg From: "Andrew Edwards" To: , X-OriginalArrivalTime: 18 May 2007 22:43:35.0785 (UTC) FILETIME=[F849CD90:01C7999D] Cc: Subject: RE: Ufs dead-locks on freebsd 6.2 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 May 2007 22:43:36 -0000 Okay, I let memtest run for a full day and there has been no memory errors. What do I do next? Just to be on the safe side I'll fsck all of my fs's and try to reproduce the problem again. I also don't know what zonelimit is, I see this on similarily configured machines but running 5.4. I know it's related to network as I periodically get network connections to work i.e. ssh, ftp (both server and client side) but eventually the box will deadlock. Should I start a different thread on this? Happens about once every 30 days on two server although I havn't checked the exact timing. -----Original Message----- From: owner-freebsd-fs@freebsd.org [mailto:owner-freebsd-fs@freebsd.org] On Behalf Of Eric Anderson Sent: Friday, May 18, 2007 3:09 PM To: Kris Kennaway Cc: freebsd-fs@freebsd.org Subject: Re: Ufs dead-locks on freebsd 6.2 On 05/18/07 14:00, Kris Kennaway wrote: > On Thu, May 17, 2007 at 11:38:20PM -0500, Eric Anderson wrote: >> On 05/17/07 12:47, Kostik Belousov wrote: >>> On Thu, May 17, 2007 at 01:03:37PM -0400, Andrew Edwards wrote: >>>> Here it is. >>>> >>>> db> show vnode 0xccd47984 >>>> vnode 0xccd47984: tag ufs, type VDIR >>>> usecount 5135, writecount 0, refcount 5137 mountedhere 0 >>>> flags (VV_ROOT) >>>> v_object 0xcd02518c ref 0 pages 1 >>>> #0 0xc0593f0d at lockmgr+0x4ed >>>> #1 0xc06b8e0e at ffs_lock+0x76 >>>> #2 0xc0739787 at VOP_LOCK_APV+0x87 >>>> #3 0xc0601c28 at vn_lock+0xac >>>> #4 0xc05ee832 at lookup+0xde >>>> #5 0xc05ee4b2 at namei+0x39a >>>> #6 0xc05e2ab0 at unp_connect+0xf0 >>>> #7 0xc05e1a6a at uipc_connect+0x66 >>>> #8 0xc05d9992 at soconnect+0x4e >>>> #9 0xc05dec60 at kern_connect+0x74 >>>> #10 0xc05debdf at connect+0x2f >>>> #11 0xc0723e2b at syscall+0x25b >>>> #12 0xc070ee0f at Xint0x80_syscall+0x1f >>>> >>>> ino 2, on dev amrd0s1a >>> It seems to be the sort of things that cannot happen. VOP_LOCK()=20 >>> returned 0, but vnode was not really locked. >>> >>> Although claiming that kernel code cannot have such bug is too=20 >>> optimistic, I would first make sure that: >>> 1. You checked the memory of the machine. >>> 2. Your kernel is built from pristine sources. >> >> This looks precisely like a lock I was seeing on one of my NFS servers.=20 >> Only one of the filesystems would cause it, but it was the same one=20 >> each time, not necessarily under any kind of load. Things like=20 >> mountd would get wedged in state 'ufs', and other things would get=20 >> stuck in one of the lock states (I can't recall). >=20 > ...so you cannot conclude that it looks "precisely like" this case. >=20 > Please, don't confuse bug reports by this kind of claim unless you=20 > have made a detailed comparison of the debugging traces to yours. Understood - my mistake. Eric _______________________________________________ freebsd-fs@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-fs To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@FreeBSD.ORG Sat May 19 00:54:56 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 569C516A404 for ; Sat, 19 May 2007 00:54:56 +0000 (UTC) (envelope-from fernando@schapachnik.com.ar) Received: from mail.fac.org.ar (mail.fac.org.ar [200.59.49.21]) by mx1.freebsd.org (Postfix) with ESMTP id 666F913C45B for ; Sat, 19 May 2007 00:54:53 +0000 (UTC) (envelope-from fernando@schapachnik.com.ar) Received: from servidor1.cursosvirtuales.com.ar (www.cursosvirtuales.com.ar [200.59.46.198]) by mail.fac.org.ar (8.12.11/8.12.11) with ESMTP id l4J0dTWv002232 for ; Fri, 18 May 2007 21:39:33 -0300 (ART) (envelope-from fernando@schapachnik.com.ar) Received: from servidor1.cursosvirtuales.com.ar (localhost [127.0.0.1]) by servidor1.cursosvirtuales.com.ar (8.12.11/8.12.11) with ESMTP id l4J0dPKl004829 for ; Fri, 18 May 2007 21:39:25 -0300 (ART) (envelope-from fernando@schapachnik.com.ar) Received: from schapachnik.com.ar (uucp@localhost) by servidor1.cursosvirtuales.com.ar (8.12.11/8.12.11/Submit) with UUCP id l4J0dPbO004828 for freebsd-fs@freebsd.org; Fri, 18 May 2007 21:39:25 -0300 (ART) (envelope-from fernando@schapachnik.com.ar) Received: from funes.schapachnik.com.ar (localhost [127.0.0.1]) by funes.schapachnik.com.ar (8.13.8/8.13.8) with ESMTP id l4J0dG1N003326 for ; Fri, 18 May 2007 21:39:16 -0300 (ART) (envelope-from fernando@schapachnik.com.ar) Received: (from fpscha@localhost) by funes.schapachnik.com.ar (8.13.8/8.13.8/Submit) id l4J0dBMt003325 for freebsd-fs@freebsd.org; Fri, 18 May 2007 21:39:11 -0300 (ART) (envelope-from fernando@schapachnik.com.ar) X-Authentication-Warning: funes.schapachnik.com.ar: fpscha set sender to fernando@schapachnik.com.ar using -f Date: Fri, 18 May 2007 21:39:11 -0300 From: Fernando Schapachnik To: freebsd-fs@freebsd.org Message-ID: <20070519003910.GA3254@funes.schapachnik.com.ar> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.1i X-OS: FreeBSD 6.2 - http://www.freebsd.org Subject: Recovering a badly broken disk X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 May 2007 00:54:56 -0000 Hi, Please excuse if this is out of topic, but I guess is too specific for -questions. A disk with valuable data broke. The system used to perform backups in a BSD partition on the same disk. Among other things, the first sectors were damaged and a professional data recovery shop was able to recover an image of the remaining disk. I fdisk'ed as it was originally, then ran scan_ffs, which was able to found my backup partition, in mountable state. I tried to copy the ~170 MB backup tbz files to another place, but after ~70MB the kernel gives ICRC errors and the copying stops. The data recovery shop is not able to extract a better image, so my last chance is to skip the sectors with bad CRC and try bzip2recover. Unluckily, both cp and bzip2 abort on reading the faulty part. Any ideas on how to skip the faulty part and copy as much as possible? The partition in question is plain UFS1 on an ATA 80 GB disk. Thanks for any help. Fernando P. Schapachnik fernando@schapachnik.com.ar From owner-freebsd-fs@FreeBSD.ORG Sat May 19 04:35:27 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3068116A400; Sat, 19 May 2007 04:35:27 +0000 (UTC) (envelope-from aedwards@sandvine.com) Received: from gw.sandvine.com (gw.sandvine.com [199.243.201.138]) by mx1.freebsd.org (Postfix) with ESMTP id D196813C459; Sat, 19 May 2007 04:35:26 +0000 (UTC) (envelope-from aedwards@sandvine.com) Received: from exchange-2.sandvine.com ([192.168.16.12]) by gw.sandvine.com with Microsoft SMTPSVC(6.0.3790.1830); Sat, 19 May 2007 00:34:25 -0400 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: quoted-printable Date: Sat, 19 May 2007 00:34:25 -0400 Message-ID: <5230D3C40B842D4F9FB3CD368021BEF72F092A@exchange-2.sandvine.com> In-Reply-To: <5230D3C40B842D4F9FB3CD368021BEF72F0926@exchange-2.sandvine.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Ufs dead-locks on freebsd 6.2 Thread-Index: AceZgA8XJQM9a6noQX+h86ioyzHh9wAHOkwgAAtyb/A= From: "Andrew Edwards" To: , X-OriginalArrivalTime: 19 May 2007 04:34:25.0729 (UTC) FILETIME=[FB085B10:01C799CE] Cc: Subject: RE: Ufs dead-locks on freebsd 6.2 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 May 2007 04:35:27 -0000 Fsck didn't help but below is a list of processes that were stuck in disk. Also, one potential problem I've hit is I have mrtg scripts that get launched from cron every min. MRTG is supposed to have a locking mechanism to prevent the same script from running at the same time but I suspect since the filesystem was unaccessible the cron jobs just kept piling up and piling up until the system would eventually crash. I caught it when the load avg. was at 620 and killed all the cron's I could. That brought the load avg. down to under 1 however system is still taking up 30% of the processor time and the disks are basically idle. I can still do an ls -l on the root of all my mounted ufs and nfs filesystems but on one it's taking a considerable amount longer than the rest. This particular rsync that I was running is copying into the /d2 fs. The system is still running and I can make tpc connections and somethings I have running from inetd work but ssh stops responding right away and I can't logon via the console. So, I've captured a core dump of the system and rebooted so that I could use it again. Are there any suggestion as to what to do next? I'm debaiting installing an adaptec raid and rebuilding the system to see if I get the same problem, my worry is that it's the intel raid drivers that are causing this problem and I have 4 other systems with the same card. PID TT STAT TIME COMMAND 2 ?? DL 0:04.86 [g_event] 3 ?? DL 2:05.90 [g_up] 4 ?? DL 1:07.95 [g_down] 5 ?? DL 0:00.00 [xpt_thrd] 6 ?? DL 0:00.00 [kqueue taskq] 7 ?? DL 0:00.00 [thread taskq] 8 ?? DL 0:06.96 [pagedaemon] 9 ?? DL 0:00.00 [vmdaemon] 15 ?? DL 0:22.28 [yarrow] 24 ?? DL 0:00.01 [usb0] 25 ?? DL 0:00.00 [usbtask] 27 ?? DL 0:00.01 [usb1] 29 ?? DL 0:00.01 [usb2] 36 ?? DL 1:28.73 [pagezero] 37 ?? DL 0:08.76 [bufdaemon] 38 ?? DL 0:00.54 [vnlru] 39 ?? DL 1:08.12 [syncer] 40 ?? DL 0:04.00 [softdepflush] 41 ?? DL 0:11.05 [schedcpu] 27182 ?? Ds 0:05.75 /usr/sbin/syslogd -l /var/run/log -l /var/named/var/run/log -b 127.0.0.1 -a 10.128.0.0/10 27471 ?? Is 0:01.10 /usr/local/bin/postmaster -D /usr/local/pgsql/data (postgres) 27594 ?? Is 0:00.04 /usr/libexec/ftpd -m -D -l -l 27602 ?? DL 0:00.28 [smbiod1] 96581 ?? D 0:00.00 cron: running job (cron) 96582 ?? D 0:00.00 cron: running job (cron) 96583 ?? D 0:00.00 cron: running job (cron) 96585 ?? D 0:00.00 cron: running job (cron) 96586 ?? D 0:00.00 cron: running job (cron) 96587 ?? D 0:00.00 cron: running job (cron) 96588 ?? D 0:00.00 cron: running job (cron) 96589 ?? D 0:00.00 cron: running job (cron) 96590 ?? D 0:00.00 cron: running job (cron) 96591 ?? D 0:00.00 cron: running job (cron) 96592 ?? D 0:00.00 cron: running job (cron) 96593 ?? D 0:00.00 cron: running job (cron) 96594 ?? D 0:00.00 cron: running job (cron) 96607 ?? D 0:00.00 cron: running job (cron) 96608 ?? D 0:00.00 cron: running job (cron) 96609 ?? D 0:00.00 cron: running job (cron) 96610 ?? D 0:00.00 cron: running job (cron) 96611 ?? D 0:00.00 cron: running job (cron) 96612 ?? D 0:00.00 cron: running job (cron) 96613 ?? D 0:00.00 cron: running job (cron) 96614 ?? D 0:00.00 cron: running job (cron) 96615 ?? D 0:00.00 cron: running job (cron) 96616 ?? D 0:00.00 cron: running job (cron) 96617 ?? D 0:00.00 cron: running job (cron) 96631 ?? D 0:00.00 cron: running job (cron) 96632 ?? D 0:00.00 cron: running job (cron) 96633 ?? D 0:00.00 cron: running job (cron) 96634 ?? D 0:00.00 cron: running job (cron) 96635 ?? D 0:00.00 cron: running job (cron) 96636 ?? D 0:00.00 cron: running job (cron) 96637 ?? D 0:00.00 cron: running job (cron) 96638 ?? D 0:00.00 cron: running job (cron) 96639 ?? D 0:00.00 cron: running job (cron) 96642 ?? D 0:00.00 cron: running job (cron) 96650 ?? D 0:00.00 cron: running job (cron) 29393 p0 D+ 22:04.58 /usr/local/bin/rsync real 0m0.012s user 0m0.000s sys 0m0.010s / real 0m0.019s user 0m0.000s sys 0m0.016s /var real 0m0.028s user 0m0.008s sys 0m0.018s /diskless real 0m0.017s user 0m0.008s sys 0m0.007s /usr real 0m0.016s user 0m0.000s sys 0m0.015s /d2 real 0m0.024s user 0m0.000s sys 0m0.023s /exports/home real 0m2.559s user 0m0.216s sys 0m2.307s -----Original Message----- From: owner-freebsd-fs@freebsd.org [mailto:owner-freebsd-fs@freebsd.org] On Behalf Of Andrew Edwards Sent: Friday, May 18, 2007 6:44 PM To: freebsd-fs@freebsd.org; freebsd-performance@freebsd.org Subject: RE: Ufs dead-locks on freebsd 6.2 Okay, I let memtest run for a full day and there has been no memory errors. What do I do next? Just to be on the safe side I'll fsck all of my fs's and try to reproduce the problem again. I also don't know what zonelimit is, I see this on similarily configured machines but running 5.4. I know it's related to network as I periodically get network connections to work i.e. ssh, ftp (both server and client side) but eventually the box will deadlock. Should I start a different thread on this? Happens about once every 30 days on two server although I havn't checked the exact timing. -----Original Message----- From: owner-freebsd-fs@freebsd.org [mailto:owner-freebsd-fs@freebsd.org] On Behalf Of Eric Anderson Sent: Friday, May 18, 2007 3:09 PM To: Kris Kennaway Cc: freebsd-fs@freebsd.org Subject: Re: Ufs dead-locks on freebsd 6.2 On 05/18/07 14:00, Kris Kennaway wrote: > On Thu, May 17, 2007 at 11:38:20PM -0500, Eric Anderson wrote: >> On 05/17/07 12:47, Kostik Belousov wrote: >>> On Thu, May 17, 2007 at 01:03:37PM -0400, Andrew Edwards wrote: >>>> Here it is. >>>> >>>> db> show vnode 0xccd47984 >>>> vnode 0xccd47984: tag ufs, type VDIR >>>> usecount 5135, writecount 0, refcount 5137 mountedhere 0 >>>> flags (VV_ROOT) >>>> v_object 0xcd02518c ref 0 pages 1 >>>> #0 0xc0593f0d at lockmgr+0x4ed >>>> #1 0xc06b8e0e at ffs_lock+0x76 >>>> #2 0xc0739787 at VOP_LOCK_APV+0x87 >>>> #3 0xc0601c28 at vn_lock+0xac >>>> #4 0xc05ee832 at lookup+0xde >>>> #5 0xc05ee4b2 at namei+0x39a >>>> #6 0xc05e2ab0 at unp_connect+0xf0 >>>> #7 0xc05e1a6a at uipc_connect+0x66 >>>> #8 0xc05d9992 at soconnect+0x4e >>>> #9 0xc05dec60 at kern_connect+0x74 >>>> #10 0xc05debdf at connect+0x2f >>>> #11 0xc0723e2b at syscall+0x25b >>>> #12 0xc070ee0f at Xint0x80_syscall+0x1f >>>> >>>> ino 2, on dev amrd0s1a >>> It seems to be the sort of things that cannot happen. VOP_LOCK()=20 >>> returned 0, but vnode was not really locked. >>> >>> Although claiming that kernel code cannot have such bug is too=20 >>> optimistic, I would first make sure that: >>> 1. You checked the memory of the machine. >>> 2. Your kernel is built from pristine sources. >> >> This looks precisely like a lock I was seeing on one of my NFS servers.=20 >> Only one of the filesystems would cause it, but it was the same one=20 >> each time, not necessarily under any kind of load. Things like=20 >> mountd would get wedged in state 'ufs', and other things would get=20 >> stuck in one of the lock states (I can't recall). >=20 > ...so you cannot conclude that it looks "precisely like" this case. >=20 > Please, don't confuse bug reports by this kind of claim unless you=20 > have made a detailed comparison of the debugging traces to yours. Understood - my mistake. Eric _______________________________________________ freebsd-fs@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-fs To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" _______________________________________________ freebsd-fs@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-fs To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@FreeBSD.ORG Sat May 19 04:55:43 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 7222E16A400 for ; Sat, 19 May 2007 04:55:43 +0000 (UTC) (envelope-from anderson@freebsd.org) Received: from mh2.centtech.com (moat3.centtech.com [64.129.166.50]) by mx1.freebsd.org (Postfix) with ESMTP id 4722113C4C9 for ; Sat, 19 May 2007 04:55:43 +0000 (UTC) (envelope-from anderson@freebsd.org) Received: from neutrino.centtech.com (andersonbox1.centtech.com [192.168.42.21]) by mh2.centtech.com (8.13.8/8.13.8) with ESMTP id l4J4teNI079243; Fri, 18 May 2007 23:55:41 -0500 (CDT) (envelope-from anderson@freebsd.org) Message-ID: <464E834C.7060407@freebsd.org> Date: Fri, 18 May 2007 23:55:40 -0500 From: Eric Anderson User-Agent: Thunderbird 2.0.0.0 (X11/20070420) MIME-Version: 1.0 To: Fernando Schapachnik References: <20070519003910.GA3254@funes.schapachnik.com.ar> In-Reply-To: <20070519003910.GA3254@funes.schapachnik.com.ar> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.88.4/3269/Fri May 18 21:36:41 2007 on mh2.centtech.com X-Virus-Status: Clean X-Spam-Status: No, score=-2.5 required=8.0 tests=AWL,BAYES_00 autolearn=ham version=3.1.6 X-Spam-Checker-Version: SpamAssassin 3.1.6 (2006-10-03) on mh2.centtech.com Cc: freebsd-fs@freebsd.org Subject: Re: Recovering a badly broken disk X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 May 2007 04:55:43 -0000 On 05/18/07 19:39, Fernando Schapachnik wrote: > Hi, > Please excuse if this is out of topic, but I guess is too > specific for -questions. > > A disk with valuable data broke. The system used to perform > backups in a BSD partition on the same disk. Among other things, the > first sectors were damaged and a professional data recovery shop was > able to recover an image of the remaining disk. > > I fdisk'ed as it was originally, then ran scan_ffs, which > was able to found my backup partition, in mountable state. > > I tried to copy the ~170 MB backup tbz files to another > place, but after ~70MB the kernel gives ICRC errors and the copying > stops. The data recovery shop is not able to extract a better image, > so my last chance is to skip the sectors with bad CRC and try > bzip2recover. Unluckily, both cp and bzip2 abort on reading the > faulty part. > > Any ideas on how to skip the faulty part and copy as much as > possible? The partition in question is plain UFS1 on an ATA 80 GB > disk. You might be able to use dd's noerror option to conv to have it skip over any errors. Eric From owner-freebsd-fs@FreeBSD.ORG Sat May 19 05:58:30 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D580816A400 for ; Sat, 19 May 2007 05:58:30 +0000 (UTC) (envelope-from gore_jarold@yahoo.com) Received: from web63006.mail.re1.yahoo.com (web63006.mail.re1.yahoo.com [69.147.96.217]) by mx1.freebsd.org (Postfix) with SMTP id 72A6813C45A for ; Sat, 19 May 2007 05:58:30 +0000 (UTC) (envelope-from gore_jarold@yahoo.com) Received: (qmail 3524 invoked by uid 60001); 19 May 2007 05:58:26 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:Date:From:Subject:To:MIME-Version:Content-Type:Content-Transfer-Encoding:Message-ID; b=PxSofQCRNEvHqWzEuWzS8kxSfUvKOQ1q3cWoehBCj7mUioiK58CARdKMc+1at9KLQRypZyYvgc+ZhBMd3lDHzfOD1i9QWVeK0OhzipnzX2iAdiNTI1GntqEibcCjlUzZEx4TR/gQHjOUK0ZbdMSV07ZcSPMbRQMvQv4K06rMjTA=; X-YMail-OSG: MBiqt6IVM1nw.DJuirLG6UbCc4SLkw584UH9pasnDnV1mAenUUY1Qp_Bfi8vefWZRb.mUQLwbPIZg4YPL1o6xkyMKZXmWeBXuRR0U.IevGI0yx9.RPIkokZzZ7lwUjwt Received: from [24.118.228.153] by web63006.mail.re1.yahoo.com via HTTP; Fri, 18 May 2007 22:58:26 PDT Date: Fri, 18 May 2007 22:58:26 -0700 (PDT) From: Gore Jarold To: freebsd-fs@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Message-ID: <550589.3257.qm@web63006.mail.re1.yahoo.com> Subject: dangers of delaying an fsck on busy fileserver ? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 May 2007 05:58:30 -0000 I have a busy fileserver - 5-20 sftp/rsync processes running on it at all times. For unknown reasons, this server crashes in the middle of the night sometimes. When it does, I comment out my four big arrays in /etc/fstab, reboot, and fsck them manually (without a snapshot and BG fsck). Easy. The problem is, I need to sit around and wait for an fsck in the middle of the night and then re-edit fstab and reboot. So I am curious ... what happens if I instruct the NOC tech to just press the reset switch instead of calling me ? If he does this, the system will boot, the arrays will come online, and since I have a very very long time set until bg_fsck starts, I can then reboot the machine and foreground fsck it during sunlight hours. But it does mean that users will continue to operate on those dirty disks for 4-8 hours until I do that. Is this a dangerous strategy ? Does this put me at some increased risk of finding myself with disks that cannot be fsck'd ? (I've never seen it, but I have heard horror stories...) Will I lose a lot of the data that has been transacted during the hours that the disks were used in a dirty state ? Any comments ? ____________________________________________________________________________________ Food fight? Enjoy some healthy debate in the Yahoo! Answers Food & Drink Q&A. http://answers.yahoo.com/dir/?link=list&sid=396545367 From owner-freebsd-fs@FreeBSD.ORG Sat May 19 06:32:33 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 072BF16A400 for ; Sat, 19 May 2007 06:32:33 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.freebsd.org (Postfix) with ESMTP id B1FF513C4AD for ; Sat, 19 May 2007 06:32:32 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from phobos.samsco.home (phobos.samsco.home [192.168.254.11]) (authenticated bits=0) by pooker.samsco.org (8.13.8/8.13.8) with ESMTP id l4J6WSjN068178; Sat, 19 May 2007 00:32:29 -0600 (MDT) (envelope-from scottl@samsco.org) Message-ID: <464E99F3.3000602@samsco.org> Date: Sat, 19 May 2007 00:32:19 -0600 From: Scott Long User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.2pre) Gecko/20070111 SeaMonkey/1.1 MIME-Version: 1.0 To: Gore Jarold References: <550589.3257.qm@web63006.mail.re1.yahoo.com> In-Reply-To: <550589.3257.qm@web63006.mail.re1.yahoo.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0.2 (pooker.samsco.org [168.103.85.57]); Sat, 19 May 2007 00:32:29 -0600 (MDT) X-Spam-Status: No, score=-1.4 required=5.5 tests=ALL_TRUSTED autolearn=failed version=3.1.8 X-Spam-Checker-Version: SpamAssassin 3.1.8 (2007-02-13) on pooker.samsco.org Cc: freebsd-fs@freebsd.org Subject: Re: dangers of delaying an fsck on busy fileserver ? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 May 2007 06:32:33 -0000 Gore Jarold wrote: > I have a busy fileserver - 5-20 sftp/rsync processes > running on it at all times. > > For unknown reasons, this server crashes in the middle > of the night sometimes. When it does, I comment out > my four big arrays in /etc/fstab, reboot, and fsck > them manually (without a snapshot and BG fsck). > > Easy. The problem is, I need to sit around and wait > for an fsck in the middle of the night and then > re-edit fstab and reboot. > > So I am curious ... what happens if I instruct the NOC > tech to just press the reset switch instead of calling > me ? If he does this, the system will boot, the > arrays will come online, and since I have a very very > long time set until bg_fsck starts, I can then reboot > the machine and foreground fsck it during sunlight > hours. > > But it does mean that users will continue to operate > on those dirty disks for 4-8 hours until I do that. > > Is this a dangerous strategy ? > > Does this put me at some increased risk of finding > myself with disks that cannot be fsck'd ? (I've never > seen it, but I have heard horror stories...) > > Will I lose a lot of the data that has been transacted > during the hours that the disks were used in a dirty > state ? > > Any comments ? > In an ideal world, the only consequence of delaying bgfsck is that not all filesystem blocks will be marked free that should be. So if you deleted a large tree of files before the crash, those blocks might still show up in use until bgfsck completes. Scott From owner-freebsd-fs@FreeBSD.ORG Sat May 19 09:08:35 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 8E8B416A405 for ; Sat, 19 May 2007 09:08:35 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from relay01.kiev.sovam.com (relay01.kiev.sovam.com [62.64.120.200]) by mx1.freebsd.org (Postfix) with ESMTP id 3623313C44B for ; Sat, 19 May 2007 09:08:35 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from [212.82.216.227] (helo=fw.zoral.com.ua) by relay01.kiev.sovam.com with esmtps (TLSv1:AES256-SHA:256) (Exim 4.60) (envelope-from ) id 1HpKvR-000Fyq-Ag for freebsd-fs@freebsd.org; Sat, 19 May 2007 12:08:34 +0300 Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by fw.zoral.com.ua (8.13.4/8.13.4) with ESMTP id l4J98TZB050741 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 19 May 2007 12:08:29 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.1/8.14.1) with ESMTP id l4J98TNs024914; Sat, 19 May 2007 12:08:29 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.1/8.14.1/Submit) id l4J98TWQ024913; Sat, 19 May 2007 12:08:29 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 19 May 2007 12:08:29 +0300 From: Kostik Belousov To: Gore Jarold Message-ID: <20070519090828.GO41395@deviant.kiev.zoral.com.ua> References: <550589.3257.qm@web63006.mail.re1.yahoo.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="KIbT1ud6duwZIwNL" Content-Disposition: inline In-Reply-To: <550589.3257.qm@web63006.mail.re1.yahoo.com> User-Agent: Mutt/1.4.2.2i X-Virus-Scanned: ClamAV version 0.88.7, clamav-milter version 0.88.7 on fw.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-0.1 required=5.0 tests=ALL_TRUSTED,SPF_NEUTRAL autolearn=failed version=3.1.8 X-Spam-Checker-Version: SpamAssassin 3.1.8 (2007-02-13) on fw.zoral.com.ua X-Scanner-Signature: 1e897a521fdb5eac39b1f944f187ca57 X-DrWeb-checked: yes X-SpamTest-Envelope-From: kostikbel@gmail.com X-SpamTest-Group-ID: 00000000 X-SpamTest-Info: Profiles 1054 [May 19 2007] X-SpamTest-Info: helo_type=3 X-SpamTest-Info: {received from trusted relay: not dialup} X-SpamTest-Method: none X-SpamTest-Method: Local Lists X-SpamTest-Rate: 0 X-SpamTest-Status: Not detected X-SpamTest-Status-Extended: not_detected X-SpamTest-Version: SMTP-Filter Version 3.0.0 [0255], KAS30/Release Cc: freebsd-fs@freebsd.org Subject: Re: dangers of delaying an fsck on busy fileserver ? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 May 2007 09:08:35 -0000 --KIbT1ud6duwZIwNL Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, May 18, 2007 at 10:58:26PM -0700, Gore Jarold wrote: > I have a busy fileserver - 5-20 sftp/rsync processes > running on it at all times. >=20 > For unknown reasons, this server crashes in the middle > of the night sometimes. When it does, I comment out What exactly you mean by crashing ? There is one pending MFC that shall cure a deadlock/panic scenario for busy fs being snapshotted. It awaits re@ approval. Also, there is a situation where exceeding inode quota on softupdate fs could cause snapshotting to deadlock (fix is in progress). --KIbT1ud6duwZIwNL Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (FreeBSD) iD8DBQFGTr6MC3+MBN1Mb4gRAtM+AJwNNRDArSktDBf943FbyzVFLNpCBwCaAv8a ETwYiBQuIpFMjs+q9PYQlUY= =9Xo4 -----END PGP SIGNATURE----- --KIbT1ud6duwZIwNL-- From owner-freebsd-fs@FreeBSD.ORG Sat May 19 10:05:07 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 7128116A406 for ; Sat, 19 May 2007 10:05:07 +0000 (UTC) (envelope-from hv@tuebingen.mpg.de) Received: from tuebingen.mpg.de (smtp-out.tuebingen.mpg.de [192.124.26.249]) by mx1.freebsd.org (Postfix) with ESMTP id 0A97313C4C2 for ; Sat, 19 May 2007 10:05:06 +0000 (UTC) (envelope-from hv@tuebingen.mpg.de) Received: from [87.179.101.1] (HELO [192.168.2.32]) by tuebingen.mpg.de (CommuniGate Pro SMTP 4.2.10) with ESMTP id 34005303; Sat, 19 May 2007 12:05:04 +0200 In-Reply-To: <125170.23599.qm@web30308.mail.mud.yahoo.com> References: <125170.23599.qm@web30308.mail.mud.yahoo.com> Mime-Version: 1.0 (Apple Message framework v752.3) Content-Type: text/plain; charset=ISO-8859-1; delsp=yes; format=flowed Message-Id: Content-Transfer-Encoding: quoted-printable From: Henry Vogt Date: Sat, 19 May 2007 12:05:00 +0200 To: =?ISO-8859-1?Q?Arne_W=F6rner?= X-Pgp-Agent: GPGMail 1.1.2 (Tiger) X-Mailer: Apple Mail (2.752.3) Cc: freebsd-fs@freebsd.org Subject: Re: growfs filesystem size limits ? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 May 2007 10:05:07 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Am 18.05.2007 um 20:47 schrieb Arne W=F6rner: > ... > Maybe u r right and it is just the output... > It should be "%jd"... :-) > >> ... >> /dev/da1p1 6.6T 3.5T 2.5T 58% /mnt >> > Have u done a "fsck /dev/da1p1", too? Aehm, it's still running (the first attempt failed and i aborted it, =20 because the logfile filled up the disk. There were a lot of unexpected softupdate inconsistencies, though. I'm now logging to a bigger disk;-) Henry - -- Netzwerk- und System Administration am MPI Campus T=FCbingen. Email: hv@tuebingen.mpg.de Tel. 07071/601-511 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (Darwin) iD8DBQFGTsvMiF3PvXvQ0FARAkz9AKCTdBsvoW0mU3Vj/U+EF+fsq8yehwCguN3m HnziormBOvZgP6B+7ORsD1k=3D =3DAVlG -----END PGP SIGNATURE----- From owner-freebsd-fs@FreeBSD.ORG Sat May 19 15:22:20 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id BF56016A41F for ; Sat, 19 May 2007 15:22:20 +0000 (UTC) (envelope-from gore_jarold@yahoo.com) Received: from web63014.mail.re1.yahoo.com (web63014.mail.re1.yahoo.com [69.147.96.241]) by mx1.freebsd.org (Postfix) with SMTP id 5C30A13C455 for ; Sat, 19 May 2007 15:22:20 +0000 (UTC) (envelope-from gore_jarold@yahoo.com) Received: (qmail 73604 invoked by uid 60001); 19 May 2007 15:22:19 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding:Message-ID; b=wguuLOwNONCE/13na6JDJ7d06oNlHTH+sOKd2eEP6r+zvY+9W6XFPHu9l12xlhzjvuX0JxtdNf+pqz1RmoipW3gHyb49p3gO6LukufoVm89pOZXsN0xApL9kK2UdF/Vf1eqkfwDnvW7IVfwvnJLZhWB0OcRdKC0mJp2/ubhhPgs=; X-YMail-OSG: Id8AWtgVM1kduG6JxpDaWai43hiqRAaWf22I2VLbzTe5UJ0WMQV7PGaXtxhfnjdB24hjP.tvZG0qKjsGadj8Tis9Fg6FM8ZDsGA2 Received: from [24.118.228.153] by web63014.mail.re1.yahoo.com via HTTP; Sat, 19 May 2007 08:22:19 PDT Date: Sat, 19 May 2007 08:22:19 -0700 (PDT) From: Gore Jarold To: Scott Long In-Reply-To: <464E99F3.3000602@samsco.org> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Message-ID: <620211.71116.qm@web63014.mail.re1.yahoo.com> Cc: freebsd-fs@freebsd.org Subject: Re: dangers of delaying an fsck on busy fileserver ? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 May 2007 15:22:20 -0000 --- Scott Long wrote: > In an ideal world, the only consequence of delaying > bgfsck is that > not all filesystem blocks will be marked free that > should be. So > if you deleted a large tree of files before the > crash, those blocks > might still show up in use until bgfsck completes. Thank you. Would _you_ do this with valuable data ? ____________________________________________________________________________________ Need Mail bonding? Go to the Yahoo! Mail Q&A for great tips from Yahoo! Answers users. http://answers.yahoo.com/dir/?link=list&sid=396546091 From owner-freebsd-fs@FreeBSD.ORG Sat May 19 17:19:01 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 328F316A421 for ; Sat, 19 May 2007 17:19:01 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.freebsd.org (Postfix) with ESMTP id E154813C45D for ; Sat, 19 May 2007 17:19:00 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from phobos.samsco.home (phobos.samsco.home [192.168.254.11]) (authenticated bits=0) by pooker.samsco.org (8.13.8/8.13.8) with ESMTP id l4JHIvDx071135; Sat, 19 May 2007 11:18:58 -0600 (MDT) (envelope-from scottl@samsco.org) Message-ID: <464F3178.1020909@samsco.org> Date: Sat, 19 May 2007 11:18:48 -0600 From: Scott Long User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.2pre) Gecko/20070111 SeaMonkey/1.1 MIME-Version: 1.0 To: Gore Jarold References: <620211.71116.qm@web63014.mail.re1.yahoo.com> In-Reply-To: <620211.71116.qm@web63014.mail.re1.yahoo.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0.2 (pooker.samsco.org [168.103.85.57]); Sat, 19 May 2007 11:18:58 -0600 (MDT) X-Spam-Status: No, score=-1.4 required=5.5 tests=ALL_TRUSTED autolearn=failed version=3.1.8 X-Spam-Checker-Version: SpamAssassin 3.1.8 (2007-02-13) on pooker.samsco.org Cc: freebsd-fs@freebsd.org Subject: Re: dangers of delaying an fsck on busy fileserver ? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 May 2007 17:19:01 -0000 Gore Jarold wrote: > --- Scott Long wrote: > > >> In an ideal world, the only consequence of delaying >> bgfsck is that >> not all filesystem blocks will be marked free that >> should be. So >> if you deleted a large tree of files before the >> crash, those blocks >> might still show up in use until bgfsck completes. > > > Thank you. Would _you_ do this with valuable data ? > Very good question =-) If you're using softupdates then any damage will have been done when the hard shutdown happens; bgfsck won't create any new damage. The biggest problem of bgfsck beyond the i/o slowness and near deadlocks that it can create (modulo the fixes that the Kostik is working on) is that if it does encounter damage that it can't fix automatically, it exits and leaves the filesystem inconsistent. So you need to keep a very close eye on your logs and check for this, then schedule downtime when it happens so you can babysit a full fsck. Scott From owner-freebsd-fs@FreeBSD.ORG Sat May 19 17:33:04 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 060CD16A41F for ; Sat, 19 May 2007 17:33:04 +0000 (UTC) (envelope-from fernando@schapachnik.com.ar) Received: from mail.fac.org.ar (mail.fac.org.ar [200.59.49.21]) by mx1.freebsd.org (Postfix) with ESMTP id EF51D13C447 for ; Sat, 19 May 2007 17:33:01 +0000 (UTC) (envelope-from fernando@schapachnik.com.ar) Received: from servidor1.cursosvirtuales.com.ar (www.cursosvirtuales.com.ar [200.59.46.198]) by mail.fac.org.ar (8.12.11/8.12.11) with ESMTP id l4JHWlNF039183; Sat, 19 May 2007 14:32:56 -0300 (ART) (envelope-from fernando@schapachnik.com.ar) Received: from servidor1.cursosvirtuales.com.ar (localhost [127.0.0.1]) by servidor1.cursosvirtuales.com.ar (8.12.11/8.12.11) with ESMTP id l4JHWhGB061913; Sat, 19 May 2007 14:32:44 -0300 (ART) (envelope-from fernando@schapachnik.com.ar) Received: from schapachnik.com.ar (uucp@localhost) by servidor1.cursosvirtuales.com.ar (8.12.11/8.12.11/Submit) with UUCP id l4JHWhvw061912; Sat, 19 May 2007 14:32:43 -0300 (ART) (envelope-from fernando@schapachnik.com.ar) Received: from funes.schapachnik.com.ar (localhost [127.0.0.1]) by funes.schapachnik.com.ar (8.13.8/8.13.8) with ESMTP id l4JHWewf003045; Sat, 19 May 2007 14:32:40 -0300 (ART) (envelope-from fernando@schapachnik.com.ar) Received: (from fpscha@localhost) by funes.schapachnik.com.ar (8.13.8/8.13.8/Submit) id l4JHWZe2003044; Sat, 19 May 2007 14:32:35 -0300 (ART) (envelope-from fernando@schapachnik.com.ar) X-Authentication-Warning: funes.schapachnik.com.ar: fpscha set sender to fernando@schapachnik.com.ar using -f Date: Sat, 19 May 2007 14:32:35 -0300 From: Fernando Schapachnik To: Eric Anderson Message-ID: <20070519173235.GI1340@funes.schapachnik.com.ar> References: <20070519003910.GA3254@funes.schapachnik.com.ar> <464E834C.7060407@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <464E834C.7060407@freebsd.org> User-Agent: Mutt/1.4.2.1i X-OS: FreeBSD 6.2 - http://www.freebsd.org Cc: freebsd-fs@freebsd.org Subject: Re: Recovering a badly broken disk X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 May 2007 17:33:04 -0000 En un mensaje anterior, Eric Anderson escribió: > On 05/18/07 19:39, Fernando Schapachnik wrote: > > Any ideas on how to skip the faulty part and copy as much as > >possible? The partition in question is plain UFS1 on an ATA 80 GB > >disk. > > > You might be able to use dd's noerror option to conv to have it skip > over any errors. Thanks, I'll try that! Fernando P. Schapachnik fernando@schapachnik.com.ar From owner-freebsd-fs@FreeBSD.ORG Sat May 19 18:49:08 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 94B6E16A41F for ; Sat, 19 May 2007 18:49:08 +0000 (UTC) (envelope-from julian@elischer.org) Received: from outR.internet-mail-service.net (outR.internet-mail-service.net [216.240.47.241]) by mx1.freebsd.org (Postfix) with ESMTP id 811DB13C489 for ; Sat, 19 May 2007 18:49:08 +0000 (UTC) (envelope-from julian@elischer.org) Received: from mx0.idiom.com (HELO idiom.com) (216.240.32.160) by out.internet-mail-service.net (qpsmtpd/0.32) with ESMTP; Sat, 19 May 2007 11:49:08 -0700 Received: from julian-mac.elischer.org (unknown [137.122.39.10]) by idiom.com (Postfix) with ESMTP id 04A98125B3F; Sat, 19 May 2007 11:49:06 -0700 (PDT) Message-ID: <464F46A3.3030509@elischer.org> Date: Sat, 19 May 2007 14:49:07 -0400 From: Julian Elischer User-Agent: Thunderbird 2.0.0.0 (Macintosh/20070326) MIME-Version: 1.0 To: Fernando Schapachnik References: <20070519003910.GA3254@funes.schapachnik.com.ar> <464E834C.7060407@freebsd.org> <20070519173235.GI1340@funes.schapachnik.com.ar> In-Reply-To: <20070519173235.GI1340@funes.schapachnik.com.ar> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit Cc: freebsd-fs@freebsd.org Subject: Re: Recovering a badly broken disk X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 May 2007 18:49:08 -0000 Fernando Schapachnik wrote: > En un mensaje anterior, Eric Anderson escribió: >> On 05/18/07 19:39, Fernando Schapachnik wrote: >>> Any ideas on how to skip the faulty part and copy as much as >>> possible? The partition in question is plain UFS1 on an ATA 80 GB >>> disk. >> >> You might be able to use dd's noerror option to conv to have it skip >> over any errors. > > Thanks, I'll try that! > > Fernando P. Schapachnik > fernando@schapachnik.com.ar > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" check recoverdisk http://cvsup.pt.freebsd.org/cgi-bin/cvsweb/cvsweb.cgi/src/tools/tools/recoverdisk/