From owner-freebsd-fs@FreeBSD.ORG Mon Sep 8 15:44:53 2003 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3F2E916A4BF for ; Mon, 8 Sep 2003 15:44:53 -0700 (PDT) Received: from smtp5.andrew.cmu.edu (SMTP5.andrew.cmu.edu [128.2.10.85]) by mx1.FreeBSD.org (Postfix) with ESMTP id 547C343FAF for ; Mon, 8 Sep 2003 15:44:52 -0700 (PDT) (envelope-from eno@andrew.cmu.edu) Received: from andrew.cmu.edu (BLISS.PDL.CMU.EDU [128.2.134.47]) (user=eno mech=PLAIN (0 bits))h88MiouO032001 (version=TLSv1/SSLv3 cipher=RC4-MD5 bits=128 verify=NOT) for ; Mon, 8 Sep 2003 18:44:50 -0400 Message-ID: <3F5D0663.7020902@andrew.cmu.edu> Date: Mon, 08 Sep 2003 18:44:51 -0400 From: Eno Thereska User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.0.0) Gecko/20020529 X-Accept-Language: en-us, en MIME-Version: 1.0 To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: free block bitmap X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Sep 2003 22:44:53 -0000 Hi, I am using FreeBSD4.4 with FFS as the underlying file system. I need to examine which blocks in the file system are currently in use. These blocks can be metadata or data blocks. I have looked at the FFS code and it seems the only way to get the list of free blocks (or busy blocks) is by querying each cylinder group's (cg) bitmap of free blocks. Is there a global bitmap that keeps track of busy/free blocks over the entire file system or is the only option the cg-by-cg lookup? Thanks a lot Eno From owner-freebsd-fs@FreeBSD.ORG Mon Sep 8 16:27:09 2003 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7BB8016A4BF for ; Mon, 8 Sep 2003 16:27:09 -0700 (PDT) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.FreeBSD.org (Postfix) with ESMTP id BF1E243FDD for ; Mon, 8 Sep 2003 16:27:08 -0700 (PDT) (envelope-from bright@elvis.mu.org) Received: by elvis.mu.org (Postfix, from userid 1192) id B12372ED439; Mon, 8 Sep 2003 16:27:08 -0700 (PDT) Date: Mon, 8 Sep 2003 16:27:08 -0700 From: Alfred Perlstein To: Eno Thereska Message-ID: <20030908232708.GE15671@elvis.mu.org> References: <3F5D0663.7020902@andrew.cmu.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3F5D0663.7020902@andrew.cmu.edu> User-Agent: Mutt/1.4.1i cc: freebsd-fs@freebsd.org Subject: Re: free block bitmap X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Sep 2003 23:27:09 -0000 * Eno Thereska [030908 15:45] wrote: > Hi, > > I am using FreeBSD4.4 with FFS as the underlying file system. > I need to examine which blocks in the file system > are currently in use. These blocks can be metadata or data blocks. > > I have looked at the FFS code and it seems the only way to get > the list of free blocks (or busy blocks) is by querying > each cylinder group's (cg) bitmap of free blocks. > > Is there a global bitmap that keeps track of busy/free blocks > over the entire file system or is the only option > the cg-by-cg lookup? afaik, cg by cg is how you'll have to do it. -- - Alfred Perlstein - Research Engineering Development Inc. - email: bright@mu.org cell: 408-480-4684 From owner-freebsd-fs@FreeBSD.ORG Wed Sep 10 01:30:33 2003 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3EEE516A4BF for ; Wed, 10 Sep 2003 01:30:33 -0700 (PDT) Received: from stork.mail.pas.earthlink.net (stork.mail.pas.earthlink.net [207.217.120.188]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8A8B743FD7 for ; Wed, 10 Sep 2003 01:30:32 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from user-2ivfl2g.dialup.mindspring.com ([165.247.212.80] helo=mindspring.com) by stork.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 19x0Mw-0002xH-00; Wed, 10 Sep 2003 01:30:31 -0700 Message-ID: <3F5EE0E8.40BF5801@mindspring.com> Date: Wed, 10 Sep 2003 01:29:28 -0700 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Eno Thereska References: <3F5D0663.7020902@andrew.cmu.edu> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a4b640fbc401a98c39e1d7cbb67e53e9dc667c3043c0873f7e350badd9bab72f9c350badd9bab72f9c cc: freebsd-fs@freebsd.org Subject: Re: free block bitmap X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Sep 2003 08:30:33 -0000 Eno Thereska wrote: > I am using FreeBSD4.4 with FFS as the underlying file system. > I need to examine which blocks in the file system > are currently in use. These blocks can be metadata or data blocks. > > I have looked at the FFS code and it seems the only way to get > the list of free blocks (or busy blocks) is by querying > each cylinder group's (cg) bitmap of free blocks. > > Is there a global bitmap that keeps track of busy/free blocks > over the entire file system or is the only option > the cg-by-cg lookup? You can do it two ways: 1) Iterate over every file in the filesystem, and examine their direct and indirect blocks. 2) Use the CG bitmaps. Out of curiosity, why aren't the CG bitmaps global enough for you? Is your problem that they are logically rather than physically contiguous? -- Terry From owner-freebsd-fs@FreeBSD.ORG Wed Sep 10 11:40:13 2003 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1904416A4BF for ; Wed, 10 Sep 2003 11:40:13 -0700 (PDT) Received: from eternal.slumber.org (eternal.slumber.org [68.98.216.195]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4F2DE43F3F for ; Wed, 10 Sep 2003 11:40:12 -0700 (PDT) (envelope-from flagg@eternal.slumber.org) Received: from eternal.slumber.org (localhost [127.0.0.1]) h8AIX1bT017400 for ; Wed, 10 Sep 2003 11:33:01 -0700 Received: (from flagg@localhost) by eternal.slumber.org (8.12.9/8.12.9/Debian-5) id h8AIX1if017398 for freebsd-fs@freebsd.org; Wed, 10 Sep 2003 11:33:01 -0700 Date: Wed, 10 Sep 2003 11:33:01 -0700 From: Shawn Ostapuk To: freebsd-fs@freebsd.org Message-ID: <20030910183301.GA17366@slumber.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.4i Subject: terabyte limit problem with vinum/ccd X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Sep 2003 18:40:13 -0000 I've tried freebsd-questions but didnt find much help there, so now I'm trying here. I have around 10 IDE drives which add up to over a terabyte. My goal is to use them all as one big drive using any means necessary (I have a backup so redundency is not needed, only space in this situation) I used to use vinum (and still would like to), i hit the terabyte limit with UFS and was told i would have to upgrade to 5.1 in order to take advantage of UFS2 and > 1TB filesystem -- so thats what i've done. However I still seem to have the exactly same problems. I'm now trying it on a whole new box and set of drives with the same set of problems. It doesn't matter if i use vinum or ccdconfig -- they all work fine and predictably, until I make it larger than a terabyte then i get the following on freebsd 5.1 RELEASE: with ccdconfig # ccdconfig -cv ccd0 16 none /dev/ad1s1e .. /dev/ad10s1e ccd0: 10 componets (ad1s1e, .., ad10s1e), 2223956864 blocks interleaved at 16 blocks # newfs /dev/ccd0 newfs: wtfs: 512 bytes at sector 2223956863: Invalid argument with vinum # newfs /dev/vinum/vinum0 /dev/vinum/vinum0: 1085915.5MB (2223954992 sectors) block size 16384, fragment size 2048 using 5910 cylinder groups of 183.77MB, 11761 blks, 23552 inodes. newfs: can't read old UFS1 superblock: read error from block device: Invalid arguement If i remove any single drive in the configuration making it < 1TB it works fine, i can newfs and what not. Add it back up to >1TB and newfs breaks. It seems to break when it calls wtfs, which just called bwrite, which seems to just be a libufs front end for calling pwrite. Unfortunately at this point i'm at a loss exactly what the values should be and why its failing still. (newfs wrong? vinum backend?) I've heard vinum has no practical limits, and should work fine beyond a terabyte, and i've spoken directly with people who have newfs'd over a terabyte using hardware raid (3ware) -- so I'm just baffled. Am I doing something wrong or is this configuration just simply not supported yet? It may be intestersting to note if I do new newfs -N for testing i get no errors and the output looks great, mainly because it never calls bwrite. Any suggestions or confirmation on weather or not this should work or not or alternative ideas? 12channel IDE raid controller is about $700 more than i got right now :) Thanks for any help -- been driving me nuts for a while now. From owner-freebsd-fs@FreeBSD.ORG Wed Sep 10 11:43:45 2003 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8D91416A4BF for ; Wed, 10 Sep 2003 11:43:45 -0700 (PDT) Received: from obsecurity.dyndns.org (adsl-64-169-107-253.dsl.lsan03.pacbell.net [64.169.107.253]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9BE0643FAF for ; Wed, 10 Sep 2003 11:43:44 -0700 (PDT) (envelope-from kris@obsecurity.org) Received: from rot13.obsecurity.org (rot13.obsecurity.org [10.0.0.5]) by obsecurity.dyndns.org (Postfix) with ESMTP id 1DBB266D32; Wed, 10 Sep 2003 11:43:44 -0700 (PDT) Received: by rot13.obsecurity.org (Postfix, from userid 1000) id D7F5F836; Wed, 10 Sep 2003 11:43:43 -0700 (PDT) Date: Wed, 10 Sep 2003 11:43:43 -0700 From: Kris Kennaway To: Shawn Ostapuk Message-ID: <20030910184343.GA787@rot13.obsecurity.org> References: <20030910183301.GA17366@slumber.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="azLHFNyN32YCQGCU" Content-Disposition: inline In-Reply-To: <20030910183301.GA17366@slumber.org> User-Agent: Mutt/1.4.1i cc: freebsd-fs@freebsd.org Subject: Re: terabyte limit problem with vinum/ccd X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Sep 2003 18:43:45 -0000 --azLHFNyN32YCQGCU Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Sep 10, 2003 at 11:33:01AM -0700, Shawn Ostapuk wrote: > I used to use vinum (and still would like to), i hit the terabyte limit > with UFS and was told i would have to upgrade to 5.1 in order to take > advantage of UFS2 and > 1TB filesystem -- so thats what i've done. > However I still seem to have the exactly same problems. I'm now trying > it on a whole new box and set of drives with the same set of problems. >=20 > It doesn't matter if i use vinum or ccdconfig -- they all work fine and > predictably, until I make it larger than a terabyte then i get the=20 > following on freebsd 5.1 RELEASE: Try 5.1-CURRENT (if you dare, or wait until 5.2-REL) - there were changes in this area recently. Kris --azLHFNyN32YCQGCU Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.3 (FreeBSD) iD8DBQE/X3DfWry0BWjoQKURAoPRAKCZdCW7v4eZ5aVefoi/0HlNYx5CmQCePzfK TCd5rk5QUAj2PFBnoJ4P+CE= =naQR -----END PGP SIGNATURE----- --azLHFNyN32YCQGCU-- From owner-freebsd-fs@FreeBSD.ORG Wed Sep 10 12:11:00 2003 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EF3F216A4BF for ; Wed, 10 Sep 2003 12:11:00 -0700 (PDT) Received: from eternal.slumber.org (eternal.slumber.org [68.98.216.195]) by mx1.FreeBSD.org (Postfix) with ESMTP id 451F543FDF for ; Wed, 10 Sep 2003 12:11:00 -0700 (PDT) (envelope-from flagg@eternal.slumber.org) Received: from eternal.slumber.org (localhost [127.0.0.1]) h8AJ3qbT017462 for ; Wed, 10 Sep 2003 12:03:52 -0700 Received: (from flagg@localhost) by eternal.slumber.org (8.12.9/8.12.9/Debian-5) id h8AJ3olV017460 for freebsd-fs@freebsd.org; Wed, 10 Sep 2003 12:03:50 -0700 Date: Wed, 10 Sep 2003 12:03:50 -0700 From: Shawn Ostapuk To: freebsd-fs@freebsd.org Message-ID: <20030910190350.GB17366@slumber.org> References: <20030910183301.GA17366@slumber.org> <20030910184343.GA787@rot13.obsecurity.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030910184343.GA787@rot13.obsecurity.org> User-Agent: Mutt/1.5.4i Subject: Re: terabyte limit problem with vinum/ccd X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Sep 2003 19:11:01 -0000 Actually since the origional post, i've upgraded to -CURRENT as of Sep 2nd, because i did see changes to the vinum files -- but no luck. I haven't noticed anymore updates since then either. Thanks tho. On Wed, Sep 10, 2003 at 11:43:43AM -0700, Kris Kennaway wrote: > On Wed, Sep 10, 2003 at 11:33:01AM -0700, Shawn Ostapuk wrote: > > > I used to use vinum (and still would like to), i hit the terabyte limit > > with UFS and was told i would have to upgrade to 5.1 in order to take > > advantage of UFS2 and > 1TB filesystem -- so thats what i've done. > > However I still seem to have the exactly same problems. I'm now trying > > it on a whole new box and set of drives with the same set of problems. > > > > It doesn't matter if i use vinum or ccdconfig -- they all work fine and > > predictably, until I make it larger than a terabyte then i get the > > following on freebsd 5.1 RELEASE: > > Try 5.1-CURRENT (if you dare, or wait until 5.2-REL) - there were > changes in this area recently. > > Kris From owner-freebsd-fs@FreeBSD.ORG Wed Sep 10 18:24:07 2003 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3F44916A4BF; Wed, 10 Sep 2003 18:24:07 -0700 (PDT) Received: from smtp6.andrew.cmu.edu (SMTP6.andrew.cmu.edu [128.2.10.86]) by mx1.FreeBSD.org (Postfix) with ESMTP id 66FF343F75; Wed, 10 Sep 2003 18:24:06 -0700 (PDT) (envelope-from eno@andrew.cmu.edu) Received: from andrew.cmu.edu (BLISS.PDL.CMU.EDU [128.2.134.47]) (user=eno mech=PLAIN (0 bits))h8B1O4il015479 (version=TLSv1/SSLv3 cipher=RC4-MD5 bits=128 verify=NOT); Wed, 10 Sep 2003 21:24:05 -0400 Message-ID: <3F5FCEB5.9010407@andrew.cmu.edu> Date: Wed, 10 Sep 2003 21:24:05 -0400 From: Eno Thereska User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.0.0) Gecko/20020529 X-Accept-Language: en-us, en MIME-Version: 1.0 To: freebsd-hackers@freebsd.org, freebsd-fs@freebsd.org Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: flush on close X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Sep 2003 01:24:07 -0000 In FreeBSD 4.4, I am noticing a huge number of calls to ffs_fsync() (in /sys/ufs/ffs/ffs_vnops.c) when running a benchmark like Postmark. ffs_fsync() flushes all dirty buffers with an open file to disk. Normally this function would be called either because the application writer explicitly flushes the file, or if the syncer daemon or buffer daemon decide it's time for the dirty blocks to go to disk. Neither of these two options is happening. Files are opened and closed very frequently though. I have a suspicion that BSD is using the "flush-on-close" semantic. Could someone confirm or reject this claim? If confirmed, I am wondering how to get rid of it... Thanks Eno From owner-freebsd-fs@FreeBSD.ORG Thu Sep 11 07:12:50 2003 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 345C916A4BF for ; Thu, 11 Sep 2003 07:12:50 -0700 (PDT) Received: from eternal.slumber.org (eternal.slumber.org [68.98.216.195]) by mx1.FreeBSD.org (Postfix) with ESMTP id B002D4400F for ; Thu, 11 Sep 2003 07:12:48 -0700 (PDT) (envelope-from flagg@eternal.slumber.org) Received: from eternal.slumber.org (localhost [127.0.0.1]) h8BE5lbT025170; Thu, 11 Sep 2003 07:05:47 -0700 Received: (from flagg@localhost) by eternal.slumber.org (8.12.9/8.12.9/Debian-5) id h8BE5kh7025168; Thu, 11 Sep 2003 07:05:46 -0700 Date: Thu, 11 Sep 2003 07:05:46 -0700 From: Shawn Ostapuk To: Chris Jackman Message-ID: <20030911140546.GE1888@slumber.org> References: <20030910183301.GA17366@slumber.org> <20030911070248.GA69505@collab.or8.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030911070248.GA69505@collab.or8.net> User-Agent: Mutt/1.5.4i cc: freebsd-fs@freebsd.org Subject: Re: terabyte limit problem with vinum/ccd X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Sep 2003 14:12:50 -0000 > I'm just curious about this. > in /usr/src/sbin/newfs/mkfs.c, have you tried > commenting out the if block around line 438 ? > > > * Wipe out old UFS1 superblock(s) if necessary. > */ > if (!Nflag && Oflag != 1) { > i = bread(&disk, SBLOCK_UFS1 / disk.d_bsize, chdummy, SBLOCKSIZE); > if (i == -1) > err(1, "can't read old UFS1 superblock: %s", disk.d_error); > ..... (continues) > > > I have zero knowledge of this code, but I'm still curious if it'd work =). > Actually that's something I already tried, If i take it out it fails with something like "disk write error: invalid arguement" similar to the ccdconfig :) It seems any call to bwrite fails when the drive is larger than 1TB. (which is why -N seems to work with no issues, calls wtfs which doesnt call bwrite when the Nflag is set) -shawn From owner-freebsd-fs@FreeBSD.ORG Thu Sep 11 09:48:59 2003 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E4B4716A4BF for ; Thu, 11 Sep 2003 09:48:59 -0700 (PDT) Received: from rs6000.univie.ac.at (rs6000.univie.ac.at [131.130.1.14]) by mx1.FreeBSD.org (Postfix) with ESMTP id B57C043F93 for ; Thu, 11 Sep 2003 09:48:58 -0700 (PDT) (envelope-from l.ertl@univie.ac.at) Received: from dialin202.cc.univie.ac.at (dialin202.cc.univie.ac.at [131.130.202.202]) by rs6000.univie.ac.at (8.12.2/8.12.2) with ESMTP id h8BGmZKs071880; Thu, 11 Sep 2003 18:48:45 +0200 Date: Thu, 11 Sep 2003 18:44:23 +0200 (CEST) From: Lukas Ertl To: Shawn Ostapuk In-Reply-To: <20030911140546.GE1888@slumber.org> Message-ID: <20030911184134.S670@korben.in.tern> References: <20030910183301.GA17366@slumber.org> <20030911070248.GA69505@collab.or8.net> <20030911140546.GE1888@slumber.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-fs@freebsd.org cc: Chris Jackman Subject: Re: terabyte limit problem with vinum/ccd X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Sep 2003 16:49:00 -0000 On Thu, 11 Sep 2003, Shawn Ostapuk wrote: > Actually that's something I already tried, If i take it out it fails > with something like "disk write error: invalid arguement" similar to the > ccdconfig :) It seems any call to bwrite fails when the drive is > larger than 1TB. (which is why -N seems to work with no issues, calls > wtfs which doesnt call bwrite when the Nflag is set) The interesting thing is that you get to the UFS1 check (which fails on the bread operation) on a vinum volume, but survive the first write which happens before this UFS1 check. regards, le -- Lukas Ertl eMail: l.ertl@univie.ac.at UNIX Systemadministrator Tel.: (+43 1) 4277-14073 Vienna University Computer Center Fax.: (+43 1) 4277-9140 University of Vienna http://mailbox.univie.ac.at/~le/ From owner-freebsd-fs@FreeBSD.ORG Thu Sep 11 12:38:22 2003 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B487716A4BF; Thu, 11 Sep 2003 12:38:22 -0700 (PDT) Received: from mailman.zeta.org.au (mailman.zeta.org.au [203.26.10.16]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0104843FEC; Thu, 11 Sep 2003 12:38:21 -0700 (PDT) (envelope-from bde@zeta.org.au) Received: from gamplex.bde.org (katana.zip.com.au [61.8.7.246]) by mailman.zeta.org.au (8.9.3p2/8.8.7) with ESMTP id FAA20462; Fri, 12 Sep 2003 05:38:13 +1000 Date: Fri, 12 Sep 2003 05:38:12 +1000 (EST) From: Bruce Evans X-X-Sender: bde@gamplex.bde.org To: Eno Thereska In-Reply-To: <3F5FCEB5.9010407@andrew.cmu.edu> Message-ID: <20030912051818.B1339@gamplex.bde.org> References: <3F5FCEB5.9010407@andrew.cmu.edu> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-fs@freebsd.org cc: freebsd-hackers@freebsd.org Subject: Re: flush on close X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Sep 2003 19:38:22 -0000 On Wed, 10 Sep 2003, Eno Thereska wrote: > In FreeBSD 4.4, I am noticing a huge number of calls > to ffs_fsync() (in /sys/ufs/ffs/ffs_vnops.c) > when running a benchmark like Postmark. > > ffs_fsync() flushes all dirty buffers with an open file > to disk. Normally this function would be called > either because the application writer explicitly > flushes the file, or if the syncer daemon or buffer daemon > decide it's time for the dirty blocks to go to disk. > > Neither of these two options is happening. Files are opened and closed > very frequently though. I have a suspicion that BSD is using the > "flush-on-close" semantic. > > Could someone confirm or reject this claim? > If confirmed, I am wondering how to get rid of it... ffs_fsync() is (or should be) rarely called except as a result of applications calling fsync(2) or sync(2). It is not normally called by the syncer daemon or buffer daemon (seems to be not at all in 4.4, though -current calls it from vfs-bio.c when there are too many dirty buffers, and benchmarks like postmark might trigger this). In 4.4 the only relevant VOP_FSYNC() seems to be the one in vinvalbuf(). Using lots of vnodes might cause this to be called a lot, but this should only cause a lot of i/o in ffs_fsync() if a lot is really needed. Dirty buffers for vnodes which will soon be deleted are supposed to be discarded in ffs_fsync(). Benchmarks that do lots of i/o to short-lived files tend to cause too much physical i/o, but this is because the i/o is done by the buffer (?) deamon before ffs_fsync() can discard it. Bruce From owner-freebsd-fs@FreeBSD.ORG Thu Sep 11 23:58:42 2003 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id ADACF16A4BF for ; Thu, 11 Sep 2003 23:58:42 -0700 (PDT) Received: from do.ru (p139.tc2.snt.ru [217.70.127.139]) by mx1.FreeBSD.org (Postfix) with ESMTP id 34DB443FE1 for ; Thu, 11 Sep 2003 23:58:41 -0700 (PDT) (envelope-from anton@do.ru) Received: from do.ru (IDENT:smmsp@localhost [127.0.0.1]) by do.ru (8.12.2/8.12.2) with ESMTP id h8C6wZ3d031317; Fri, 12 Sep 2003 13:58:35 +0700 Received: (from root@localhost) by do.ru (8.12.2/8.12.2/Submit) id h8C6wZqb031316; Fri, 12 Sep 2003 13:58:35 +0700 Received: from a-serbulov.do.ru (a-serbulov.do.ru [192.168.1.36]) by do.ru (8.12.2/8.12.2) with ESMTP id h8C6wY3d031313 for ; Fri, 12 Sep 2003 13:58:34 +0700 Date: Fri, 12 Sep 2003 13:58:33 +0700 From: anton X-Mailer: The Bat! (v1.61) Organization: do X-Priority: 3 (Normal) Message-ID: <8114690253.20030912135833@do.ru> To: freebsd-fs@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: mount_smbfs and lowercase X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: anton List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Sep 2003 06:58:42 -0000 I using FreBSD-4.8 I was mount smb file system with mount_smbfs -c l, but file and folder names are don't convert to lowercase, whats wrong? I was build kernel with options: options NETSMB options NETSMBCRYPTO options LIBMCHAIN options LIBICONV options SMBFS And I was install pakage libiconv-1.8_2 I was mount smbfs: #mount_smbfs -I server.mydomain -E koi8-r:cp866 -c l //user@host/fold ~/smbfs -- Best regards, anton mailto:anton@do.ru From owner-freebsd-fs@FreeBSD.ORG Sat Sep 13 05:26:03 2003 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4A32416A4BF for ; Sat, 13 Sep 2003 05:26:03 -0700 (PDT) Received: from mail.ryu16.org (YahooBB219005044040.bbtec.net [219.5.44.40]) by mx1.FreeBSD.org (Postfix) with ESMTP id E833243FDF for ; Sat, 13 Sep 2003 05:26:00 -0700 (PDT) (envelope-from imura@ryu16.org) Received: from redeye.xt.ryu16.org (localhost [127.0.0.1]) by mail.ryu16.org (8.12.9/8.12.9) with ESMTP id h8DCPjl8098596; Sat, 13 Sep 2003 21:25:45 +0900 (JST) (envelope-from imura@redeye.xt.ryu16.org) Received: (from imura@localhost) by redeye.xt.ryu16.org (8.12.9/8.12.9/Submit) id h8DCPhEY098595; Sat, 13 Sep 2003 21:25:43 +0900 (JST) Date: Sat, 13 Sep 2003 21:25:43 +0900 From: "R. Imura" To: anton Message-ID: <20030913212543.A98493@ryu16.org> References: <8114690253.20030912135833@do.ru> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Mutt/1.2.5i-jp2 In-Reply-To: <8114690253.20030912135833@do.ru>; from anton@do.ru on Fri, Sep 12, 2003 at 01:58:33PM +0700 cc: freebsd-fs@freebsd.org Subject: Re: mount_smbfs and lowercase X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Sep 2003 12:26:03 -0000 On Fri, Sep 12, 2003 at 01:58:33PM +0700, anton wrote: > I using FreBSD-4.8 > I was mount smb file system with mount_smbfs -c l, but file and > folder names are don't convert to lowercase, whats wrong? It seems -c feature has been disabled. I don't know why but you may need to un-comment the code. Index: smbfs_subr.c =================================================================== RCS file: /home/ncvs/src/sys/fs/smbfs/smbfs_subr.c,v retrieving revision 1.1.2.2 diff -u -r1.1.2.2 smbfs_subr.c --- smbfs_subr.c 17 Jan 2003 08:20:26 -0000 1.1.2.2 +++ smbfs_subr.c 13 Sep 2003 12:20:41 -0000 @@ -316,10 +316,10 @@ int smbfs_fname_tolocal(struct smb_vc *vcp, char *name, int nmlen, int caseopt) { -/* if (caseopt & SMB_CS_UPPER) + if (caseopt & SMB_CS_UPPER) iconv_convmem(vcp->vc_toupper, name, name, nmlen); else if (caseopt & SMB_CS_LOWER) - iconv_convmem(vcp->vc_tolower, name, name, nmlen);*/ + iconv_convmem(vcp->vc_tolower, name, name, nmlen); if (vcp->vc_tolocal) iconv_convmem(vcp->vc_tolocal, name, name, nmlen); return 0; - R. Imura