From owner-freebsd-fs Sun Sep 12 1:27:47 1999 Delivered-To: freebsd-fs@freebsd.org Received: from mail.du.gtn.com (mail.du.gtn.com [194.77.9.57]) by hub.freebsd.org (Postfix) with ESMTP id 3342514CC8 for ; Sun, 12 Sep 1999 01:27:24 -0700 (PDT) (envelope-from ticso@cicely8.cicely.de) Received: from mail.cicely.de (cicely.de [194.231.9.142]) by mail.du.gtn.com (8.9.3/8.9.3) with ESMTP id KAA00561 for ; Sun, 12 Sep 1999 10:26:36 +0200 (MET DST) Received: from cicely8.cicely.de (cicely8.cicely.de [10.1.2.10]) by mail.cicely.de (8.9.0/8.9.0) with ESMTP id KAA68795 for ; Sun, 12 Sep 1999 10:27:36 +0200 (CEST) Received: (from ticso@localhost) by cicely8.cicely.de (8.9.3/8.9.2) id KAA37625 for freebsd-fs@freebsd.org; Sun, 12 Sep 1999 10:27:38 +0200 (CEST) (envelope-from ticso) Date: Sun, 12 Sep 1999 10:27:37 +0200 From: Bernd Walter To: freebsd-fs@freebsd.org Subject: Anyone working on HSM? Message-ID: <19990912102737.B37555@cicely8.cicely.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 0.95.3i Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org I'm in the situation to learn how the ffs code works to get mounted ffs-filesystems bigger and got some basic ideas about implementing HSM funktionality. I own a MO Jukebox so I have the needed hardware available. Is already anyone already working on an implementation or am I the first? -- B.Walter COSMO-Project http://www.cosmo-project.de ticso@cicely.de Usergroup info@cosmo-project.de To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Sun Sep 12 2:58:57 1999 Delivered-To: freebsd-fs@freebsd.org Received: from mail.du.gtn.com (mail.du.gtn.com [194.77.9.57]) by hub.freebsd.org (Postfix) with ESMTP id 1DC1A1525C for ; Sun, 12 Sep 1999 02:58:54 -0700 (PDT) (envelope-from ticso@cicely8.cicely.de) Received: from mail.cicely.de (cicely.de [194.231.9.142]) by mail.du.gtn.com (8.9.3/8.9.3) with ESMTP id LAA03064 for ; Sun, 12 Sep 1999 11:58:05 +0200 (MET DST) Received: from cicely8.cicely.de (cicely8.cicely.de [10.1.2.10]) by mail.cicely.de (8.9.0/8.9.0) with ESMTP id LAA68935 for ; Sun, 12 Sep 1999 11:59:07 +0200 (CEST) Received: (from ticso@localhost) by cicely8.cicely.de (8.9.3/8.9.2) id LAA37722 for freebsd-fs@freebsd.org; Sun, 12 Sep 1999 11:59:04 +0200 (CEST) (envelope-from ticso) Date: Sun, 12 Sep 1999 11:59:04 +0200 From: Bernd Walter To: freebsd-fs@freebsd.org Subject: FFS-questions Message-ID: <19990912115904.A37697@cicely8.cicely.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 0.95.3i Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org I thought that a directory always contains full blocks and never contain a fragment. Now I have the following situation: dirsize = 512, fragsize = 1024, blocksize = 4096 The dirrectory contains including . and .. 5 Entries. It seems to me that the size of a directory is always a multiple of 512. My problem is that I need to know how big the last datareference of a given inode is: How many frags does it contain or is the last reference a complete block. Another problem might be the situation if the fs is mounted: AFAIK FFS allocates complete blocks (or clusters?) in case the file gets bigger. I can't find out if it's a block or a couple of frags depending on the ino->di_size. Is there a relyable way for doing such things? Do I still have any missunderstandings about fileendings? -- B.Walter COSMO-Project http://www.cosmo-project.de ticso@cicely.de Usergroup info@cosmo-project.de To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Sun Sep 12 12: 2: 8 1999 Delivered-To: freebsd-fs@freebsd.org Received: from sasami.jurai.net (sasami.jurai.net [63.67.141.99]) by hub.freebsd.org (Postfix) with ESMTP id 9D24114D11 for ; Sun, 12 Sep 1999 12:02:06 -0700 (PDT) (envelope-from winter@jurai.net) Received: from localhost (winter@localhost) by sasami.jurai.net (8.8.8/8.8.7) with ESMTP id PAA24174; Sun, 12 Sep 1999 15:01:58 -0400 (EDT) Date: Sun, 12 Sep 1999 15:01:58 -0400 (EDT) From: "Matthew N. Dodd" To: Bernd Walter Cc: freebsd-fs@FreeBSD.ORG Subject: Re: Anyone working on HSM? In-Reply-To: <19990912102737.B37555@cicely8.cicely.de> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Sun, 12 Sep 1999, Bernd Walter wrote: > I'm in the situation to learn how the ffs code works to get mounted > ffs-filesystems bigger and got some basic ideas about implementing HSM > funktionality. > I own a MO Jukebox so I have the needed hardware available. You might want to look at the code SGI released a while back; I'm not sure that its and out and out HSM but it does have bits that you'll need. -- | Matthew N. Dodd | '78 Datsun 280Z | '75 Volvo 164E | FreeBSD/NetBSD | | winter@jurai.net | 2 x '84 Volvo 245DL | ix86,sparc,pmax | | http://www.jurai.net/~winter | This Space For Rent | ISO8802.5 4ever | To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Sun Sep 12 12:54:57 1999 Delivered-To: freebsd-fs@freebsd.org Received: from mail.du.gtn.com (mail.du.gtn.com [194.77.9.57]) by hub.freebsd.org (Postfix) with ESMTP id 11BE81519C for ; Sun, 12 Sep 1999 12:54:13 -0700 (PDT) (envelope-from ticso@cicely8.cicely.de) Received: from mail.cicely.de (cicely.de [194.231.9.142]) by mail.du.gtn.com (8.9.3/8.9.3) with ESMTP id VAA22909; Sun, 12 Sep 1999 21:53:19 +0200 (MET DST) Received: from cicely8.cicely.de (cicely8.cicely.de [10.1.2.10]) by mail.cicely.de (8.9.0/8.9.0) with ESMTP id VAA70262; Sun, 12 Sep 1999 21:54:21 +0200 (CEST) Received: (from ticso@localhost) by cicely8.cicely.de (8.9.3/8.9.2) id VAA38384; Sun, 12 Sep 1999 21:54:19 +0200 (CEST) (envelope-from ticso) Date: Sun, 12 Sep 1999 21:54:19 +0200 From: Bernd Walter To: "Matthew N. Dodd" Cc: Bernd Walter , freebsd-fs@FreeBSD.ORG Subject: Re: Anyone working on HSM? Message-ID: <19990912215419.A38366@cicely8.cicely.de> References: <19990912102737.B37555@cicely8.cicely.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 0.95.3i In-Reply-To: ; from Matthew N. Dodd on Sun, Sep 12, 1999 at 03:01:58PM -0400 Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Sun, Sep 12, 1999 at 03:01:58PM -0400, Matthew N. Dodd wrote: > On Sun, 12 Sep 1999, Bernd Walter wrote: > > I'm in the situation to learn how the ffs code works to get mounted > > ffs-filesystems bigger and got some basic ideas about implementing HSM > > funktionality. > > I own a MO Jukebox so I have the needed hardware available. > > You might want to look at the code SGI released a while back; I'm not sure > that its and out and out HSM but it does have bits that you'll need. > XFS is a nice fs but I don't beleave it can come near the performance of a softupdated FFS. HSM does not sound to be very fast by principle but it's usefull to have a really fast FS at first stage if the stagedimensions are selected properly. The next point is that I found FFS with softupdates idealy for writing MOs if you don't want to use a streaming method. -- B.Walter COSMO-Project http://www.cosmo-project.de ticso@cicely.de Usergroup info@cosmo-project.de To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Sun Sep 12 13:30:27 1999 Delivered-To: freebsd-fs@freebsd.org Received: from sasami.jurai.net (sasami.jurai.net [63.67.141.99]) by hub.freebsd.org (Postfix) with ESMTP id 61ACF1548B for ; Sun, 12 Sep 1999 13:30:17 -0700 (PDT) (envelope-from winter@jurai.net) Received: from localhost (winter@localhost) by sasami.jurai.net (8.8.8/8.8.7) with ESMTP id QAA04069; Sun, 12 Sep 1999 16:29:57 -0400 (EDT) Date: Sun, 12 Sep 1999 16:29:56 -0400 (EDT) From: "Matthew N. Dodd" To: Bernd Walter Cc: freebsd-fs@FreeBSD.ORG Subject: Re: Anyone working on HSM? In-Reply-To: <19990912215419.A38366@cicely8.cicely.de> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Sun, 12 Sep 1999, Bernd Walter wrote: > XFS is a nice fs but I don't beleave it can come near the performance > of a softupdated FFS. HSM does not sound to be very fast by principle > but it's usefull to have a really fast FS at first stage if the > stagedimensions are selected properly. The next point is that I found > FFS with softupdates idealy for writing MOs if you don't want to use a > streaming method. I wasn't talking about XFS. http://oss.sgi.com/projects/openvault/ -- | Matthew N. Dodd | '78 Datsun 280Z | '75 Volvo 164E | FreeBSD/NetBSD | | winter@jurai.net | 2 x '84 Volvo 245DL | ix86,sparc,pmax | | http://www.jurai.net/~winter | This Space For Rent | ISO8802.5 4ever | To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Tue Sep 14 15: 0:43 1999 Delivered-To: freebsd-fs@freebsd.org Received: from smtp02.primenet.com (smtp02.primenet.com [206.165.6.132]) by hub.freebsd.org (Postfix) with ESMTP id 6FB9514CCD for ; Tue, 14 Sep 1999 15:00:37 -0700 (PDT) (envelope-from tlambert@usr09.primenet.com) Received: (from daemon@localhost) by smtp02.primenet.com (8.8.8/8.8.8) id PAA05989; Tue, 14 Sep 1999 15:00:30 -0700 (MST) Received: from usr09.primenet.com(206.165.6.209) via SMTP by smtp02.primenet.com, id smtpd005976; Tue Sep 14 15:00:28 1999 Received: (from tlambert@localhost) by usr09.primenet.com (8.8.5/8.8.5) id PAA22155; Tue, 14 Sep 1999 15:00:26 -0700 (MST) From: Terry Lambert Message-Id: <199909142200.PAA22155@usr09.primenet.com> Subject: Re: FFS-questions To: ticso@cicely.de (Bernd Walter) Date: Tue, 14 Sep 1999 22:00:26 +0000 (GMT) Cc: freebsd-fs@FreeBSD.ORG In-Reply-To: <19990912115904.A37697@cicely8.cicely.de> from "Bernd Walter" at Sep 12, 99 11:59:04 am X-Mailer: ELM [version 2.4 PL25] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > I thought that a directory always contains full blocks and never > contain a fragment. Physical device blocks. > Now I have the following situation: > dirsize = 512, fragsize = 1024, blocksize = 4096 dirsize = 1 physical device block fragsize = 2 physical device blocks blocksize = 8 physical device blocks > The dirrectory contains including . and .. 5 Entries. > > It seems to me that the size of a directory is always a multiple > of 512. The size of a directory entry block (dirsize) is always a physical device block. At a fragsize of 1024 bytes, directories are allocates in 1024 byte fragments, which act as an index reference for 2 physical device blocks each. > My problem is that I need to know how big the last datareference of > a given inode is: How many frags does it contain or is the last > reference a complete block. This can be determined via the standard fragment lookup code on directories, just as with files. > Another problem might be the situation if the fs is mounted: > AFAIK FFS allocates complete blocks (or clusters?) in case > the file gets bigger. I can't find out if it's a block or a > couple of frags depending on the ino->di_size. This is problematic, because files can be sparse. The value of ino->di_size is the file size, in bytes. If you take the remainder modulo the physical block size, you will get the size of the remainder. If this value is less than the file system block size minus the fragment size (4096 - 1024 = 3072), then there are frag(s) involved, unless it has been truncated back. The correct place to look for this information is /usr/src/sys/ufs/ffs/fs.h, since frags are FFS specific. > Is there a relyable way for doing such things? > Do I still have any missunderstandings about fileendings? Look also in ffs_alloc.c and ffs_inode.c; pay special attention to the macro "numfrags". Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Tue Sep 14 23: 3: 8 1999 Delivered-To: freebsd-fs@freebsd.org Received: from mail.du.gtn.com (mail.du.gtn.com [194.77.9.57]) by hub.freebsd.org (Postfix) with ESMTP id 4131014F01 for ; Tue, 14 Sep 1999 23:03:04 -0700 (PDT) (envelope-from ticso@cicely8.cicely.de) Received: from mail.cicely.de (cicely.de [194.231.9.142]) by mail.du.gtn.com (8.9.3/8.9.3) with ESMTP id HAA06844; Wed, 15 Sep 1999 07:59:27 +0200 (MET DST) Received: from cicely8.cicely.de (cicely8.cicely.de [10.1.2.10]) by mail.cicely.de (8.9.0/8.9.0) with ESMTP id IAA78509; Wed, 15 Sep 1999 08:03:18 +0200 (CEST) Received: (from ticso@localhost) by cicely8.cicely.de (8.9.3/8.9.2) id IAA42227; Wed, 15 Sep 1999 08:03:17 +0200 (CEST) (envelope-from ticso) Date: Wed, 15 Sep 1999 08:03:17 +0200 From: Bernd Walter To: Terry Lambert Cc: Bernd Walter , freebsd-fs@FreeBSD.ORG Subject: Re: FFS-questions Message-ID: <19990915080317.A42200@cicely8.cicely.de> References: <19990912115904.A37697@cicely8.cicely.de> <199909142200.PAA22155@usr09.primenet.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 0.95.3i In-Reply-To: <199909142200.PAA22155@usr09.primenet.com>; from Terry Lambert on Tue, Sep 14, 1999 at 10:00:26PM +0000 Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Tue, Sep 14, 1999 at 10:00:26PM +0000, Terry Lambert wrote: Thank you for explaining the directory specific behavours. During search in the source I got the impression that softlinks uses the di_ib[] array of struct dinode as direct pointers. Is that true or is my impression wrong? > > > Another problem might be the situation if the fs is mounted: > > AFAIK FFS allocates complete blocks (or clusters?) in case > > the file gets bigger. I can't find out if it's a block or a > > couple of frags depending on the ino->di_size. > > This is problematic, because files can be sparse. > > The value of ino->di_size is the file size, in bytes. > > If you take the remainder modulo the physical block size, you > will get the size of the remainder. > > If this value is less than the file system block size minus the > fragment size (4096 - 1024 = 3072), then there are frag(s) involved, > unless it has been truncated back. > That's the way I already do it. My problem was special for the mounted case that the kernel might allocate more than the needed frags but write the real filesize to the inode. It doesn't seem to be a real problem since may need to sync before or use the incore informations anyway. -- B.Walter COSMO-Project http://www.cosmo-project.de ticso@cicely.de Usergroup info@cosmo-project.de To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message