From owner-freebsd-fs Tue May 23 13:53: 7 2000 Delivered-To: freebsd-fs@freebsd.org Received: from arjun.niksun.com (gwnew.niksun.com [206.20.52.130]) by hub.freebsd.org (Postfix) with ESMTP id CEFCC37B8B3 for ; Tue, 23 May 2000 13:53:01 -0700 (PDT) (envelope-from joy@niksun.com) Received: from falcon.niksun.com (falcon.niksun.com [10.0.0.167]) by arjun.niksun.com (8.9.3/8.9.3) with ESMTP id QAA60727 for ; Tue, 23 May 2000 16:52:47 -0400 (EDT) (envelope-from joy@falcon.niksun.com) Message-ID: <392AEFB6.F0EC04F0@falcon.niksun.com> Date: Tue, 23 May 2000 16:53:10 -0400 From: Joy Ganguly X-Mailer: Mozilla 4.51 [en] (X11; I; FreeBSD 3.2-RELEASE i386) X-Accept-Language: en MIME-Version: 1.0 To: fs Subject: blocks and fragments??? Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org hi all, do disk addresses in struct dinode ( di_db[] array) address block addresses or fragment addresses?? the comment in dinode.h says they are block addresses. but fs.h says addresses are capable of addressing fragments. will somebody please explain this?? thanx all joy To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Thu May 25 23:40: 3 2000 Delivered-To: freebsd-fs@freebsd.org Received: from aplcenMP.apl.jhu.edu (apl.jhu.edu [128.220.101.100]) by hub.freebsd.org (Postfix) with ESMTP id 275C137B82A for ; Thu, 25 May 2000 23:40:00 -0700 (PDT) (envelope-from mccrobi@aplcenMP.apl.jhu.edu) Received: from apl.jhu.edu (kslip27.apl.jhu.edu [128.220.108.37]) by aplcenMP.apl.jhu.edu (8.9.3/8.9.1) with ESMTP id CAA25505 for ; Fri, 26 May 2000 02:39:36 -0400 (EDT) Message-ID: <392E1E19.5E2EFC51@apl.jhu.edu> Date: Fri, 26 May 2000 02:47:53 -0400 From: Chuck McCrobie X-Mailer: Mozilla 4.72 [en] (X11; I; FreeBSD 4.0-RELEASE i386) X-Accept-Language: en MIME-Version: 1.0 To: freebsd-fs@FreeBSD.ORG Subject: Changes in VFS for 4.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Hello, please forgive this message: I am working on a file system under FreeBSD 3.3. I just got a new disk and decided to install 4.0. But my file system doesn't compile anymore :( - nothing too serious ;) Is there a list of VFS interface changes (such as the "new" vn_isdisk routine) for 4.0? Thanks, Chuck McCrobie mccrobi@apl.jhu.edu To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Thu May 25 23:50:52 2000 Delivered-To: freebsd-fs@freebsd.org Received: from aplcenMP.apl.jhu.edu (apl.jhu.edu [128.220.101.100]) by hub.freebsd.org (Postfix) with ESMTP id 9816D37B7DD for ; Thu, 25 May 2000 23:50:44 -0700 (PDT) (envelope-from mccrobi@aplcenMP.apl.jhu.edu) Received: from apl.jhu.edu (kslip27.apl.jhu.edu [128.220.108.37]) by aplcenMP.apl.jhu.edu (8.9.3/8.9.1) with ESMTP id CAA25704 for ; Fri, 26 May 2000 02:50:12 -0400 (EDT) Message-ID: <392E2096.FED6F59B@apl.jhu.edu> Date: Fri, 26 May 2000 02:58:30 -0400 From: Chuck McCrobie X-Mailer: Mozilla 4.72 [en] (X11; I; FreeBSD 4.0-RELEASE i386) X-Accept-Language: en MIME-Version: 1.0 To: freebsd-fs@FreeBSD.ORG Subject: Opinion on File System Implementation Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org I am working on readability of the OpenVMS Files-11 On-Disk Structure 2 file system under FreeBSD. As many of you may know, Files-11 ODS-2 has a "wealth" of record attributes and file organizations to choose from. The actual file data for some of these record types contains record level meta-data. Simply reading the file "blindly" will result in both the "intended" data and the meta-data being returned. I'm wondering if I should just return the "raw" data, including embedded record attributes back during the "read" routine. If so, I would like to provide some mechanism so that user mode programs can get the record attributes and "do the right thing". Is there such a thing as an "ioctl" interface to a file system? If I approach the embedded meta-data in this manner, what would be the best mechanism for returning the record attributes? Perhaps a special file, which when opened and read, will return a fixed structure describing the file/record attributes? It doesn't seem too hard to handle special files inside the operating system. I suspect that the majority of customers (I plan on commericalizing my work) would like to have the file system interpret the record attributes and file organizations. Thus, the user mode programs would see the "intended" data, not the embedded meta-data. Thanks for your time, Chuck McCrobie (** MAD VAX **) mccrobi@apl.jhu.edu To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Fri May 26 7:34: 0 2000 Delivered-To: freebsd-fs@freebsd.org Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by hub.freebsd.org (Postfix) with ESMTP id 3249A37B6B3 for ; Fri, 26 May 2000 07:33:50 -0700 (PDT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (robert@fledge.pr.watson.org [192.0.2.3]) by fledge.watson.org (8.9.3/8.9.3) with SMTP id KAA37463; Fri, 26 May 2000 10:33:34 -0400 (EDT) (envelope-from robert@fledge.watson.org) Date: Fri, 26 May 2000 10:33:34 -0400 (EDT) From: Robert Watson X-Sender: robert@fledge.watson.org To: Chuck McCrobie Cc: freebsd-fs@freebsd.org Subject: Re: Opinion on File System Implementation In-Reply-To: <392E2096.FED6F59B@apl.jhu.edu> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Chuck, Responding briefly as I'm currently on travel at a DARPA PI meeting and checking out shortly -- in 4.0-RELEASE/STABLE and 5.0-CURRENT, there is an interface for "extended attributes" on file system objects. Take a look at extattr(9), VOP_GETEXTATTR(9) and VOP_SETEXTATTR(9). There are exposed syscalls to userland also, but in 4.x they don't have man pages as there are no tools using them. In 5.0-CURRENT, I introduced rudimentary support for extended attributes in UFS/FFS to support security extensions I have been working on, so we also pushed in simple command line tools to allow userland to read/set/control extended attributes (setextattr(8), getextattr(8), extattrctl(8)). It may be that this interface is the appropriate interface to read, if not write, this meta-data. I believe that Boris Popov was looking at exposing NWFS attributes through this interface, as well as DOS file attributes that don't easily map to UNIX file permissions/mode. I believe that our current HPFS implementation uses an ioctl in the style that you describe, but it might be appropriate to modify it to use the extended attribute interface. It is my intent to commit syscall manpages for these interfaces over the next couple of weeks once I'm back from my trip. Robert N M Watson robert@fledge.watson.org http://www.watson.org/~robert/ PGP key fingerprint: AF B5 5F FF A6 4A 79 37 ED 5F 55 E9 58 04 6A B1 TIS Labs at Network Associates, Safeport Network Services To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Fri May 26 7:49:24 2000 Delivered-To: freebsd-fs@freebsd.org Received: from bingnet2.cc.binghamton.edu (bingnet2.cc.binghamton.edu [128.226.1.18]) by hub.freebsd.org (Postfix) with ESMTP id 14BB537BBF0; Fri, 26 May 2000 07:49:20 -0700 (PDT) (envelope-from zzhang@cs.binghamton.edu) Received: from sol.cs.binghamton.edu (sol.cs.binghamton.edu [128.226.123.100]) by bingnet2.cc.binghamton.edu (8.9.3/8.9.3) with ESMTP id KAA23285; Fri, 26 May 2000 10:49:18 -0400 (EDT) Date: Fri, 26 May 2000 10:48:16 -0400 (EDT) From: Zhihui Zhang To: Robert Watson Cc: Chuck McCrobie , freebsd-fs@FreeBSD.ORG Subject: Re: Opinion on File System Implementation In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Fri, 26 May 2000, Robert Watson wrote: > > Chuck, > > Responding briefly as I'm currently on travel at a DARPA PI meeting and > checking out shortly -- in 4.0-RELEASE/STABLE and 5.0-CURRENT, there is an > interface for "extended attributes" on file system objects. Take a look > at extattr(9), VOP_GETEXTATTR(9) and VOP_SETEXTATTR(9). There are exposed > syscalls to userland also, but in 4.x they don't have man pages as there > are no tools using them. > > In 5.0-CURRENT, I introduced rudimentary support for extended attributes > in UFS/FFS to support security extensions I have been working on, so we > also pushed in simple command line tools to allow userland to > read/set/control extended attributes (setextattr(8), getextattr(8), > extattrctl(8)). It may be that this interface is the appropriate > interface to read, if not write, this meta-data. > I am interested in where you are going to put those extended attributes, in the extended inode, in the file data, or in a separate associated file? -Zhihui To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Fri May 26 15: 8:45 2000 Delivered-To: freebsd-fs@freebsd.org Received: from aplcenMP.apl.jhu.edu (apl.jhu.edu [128.220.101.100]) by hub.freebsd.org (Postfix) with ESMTP id C0DEF37B90A; Fri, 26 May 2000 15:08:42 -0700 (PDT) (envelope-from mccrobi@aplcenMP.apl.jhu.edu) Received: from apl.jhu.edu (kslip14.apl.jhu.edu [128.220.108.24]) by aplcenMP.apl.jhu.edu (8.9.3/8.9.1) with ESMTP id SAA21177; Fri, 26 May 2000 18:08:18 -0400 (EDT) Message-ID: <392EF7C3.E8930B01@apl.jhu.edu> Date: Fri, 26 May 2000 18:16:35 -0400 From: Chuck McCrobie X-Mailer: Mozilla 4.72 [en] (X11; I; FreeBSD 4.0-RELEASE i386) X-Accept-Language: en MIME-Version: 1.0 To: Zhihui Zhang Cc: Robert Watson , freebsd-fs@FreeBSD.ORG Subject: Re: Opinion on File System Implementation References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Zhihui Zhang wrote: > > On Fri, 26 May 2000, Robert Watson wrote: As for UFS/FFS, that's upto someone else to figure out ;) As for ODS-2, this VOP_GETEXTATTR and VOP_SETEXTATTR would be the best choice. In ODS-2, these attributes are stored as part of the "inode" (file header). I believe that I will, however, have the file system perform the appropriate record minipulations - although for those record types/file types I don't support, it would make sense to expose the attributes via this interface. In fact, it might make sense for a consistent presentation of attributes for both "supported" and unsupported attributes... Thank you very much, I'll be checking out the new interfaces! > > > > > Chuck, > > > > Responding briefly as I'm currently on travel at a DARPA PI meeting and > > checking out shortly -- in 4.0-RELEASE/STABLE and 5.0-CURRENT, there is an > > interface for "extended attributes" on file system objects. Take a look > > at extattr(9), VOP_GETEXTATTR(9) and VOP_SETEXTATTR(9). There are exposed > > syscalls to userland also, but in 4.x they don't have man pages as there > > are no tools using them. > > > > In 5.0-CURRENT, I introduced rudimentary support for extended attributes > > in UFS/FFS to support security extensions I have been working on, so we > > also pushed in simple command line tools to allow userland to > > read/set/control extended attributes (setextattr(8), getextattr(8), > > extattrctl(8)). It may be that this interface is the appropriate > > interface to read, if not write, this meta-data. > > > > I am interested in where you are going to put those extended attributes, > in the extended inode, in the file data, or in a separate associated file? > > -Zhihui > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-fs" in the body of the message To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Fri May 26 22:36:17 2000 Delivered-To: freebsd-fs@freebsd.org Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by hub.freebsd.org (Postfix) with ESMTP id EF07437B57A for ; Fri, 26 May 2000 22:36:13 -0700 (PDT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (robert@fledge.pr.watson.org [192.0.2.3]) by fledge.watson.org (8.9.3/8.9.3) with SMTP id BAA40661; Sat, 27 May 2000 01:36:11 -0400 (EDT) (envelope-from robert@fledge.watson.org) Date: Sat, 27 May 2000 01:36:10 -0400 (EDT) From: Robert Watson X-Sender: robert@fledge.watson.org To: Zhihui Zhang Cc: Chuck McCrobie , freebsd-fs@FreeBSD.ORG Subject: Re: Opinion on File System Implementation In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Fri, 26 May 2000, Zhihui Zhang wrote: > On Fri, 26 May 2000, Robert Watson wrote: > > > In 5.0-CURRENT, I introduced rudimentary support for extended attributes > > in UFS/FFS to support security extensions I have been working on, so we > > also pushed in simple command line tools to allow userland to > > read/set/control extended attributes (setextattr(8), getextattr(8), > > extattrctl(8)). It may be that this interface is the appropriate > > interface to read, if not write, this meta-data. > > > > I am interested in where you are going to put those extended attributes, > in the extended inode, in the file data, or in a separate associated file? Currently, the attributes are backed to normal files treated as indexed arrays under UFS. I.e., very similar to the quota implementation--a userland process invokes extattrctl() to inform the kernel of mappings between particular named attributes and particular backing files. I posted a relatively detailed description of this to freebsd-fs/freebsd-arch a few months ago before committing, you may want to check the archive for full details. In the long term, an alternative solution needs to be found--right now, from the perspective of softupdates/et al, these attributes are considered "data" not "meta-data", so in the event of a crash, there are few guarantees about consistency and event ordering. Unlike quotas, you can't just inspect the file system to see what the correct values are. I have some safety checks to determine if an extended attribute is appropriate for the inode it's mapped against based on the generation number, which catches use of the file system prior to the mapping being configured, and most results of fsck. In the long term, presumably the correct answer is to make use of some HPFS-like mechanism, although I like to make a distinction between extended attributes (FreeBSD, HPFS, ...) and file forks (NTFS, HFS, ...), where extended attributes have fairly stringent semantics and a limited API (get, replace, remove), vs. a complete address space supporting its own file descriptor, mmap, etc. Switching to an alternative mechanism would also improve performance, especially if the file system/store maintained spacial locality for attributes and inodes they are assigned to. Needless to say, there are lots of design choices in this arena--I posted an overview of common choices for backing ACLs in various UNIX-esque file systems last year at some point. I can repost if desired. The current mechanism is sufficient to support the TrustedBSD labeling requirements during development, which was the goal. :-) Robert N M Watson robert@fledge.watson.org http://www.watson.org/~robert/ PGP key fingerprint: AF B5 5F FF A6 4A 79 37 ED 5F 55 E9 58 04 6A B1 TIS Labs at Network Associates, Safeport Network Services To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message