Date: Sun, 05 May 2002 12:05:20 -0700 From: Terry Lambert <tlambert2@mindspring.com> To: dak <aurelien.nephtali@wanadoo.fr> Cc: freebsd-fs@freebsd.org Subject: Re: Implementing a new FS with loadable modules Message-ID: <3CD58270.DF9B8F06@mindspring.com> References: <20020505173855.GA528@nitrogen.WorkGroup>
next in thread | previous in thread | raw e-mail | index | archive | help
dak wrote: > I'm making my own file system (just for fun and to understand how it works) > and I want to implement it in a module. > > Can anybody points me to a skeleton code which regroups code to mount and > everything the system needs to 'understand' my module ? > > I've already looked at 'ntfs' module and others but I don't realy understand > the way to follow to make a module that works :/ There is no such thing as "skeleton code" for a working FS. The FreeBSD VFS layer doesn't really support the concept of skeletonizing FS code very well, and has actually evloved further from that goal than it was originally, imeediately after 4.4.-Lite and 4.4.-Lite2. In simple terms, there are three types of VFS implementations: 1) Local media FS Examples: FFS, NTFS, EXT2FS This implementation writes data to local storage, and retrieves it from local storage. In most cases, this local storage is a disk drive. To write this type of FS, you will need a good understanding of the VM system. The simplest implementation is probably the MS-DOS FS. This is because it does not address the NFS export, the root mount, or other important issues, and it does not address a lot of the UNIX permissions and other access semantics, including the idea of file ownership, etc. (all of which are not supported by the on disk structure of this FS). 2) Stacking VFS layer Examples: nullfs; there are no other working examples. This implementation calls down to the underlying layer. A stacking VFS layer has the same interface on the bottom that it has on the top. Rather than being a consumer of the VM services, it's a consumer of the VFS interface. Because there are cache coherency issues when you attempt to do anything with the data, since each vnode has a pointer to a vm_object_t, if you do anything more complicated than a pass through layer (e.g. "nullfs"), then you will need a good understanding of the VM system. The simplest possible fully functional VFS stacking layer implementation is nullfs; anythiing more complicates could require some changes the the FreeBSD VFS interface itself, to actually address, rahter than work around, the coherency issues. In particular, this would be required of any VFS that wanted to manipulate data, rather than passing it through without any modification. This effectively limits stacking layers to semantic enforcement. 3) Pseudo VFS layer Examples: procfs, devfs, specfs This implementation does not consume the lower layer, and it is not (normally) a consumer of the VM system. Operations on it are handled within the VFS itself. Because it does not have dependencies on more complex interfaces (unless the implementation demands it), this is the simplest VFS layer. #3 is easiest to write, but least useful. A minimally full functional FS is a complicated thing to write. It implements a "struct vfsops" , which contains pointers to the functions that operate on file systems. It exports it both as a module interface list, and as a linker set entry, so that it can be automatically aggregated into the list of system file system types when statically linked into the kernel. These are generally called "VFSOPs". It also implements a "struct vnodeopv_desc", which contains a pointer to the descriptor list "struct vnodeopv_entry_desc", which contains pointers to the functions that operate on objects within a single instance of a file system (files and directories). If the FS supports FIFOs and special device nodes as file system objects, then it will also implement a "struct vnodeopv_desc" for each of these. In effect, the specfs and fifofs are implicitly stacked already, rather than being explicitly stacked (see /sys/ufs/ffs/ffs_vnops.c). This works because the stacking layers *replace*, rather than *augment and pass through* in this case. In other words, they are a tiny subset of what you are supposed to be able to do with stacking layers. The operations that are defined here are generally called "VOPs" (or "VNOPs", if you have an SVR4 background). The nullfs is more than "the minimum possible FS"; it implements all of the necessary VFSOPS and VOPS to be able to pass through all requests to an underlying layer. A "minimum possible FS" would implement only the VFSOPs necessary to support mount and unmount (not FS export, rout mount, file handle conversion, or extended attribute definition, etc. ...the VFSOPs have become overly complicated recently, actually). It would also implement only VOPs necessary for minimum mount related directory operations... thus allowing you to mount and unmount the FS, cd into the root (since the same operations are required to support access to the root vnode of the FS for the mount), but nothing else. A "somewhat more than minimum possible FS" would implement directory elelment lookup and iteration operations. You would be able to "cd .." to get out of the mounted FS, without having to cd "/", since the ".." relative path would then exist (and reference the parent directory). You would also be able to "ls -la" (to see the "." and ".." entries), and you would be able to stat the "." inode to see its size, etc.. But you would not be able to "cat .", as would would on another FS, since the read/write/open/close on files themselves would not exist. Really, FSs are complicated things. They touch on many of the services exported by the operating system. It's important to understand these services before you go off and try to write an FS. You will be lucky to get off with implementing 30 functions, plus the supporting data structures to make these functions visible to the OS. Also, last time I looked, FFS consumes 148 kernel interfaces -- different functions -- to get its work done (you can "look", too: compile a kernel, then go into the kernel build directory, and link all the ufs_*.o and ffs_*.o files together, and count the number of unique unresolved externals). I *would* suggest that you look at FreeBSD's DDI/DKI (device driver interface, device/kernel interface) documentation, but the DDI and DKI change quickly enough that they are very poorly documented. It would be incredibly beneficial if they were designed based on some foresight into future use, *then left the heck alone*, so they could be adequately documented. Basically, this means that you need to be pretty good at reading source code and chasing a moving target, if you want to be able to write an FS and have it work when you are done. Most people start with an existing FS similar to their target FS, and then modify the code. Not a good example of the use of "first principles", I know. 8-(. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3CD58270.DF9B8F06>