From owner-freebsd-fs Sun Aug 9 14:37:52 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id OAA26650 for freebsd-fs-outgoing; Sun, 9 Aug 1998 14:37:52 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from smtp03.primenet.com (smtp03.primenet.com [206.165.6.133]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id OAA26645 for ; Sun, 9 Aug 1998 14:37:49 -0700 (PDT) (envelope-from tlambert@usr04.primenet.com) Received: (from daemon@localhost) by smtp03.primenet.com (8.8.8/8.8.8) id OAA11845; Sun, 9 Aug 1998 14:37:30 -0700 (MST) Received: from usr04.primenet.com(206.165.6.204) via SMTP by smtp03.primenet.com, id smtpd011834; Sun Aug 9 14:37:26 1998 Received: (from tlambert@localhost) by usr04.primenet.com (8.8.5/8.8.5) id OAA18972; Sun, 9 Aug 1998 14:37:18 -0700 (MST) From: Terry Lambert Message-Id: <199808092137.OAA18972@usr04.primenet.com> Subject: Re: Filesystem locking during lookups To: dillon@backplane.com (Matthew Dillon) Date: Sun, 9 Aug 1998 21:37:18 +0000 (GMT) Cc: tlambert@primenet.com, wollman@khavrinen.lcs.mit.edu, freebsd-fs@FreeBSD.ORG In-Reply-To: <199808081748.KAA19458@apollo.backplane.com> from "Matthew Dillon" at Aug 8, 98 10:48:43 am X-Mailer: ELM [version 2.4 PL25] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > However, I am still seeing significant VM/fs corruption where > the mmaping of files being actively appended to (in this case, > the spool files) can cause physical corruption of the file. Whistle has a test bed capable of reproducing this and several (seemingly) related problems. Unfortunately, it's statistical, so it's not useful for diagnostics, only for verification. The "seemingly" comes from the fact that we have seen file corruption, but we can repeatably diagnose the missing page problems. The append related problems seem to be a seperate symptom of the same problem, but are not reliably diagnosible except over several days for us (so it looks like you have the best test-jig for that one). We're willing to test any proposed fixes, with a (comparatively) short turn around. Note that these problems are *not* specific to 3.0-current. In the diagnostic vein, can I ask if the appends are occurring on page or FS block boundries? If you could ensure that the file extension was always a page and on a page boundry, then this would guarantee to reliably diagnose whether or not this is related to partial page behaviour. Is your FS block size 8k? If the allocations were forced to 8k instead of one page, this would distinguish whether or not the problems were realted to the treatment of FS frags or not (presuming they still occur when the 4k page size boundary and extension is used). If the problems disappear in either one of these cases, the next thing to try is to set the FS frag size of an 8k for to 4k (yes, I realize this will cause fragmentation aut the wazoo); if the problem still occurs, then disable fragging altogether. If none of these (unacceptable; merely diagnostic) workarounds fix the problem, then that also eliminates a large amount of code from consideration. Meanwhile, I will coinue my instrumentation of the VM system at home, and start triggerring panics on alias references as soon as I'm able, and work at it from that angle. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Thu Aug 13 08:38:06 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id IAA13982 for freebsd-fs-outgoing; Thu, 13 Aug 1998 08:38:06 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from gatewaya.anheuser-busch.com (gatewaya.anheuser-busch.com [151.145.250.252]) by hub.freebsd.org (8.8.8/8.8.8) with SMTP id IAA13913 for ; Thu, 13 Aug 1998 08:37:52 -0700 (PDT) (envelope-from Matthew.Alton@anheuser-busch.com) Received: by gatewaya.anheuser-busch.com; id KAA24138; Thu, 13 Aug 1998 10:34:45 -0500 Received: from stlabcexg004.anheuser-busch.com(stlabcexg004 151.145.101.160) by gatewaya via smap (V2.1) id xma024111; Thu, 13 Aug 98 10:34:16 -0500 Received: by stlabcexg004.anheuser-busch.com with Internet Mail Service (5.5.1960.3) id ; Thu, 13 Aug 1998 16:36:08 +0100 Message-ID: <31B3F0BF1C40D11192A700805FD48BF901776634@STLABCEXG011> From: "Alton, Matthew" To: "'FreeBSD-fs@FreeBSD.ORG'" Subject: LFS Hacking Date: Thu, 13 Aug 1998 16:36:10 +0100 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.1960.3) Content-Type: text/plain Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org What function call is used in the kernel to syncronously write data to a disk? I mean really actually write it and change the polarity in the little magnetic domains and everything such that I may assume that the data is ON THE DISK when the call returns success. How should I impliment "callback" in the kernel buffer cache flushing mechanism so that the vm dosn't interfere with my write scheduling? I need to know how to properly register a filesystem with the kernel at boot time and how to register vm operations including union-of-set-preserving NOOPs and how to "intercept" vm-issued fsyncs and whatever else. A sort of pseudocode stack-trace of the whole shebang from userland write(2) to data-on-the-disk would be wonderful. What are we using instead of SYSV FSS-switch? Once I get all this sorted out, I plan to write up a nice, pretty FS Hacker's Guide for the Handbook. Please feel free to carry on at length about any and all FS implementation particulars. I will give you credit in the Guide. Matthew Alton Computer Services - UNIX Systems Administration (314)632-6644 matthew.alton@anheuser-busch.com alton@plantnet.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Thu Aug 13 22:27:38 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id WAA22998 for freebsd-fs-outgoing; Thu, 13 Aug 1998 22:27:38 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from parkplace.cet.co.jp (parkplace.cet.co.jp [202.32.64.1]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id WAA22974 for ; Thu, 13 Aug 1998 22:27:34 -0700 (PDT) (envelope-from michaelh@cet.co.jp) Received: from localhost (michaelh@localhost) by parkplace.cet.co.jp (8.8.8/CET-v2.2) with SMTP id FAA10021; Fri, 14 Aug 1998 05:24:47 GMT Date: Fri, 14 Aug 1998 14:24:47 +0900 (JST) From: Michael Hancock To: "Alton, Matthew" cc: "'FreeBSD-fs@FreeBSD.ORG'" Subject: Re: LFS Hacking In-Reply-To: <31B3F0BF1C40D11192A700805FD48BF901776634@STLABCEXG011> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Thu, 13 Aug 1998, Alton, Matthew wrote: > What function call is used in the kernel to syncronously write data to a disk? > I mean really actually write it and change the polarity in the little > magnetic domains and everything such that I may assume that the > data is ON THE DISK when the call returns success. Have a look at bwrite() and compare it to bawrite() and bdwrite(). > How should I impliment "callback" in the kernel buffer cache flushing > mechanism so that the vm dosn't interfere with my write scheduling? I > need to know how to properly register a filesystem with the kernel at > boot time and how to register vm operations including > union-of-set-preserving NOOPs and how to "intercept" vm-issued fsyncs > and whatever else. Sorry I've only spent time on VOP glue, but concerning registering an fs poke around in kern. A grep for mount.h should narrow it down. Regards, Mike Hancock To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Fri Aug 14 12:38:13 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id MAA27787 for freebsd-fs-outgoing; Fri, 14 Aug 1998 12:38:13 -0700 (PDT) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from mail-out1.apple.com (mail-out1.apple.com [17.254.0.52]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id MAA27696 for ; Fri, 14 Aug 1998 12:37:59 -0700 (PDT) (envelope-from conrad@apple.com) Received: from mailgate.apple.com (A17-128-100-225.apple.com [17.128.100.225]) by mail-out1.apple.com (8.8.5/8.8.5) with ESMTP id MAA19126 for ; Fri, 14 Aug 1998 12:24:22 -0700 Received: from scv2.apple.com (scv2.apple.com [17.128.100.140]) by mailgate.apple.com (mailgate.apple.com2.0.15) with ESMTP id ; Fri, 14 Aug 1998 12:20:53 -0700 Received: from [17.202.43.185] (wa.apple.com [17.202.43.185]) by scv2.apple.com (8.8.5/8.8.5) with ESMTP id MAA07574; Fri, 14 Aug 1998 12:20:42 -0700 X-Sender: conrad@mail.apple.com Message-Id: In-Reply-To: <483.903079157@gjp.erols.com> References: "Your message of Fri, 14 Aug 1998 02:02:02 EDT." MIME-Version: 1.0 Date: Fri, 14 Aug 1998 12:20:41 -0700 To: freebsd-fs@FreeBSD.ORG From: Conrad Minshall Subject: Re: UDF Filesystem Cc: mturpin@shadow.spel.com Content-Type: text/plain; charset="us-ascii" Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org >Mark turpin wrote in message ID >: >> >> Is any body working on implementing UDF on FreeBSD? I looked in the >> archives and didn't see anything about it? > >URL with details? I'm working on UDF, but not UDF on FreeBSD :-( Here are some potentially useful UDF & DVD sites. Please let me know if you find others. UDF and related DVD standards: http://www.osta.org/ DVD newsgroup FAQ: http://www.videodiscovery.com/vdyweb/dvd/dvdfaq.html DVD industry news: http://www.unik.no/~robert/hifi/dvd/ DVD Video and DVD Audio info: http://www.dvdforum.com/ HP toolkit (a UDF 1.02 source base): http://www.hp.com/storage/optical/udf/ Apple does DVD! http://www.apple.com/dvd/ Also there was a Linux UDF effort. Anyone know its status? >Gary Palmer FreeBSD Core Team Member -- Conrad Minshall mailto:conrad@apple.com If "conrad@apple.com" doesn't work, try using rad@acm.org. Picon viewable at: http://facesaver.usenix.org/faces/h/49/4974.htm To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message