From owner-freebsd-fs Mon Jan 26 11:38:38 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id LAA11141 for freebsd-fs-outgoing; Mon, 26 Jan 1998 11:38:38 -0800 (PST) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from gate.iterated.com (gate.iterated.com [206.30.188.2]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id LAA11133 for ; Mon, 26 Jan 1998 11:38:36 -0800 (PST) (envelope-from CSagar@exchange.iterated.com) Received: from [198.242.115.150] by gate.iterated.com for id OAA26965; Mon Jan 26 14:30:13 1998 Received: by exchange.iterated.com with Internet Mail Service (5.5.1664.3) id ; Mon, 26 Jan 1998 14:30:12 -0500 Message-ID: Subject: help with ccd (again) Date: Mon, 26 Jan 1998 14:30:11 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.1664.3) Content-Type: text/plain; charset="iso-8859-1" To: "'freebsd-fs@freebsd.org'" From: Chris Sagar Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Seems I have overestimated my capability or I have underestimated the ccd/ccdconfig installation process. Each time I get past one problem, I hit another where there is a lacking of documentation. So, please forgive me as I ask some of the same questions again, this time with the entire picture outlined. My goal is to bring online two Adaptec 2944 (PCI) controllers, each with 8 4-GB Seagate disks (SCSI id 1,2,3,4,11,12,13,14) configured as a ccd device. I want to end up with two, 32 GB disk arrays available for usage. These arrays are not boot drives. My boot disk is an IDE drive. Thanks to the help of several of you, I have gotten one of my disk arrays running, but here are my current problems. First, the array configuration does not survive being booted. I have the array being mounted in fstab as /disk1. When the system reboots, it barks about the ccd not being configured. I can not do a ccdconfig, then newfs, then fsck each time I need to reboot the server. What am I missing here? What preserves the configuration information of the ccd array? If I manually do ccdconfig/newfs/fsck/mount on the drive, it is there and usable. But after reboot it is gone. Second, I need to get the second Adaptec controller setup. THe cards are seen during bootup (ahc0 and ahc1). They have different IRQ and addresses. I have run SCSI-Select and this also finds both cards. I am assuming (I know, I know) that the OS is happy, but my question regards the fully qualified device name for the disks. As each disk array will the same scsi ID numbering scheme (scsi id 1,2,3,4,11,12,13,and 14 in each array) where/how do you define in the fully qualified device name the scsi bus that the disk in on? I have not yet connected any disks to the second controller. Are there going to be any other nasties waiting to bite me on this. I have done kernel rebuilds for these items. Here is my kernel config file. # GENERIC -- Generic machine with WD/AHx/NCR/BTx family disks # # For more information read the handbook part System Administration -> # Configuring the FreeBSD Kernel -> The Configuration File. # The handbook is available in /usr/share/doc/handbook or online as # latest version from the FreeBSD World Wide Web server # # # An exhaustive list of options and more detailed explanations of the # device lines is present in the ./LINT configuration file. If you are # in doubt as to the purpose or necessity of a line, check first in LINT. # # $Id: GENERIC,v 1.77.2.12 1997/10/18 11:03:10 joerg Exp $ machine "i386" cpu "I586_CPU" ident ISI maxusers 10 options MATH_EMULATE #Support for x87 emulation options INET #InterNETworking options FFS #Berkeley Fast Filesystem options NFS #Network Filesystem options MSDOSFS #MSDOS Filesystem options "CD9660" #ISO 9660 Filesystem options PROCFS #Process filesystem options "COMPAT_43" #Compatible with BSD 4.3 [KEEP THIS!] options SCSI_DELAY=15 #Be pessimistic about Joe SCSI device options BOUNCE_BUFFERS #include support for DMA bounce buffers options UCONSOLE #Allow users to grab the console options FAILSAFE #Be conservative options USERCONFIG #boot -c editor options VISUAL_USERCONFIG #visual boot -c editor config kernel root on wd0 controller isa0 controller eisa0 controller pci0 controller fdc0 at isa? port "IO_FD1" bio irq 6 drq 2 vector fdintr disk fd0 at fdc0 drive 0 disk fd1 at fdc0 drive 1 tape ft0 at fdc0 drive 2 options "CMD640" # work around CMD640 chip deficiency controller wdc0 at isa? port "IO_WD1" bio irq 14 vector wdintr disk wd0 at wdc0 drive 0 disk wd1 at wdc0 drive 1 controller wdc1 at isa? port "IO_WD2" bio irq 15 vector wdintr disk wd2 at wdc1 drive 0 disk wd3 at wdc1 drive 1 options ATAPI #Enable ATAPI support for IDE bus options ATAPI_STATIC #Don't do it as an LKM device wcd0 #IDE CD-ROM # A single entry for any of these controllers (ncr, ahb, ahc, amd) is # sufficient for any number of installed devices. controller ncr0 controller amd0 controller ahb0 controller ahc0 controller ahc1 controller bt0 at isa? port "IO_BT0" bio irq ? vector bt_isa_intr controller uha0 at isa? port "IO_UHA0" bio irq ? drq 5 vector uhaintr controller aha0 at isa? port "IO_AHA0" bio irq ? drq 5 vector ahaintr controller aic0 at isa? port 0x340 bio irq 11 vector aicintr controller nca0 at isa? port 0x1f88 bio irq 10 vector ncaintr controller nca1 at isa? port 0x350 bio irq 5 vector ncaintr controller sea0 at isa? bio irq 5 iomem 0xc8000 iosiz 0x2000 vector seaintr controller scbus0 at ahc0 controller scbus1 at ahc1 disk sd1 at scbus0 target 1 disk sd2 at scbus0 target 2 disk sd3 at scbus0 target 3 disk sd4 at scbus0 target 4 disk sd11 at scbus0 target 11 disk sd12 at scbus0 target 12 disk sd13 at scbus0 target 13 disk sd14 at scbus0 target 14 disk sd1 at scbus1 target 1 disk sd2 at scbus1 target 2 disk sd3 at scbus1 target 3 disk sd4 at scbus1 target 4 disk sd11 at scbus1 target 11 disk sd12 at scbus1 target 12 disk sd13 at scbus1 target 13 disk sd14 at scbus1 target 14 device sd0 device od0 #See LINT for possible `od' options. device st0 device cd0 #Only need one of these, the code dynamically grows device wt0 at isa? port 0x300 bio irq 5 drq 1 vector wtintr device mcd0 at isa? port 0x300 bio irq 10 vector mcdintr controller matcd0 at isa? port 0x230 bio device scd0 at isa? port 0x230 bio # syscons is the default console driver, resembling an SCO console device sc0 at isa? port "IO_KBD" tty irq 1 vector scintr # Enable this and PCVT_FREEBSD for pcvt vt220 compatible console driver #device vt0 at isa? port "IO_KBD" tty irq 1 vector pcrint #options XSERVER # support for X server #options FAT_CURSOR # start with block cursor # If you have a ThinkPAD, uncomment this along with the rest of the PCVT lines #options PCVT_SCANSET=2 # IBM keyboards are non-std # Mandatory, don't remove device npx0 at isa? port "IO_NPX" flags 0x1 irq 13 vector npxintr # # Laptop support (see LINT for more options) # device apm0 at isa? disable # Advanced Power Management options APM_BROKEN_STATCLOCK # Workaround some buggy APM BIOS # PCCARD (PCMCIA) support #controller crd0 #device pcic0 at crd? #device pcic1 at crd? device sio0 at isa? port "IO_COM1" tty irq 4 vector siointr device sio1 at isa? port "IO_COM2" tty irq 3 vector siointr device sio2 at isa? disable port "IO_COM3" tty irq 5 vector siointr device sio3 at isa? disable port "IO_COM4" tty irq 9 vector siointr device lpt0 at isa? port? tty irq 7 vector lptintr device lpt1 at isa? port? tty device mse0 at isa? port 0x23c tty irq 5 vector mseintr device psm0 at isa? disable port "IO_KBD" conflicts tty irq 12 vector psmintr # Order is important here due to intrusive probes, do *not* alphabetize # this list of network interfaces until the probes have been fixed. # Right now it appears that the ie0 must be probed before ep0. See # revision 1.20 of this file. device de0 device fxp0 device vx0 device ed0 at isa? port 0x280 net irq 5 iomem 0xd8000 vector edintr device ed1 at isa? port 0x300 net irq 5 iomem 0xd8000 vector edintr device ie0 at isa? port 0x300 net irq 10 iomem 0xd0000 vector ieintr device ie1 at isa? port 0x360 net irq 7 iomem 0xd0000 vector ieintr device ep0 at isa? port 0x300 net irq 10 vector epintr device ex0 at isa? port 0x300 net irq 10 iomem 0xd0000 vector exintr device fe0 at isa? port 0x300 net irq ? vector feintr device le0 at isa? port 0x300 net irq 5 iomem 0xd0000 vector le_intr device lnc0 at isa? port 0x280 net irq 10 drq 0 vector lncintr device ze0 at isa? port 0x300 net irq 5 iomem 0xd8000 vector zeintr device zp0 at isa? port 0x300 net irq 10 iomem 0xd8000 vector zpintr pseudo-device loop pseudo-device ether pseudo-device log pseudo-device sl 1 # ijppp uses tun instead of ppp device #pseudo-device ppp 1 pseudo-device vn 1 pseudo-device tun 1 pseudo-device pty 16 pseudo-device gzip # Exec gzipped a.out's pseudo-device ccd 8 # KTRACE enables the system-call tracing facility ktrace(2). # This adds 4 KB bloat to your kernel, and slightly increases # the costs of each syscall. options KTRACE #kernel tracing # This provides support for System V shared memory. # options SYSVSHM From owner-freebsd-fs Mon Jan 26 12:37:26 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id MAA22845 for freebsd-fs-outgoing; Mon, 26 Jan 1998 12:37:26 -0800 (PST) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from pluto.plutotech.com (mail.plutotech.com [206.168.67.137]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id MAA22817 for ; Mon, 26 Jan 1998 12:37:24 -0800 (PST) (envelope-from gibbs@plutotech.com) Received: from narnia.plutotech.com (narnia.plutotech.com [206.168.67.130]) by pluto.plutotech.com (8.8.7/8.8.5) with ESMTP id NAA20688; Mon, 26 Jan 1998 13:11:09 -0700 (MST) Message-Id: <199801262011.NAA20688@pluto.plutotech.com> X-Mailer: exmh version 2.0.1 12/23/97 To: Chris Sagar cc: "'freebsd-fs@freebsd.org'" Subject: Re: help with ccd (again) In-reply-to: Your message of "Mon, 26 Jan 1998 14:30:11 EST." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Mon, 26 Jan 1998 13:08:38 -0700 From: "Justin T. Gibbs" Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org >First, the array configuration does not survive being booted. I have the >array being mounted in fstab as /disk1. When the system reboots, it >barks about the ccd not being configured. I can not do a ccdconfig, then >newfs, then fsck each time I need to reboot the server. What am I >missing here? What preserves the configuration information of the ccd >array? If I manually do ccdconfig/newfs/fsck/mount on the drive, it is >there and usable. But after reboot it is gone. You need to define the ccd array in /etc/ccd.conf if you want the system to automatically reinstate the array with each boot. Read the ccdconfig man page for details on how to do this. As with all file systems, you only need to newfs them once (assuming you don't have a catastrophic failure that requires the file system to be re-initialized). ccdconfig simply creates a striped/mirrored device out of sub devices. Once that device is available, you treat it like any other non-ccd device. Think of it as a puzzle made out of component drives. You use ccdconfig to assemble the puzzle. If you write a pattern on the assembled puzzle, the pattern will survive if you pull the pieces apart (reboot), and then reassemble them (ccdconfig). >Second, I need to get the second Adaptec controller setup. THe cards are >seen during bootup (ahc0 and ahc1). They have different IRQ and >addresses. I have run SCSI-Select and this also finds both cards. I am >assuming (I know, I know) that the OS is happy, but my question regards >the fully qualified device name for the disks. As each disk array will >the same scsi ID numbering scheme (scsi id 1,2,3,4,11,12,13,and 14 in >each array) where/how do you define in the fully qualified device name >the scsi bus that the disk in on? I have not yet connected any disks to >the second controller. Unless you hardwire the devices to specific unit numbers, the system will increment the unit number by one for each "sd" device found. The target id of the device will not matter in an "unwired" configuration other than the fact that devices are found by scanning from lowest to highest target id and will be attached by the system in this order. To find out information about "hardwiring" particular devices to explicit unit numbers read the scsi(4) man page. To specify the devices to ccdconfig, you use their unit number/slice number/ BSD partition number: /dev/sd14s1a Ensure that you create device nodes for the devices that you will use via the /dev/MAKEDEV script: cd /dev sh MAKEDEV sd14s1a -- Justin From owner-freebsd-fs Tue Jan 27 20:51:20 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id UAA12199 for freebsd-fs-outgoing; Tue, 27 Jan 1998 20:51:20 -0800 (PST) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from gate.iterated.com (gate.iterated.com [206.30.188.2]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id UAA12172 for ; Tue, 27 Jan 1998 20:51:13 -0800 (PST) (envelope-from CSagar@exchange.iterated.com) Received: from [198.242.115.150] by gate.iterated.com for id XAA27150; Tue Jan 27 23:26:27 1998 Received: by exchange.iterated.com with Internet Mail Service (5.5.1664.3) id ; Tue, 27 Jan 1998 23:26:26 -0500 Message-ID: Subject: ccd problems still Date: Tue, 27 Jan 1998 23:26:19 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.1664.3) Content-Type: text/plain; charset="iso-8859-1" To: "'freebsd-fs@freebsd.org'" From: Chris Sagar Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Is there anyone who is running CCD devices successfully? I have just spent 8 hrs loading data from disks to the ccd devices I built and tested. As soon as I got the 15 GB of data loaded, I sync-ed the system and rebooted. Lo and behold the ccd device would not come up. I run fsck on it and it tells me bad super block, values disagree with those in first alternate. I give it an alternate superblock number (32) to fsck from, and tells me bad super block, magic number wrong. I just can't do another newfs on it to find the other super block numbers. All you guys that have responded have been a great help, but I just don't get any confidence in this system. I have also, in the last 24 hours, had the system lose the device files for my /var /usr, and / file systems. Any ideas. What is wrong with this disk system. I am about to trash it and go with NT. I believe I could have had an NT server up and running in half the time a 1/10 the frustration of fooling with this ccd system. Pardon the venting, but I have been working on this system now for almost 18 hrs today, and I am now at start-over. Chris Sagar From owner-freebsd-fs Tue Jan 27 21:00:22 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id VAA15166 for freebsd-fs-outgoing; Tue, 27 Jan 1998 21:00:22 -0800 (PST) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from allegro.lemis.com (allegro.lemis.com [192.109.197.134]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id VAA15113 for ; Tue, 27 Jan 1998 21:00:06 -0800 (PST) (envelope-from grog@lemis.com) Received: from freebie.lemis.com (freebie.lemis.com [192.109.197.137]) by allegro.lemis.com (8.8.7/8.8.5) with ESMTP id PAA21873; Wed, 28 Jan 1998 15:29:49 +1030 (CST) Received: (from grog@localhost) by freebie.lemis.com (8.8.8/8.8.7) id PAA17490; Wed, 28 Jan 1998 15:29:48 +1030 (CST) (envelope-from grog) Message-ID: <19980128152948.51356@lemis.com> Date: Wed, 28 Jan 1998 15:29:48 +1030 From: Greg Lehey To: Chris Sagar Cc: "'freebsd-fs@freebsd.org'" Subject: Re: ccd problems still References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 0.84e In-Reply-To: ; from Chris Sagar on Tue, Jan 27, 1998 at 11:26:19PM -0500 Organization: LEMIS, PO Box 460, Echunga SA 5153, Australia Phone: +61-8-8388-8286 Fax: +61-8-8388-8725 Mobile: +61-41-739-7062 WWW-Home-Page: http://www.lemis.com/~grog Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Tue, Jan 27, 1998 at 11:26:19PM -0500, Chris Sagar wrote: > Is there anyone who is running CCD devices successfully? I have done so, and I know others. > I have just spent 8 hrs loading data from disks to the ccd devices I > built and tested. As soon as I got the 15 GB of data loaded, I sync-ed > the system and rebooted. Lo and behold the ccd device would not come up. > I run fsck on it and it tells me bad super block, values disagree with > those in first alternate. I give it an alternate superblock number (32) > to fsck from, and tells me bad super block, magic number wrong. I just > can't do another newfs on it to find the other super block numbers. I would guess that you've screwed up the configuration file somewhere along the way. You *are* talking to /dev/ccd? and not the component disks, aren't you? > All you guys that have responded have been a great help, but I just > don't get any confidence in this system. I'm not overly impressed with the resilience of CCD either, if it makes you any happer. But it does work. > I have also, in the last 24 hours, had the system lose the device files > for my /var /usr, and / file systems. Any ideas. No. > What is wrong with this disk system. I am about to trash it and go with > NT. It's difficult to tell if you don't give any details. What kind of CCD are you building? What version of FreeBSD are you running? What happens if you do a newfs, copy a couple of files to it, and then reboot? > I believe I could have had an NT server up and running in half the time > a 1/10 the frustration of fooling with this ccd system. With CCD? Of course, you'd also get 1/10th the performance, with or without CCD. > Pardon the venting, but I have been working on this system now for > almost 18 hrs today, and I am now at start-over. My sympathies. But you'll get more help if you give a few more details. Greg From owner-freebsd-fs Wed Jan 28 03:59:18 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id DAA16470 for freebsd-fs-outgoing; Wed, 28 Jan 1998 03:59:18 -0800 (PST) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from silvia.HIP.Berkeley.EDU (wck-ca22-16.ix.netcom.com [207.92.174.144]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id DAA16464 for ; Wed, 28 Jan 1998 03:59:15 -0800 (PST) (envelope-from asami@vader.cs.berkeley.edu) Received: (from asami@localhost) by silvia.HIP.Berkeley.EDU (8.8.8/8.6.9) id DAA05861; Wed, 28 Jan 1998 03:59:10 -0800 (PST) Date: Wed, 28 Jan 1998 03:59:10 -0800 (PST) Message-Id: <199801281159.DAA05861@silvia.HIP.Berkeley.EDU> To: CSagar@exchange.iterated.com CC: freebsd-fs@FreeBSD.ORG In-reply-to: (message from Chris Sagar on Tue, 27 Jan 1998 23:26:19 -0500) Subject: Re: ccd problems still From: asami@cs.berkeley.edu (Satoshi Asami) Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org * Is there anyone who is running CCD devices successfully? I've been running it for many years. ;) * I have just spent 8 hrs loading data from disks to the ccd devices I * built and tested. As soon as I got the 15 GB of data loaded, I sync-ed * the system and rebooted. Lo and behold the ccd device would not come up. * I run fsck on it and it tells me bad super block, values disagree with * those in first alternate. I give it an alternate superblock number (32) * to fsck from, and tells me bad super block, magic number wrong. That shouldn't happen. Are you sure the array is configured? It is conceivable that the ccd is just not being configured and fsck can't read anything at all. * I just * can't do another newfs on it to find the other super block numbers. I don't think this will help in this case, but according to newfs's man page, "newfs -N" will print out all the stuff without actually doing anything. * All you guys that have responded have been a great help, but I just * don't get any confidence in this system. Sorry you feel that way. Please send me the output of "cat /etc/ccd.conf", "ccdconfig -g" (after you get the ccd up, of course), and "disklabel ccd0" (or ccd1 or whatever you are using). Satoshi From owner-freebsd-fs Thu Jan 29 06:23:35 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id GAA14037 for freebsd-fs-outgoing; Thu, 29 Jan 1998 06:23:35 -0800 (PST) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from gate.iterated.com (gate.iterated.com [206.30.188.2]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id GAA14030 for ; Thu, 29 Jan 1998 06:23:33 -0800 (PST) (envelope-from CSagar@exchange.iterated.com) Received: from [198.242.115.150] by gate.iterated.com for id JAA26506; Thu Jan 29 09:23:33 1998 Received: by exchange.iterated.com with Internet Mail Service (5.5.1664.3) id ; Thu, 29 Jan 1998 09:23:32 -0500 Message-ID: Subject: ccd problems / part 4 Date: Thu, 29 Jan 1998 09:23:31 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.1664.3) Content-Type: text/plain; charset="iso-8859-1" To: "'freebsd-fs@freebsd.org'" From: Chris Sagar Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org X-To-Unsubscribe: mail to majordomo@FreeBSD.org "unsubscribe freebsd-fs" OK guys, I have finally gotten one ccd volume up and stable. Now my next problem - I am setting up the second ccd volume and I can not get all the disks labeled. Here is what I've got. 8 4-GB scsi disks in a rack mounted chassis that is back-planed. No interconnecting cables between disks. Disk scsi ID numbers are: 1 2 3 4 11 12 13 14 At bootup, BIOS sees the disks correctly, Adaptec SCSI-Select sees the disks correctly, OS boot sees the disks correctly. When I do disklabel on disk sd001 thru sd004, they are tagged ( as evidenced by the activity light) correctly. When I disklabel sd011, the target disk of ID 13 activates. When I disklabel sd012, disk ID 14 lights. I can not get disk Ids 11 and 12 to light. I have the following wired-down statements in kernel config file - controller scbus0 at ahc0 controller scbus1 at ahc1 disk sd001 at scbus0 target 1 disk sd002 at scbus0 target 2 disk sd003 at scbus0 target 3 disk sd004 at scbus0 target 4 disk sd011 at scbus0 target 11 disk sd012 at scbus0 target 12 disk sd013 at scbus0 target 13 disk sd014 at scbus0 target 14 These are disks that have never been attached to the FreeBSD system before. They were on an SGI server. The other ccd volume is exactly the same configuration, with devices names of /dev/sd1xx where xx is the target ID number. It is working fine. How do I get the disklabeling to access the drives I want. I have moved the drives around in the mounting chassis, I have changed the cable connecting to the chassis. Thanks again for your help and patience in dealing with an old SVR4 person. From owner-freebsd-fs Thu Jan 29 12:20:27 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id MAA11469 for freebsd-fs-outgoing; Thu, 29 Jan 1998 12:20:27 -0800 (PST) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from frmug.org (frmug-gw.frmug.org [193.56.58.252]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id MAA11434 for ; Thu, 29 Jan 1998 12:20:17 -0800 (PST) (envelope-from roberto@keltia.freenix.fr) Received: (from uucp@localhost) by frmug.org (8.8.8/frmug-2.2/nospam) with UUCP id VAA28013 for fs@freebsd.org; Thu, 29 Jan 1998 21:20:03 +0100 (CET) (envelope-from roberto@keltia.freenix.fr) Received: (from roberto@localhost) by keltia.freenix.fr (8.8.8/keltia-2.13/nospam) id UAA16386; Thu, 29 Jan 1998 20:13:58 +0100 (CET) (envelope-from roberto) Message-ID: <19980129201358.52193@keltia.freenix.fr> Date: Thu, 29 Jan 1998 20:13:58 +0100 From: Ollivier Robert To: fs@FreeBSD.ORG Subject: Re: ccd problems / part 4 Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 0.89i X-Operating-System: FreeBSD 3.0-CURRENT ctm#4019 AMD-K6 MMX @ 225 MHz Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org X-To-Unsubscribe: mail to majordomo@FreeBSD.org "unsubscribe freebsd-fs" Wait, you labelled your drives with a starting "0". How much are you willing to bet that the number is interpreted as an octal number ? 010 | ID #8 can't be labelled 011 | ID #9 idem 012 | ID #10 idem 013 | ID #11 can be labelled 014 | ID #12 idem Just gropping(sp?) at straws... [ sorry I didn't quote your message, I already deleted it and thought about that explanation after so I'm working from memory... ] -- Ollivier ROBERT -=- FreeBSD: The Power to Serve! -=- roberto@keltia.freenix.fr Usenet Canal Historique From owner-freebsd-fs Fri Jan 30 12:53:02 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id MAA16413 for freebsd-fs-outgoing; Fri, 30 Jan 1998 12:53:02 -0800 (PST) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from gatewayb.anheuser-busch.com (gatewayb.anheuser-busch.com [151.145.250.253]) by hub.freebsd.org (8.8.8/8.8.8) with SMTP id MAA16377; Fri, 30 Jan 1998 12:53:00 -0800 (PST) (envelope-from Matthew.Alton@anheuser-busch.com) Received: by gatewayb.anheuser-busch.com; id OAA20841; Fri, 30 Jan 1998 14:49:30 -0600 Received: from stlabcexg002.anheuser-busch.com(151.145.101.152) by gatewayb.anheuser-busch.com via smap (3.2) id xma020634; Fri, 30 Jan 98 14:49:15 -0600 Received: by STLABCEXG002 with Internet Mail Service (5.0.1458.49) id ; Fri, 30 Jan 1998 14:52:43 -0600 Message-ID: <31B3F0BF1C40D11192A700805FD48BF9C33287@STLABCEXG011> From: "Alton, Matthew" To: fs@FreeBSD.ORG Cc: hackers@FreeBSD.ORG Subject: Filesystem hacking Date: Fri, 30 Jan 1998 14:52:27 -0600 X-Priority: 3 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.0.1458.49) Content-Type: text/plain Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org X-To-Unsubscribe: mail to majordomo@FreeBSD.org "unsubscribe freebsd-fs" I have decided to code up an IBM-style journaling filesystem (jfs) with maximum portability for free unices. While I'm at it I had might as well have the clean-bits map to a PP/extent disk arrangement which will act as a useful abstraction for a Logical Volume Manager / Veritas -esque disk management system which I also find interesting enough to code up. This is not an attempt to morph FreeBSD into AIX by any means. It is just an interesting project which I think may be of use to hackers. This month (02/98) I will explore the GNU HURD OS to see if their goal of creating an OS which would allow me to imple- ment a filesystem in user space has been realized. If it has not, I will use my normal FreeBSD 2.2.2 i386/40 and NetBSD 1.3 Sun 3/50 & 3/80 development boxes. Please let me know if I am duplicating effort and provide me with relevant pointers. From owner-freebsd-fs Fri Jan 30 21:57:33 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id VAA20684 for freebsd-fs-outgoing; Fri, 30 Jan 1998 21:57:33 -0800 (PST) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from word.smith.net.au (ppp6.portal.net.au [202.12.71.106]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id VAA20631; Fri, 30 Jan 1998 21:57:16 -0800 (PST) (envelope-from mike@word.smith.net.au) Received: from word (localhost [127.0.0.1]) by word.smith.net.au (8.8.8/8.8.5) with ESMTP id QAA00595; Sat, 31 Jan 1998 16:19:56 +1030 (CST) Message-Id: <199801310549.QAA00595@word.smith.net.au> X-Mailer: exmh version 2.0zeta 7/24/97 To: "Alton, Matthew" cc: fs@FreeBSD.ORG, hackers@FreeBSD.ORG Subject: Re: Filesystem hacking In-reply-to: Your message of "Fri, 30 Jan 1998 14:52:27 MDT." <31B3F0BF1C40D11192A700805FD48BF9C33287@STLABCEXG011> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Sat, 31 Jan 1998 16:19:55 +1030 From: Mike Smith Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org X-To-Unsubscribe: mail to majordomo@FreeBSD.org "unsubscribe freebsd-fs" > This month (02/98) I will explore the GNU HURD OS to see if > their goal of creating an OS which would allow me to imple- > ment a filesystem in user space has been realized. If it has > not, I will use my normal FreeBSD 2.2.2 i386/40 and NetBSD > 1.3 Sun 3/50 & 3/80 development boxes. > > Please let me know if I am duplicating effort and provide me > with relevant pointers. You can implement a filesystem in userspace on FreeBSD (and presumably NetBSD as well); see the 'rumba' port for an example of this. -- \\ Sometimes you're ahead, \\ Mike Smith \\ sometimes you're behind. \\ mike@smith.net.au \\ The race is long, and in the \\ msmith@freebsd.org \\ end it's only with yourself. \\ From owner-freebsd-fs Sat Jan 31 09:25:48 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id JAA07499 for freebsd-fs-outgoing; Sat, 31 Jan 1998 09:25:48 -0800 (PST) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from x115-105.reshalls.umn.edu (x115-105.reshalls.umn.edu [134.84.115.105]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id JAA07469; Sat, 31 Jan 1998 09:25:36 -0800 (PST) (envelope-from chris@x115-105.reshalls.umn.edu) Received: from x115-105.reshalls.umn.edu (localhost [127.0.0.1]) by x115-105.reshalls.umn.edu (8.8.7/8.8.7) with ESMTP id LAA13881; Sat, 31 Jan 1998 11:27:19 -0600 (CST) (envelope-from chris@x115-105.reshalls.umn.edu) Message-Id: <199801311727.LAA13881@x115-105.reshalls.umn.edu> From: mikk0022@maroon.tc.umn.edu To: "Alton, Matthew" cc: fs@FreeBSD.ORG, hackers@FreeBSD.ORG Subject: Re: Filesystem hacking In-reply-to: Your message of "Fri, 30 Jan 1998 14:52:27 CST." <31B3F0BF1C40D11192A700805FD48BF9C33287@STLABCEXG011> References: <31B3F0BF1C40D11192A700805FD48BF9C33287@STLABCEXG011> Date: Sat, 31 Jan 1998 11:27:19 -0600 Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org X-To-Unsubscribe: mail to majordomo@FreeBSD.org "unsubscribe freebsd-fs" On Fri, 30 Jan 1998 14:52:27 -0600 "Alton, Matthew" wrote >I have decided to code up an IBM-style journaling filesystem >(jfs) with maximum portability for free unices. While I'm at it >I had might as well have the clean-bits map to a PP/extent >disk arrangement which will act as a useful abstraction for a >Logical Volume Manager / Veritas -esque disk management >system which I also find interesting enough to code up. What do you know about LFS for FreeBSD. I haven't used it, but from what I understand, it was an early implementation of a "log-structured filesystem" for BSD. Are "log-structured" and "journaling" synonymous? I know that SGI's XFS is a hybrid, where each filesystem has a log which stores committed operations on the filesystem. The filesystem is a fairly normal filesystem from what I understand. The advantage to the separate log is flexibility -- some installations store the log on a separate, high speed disk, but most just use an "internal log" on the same partition as the filesystem. As far as Logical Volume Management, SGI's XLV is a good target (can you tell what kind of UNIXen I use at work yet? :-). In my understanding, the system marks each disk with its place in the volume, so the logical volume can be automagically composed on boot-up. This is nice, because there is no configuration file to worry about, and you can move around the disks on the SCSI chain without affecting the volume. >This is not an attempt to morph FreeBSD into AIX by any >means. I have zero real experience with AIX, but know enough to thank you kindly for this :-) Sorry 'bout going on about SGI. I'm really not a salesman, just a not-very-annoyed user tossing out ideas.... :-) I like the idea of implementing multiple filesystems on FreeBSD. I have a currently unused disk, if you'd like a beta tester (hint, hint... :-) -- Chris Mikkelson mikk0022@maroon.tc.umn.edu U of M Tuba and Student "Life is too short for windows..." '94-present From owner-freebsd-fs Sat Jan 31 14:52:37 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id OAA19790 for freebsd-fs-outgoing; Sat, 31 Jan 1998 14:52:37 -0800 (PST) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from friley585.res.iastate.edu (friley585.res.iastate.edu [129.186.167.85]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id OAA19754 for ; Sat, 31 Jan 1998 14:52:33 -0800 (PST) (envelope-from ccsanady@friley585.res.iastate.edu) Received: from friley585.res.iastate.edu (loopback [127.0.0.1]) by friley585.res.iastate.edu (8.8.8/8.8.5) with ESMTP id QAA10636; Sat, 31 Jan 1998 16:52:52 -0600 (CST) Message-Id: <199801312252.QAA10636@friley585.res.iastate.edu> X-Mailer: exmh version 2.0.1 12/23/97 To: "Alton, Matthew" cc: fs@FreeBSD.ORG Subject: Re: Filesystem hacking In-reply-to: Your message of "Fri, 30 Jan 1998 14:52:27 CST." <31B3F0BF1C40D11192A700805FD48BF9C33287@STLABCEXG011> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Sat, 31 Jan 1998 16:52:51 -0600 From: Chris Csanady Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org X-To-Unsubscribe: mail to majordomo@FreeBSD.org "unsubscribe freebsd-fs" >I have decided to code up an IBM-style journaling filesystem >(jfs) with maximum portability for free unices. While I'm at it >I had might as well have the clean-bits map to a PP/extent >disk arrangement which will act as a useful abstraction for a >Logical Volume Manager / Veritas -esque disk management >system which I also find interesting enough to code up. This is quite nice to hear, I would love to see a journaling file system for FreeBSD. If you were to base it on JFS, you might want to talk to Terry as I believe that he has written a read only version. If you were to make it compatible with as existing fs, I think that SGI's XFS would be the way to go. I think that it is considerably better than JFS. See the white paper.. http://www.sgi.com/Technology/xfs-whitepaper.html This makes me curious though--how would one go about creating an fs like jfs, ntfs, hfs or something? Do you need to reverse engineer the whole thing? Chris Csanady From owner-freebsd-fs Sat Jan 31 15:20:42 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id PAA24407 for freebsd-fs-outgoing; Sat, 31 Jan 1998 15:20:42 -0800 (PST) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from freefall.pipeline.ch (intranet.pipeline.ch [195.134.128.66]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id PAA24279 for ; Sat, 31 Jan 1998 15:19:27 -0800 (PST) (envelope-from andre@pipeline.ch) Received: from pipeline.ch ([195.134.128.41]) by freefall.pipeline.ch (Netscape Mail Server v2.02) with ESMTP id AAA316 for ; Sun, 1 Feb 1998 00:07:22 +0100 Message-ID: <34D3AECD.218E85DB@pipeline.ch> Date: Sun, 01 Feb 1998 00:07:57 +0100 From: "IBS / Andre Oppermann" Organization: Internet Business Solutions Ltd. (AG) X-Mailer: Mozilla 4.03 [en] (WinNT; U) MIME-Version: 1.0 To: fs@FreeBSD.ORG Subject: Re: Filesystem hacking References: <199801312252.QAA10636@friley585.res.iastate.edu> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org X-To-Unsubscribe: mail to majordomo@FreeBSD.org "unsubscribe freebsd-fs" Chris Csanady wrote: > > >I have decided to code up an IBM-style journaling filesystem > >(jfs) with maximum portability for free unices. While I'm at it > >I had might as well have the clean-bits map to a PP/extent > >disk arrangement which will act as a useful abstraction for a > >Logical Volume Manager / Veritas -esque disk management > >system which I also find interesting enough to code up. > > This is quite nice to hear, I would love to see a journaling file system > for FreeBSD. If you were to base it on JFS, you might want to talk > to Terry as I believe that he has written a read only version. If > you were to make it compatible with as existing fs, I think that SGI's > XFS would be the way to go. I think that it is considerably better > than JFS. See the white paper.. > > http://www.sgi.com/Technology/xfs-whitepaper.html This is the whitepaper of the new Novell Netware Filesystem, it's a hybrid between filesystem and database. http://www.novell.com/products/nss/whtpaper.html This paper shows very good the concept of the Novell Storage Services. -- Andre Oppermann CEO / Geschaeftsfuehrer Internet Business Solutions Ltd. (AG) Hardstrasse 235, 8005 Zurich, Switzerland Fon +41 1 277 75 75 / Fax +41 1 277 75 77 http://www.pipeline.ch ibs@pipeline.ch From owner-freebsd-fs Sat Jan 31 16:49:46 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id QAA10654 for freebsd-fs-outgoing; Sat, 31 Jan 1998 16:49:46 -0800 (PST) (envelope-from owner-freebsd-fs@FreeBSD.ORG) Received: from smtp01.primenet.com (smtp01.primenet.com [206.165.6.131]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id QAA10638; Sat, 31 Jan 1998 16:49:31 -0800 (PST) (envelope-from tlambert@usr06.primenet.com) Received: (from daemon@localhost) by smtp01.primenet.com (8.8.8/8.8.8) id RAA22566; Sat, 31 Jan 1998 17:49:28 -0700 (MST) Received: from usr06.primenet.com(206.165.6.206) via SMTP by smtp01.primenet.com, id smtpd022520; Sat Jan 31 17:49:21 1998 Received: (from tlambert@localhost) by usr06.primenet.com (8.8.5/8.8.5) id RAA18732; Sat, 31 Jan 1998 17:49:17 -0700 (MST) From: Terry Lambert Message-Id: <199802010049.RAA18732@usr06.primenet.com> Subject: Re: Filesystem hacking To: mikk0022@maroon.tc.umn.edu Date: Sun, 1 Feb 1998 00:49:17 +0000 (GMT) Cc: Matthew.Alton@anheuser-busch.com, fs@FreeBSD.ORG, hackers@FreeBSD.ORG In-Reply-To: <199801311727.LAA13881@x115-105.reshalls.umn.edu> from "mikk0022@maroon.tc.umn.edu" at Jan 31, 98 11:27:19 am X-Mailer: ELM [version 2.4 PL25] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org X-To-Unsubscribe: mail to majordomo@FreeBSD.org "unsubscribe freebsd-fs" > What do you know about LFS for FreeBSD. I haven't used > it, but from what I understand, it was an early implementation > of a "log-structured filesystem" for BSD. Are "log-structured" > and "journaling" synonymous? No, they are not. A log-structured FS logs data; a journalling FS logs data and transactions in a transaction journal. A log structured FS can only roll transactions back to recover from failures. A journalling FS can roll transactions forward. A Journalling FS also allows you to expose a transactioning interface to allow you to group transactions. For example, if you have a relational database consisting of an index file and a records file, you can be guaranteed that a user transaction resulting in both the record file and the index file being modified is atomic, even if the individual operations are not idempotent. This saves the user process from having to process the roll-forward/roll-back themselves, using a two stage commit. It also saves you from reference problems in case of a failure in the middle of the index write for the second stage of the commit. > I know that SGI's XFS is a hybrid, where each filesystem has a log > which stores committed operations on the filesystem. The filesystem > is a fairly normal filesystem from what I understand. The advantage > to the separate log is flexibility -- some installations store > the log on a separate, high speed disk, but most just use an > "internal log" on the same partition as the filesystem. XFS internally seems to look a lot like NTFS internally. That is, it seems to journal. I've only looked at images of small XFS's snapshotted before and after transactions, I haven't really snooped out the structure. > As far as Logical Volume Management, SGI's XLV is a good target > (can you tell what kind of UNIXen I use at work yet? :-). In my > understanding, the system marks each disk with its place in the > volume, so the logical volume can be automagically composed on > boot-up. This is nice, because there is no configuration file to > worry about, and you can move around the disks on the > SCSI chain without affecting the volume. CCD can do this as well. The problem with most logical volume management strategy is that the FS types you could use if it were implemented in BSD can be grown (at the expense of fragmentation), but can't really be shrunk. In both the grow/shrink case, you need a defragmenter that can be run after/before the fact, and can be told to not reuse an area (for the shrink case). This is a pain, since the fragmentation has never really been a problem until you wanted to do this, so nobody has written much code to deal with it. Other than "tar"/"cpio" and using other media to store the image in the interim. 8-(. LFS/JFS/XFS/NTFS can deal with the changing volumes sizes, but only because they already have the concept of a garbage collection process which can be subverted into the role of an FS expander or contracter which can deal with the problem. Logical volume management is one of the reasons IO was unhappy to see LFS go into the attic. 8-(. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers.