From owner-freebsd-questions Tue May 29 10:47:42 2001 Delivered-To: freebsd-questions@freebsd.org Received: from sabre.velocet.net (sabre.velocet.net [198.96.118.66]) by hub.freebsd.org (Postfix) with ESMTP id E518137B424 for ; Tue, 29 May 2001 10:47:33 -0700 (PDT) (envelope-from dgilbert@office.tor.velocet.net) Received: from office.tor.velocet.net (trooper.velocet.net [204.138.45.2]) by sabre.velocet.net (Postfix) with ESMTP id 33CF8137F32; Tue, 29 May 2001 13:47:29 -0400 (EDT) Received: (from dgilbert@localhost) by office.tor.velocet.net (8.11.2/8.9.3) id f4THlPw43383; Tue, 29 May 2001 13:47:25 -0400 (EDT) (envelope-from dgilbert) From: David Gilbert MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15123.57516.832209.744900@trooper.velocet.net> Date: Tue, 29 May 2001 13:47:24 -0400 To: doug.poland@omniresources.com Cc: avatar+freebsd@huey.kew.com, freebsd-questions@freebsd.org Subject: [FBSD-Q] RE: Building a Raid 1 or Raid 5 system? In-Reply-To: References: X-Mailer: VM 6.92 under 20.4 "Emerald" XEmacs Lucid Sender: owner-freebsd-questions@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG [note: anyone has permission to put this in an FAQ somewhere as long as I get credit] >>>>> "doug" == doug poland writes: >> Does someone have a cookbook procedure for building and/or >> upgrading to a Raid 1 or Raid 5 configuration using VINUM? I want >> as much as of the system on RAID as possible, specifically >> including /usr and (if possible) root (/). doug> This will get you going with vinum: doug> http://www.freebsddiary.org/vinum.php I have a different recipe that I've used successfully on 2 drive (raid-1) and 8 drive (raid-5) systems. For the following ad? will be a 2 drive raid-1 and da? will be an 8 drive raid-5. 1. install the bare minimum on drive da0 (or ad0). 1a. Use the custom install 1b. fdisk all drives. 1c. (ad) disklabel ad0s1a 128M / ad0s1b 256M swap ad0s1g rest of disk /d1 Disklabel ad1 with anything, we'll change it in step 2. (da) disklabel da0s1a 128M / da0s1g rest of disk /d1 disklabel rest with anything, we'll change it later. 1d. install minimum from menu. 2. Boot from your minimum install, edit disklabels 2a. unmount /d1 and any other partitions you've made 2b. edit your disk 0 partion (disklabel -e) to change the g partition to be "vinum" instead of "4.2BSD" 2c. (ad) "disklabel ad0 >ad0.label" "disklabel -R -r ad1 ad0.label" (da) "disklabel da0 >da0.label" "disklabel -R -r da1 da0.label" "disklabel -R -r da2 da0.label" edit (disklabel -e) your da2 label. Change the 'a' partition to a 'b' partition and change it's type to swap. Then copy it to da3 thru da7 (or whatever) using the same method above. 2d. (ad) you have two swap partitions now ... ad0s1b and ad1s1b. Add them to fstab and swapon -a (da) you have 6 possible swap partitions (da2s1b thru da7s1b). Add 4 of them to fstab and swapon -a. I don't know why you can't add six yet, but your raid sever probably shouldn't swap that much :). 3. vinum config. Run "vinum create" and add all your disks together into whatever RAID(s) you want. Make sure you start them, etc. Also... use mkfs. Finally mount your new vinum partition on /mnt. (if you're creating multiple raid partitions, you might mkdir /mnt/usr /mnt/var /mnt/u, etc.) Put "vinum_start="YES"" in your /etc/rc.conf 4. Copy /usr and/or /var to the new raid partitions. Edit fstab to put the new partitions into the right place. mv old /usr and /var to /usr-old and /var-old. Reboot. 5. Fire up /stand/sysinstall and install everything _but_ bin that you want on the server. If everything's working, you can also remove /usr-old and /var-old. (bonus steps) Your server is now working, but there are a few bonus steps: 6. (ad) copy / to ad1s1a and make ad1s1a also bootable. 6. (da) copy / to da1s1a and make da1s1a also bootable. You could also have more than one other drive that was bootable. It's also possible to use the other 2 disks that don't have swap as another raid partition for /var (raid-0 or raid-1). 7. cvsup and make world (it is FreeBSD, not Linux, after all)! Here are the two configurations I have: -- 8 drive SCSI RAID-5 -- [1:14:2014]root@arbiter:/usr/ports/lang/python-doc-html> vinum dumpconfig Drive d0: Device /dev/da0s1g Created on at Sat May 5 12:42:19 2001 Config last updated Mon May 28 17:09:41 2001 Size: 18232800256 bytes (17388 MB) volume usr state up plex name usr.p0 state up org raid5 128s vol usr sd name usr.p0.s0 drive d0 plex usr.p0 state up len 35610624s driveoffset 265s plexoffset 0s sd name usr.p0.s1 drive d1 plex usr.p0 state up len 35610624s driveoffset 265s plexoffset 128s sd name usr.p0.s2 drive d2 plex usr.p0 state up len 35610624s driveoffset 265s plexoffset 256s sd name usr.p0.s3 drive d3 plex usr.p0 state up len 35610624s driveoffset 265s plexoffset 384s sd name usr.p0.s4 drive d4 plex usr.p0 state up len 35610624s driveoffset 265s plexoffset 512s sd name usr.p0.s5 drive d5 plex usr.p0 state up len 35610624s driveoffset 265s plexoffset 640s sd name usr.p0.s6 drive d6 plex usr.p0 state up len 35610624s driveoffset 265s plexoffset 768s sd name usr.p0.s7 drive d7 plex usr.p0 state up len 35610624s driveoffset 265s plexoffset 896s [1:7:2007]root@arbiter:/usr/ports/lang/python-doc-html> vinum list 8 drives: D d0 State: up Device /dev/da0s1g Avail: 0/17388 MB (0%) D d1 State: up Device /dev/da1s1g Avail: 0/17388 MB (0%) D d2 State: up Device /dev/da2s1g Avail: 0/17388 MB (0%) D d3 State: up Device /dev/da3s1g Avail: 0/17388 MB (0%) D d4 State: up Device /dev/da4s1g Avail: 0/17388 MB (0%) D d5 State: up Device /dev/da5s1g Avail: 0/17388 MB (0%) D d6 State: up Device /dev/da6s1g Avail: 0/17388 MB (0%) D d7 State: up Device /dev/da7s1g Avail: 0/17388 MB (0%) 1 volumes: V usr State: up Plexes: 1 Size: 118 GB 1 plexes: P usr.p0 R5 State: up Subdisks: 8 Size: 118 GB 8 subdisks: S usr.p0.s0 State: up PO: 0 B Size: 16 GB S usr.p0.s1 State: up PO: 64 kB Size: 16 GB S usr.p0.s2 State: up PO: 128 kB Size: 16 GB S usr.p0.s3 State: up PO: 192 kB Size: 16 GB S usr.p0.s4 State: up PO: 256 kB Size: 16 GB S usr.p0.s5 State: up PO: 320 kB Size: 16 GB S usr.p0.s6 State: up PO: 384 kB Size: 16 GB S usr.p0.s7 State: up PO: 448 kB Size: 16 GB [1:8:2008]root@arbiter:/usr/ports/lang/python-doc-html> cat /etc/rc.conf inetd_enable="NO" sendmail_enable="NO" sshd_enable="YES" usbd_enable="NO" start_vinum="YES" nfs_server_enable="YES" nfs_server_flags="-u -t -n 8" nfs_client_enable="YES" ifconfig_rl0="inet blah netmask 255.255.255.252 up" defaultrouter="blah" hostname="blah.velocet.net" [1:9:2009]root@arbiter:/usr/ports/lang/python-doc-html> disklabel da0 8 partitions: # size offset fstype [fsize bsize bps/cpg] a: 262144 0 4.2BSD 1024 8192 16 # (Cyl. 0 - 16*) c: 35873082 0 unused 0 0 # (Cyl. 0 - 2232*) g: 35610938 262144 vinum # (Cyl. 16*- 2232*) [1:11:2011]root@arbiter:/usr/ports/lang/python-doc-html> disklabel da2 8 partitions: # size offset fstype [fsize bsize bps/cpg] b: 262144 0 swap # (Cyl. 0 - 16*) c: 35873082 0 unused 0 0 # (Cyl. 0 - 2232*) g: 35610938 262144 vinum # (Cyl. 16*- 2232*) [1:12:2012]root@arbiter:/usr/ports/lang/python-doc-html> df -k Filesystem 1K-blocks Used Avail Capacity Mounted on /dev/da0s1a 127023 112459 4403 96% / /dev/vinum/usr 120802676 10283498 100854964 9% /usr [1:13:2013]root@arbiter:/usr/ports/lang/python-doc-html> pstat -s Device 1K-blocks Used Avail Capacity Type /dev/da2s1b 130944 8 130936 0% Interleaved /dev/da3s1b 130944 0 130944 0% Interleaved /dev/da4s1b 130944 0 130944 0% Interleaved /dev/da5s1b 130944 0 130944 0% Interleaved Total 523776 8 523768 0% -- 2 drive IDE RAID-1 -- [1:13:326]root@sheppard:~> vinum dumpconfig Drive d0: Device /dev/ad0s1f Created on freenet.velocet.ca at Fri Nov 3 13:52:16 2000 Config last updated Tue May 29 12:26:36 2001 Size: 30064460288 bytes (28671 MB) volume var state up volume usr state up volume u state up plex name var.p0 state up org concat vol var plex name var.p1 state up org concat vol var plex name usr.p0 state up org concat vol usr plex name usr.p1 state up org concat vol usr plex name u.p0 state up org concat vol u plex name u.p1 state up org concat vol u sd name var.p0.s0 drive d0 plex var.p0 state up len 4194304s driveoffset 265s plexoffset 0s sd name var.p1.s0 drive d1 plex var.p1 state up len 4194304s driveoffset 265s plexoffset 0s sd name usr.p0.s0 drive d0 plex usr.p0 state up len 8388608s driveoffset 4194569s plexoffset 0s sd name usr.p1.s0 drive d1 plex usr.p1 state up len 8388608s driveoffset 4194569s plexoffset 0s sd name u.p0.s0 drive d0 plex u.p0 state up len 46136472s driveoffset 12583177s plexoffset 0s sd name u.p1.s0 drive d1 plex u.p1 state up len 46136472s driveoffset 12583177s plexoffset 0s [1:4:317]root@sheppard:~> vinum list 2 drives: D d0 State: up Device /dev/ad0s1f Avail: 0/28671 MB (0%) D d1 State: up Device /dev/ad1s1f Avail: 0/28671 MB (0%) 3 volumes: V var State: up Plexes: 2 Size: 2048 MB V usr State: up Plexes: 2 Size: 4096 MB V u State: up Plexes: 2 Size: 21 GB 6 plexes: P var.p0 C State: up Subdisks: 1 Size: 2048 MB P var.p1 C State: up Subdisks: 1 Size: 2048 MB P usr.p0 C State: up Subdisks: 1 Size: 4096 MB P usr.p1 C State: up Subdisks: 1 Size: 4096 MB P u.p0 C State: up Subdisks: 1 Size: 21 GB P u.p1 C State: up Subdisks: 1 Size: 21 GB 6 subdisks: S var.p0.s0 State: up PO: 0 B Size: 2048 MB S var.p1.s0 State: up PO: 0 B Size: 2048 MB S usr.p0.s0 State: up PO: 0 B Size: 4096 MB S usr.p1.s0 State: up PO: 0 B Size: 4096 MB S u.p0.s0 State: up PO: 0 B Size: 21 GB S u.p1.s0 State: up PO: 0 B Size: 21 GB [1:5:318]root@sheppard:~> cat /etc/rc.conf # This file now contains just the overrides from /etc/defaults/rc.conf # please make all changes to this file. # -- sysinstall generated deltas -- # ifconfig_rl0="inet blah netmask 255.255.255.0" defaultrouter="199.71.188.30" inetd_enable="YES" #ifconfig_rl0="DHCP" hostname="blah" sendmail_enable="YES" sshd_enable="YES" usbd_enable="NO" start_vinum="YES" portmap_enable="NO" [1:7:320]root@sheppard:~> disklabel ad0 8 partitions: # size offset fstype [fsize bsize bps/cpg] a: 262144 0 4.2BSD 1024 8192 16 # (Cyl. 0 - 260*) b: 1048576 262144 swap # (Cyl. 260*- 1300*) c: 60030369 0 unused 0 0 # (Cyl. 0 - 59553*) f: 58719649 1310720 vinum # (Cyl. 1300*- 59553*) Filesystem 1K-blocks Used Avail Capacity Mounted on /dev/ad0s1a 127023 114415 2447 98% / mfs:29 507639 3 467025 0% /tmp /dev/vinum/var 2032623 34395 1835619 2% /var /dev/vinum/usr 4065262 1523152 2216890 41% /usr /dev/vinum/u 22358577 4457713 16112178 22% /u [1:10:323]root@sheppard:~> pstat -s Device 1K-blocks Used Avail Capacity Type /dev/ad0s1b 524160 0 524160 0% Interleaved /dev/ad1s1b 524160 0 524160 0% Interleaved Total 1048320 0 1048320 0% Dave. -- ============================================================================ |David Gilbert, Velocet Communications. | Two things can only be | |Mail: dgilbert@velocet.net | equal if and only if they | |http://www.velocet.net/~dgilbert | are precisely opposite. | =========================================================GLO================ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-questions" in the body of the message