From owner-freebsd-stable@FreeBSD.ORG Wed Jun 6 13:53:35 2007 Return-Path: X-Original-To: freebsd-stable@freebsd.org Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5528616A46F for ; Wed, 6 Jun 2007 13:53:35 +0000 (UTC) (envelope-from x@vex.net) Received: from mail.295.ca (mail.295.ca [66.38.192.228]) by mx1.freebsd.org (Postfix) with ESMTP id 2C49813C4BF for ; Wed, 6 Jun 2007 13:53:35 +0000 (UTC) (envelope-from x@vex.net) Received: from offload2.295.ca (offload2.295.ca [66.38.192.139]) by mail.295.ca (Postfix) with ESMTP id 1560828BDAFB for ; Wed, 6 Jun 2007 09:20:56 -0400 (EDT) X-Virus-Scanned: amavisd-new at 295.ca Received: from mail.295.ca ([66.38.192.228]) by offload2.295.ca (offload2.295.ca [66.38.192.139]) (amavisd-new, port 10024) with ESMTP id aBXp+VyK1EoL for ; Wed, 6 Jun 2007 09:20:54 -0400 (EDT) Received: from bee (host-208-72-124-111.dyn.295.ca [208.72.124.111]) by mail.295.ca (Postfix) with ESMTP id 32E4628BDB26 for ; Wed, 6 Jun 2007 09:20:54 -0400 (EDT) From: "T. Middleton" To: freebsd-stable@freebsd.org User-Agent: KMail/1.9.6 MIME-Version: 1.0 Content-Disposition: inline X-Length: 3598 X-UID: 2717 X-GPG-Fingerprint: F59D 8A8C A8E1 23B5 407D DD30 9069 D1B3 6416 FFB6 X-Whee: Yes, Please. Date: Wed, 6 Jun 2007 09:20:55 -0400 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <200706060920.56186.x@vex.net> Subject: Booting Supermicro 6015V-M3 on SAS drive with LSI 1068E X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Jun 2007 13:53:35 -0000 I have this machine and I'm just trying to get it to boot from an SAS drive.... that's all. I'm not trying to use the LSI 1068E for RAID. See here if you're curious about specs: http://www.supermicro.com/products/system/1U/6015/SYS-6015V-M3.cfm Booting from the FreeBSD 6.2 install CD is not a problem; and the install kernel is, interestingly, able to see the drives attached to the LSI 1068E controller and use them. The problem is that the machine's BIOS can not seem to see the drives plugged into the LSI 1068E (raid controller), and thus those drives can not be selected in the BIOS for booting. "operating system not found". It seems the only way to get drives bootable from the BIOS that are plugged into the LSI is to put them into a raid. Once a logical disk is configured in the RAID bios one can see in the system bios "PCI SCSI: Software Raid". However, even if i did want to use this RAID device, FreeBSD doesn't see/support it (or, it doesn't support FreeBSD). As an experiment I configured the LSI raid to contain one drive (RAID0) (stripe being the size of the drive). I then installed FreeBSD on this drive, and, wonderfully enough, it worked. The BIOS was set to use the "PCI SCSI: Software raid", and the MBR was picked up from the single drive, and FreeBSD took over from there. I don't know how safe this is, but it seemed to still work after multiple reboots, and installing a bunch of software. Problems arose however when i installed a gmirror label onto the drive. Suddenly there was no more logical drive in the RAID bios on my next reboot; so i was back to "operating system not found": so it seems the LSI RAID also uses the same area as gmirror to store metadata. We want the gmirror on the SAS drives but not the LSI RAID... Other interesting ideas I had included trying to create a boot CD-ROM that would instruct the boot loader to do whatever it needs to do to load gmirror and then continue booting (using vfs.root.mountfrom?). Another idea was using a USB flash key for this. (Not really sure how to create a bootable CDRom to do this, but i'm sure it's possible. Any pointers here might be helpful.) Another idea is it seems we could possibly switch a couple of the cables currently plugged into the LSI ports to the ESB2 SATA ports (this board has both the LSI 1068E controller supporting SAS/SATA, and an ESB2 controler supporting SATA) and use gmirrored SATA drives for booting. This is just a bit of a pain because the existing cables probably aren't long enough... and everything is very tightly packed. Also it means we have to use a couple of SATA drives. And it makes my boss nervous to start moving cables... (though perhaps not as nervous as using a USB flash to boot). Anyhow, anyone played with this hardware? Any recommendations on how to handle this seemingly weird situation?