From owner-freebsd-questions@FreeBSD.ORG Thu Nov 18 17:29:43 2010 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 83189106564A for ; Thu, 18 Nov 2010 17:29:43 +0000 (UTC) (envelope-from pgiessel@mac.com) Received: from asmtpout018.mac.com (asmtpout018.mac.com [17.148.16.93]) by mx1.freebsd.org (Postfix) with ESMTP id 6C07C8FC0A for ; Thu, 18 Nov 2010 17:29:43 +0000 (UTC) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; charset=UTF-8; format=flowed Received: from Blue-Star.local ([12.12.31.115]) by asmtp018.mac.com (Sun Java(tm) System Messaging Server 6.3-8.01 (built Dec 16 2008; 32bit)) with ESMTPSA id <0LC300AZICLFP000@asmtp018.mac.com> for freebsd-questions@freebsd.org; Thu, 18 Nov 2010 09:29:42 -0800 (PST) X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 ipscore=0 suspectscore=2 phishscore=0 bulkscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx engine=6.0.2-1010190000 definitions=main-1011180128 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.2.15,1.0.148,0.0.0000 definitions=2010-11-18_08:2010-11-18, 2010-11-18, 1970-01-01 signatures=0 Date: Thu, 18 Nov 2010 08:29:41 -0900 From: "Peter A. Giessel" To: Andy Wodfer X-Priority: 3 In-reply-to: Message-id: X-Mailer: Mailsmith 2.2.5 Cc: freebsd-questions Subject: Re: FreeBSD and large harddrives X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Nov 2010 17:29:43 -0000 On 2010/11/18 at 3:16, wodfer@gmail.com (Andy Wodfer) wrote: >Total storage in version 1 of this server will >probably be 8-12 TB. ... >The processor will be a 64bit capable Intel processor and I plan on using a >Highpoint Rocketraid or 3ware Raid controller. ... >1. Which FreeBSD version should I install? ... >2. I know that the 3ware Raid controller supports larger drives than 2TB (or >was it 1TB?)... How can I create this huge partition/slice? I >don't think the installer (atleast on the standard FreeBSD version) supports >these large drives? Just chiming in with my own experience... I built a "cheap" media storage server so I don't have to keep digging DVDs out of the pile and can just stream them from the server. 4x1.5TB Raid 5 disks. I haven't updated it to FreeBSD 8.2 yet, so: [media: ~]> uname -a FreeBSD media 8.0-RELEASE FreeBSD 8.0-RELEASE #0: Sat Nov 21 15:48:17 UTC 2009 root@almeida.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC i386 (its a Pentium 4... cheap but effective hardware... the processor is hardly ever significantly used. Its just a file server). To ease booting problems, I used 3ware's drivers to create a booting volume out of the bigger unit. FYI: I used Seagate hard drives (I hear they are coming out with a 3TB internal drive any day now): _______________________________________________________ # tw_cli /c0 show all /c0 Driver Version = 3.70.05.001 /c0 Model = 9550SXU-4LP /c0 Available Memory = 112MB /c0 Firmware Version = FE9X 3.08.00.029 /c0 Bios Version = BE9X 3.10.00.003 /c0 Boot Loader Version = BL9X 3.02.00.001 /c0 Serial Number = L320912A9410199 /c0 PCB Version = Rev 032 /c0 PCHIP Version = 1.60 /c0 ACHIP Version = 1.90 /c0 Number of Ports = 4 /c0 Number of Drives = 4 /c0 Number of Units = 1 /c0 Total Optimal Units = 1 /c0 Not Optimal Units = 0 /c0 JBOD Export Policy = off /c0 Disk Spinup Policy = 1 /c0 Spinup Stagger Time Policy (sec) = 1 /c0 Auto-Carving Policy = off /c0 Auto-Carving Size = 2048 GB /c0 Auto-Rebuild Policy = on /c0 Rebuild Rate = 1 /c0 Verify Rate = 1 /c0 Controller Bus Type = PCIX /c0 Controller Bus Width = 64 bits /c0 Controller Bus Speed = 66 Mhz Unit UnitType Status %RCmpl %V/I/M Stripe Size(GB) Cache AVrfy ------------------------------------------------------------------------------ u0 RAID-5 OK - - 64K 4190.92 ON OFF Port Status Unit Size Blocks Serial --------------------------------------------------------------- p0 OK u0 1.36 TB 2930277168 9VS2Y6AV p1 OK u0 1.36 TB 2930277168 9VS2X5A2 p2 OK u0 1.36 TB 2930277168 9VS2VDDA p3 OK u0 1.36 TB 2930277168 9VS2X5P6 _______________________________________________________ Because I was using an older motherboard that doesn't have EFI, I needed a boot partition that was less than 2.2TB, so I used 3ware's bios utility to make 2 volumes: _______________________________________________________ # tw_cli /c0/u0 show Unit UnitType Status %RCmpl %V/I/M Port Stripe Size(GB) ------------------------------------------------------------------------ u0 RAID-5 OK - - - 64K 4190.92 u0-0 DISK OK - - p3 - 1396.97 u0-1 DISK OK - - p2 - 1396.97 u0-2 DISK OK - - p1 - 1396.97 u0-3 DISK OK - - p0 - 1396.97 u0/v0 Volume - - - - - 20 u0/v1 Volume - - - - - 4170.92 _______________________________________________________ To partition the larger volume I used gpart: _______________________________________________________ # gpart show => 63 41942943 da0 MBR (20G) 63 41929587 1 freebsd [active] (20G) 41929650 13356 - free - (6.5M) => 0 41929587 da0s1 BSD (20G) 0 524288 1 freebsd-ufs (256M) 524288 8388608 2 freebsd-swap (4.0G) 8912896 8388608 4 freebsd-ufs (4.0G) 17301504 8388608 5 freebsd-ufs (4.0G) 25690112 16239475 6 freebsd-ufs (7.7G) => 34 8747055038 da1 GPT (4.1T) 34 8747055038 1 freebsd-ufs (4.1T) _______________________________________________________ Which turned out thusly: _______________________________________________________ # df -h Filesystem Size Used Avail Capacity Mounted on /dev/da0s1a 248M 163M 64M 72% / devfs 1.0K 1.0K 0B 100% /dev /dev/da1p1 3.9T 3.5T 155G 96% /media /dev/da0s1e 3.9G 18K 3.6G 0% /tmp /dev/da0s1f 7.5G 3.4G 3.5G 50% /usr /dev/da0s1d 3.9G 234M 3.3G 6% /var _______________________________________________________ I just used UFS formatted slices as I was planning to share quite a few of them using NFS. It has worked quite well and I'm quite happy with my setup. I really like the 3ware raid cards. They have worked quite well for me both on this project and on a previous server project.