From owner-freebsd-isp@FreeBSD.ORG Fri Feb 4 23:42:08 2005 Return-Path: Delivered-To: freebsd-isp@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 91E7416A4CF for ; Fri, 4 Feb 2005 23:42:08 +0000 (GMT) Received: from util.inch.com (mx.inch.com [216.223.198.27]) by mx1.FreeBSD.org (Postfix) with ESMTP id E7CDA43D4C for ; Fri, 4 Feb 2005 23:42:07 +0000 (GMT) (envelope-from gcoon@inch.com) Received: from kod.inch.com (kod.inch.com [216.223.192.68]) j14NcmmF068966; Fri, 4 Feb 2005 18:38:48 -0500 (EST) (envelope-from gcoon@inch.com) Date: Fri, 4 Feb 2005 18:38:48 -0500 (EST) From: Gerald To: Simon In-Reply-To: <200502042252.j14MqSQB039609@util.inch.com> Message-ID: <20050204182048.I3975@kod.inch.com> References: <200502042252.j14MqSQB039609@util.inch.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed cc: "freebsd-isp@freebsd.org" cc: Gerald Subject: Re: SATA 3ware RAID review...sort of. X-BeenThere: freebsd-isp@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Internet Services Providers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Feb 2005 23:42:08 -0000 On Fri, 4 Feb 2005, Simon wrote: > While your review is helpful, I hope you realize that server's throughput doesn't > indicate the load you are putting on the card or the harddrives. If you were > serving large movie files, you could quickly fill up both of your 100mbps links > with a single 5.4k RPM ATA drve, given you have some RAM so FreeBSD > could cache the data being served. 30Mbps of bandwidth is merely ~3mBps > of sustained disk transfer. Most of this data could be cached by FreeBSD > without you even knowing. It could also be cached by the memory on the > control and harddrives. Quite a bit of the drive is being cached in memory. Between Apache and the OS the memory is being put to good use, but I just wanted to send out that the SATA setup is standing up to my commercial load for this particular customer. > What would be very useful is if you could mention what kind of files are being > served (their size), how many hits the server serves every second. Are they > mostly the same files or completely random, and so on... how often is the server > writing to disk, is it creating many random files? how big? how often? there is > just so many things involved, you can't merely post your hardware and say you > are pushing 30mbps of bandwidth. I have a 8x250gb using PATA and 3ware > 8port 8600 series card which can do sustained reads of over 100mBps which > would translate into ~1000mbps of bandwidth. However, this doesn't indicate > that it would be able to serve 50,000,000 small, mostly random files, a day like > a similar server using SCSI could. Don't forget, it's one thing when you just > read, but completely different when you read and write, especially with RAID5. Granted and I tried to write enough disclaimers to cover all that. In talking to some other admins I'm most interested to see the following: 1. Backups (working on adding snapshots to dump/amanda now) 2. Reads + Writes. I think there's only 2 occasions when a lot of writes are made to the disks. ...but it's a web server. There's supposed to be more reads than writes. I would hope someone reading my E-mail would be able to easily discern all of what you have correctly pointed out. Nothing is ever simple and I hope my E-mail didn't came across as an attempt to oversimplify a complex operating system and application. Top has this: Mem: 840M Active, 2452M Inact, 270M Wired, 189M Cache, 112M Buf, 138M Free Systat has this: Mem:KB REAL VIRTUAL VN PAGER SWAP PAGER Tot Share Tot Share Free in out in out Act 163180 6012 816892 11436 290788 count 57 All 3884364 8132 2855508 15612 pages 279 Interrupts Proc:r p d s w Csw Trp Sys Int Sof Flt 170 cow 1998 total 397 5390 1932 5340 3397 667 1538 275352 wire 6: fdc0 873736 act 128 8: rtc 19.0%Sys 3.6%Intr 19.0%User 0.0%Nice 58.4%Idl 2542712 inact 13: npx | | | | | | | | | | 202748 cache 15: ata =========++>>>>>>>>>> 88040 free 1704 28: em0 daefr 66 48: twa Namei Name-cache Dir-cache 2079 prcfr 100 0: clk Calls hits % hits % 78 react 5891 5847 99 13 0 pdwake 1193 zfod pdpgs Disks da0 pass0 1193 ofod intrn KB/t 17.59 0.00 %slo-z 114880 buf tps 65 0 2156 tfree 41 dirtybuf MB/s 1.12 0.00 100000 desiredvnodes % busy 38 0 90784 numvnodes 4716 freevnodes The OS is caching between 2.5 and 3 GB of I/O. As far as file sizes and getting in to really small details I have a lot of work left to do on this server to go in to too much. It's a public web server though so go to www.firstview.com and answer some of those questions yourself. Just trying to contribute what I can. (Disclaimer since it seems to be asked often when I give that link: neither I nor my company designed the web site.) Gerald