From owner-freebsd-performance@FreeBSD.ORG Mon May 2 18:27:06 2005 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 12E9216A4CE; Mon, 2 May 2005 18:27:06 +0000 (GMT) Received: from gate.bitblocks.com (bitblocks.com [209.204.185.216]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8154543D70; Mon, 2 May 2005 18:27:05 +0000 (GMT) (envelope-from bakul@bitblocks.com) Received: from bitblocks.com (localhost [127.0.0.1]) by gate.bitblocks.com (8.13.3/8.13.1) with ESMTP id j42IR3KK007482; Mon, 2 May 2005 11:27:03 -0700 (PDT) (envelope-from bakul@bitblocks.com) Message-Id: <200505021827.j42IR3KK007482@gate.bitblocks.com> To: "Poul-Henning Kamp" In-reply-to: Your message of "Mon, 02 May 2005 19:33:18 +0200." <19122.1115055198@critter.freebsd.dk> Date: Mon, 02 May 2005 11:27:03 -0700 From: Bakul Shah cc: Robert Watson cc: Eric Anderson cc: freebsd-performance@freebsd.org cc: Steven Hartland Subject: Re: Very low disk performance on 5.x X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 May 2005 18:27:06 -0000 It may make sense to look at measured bandwidth as a percentage of *guaranteed not to exceed* bandwidth of the disk setup -- what is the theoretical max bandwidth writing to a raw partition (and assuming zero cpu overhead, latency, seek time)? This will help in figuring out how to maximize end-to-end performance, taking into account filesystem overhead, data integrity tradeoffs etc. One has to measure each component to find and fix the top N bottlenecks. This sort of critical analysis is what allowed people to make TCP/IP blazingly fast. For instance, Hartland reported max read/write performance of about 260M/s. *If* his disks can do, say, 80MB/s, for a 5 disk RAID5 he'd get 320MB/s. If so, his end-to-end measured performance would be over 80% of max (for whatever data integrity guarantees). Not too shabby:-) Improving end-to-end disk/os/filesystem performance can be quite an exciting (and time consuming) project. - do we have hooks to measure performance of each component? - do we have tests for these? - do we know the top 3 bottlenecks? - do we have a framework for independently experimenting with each component? (geom definitely helps here!) - do we have a framework for experimenting with various filesystem layout schemes? with various caching strategies? with various allocation strategies? - tools for locating bottlenecks due to mistuning? - tools to help tune the system? - how about tools to auto-tune the system?:) I agree with the questions you raised, but it seems we need to move the discussion to a meta level! I fully realize that in a volunteer project a) there are never enough people b) people works on what they like.... May be someone like you can inspire & put together a team to make freebsd the best damn storage OS in the world! Actually what would be really neat is to factor out all FS code across all *BSDs so that rather than fixing the same old bugs N times, people work on different FS designs. The reality is that open OSes are still nowhere near what SGI boxes of a few years ago could do. I have heard of some NAS box vendors achieving really good performance but that doesn't help us in the open OS community (not to mention their advances will be lost when the company dies or is garbage collected by the likes of EMC, netApp).