From owner-freebsd-current@FreeBSD.ORG Fri May 21 19:09:47 2004 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6EC3B16A4CE for ; Fri, 21 May 2004 19:09:47 -0700 (PDT) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.FreeBSD.org (Postfix) with ESMTP id 07D1043D2F for ; Fri, 21 May 2004 19:09:47 -0700 (PDT) (envelope-from scottl@freebsd.org) Received: from freebsd.org (junior-wifi.samsco.home [192.168.0.11]) (authenticated bits=0) by pooker.samsco.org (8.12.11/8.12.10) with ESMTP id i4M2FXZr046991; Fri, 21 May 2004 20:15:33 -0600 (MDT) (envelope-from scottl@freebsd.org) Message-ID: <40AEB65F.2020905@freebsd.org> Date: Fri, 21 May 2004 20:09:35 -0600 From: Scott Long User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.6) Gecko/20040304 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Kris Kennaway References: <20040521224729.680285D0A@ptavv.es.net> <20040522005345.GA12594@xor.obsecurity.org> In-Reply-To: <20040522005345.GA12594@xor.obsecurity.org> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, hits=0.0 required=3.8 tests=none autolearn=no version=2.63 X-Spam-Checker-Version: SpamAssassin 2.63 (2004-01-11) on pooker.samsco.org cc: freebsd-current@freebsd.org Subject: Re: Disk performance under CURRENT X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 May 2004 02:09:47 -0000 Kris Kennaway wrote: > On Fri, May 21, 2004 at 03:47:29PM -0700, Kevin Oberman wrote: > >>I just ran test of disk writing performance under V4 (STABLE) and V5 >>(CURRENT) and was surprised at the difference. >> >>The test was simple and not at all rigorous. Just a dd bs=256k >>if=/dev/zero of=/dev/ad2. This is about the simplest way of dealing with >>a disk. No file system or anything else. Just raw data to the device. >> >>Under STABLE, I get an average of 25 MB/sec to the disk. Under CURRENT, >>it drops to 15 MB/sec. I did this because I had noted that it was now >>taking over an hour to backup my system disk (40 GB) when it was only >>taking 40 minutes when I was running V4.6. The STABLE system was built >>yesterday. The CURRENT system last Sunday. >> >>Any idea why this is so much slower? It looks to me like it must be in >>either geom or the disk driver. > > > Yes, disk performance sucks on 5.x..this is something phk is planning > to work on. > > Kris I'm not really sure what the smoking gun is here compared to 4.x. Yes, in order to do any sort of disk i/o you have to go through the VFS layer which means that you are going to run into Giant, but once you leave there and go into the block layer and below you shouldn't need Giant anymore (assuming that you are using the ATA hardware and driver). Is the problem something related to the extra contexts in GEOM, or is it an inefficiency with ATA locking, or is it that ithreads can't always preempt? I've experimented with taking out the g_up context, but it doesn't appear to make a measurable difference. Maybe g_down is causing a bad effect, though I can't imagine why. On high-end SMP hardware, the AAC driver performs considerably better than 4.x in both sequential and random I/O when multiple processes and/or threads are involved. For single threaded I/O on lower-end UP hardware, there is little difference in my testing between 4.x and 5.x, though 4.x tends to still win. PHK's plans to short-circuit direct device I/O away from VFS will be interesting and will almost certainly have a performance benefit, but that still won't address the common case of doing I/O through a filesystem. We need to really sit down and instrument the I/O path and figure out what the bottlenecks are on both UP and SMP. Maybe it is 100% the fault of VFS and Giant, but we need real measurements to find out. Scott