From owner-freebsd-geom@FreeBSD.ORG Mon Jun 25 22:37:10 2007 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id C9E5F16A421 for ; Mon, 25 Jun 2007 22:37:10 +0000 (UTC) (envelope-from etc@fluffles.net) Received: from auriate.fluffles.net (cust.95.160.adsl.cistron.nl [195.64.95.160]) by mx1.freebsd.org (Postfix) with ESMTP id 7BE2513C45B for ; Mon, 25 Jun 2007 22:37:10 +0000 (UTC) (envelope-from etc@fluffles.net) Received: from 195-241-125-45.dsl.ip.tiscali.nl ([195.241.125.45] helo=[10.0.0.18]) by auriate.fluffles.net with esmtpa (Exim 4.66 (FreeBSD)) (envelope-from ) id 1I2xB1-000N6D-DS; Tue, 26 Jun 2007 00:36:55 +0200 Message-ID: <4680438E.9030407@fluffles.net> Date: Tue, 26 Jun 2007 00:37:02 +0200 From: Fluffles User-Agent: Thunderbird 2.0.0.4 (Windows/20070604) MIME-Version: 1.0 To: freebsd-geom@freebsd.org Content-Type: text/plain; charset=ISO-8859-15; format=flowed Content-Transfer-Encoding: 7bit Cc: Pawel Jakub Dawidek Subject: gjournal performance issues X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Jun 2007 22:37:10 -0000 Hello list, I'm testing gjournal present in -CURRENT as of 13 June 2007. So far i'm not really impressed with performance, i'm writing to the list for any suggestions and information regarding gjournal. First, my setup: 8 disks in RAID5 using geom_raid5, gjournal on top where both the journal (1GB) and the data is stored on the same consumer. Since gjournal uses both metadata and file/dir for journaling, this means that, theoretically, the write speed of sequential operations is doubled. Unfortunately, it appears to have crashed. My problems: - first, throughput appears to be only 8% of the throughput when not using gjournal at all. Whereas it should be close to 50%. - second, during the 'switch' (writing the journal to its final location and starting a new journal) it appears no read operations are possible to the .journal device. If the .journal is /usr, that means the whole system basically freezes for 3 to 5 seconds. Not really sexy. Why would it block read requests? - when using one consumer for both journal and data, it appears the journal is placed at the end of the device. Why? Normally, the beginning of a disk is the fastest and therefore preferable location for the journal. - when analysing graid5 sysctl statistics, it appears gjournal is causing non-contiguous I/O which causes a lot of 2-phase I/O's (involving both reading and writing for 1 write request), the performance difficulties are most probably related to this issue. Why doesn't gjournal read a chunk of journal (50MB) and then write it? And why doesn't it write contiguously? I've tried: - playing with graid5 tunables, including disabling write-back buffer - playing with journal tunables, including disabling optimization (combining), reducing parallel operations to 1, reducing journal switch time and more - kmem is 500MB, gjournal can use 250MB of kernel memory for it's cache (more than the default) - standard UFS2 using async option and without softupdates, newfs used with -J parameter Anyone has any input? I was hoping for at least 40MB/s throughput and no blocking I/O for read requests. Regards, - Veronica