From owner-freebsd-fs@FreeBSD.ORG Fri Mar 19 14:16:47 2004 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7C72916A4CE for ; Fri, 19 Mar 2004 14:16:47 -0800 (PST) Received: from dan.emsphone.com (dan.emsphone.com [199.67.51.101]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3071343D2F for ; Fri, 19 Mar 2004 14:16:47 -0800 (PST) (envelope-from dan@dan.emsphone.com) Received: (from dan@localhost) by dan.emsphone.com (8.12.10/8.12.10) id i2JMGhtX037578; Fri, 19 Mar 2004 16:16:43 -0600 (CST) (envelope-from dan) Date: Fri, 19 Mar 2004 16:16:43 -0600 From: Dan Nelson To: Claus Assmann Message-ID: <20040319221643.GA90277@dan.emsphone.com> References: <20040317060617.GA23526@zardoc.esmtp.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20040317060617.GA23526@zardoc.esmtp.org> X-OS: FreeBSD 5.2-CURRENT X-message-flag: Outlook Error User-Agent: Mutt/1.5.6i cc: freebsd-fs@freebsd.org Subject: Re: softupdates and two different MTAs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Mar 2004 22:16:47 -0000 In the last episode (Mar 16), Claus Assmann said: > | program | FS | writes | reads | > |---------------+------------------+---------+-------| > | sm8.12.11 | UFS, softupdates | 236 | 0 | > | sm8.12.11 | UFS, no su | 8100 | 1 | > | sm9.0.0.12 | UFS, softupdates | 3500 | 4 | > | sm9.0.0.12 | UFS, no su | 6300 | 0 | > > Question: why does sm8 use so few writes? Can softupdates eliminate > or cluster most writes including fsync(2)? Why doesn't this work for > sm9? I'm not sure I understand any of those values :) 1000 mails requires at least 3000 fsyncs, doesn't it, one each for qf, df, and tf? And each fsync should write two disk blocks (inode and data). Without softupdates, that should be 12 IO/s per message: create+2*fsync+unlink, once for each of df+qf+tf. Total 12000 I/Os With softupdates, it should be 6: 2*fsync, once for each of df+qf+tf. Total 6000 I/Os So something is either not fsyncing, or there is clustering going on behind the scenes. The sm8 softupdates count is disturbingly low, even assuming good clustering. > I had the following idea why there are so many writes for sm9 > compared to sm8: sm9 has only one SMTP server process and since > fsync(2) is synchronous the process has to wait for it to return. In > sm8 there are many processes which can proceed independently and > softupdates could reorder the operations and perform a "group > commit", i.e., schedule multiple fsync(2) operations together and > then a group of processes can proceed. Theoretically possible. If the files' inodes happen to be on the same disk block, or if they are small messages and the frags happen to be in the same disk block, then an fsync() on one may automatically sync the other files for free. I don't know enough about ffs or softupdates to be able to tell from looking at the source though. You might be able to demonstrate that it's clustering by creating an fsync() wrapper that gets read and write counts before and after the real fsync() call. If you ever call fsync but the I/O counts don't change, then you know that something else synced the file already. -- Dan Nelson dnelson@allantgroup.com