From owner-freebsd-hackers@FreeBSD.ORG Sun Dec 26 17:47:27 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B5396106564A; Sun, 26 Dec 2010 17:47:27 +0000 (UTC) (envelope-from gleb.kurtsou@gmail.com) Received: from mail-ew0-f54.google.com (mail-ew0-f54.google.com [209.85.215.54]) by mx1.freebsd.org (Postfix) with ESMTP id 18BC98FC0A; Sun, 26 Dec 2010 17:47:26 +0000 (UTC) Received: by ewy24 with SMTP id 24so4142671ewy.13 for ; Sun, 26 Dec 2010 09:47:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:date:from:to:cc:subject :message-id:references:mime-version:content-type:content-disposition :content-transfer-encoding:in-reply-to:user-agent; bh=L3wR/TL/iVvWhrWIYGfnCA5cPHIiOpIqwhvW7aUoeMg=; b=HMtfzy3GUBpYQxWMmEOrpcpKtp47ybMrb2mNFNmTZxMGj5n5csG/1EAf87gR4cDvML KJ/lUf192l7jX2ZDv8+FHZIB7Wx7AqiRx2DO0nI+NLnpiWB7kdeTDPBC0lD5kgAb2J9T 1TONeTwxvqZn1oyQ/zRBmI8GOgIjS/YcjEYAY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:content-transfer-encoding :in-reply-to:user-agent; b=aM+8UMV+pRNIWoV2bnKHLdS7dxPsPJYAF6DquucanF/33rRN95t1F2iDCMDs/SAKc6 6UcHhB768yRAIOWIBOYirADkZ8Z7Z/Q7KmJ7kSX6lFhcs/dhjvS8zhHCM2WbX66+/P2r M4WkbbvlwUb5/6RgbbGAbbX/V9A5OpVLL1+kM= Received: by 10.213.27.136 with SMTP id i8mr9420743ebc.11.1293385645818; Sun, 26 Dec 2010 09:47:25 -0800 (PST) Received: from localhost ([212.98.186.134]) by mx.google.com with ESMTPS id b52sm8190561eei.7.2010.12.26.09.47.24 (version=TLSv1/SSLv3 cipher=RC4-MD5); Sun, 26 Dec 2010 09:47:25 -0800 (PST) Date: Sun, 26 Dec 2010 19:46:53 +0200 From: Gleb Kurtsou To: Ivan Voras Message-ID: <20101226174653.GA45598@tops> References: <20101223224619.GA21984@tops> <20101226132431.GA16490@tops> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-hackers@freebsd.org Subject: Re: [rfc] Replacing FNV and hash32 with Paul Hsieh's SuperFastHash X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Dec 2010 17:47:27 -0000 On (26/12/2010 15:20), Ivan Voras wrote: > On 26 December 2010 14:24, Gleb Kurtsou wrote: > > On (25/12/2010 20:29), Ivan Voras wrote: > >> On 23.12.2010 23:46, Gleb Kurtsou wrote: > >> > >> > For testing I've used dbench with 16 processes on 1 Gb swap back md > >> > device, UFS + SoftUpdates: > >> > Old hash (Mb/s): 599.94  600.096 599.536 > >> > SFH hash (Mb/s): 612.439 612.341 609.673 > >> > > >> > It's just ~1% improvement, but dbench is not a VFS metadata intensive > >> > benchmark. Subjectively it feels faster accessing maildir mailboxes > >> > with ~10.000 messages : ) > >> > >> Try blogbench if you need metadata-intensive operations, or even fsx. > > > blogbench should be good, but I've always had hard time interpreting its > > results. Besides results tend to very a lot, there is no way to set seed > > value like in fsx, so that I could run exactly the same test in different > > configurations. > > I think the exact sequence of blogbench operations depends on duration > of previous operations (it's multithreaded) so from that angle you are Why should it? Operation order in dbench or fsx doesn't depend on duration of previous operations. > right - you can't do a repeatable run except in the trivial cases. On > the other hand, it uses rand() without seeding it with > srand()/sranddev() so this part is actually very repeatable :) I've once tried to make its behaviour more predictable, I can't find the patch and can't recall any specifics, but there were architectural issues. You are right, setting seed and calling rand() should give stable results, that's what I was trying to achieve. The other way to work around such "limitation" is too run sufficiently large number of tests. Which requires patience :) > > fsx is a different beast, it reads/writes/truncates at random offsets - > > great tool for debugging mmap/truncate issues. Patch doesn't improve it > > in any way. > > It depends on what metadata operations you require - blogbench will > create, find and write files (if we ignore atime); fsx will create a > decent amount of traffic with file size and mtime changes. In your > case you'll probably need to run it on a memory file system or tmpfs > due to sensitivity to disk IO latencies (if your improvements is on > the order of few percent). I meant create/readdir/remove as metadata intensive operations -- blogbench is very good for it. fsx creates single file. Most people will only notice changes in vfs_cache.c and UFS' dirhash, that's 600 Mb/s vs 613 Mb/s improvement I've written about above. I'd appreciate if someone could benchmark if_lagg, it was using hash32 for binary data, which could result in poor hash table usage, which could possibly make most of the data go on single interface. But there would be hardly any performance improvement due to limited network bandwidth. Besides old hash32 is faster than new SFH.