From owner-freebsd-fs@FreeBSD.ORG Tue Aug 30 22:14:09 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 20945106564A for ; Tue, 30 Aug 2011 22:14:09 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from onlyone.friendlyhosting.spb.ru (onlyone.friendlyhosting.spb.ru [IPv6:2a01:4f8:131:60a2::2]) by mx1.freebsd.org (Postfix) with ESMTP id B21688FC12 for ; Tue, 30 Aug 2011 22:14:08 +0000 (UTC) Received: from lion.home.serebryakov.spb.ru (unknown [IPv6:2001:470:923f:1:6407:f3f9:7d93:d34c]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.friendlyhosting.spb.ru (Postfix) with ESMTPA id E209A4AC31; Wed, 31 Aug 2011 02:14:06 +0400 (MSD) Date: Wed, 31 Aug 2011 02:14:04 +0400 From: Lev Serebryakov Organization: FreeBSD Project X-Priority: 3 (Normal) Message-ID: <103666698.20110831021404@serebryakov.spb.ru> To: Kirk McKusick In-Reply-To: <201108302009.p7UK9CBQ085481@chez.mckusick.com> References: <317753422.20110830231815@serebryakov.spb.ru> <201108302009.p7UK9CBQ085481@chez.mckusick.com> MIME-Version: 1.0 Content-Type: text/plain; charset=windows-1251 Content-Transfer-Encoding: quoted-printable Cc: freebsd-fs@freebsd.org Subject: Re: Very inconsistent (read) speed on UFS2 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: lev@FreeBSD.org List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Aug 2011 22:14:09 -0000 Hello, Kirk. You wrote 31 =E0=E2=E3=F3=F1=F2=E0 2011 =E3., 0:09:12: > Now that you have defragmented your filesystem, we can factor that out > of the equation. What is left is the management of the memory used for Yep. > caching the files. I expect what is happening is that you are busily > reading along and run out of free memory in which to read. This triggers > a cleanup thread that churns through the memory pool to decide what > should be thrown out to make room. Your reading process is demanding > memory faster than the cleanup thread can produce it. The result is > that your read idles (e.g., appears to run slowly). It is random because > it depends on when you run out of memory. It is interesting. But this box have two real cores (yes, I know, now it is ``only two cores'' :)) and nothing to do but this test program (single-threaded). Of course, this program read data into same buffer and didn't allocate anything in process of benchmark. Yes, I know, that kernel prefer not to throw away data even if it is not needed right now, but here decision looks very simple :) And, one more detail: I use O_DIRECT flag in open(2). Other interesting observing: this program consume about 20% of one core. It is very strange for I/O bound process, isn't it? > The cleanup is complex because it has to deal with all of memory and > it wants to avoid a simple LRU which would cause the read of a large=20 > file to throw out a lot of things you would rather keep (like your=20 > window manager, browser, etc). Not sure what the best strategy is here. window manager and browser look different from one small (128Kb) buffer which is ovewritten again and again. As far as I understand, FreeBSD's VM (according to "Design and Implementation of...) should move such buffers (data read recently, not changed and not needed) into "Inact" state and it is easy task to re-use these buffers -- they don't belong to any active process, they don't need to be paged or swapped out, etc. I'll try this experiment with mmap() and touching every 4096-th byte of mapped memory instead of read(2). --=20 // Black Lion AKA Lev Serebryakov