From owner-freebsd-fs@FreeBSD.ORG Sun Oct 31 09:47:23 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7B3FB1065670; Sun, 31 Oct 2010 09:47:23 +0000 (UTC) (envelope-from ronald-freebsd8@klop.yi.org) Received: from smtp-out2.tiscali.nl (smtp-out2.tiscali.nl [195.241.79.177]) by mx1.freebsd.org (Postfix) with ESMTP id 085F78FC0A; Sun, 31 Oct 2010 09:47:22 +0000 (UTC) Received: from [212.123.145.58] (helo=sjakie.klop.ws) by smtp-out2.tiscali.nl with esmtp (Exim) (envelope-from ) id 1PCUVV-0005qu-PV; Sun, 31 Oct 2010 10:47:21 +0100 Received: from 212-123-145-58.ip.telfort.nl (localhost [127.0.0.1]) by sjakie.klop.ws (Postfix) with ESMTP id 2FDE3E08D; Sun, 31 Oct 2010 10:47:14 +0100 (CET) Content-Type: text/plain; charset=us-ascii; format=flowed; delsp=yes To: "'Andriy Gapon'" , "Alexander Zagrebin" References: <4CCAF0EB.4080001@icyb.net.ua> <20101029175105.GB18613@two.kliksys.ru> <4CCBD443.7010305@icyb.net.ua> <4CCBD46E.9030200@icyb.net.ua> <4CCBD661.3000204@freebsd.org> <4CCBEAF6.2030408@freebsd.org> <20101030101254.GB79691@two.kliksys.ru> <4CCBF45C.4080208@icyb.net.ua> <20101030112520.GD79691@two.kliksys.ru> <4CCC2F2A.7020809@icyb.net.ua> <20101030190120.GA80301@two.kliksys.ru> <4CCD0804.8010703@icyb.net.ua> Date: Sun, 31 Oct 2010 10:47:13 +0100 MIME-Version: 1.0 From: "Ronald Klop" Message-ID: In-Reply-To: User-Agent: Opera Mail/10.62 (FreeBSD) Content-Transfer-Encoding: quoted-printable Cc: freebsd-fs@freebsd.org, freebsd-stable@freebsd.org Subject: Re: 8.1-STABLE: zfs and sendfile: problem still exists X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 31 Oct 2010 09:47:23 -0000 On Sun, 31 Oct 2010 10:02:44 +0100, Alexander Zagrebin =20 wrote: >> >> I apologize for my haste, it should have been VM_ALLOC_WIRED. >> > >> > Ok, applied and tested under some load(~1200 active >> connections, outgoing >> > ~80MB/s). Patch work as expected and i has noted no side >> effects. Just one >> > question - should grow Active memory counter, if some pages >> is "hot"(during >> > multiple sendfile on one file)? >> >> Pages used by sendfile are marked as Inactive for faster >> reclamation on demand. > > I have a question. > When we transfer a file via sendfile, then current code allocates > a memory, marked inactive. For example, if the file has size 100 MB, > then 100 MB of memory will be allocated. > If we have to transfer this file again later, then this memory will use= d > as cache, and no disk io will be required. > The memory will be freed if file will be deleted or operating system > will need an additional memory. > I have correctly understood? > If it so, the i continue... > Such behaviour is good if we have files with relatively small size. > Suppose we have to transfer file with large size (for example, greater > than amount of physical memory). > While transfering, the inactive memory will grow, pressing the ARC. > When size of the ARC will fall to its minimum (vfs.zfs.arc_min), then > inactive memory will be reused. > So, when transfer is complete, we have: > 1. No free memory > 2. Size of the ARC has minimal size (it is bad) > 3. Inactive memory contains the _tail_ of the file only (it is bad too) > Now if we have to transfer this file again, then > 1. there is no (or few) file's data in ARC (ARC too small) > 2. The inactive memory doesn't contain a head part of the file > So the file's data will read from a disk again and again... > Also i've noticed that inactive memory frees relatively slowly, > so if there is a frequent access to large files, then system will run > at very unoptimal conditions. > It's imho... > Can you comment this? > Add more RAM?