From owner-freebsd-fs@FreeBSD.ORG Sun Oct 31 09:02:49 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DAAA4106566C; Sun, 31 Oct 2010 09:02:49 +0000 (UTC) (envelope-from alexz@visp.ru) Received: from mail.visp.ru (srv1.visp.ru [91.215.204.2]) by mx1.freebsd.org (Postfix) with ESMTP id 8BB238FC19; Sun, 31 Oct 2010 09:02:47 +0000 (UTC) Received: from 91-215-205-255.static.visp.ru ([91.215.205.255] helo=zagrebin) by mail.visp.ru with esmtp (Exim 4.72 (FreeBSD)) (envelope-from ) id 1PCToL-0000tt-3X; Sun, 31 Oct 2010 12:02:45 +0300 From: "Alexander Zagrebin" To: "'Andriy Gapon'" References: <4CCAF0EB.4080001@icyb.net.ua> <20101029175105.GB18613@two.kliksys.ru> <4CCBD443.7010305@icyb.net.ua><4CCBD46E.9030200@icyb.net.ua> <4CCBD661.3000204@freebsd.org><4CCBEAF6.2030408@freebsd.org> <20101030101254.GB79691@two.kliksys.ru> <4CCBF45C.4080208@icyb.net.ua> <20101030112520.GD79691@two.kliksys.ru> <4CCC2F2A.7020809@icyb.net.ua><20101030190120.GA80301@two.kliksys.ru> <4CCD0804.8010703@icyb.net.ua> Date: Sun, 31 Oct 2010 12:02:44 +0300 Keywords: freebsd-fs Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="koi8-r" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook 11 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.5512 In-Reply-To: <4CCD0804.8010703@icyb.net.ua> Thread-Index: Act4wku/wmVwJWj2TD2hR3M6uk44wAAEHgaA Cc: freebsd-fs@freebsd.org, freebsd-stable@freebsd.org Subject: RE: 8.1-STABLE: zfs and sendfile: problem still exists X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 31 Oct 2010 09:02:49 -0000 > >> I apologize for my haste, it should have been VM_ALLOC_WIRED. > > > > Ok, applied and tested under some load(~1200 active > connections, outgoing > > ~80MB/s). Patch work as expected and i has noted no side > effects. Just one > > question - should grow Active memory counter, if some pages > is "hot"(during > > multiple sendfile on one file)? > > Pages used by sendfile are marked as Inactive for faster > reclamation on demand. I have a question. When we transfer a file via sendfile, then current code allocates a memory, marked inactive. For example, if the file has size 100 MB, then 100 MB of memory will be allocated. If we have to transfer this file again later, then this memory will used as cache, and no disk io will be required. The memory will be freed if file will be deleted or operating system will need an additional memory. I have correctly understood? If it so, the i continue... Such behaviour is good if we have files with relatively small size. Suppose we have to transfer file with large size (for example, greater than amount of physical memory). While transfering, the inactive memory will grow, pressing the ARC. When size of the ARC will fall to its minimum (vfs.zfs.arc_min), then inactive memory will be reused. So, when transfer is complete, we have: 1. No free memory 2. Size of the ARC has minimal size (it is bad) 3. Inactive memory contains the _tail_ of the file only (it is bad too) Now if we have to transfer this file again, then 1. there is no (or few) file's data in ARC (ARC too small) 2. The inactive memory doesn't contain a head part of the file So the file's data will read from a disk again and again... Also i've noticed that inactive memory frees relatively slowly, so if there is a frequent access to large files, then system will run at very unoptimal conditions. It's imho... Can you comment this? -- Alexander Zagrebin