From owner-freebsd-hackers@FreeBSD.ORG Mon Dec 12 12:09:47 2005 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8396416A41F for ; Mon, 12 Dec 2005 12:09:47 +0000 (GMT) (envelope-from simon@comsys.ntu-kpi.kiev.ua) Received: from comsys.ntu-kpi.kiev.ua (comsys.ntu-kpi.kiev.ua [195.245.194.142]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2F79343D53 for ; Mon, 12 Dec 2005 12:09:16 +0000 (GMT) (envelope-from simon@comsys.ntu-kpi.kiev.ua) Received: from pm513-1.comsys.ntu-kpi.kiev.ua (pm513-1.comsys.ntu-kpi.kiev.ua [10.18.52.101]) (authenticated bits=0) by comsys.ntu-kpi.kiev.ua (8.12.10/8.12.10) with ESMTP id jBCCJeVO050226 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL); Mon, 12 Dec 2005 14:19:40 +0200 (EET) Received: by pm513-1.comsys.ntu-kpi.kiev.ua (Postfix, from userid 1001) id F3CE25C07C; Mon, 12 Dec 2005 14:09:02 +0200 (EET) Date: Mon, 12 Dec 2005 14:09:02 +0200 From: Andrey Simonenko To: Cedric Tabary Message-ID: <20051212120902.GA1143@pm513-1.comsys.ntu-kpi.kiev.ua> References: <20051212083930.GC91837@efrei.fr> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20051212083930.GC91837@efrei.fr> User-Agent: Mutt/1.5.11 X-Virus-Scanned: ClamAV 0.82/1207/Sat Dec 10 00:01:12 2005 on comsys.ntu-kpi.kiev.ua X-Virus-Status: Clean Cc: freebsd-hackers@freebsd.org Subject: Re: mmap() sendfile() X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2005 12:09:47 -0000 On Mon, Dec 12, 2005 at 09:39:30AM +0100, Cedric Tabary wrote: > > This is some sort of file cache, it works by mmap()ing some files and > keeping the mmap address in a hashtable. I suppose this is used to keep > the file in memory until munmap() is called. I guess this is used for accessing file's data directly (read automatic), without read(2) and write(2) system calls, according to source files if mmap(2) is not supported, then malloc(3) and read(2) are used for reading files (I checked source files very quickly) and write(2) for transferring data to socket. I do not understand why mmap() is called with MAP_PRIVATE and PROT_READ at the same time, because copy-objects are not supported on FreeBSD (probably on other systems too). > The patch is just removing the mmap() and keeping file descriptors open > for use by sendfile(). But I don't know if replacing the mmap() by > sendfile() has the same cache effect ?! The sendfile(2) system call is used for transferring some portion of a file to a stream socket. Since this is a system call and not library function it can implement data transfers from a file to a socket entirely in the kernel, without bringing data to the userspace as in case of read(2) or mmap(2). This technique sometimes is called "zerocopy". > Keeping the file descriptor open after using sendfile() will keep the > file in memory ?? I think no. Owners of files' pages are VM objects, even if a file (really vnode) does not have any reference and hold counter, its vnode will go to the free list, but its VM pages in corresponding VM object will be still in memory, until another pages which have higher priority (according to the logic of VM system) will cause removing of file's pages from the VM cache or until there are not enough free vnodes. Just test it: open file, write something to it, close it and { read it and close it } several times, reboot and read this file again and compare times. > If it is true, doing a sendfile() on some very big files (even if not > keeping the descriptor open after) will kill the cache ? VM system keeps pages in several queues (read classes), so reading big files will not remove frequently used pages (also wired pages cannot be removed from memory). Read corresponding paragraph in FAQ ("What do the various memory states..." and next one). > Please help me to understand why this patch ? and the difference between > sendfile() and mmap() at the memory or cache level.. sendfile(2) will transfer data from a file to a socket entirely in the kernel, without context switching, unlike read(2) which requires context switching and mmap(2) which will require context switching in case of page fault; as I understand VM cache will be used in the same way. As usually correct me if I'm wrong.