From owner-freebsd-hackers@FreeBSD.ORG Wed Jun 13 13:15:19 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (unknown [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 64EFC106564A for ; Wed, 13 Jun 2012 13:15:19 +0000 (UTC) (envelope-from freebsd@damnhippie.dyndns.org) Received: from qmta14.emeryville.ca.mail.comcast.net (qmta14.emeryville.ca.mail.comcast.net [76.96.27.212]) by mx1.freebsd.org (Postfix) with ESMTP id 41D948FC08 for ; Wed, 13 Jun 2012 13:15:19 +0000 (UTC) Received: from omta18.emeryville.ca.mail.comcast.net ([76.96.30.74]) by qmta14.emeryville.ca.mail.comcast.net with comcast id Mox61j0041bwxycAEpECkc; Wed, 13 Jun 2012 13:14:12 +0000 Received: from damnhippie.dyndns.org ([24.8.232.202]) by omta18.emeryville.ca.mail.comcast.net with comcast id MpEB1j00Y4NgCEG8epECkQ; Wed, 13 Jun 2012 13:14:13 +0000 Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id q5DDE97v033126; Wed, 13 Jun 2012 07:14:09 -0600 (MDT) (envelope-from freebsd@damnhippie.dyndns.org) From: Ian Lepore To: Konstantin Belousov In-Reply-To: <20120612204508.GP2337@deviant.kiev.zoral.com.ua> References: <1339259223.36051.328.camel@revolution.hippie.lan> <20120609165217.GO85127@deviant.kiev.zoral.com.ua> <1339512694.36051.362.camel@revolution.hippie.lan> <20120612204508.GP2337@deviant.kiev.zoral.com.ua> Content-Type: text/plain; charset="us-ascii" Date: Wed, 13 Jun 2012 07:14:09 -0600 Message-ID: <1339593249.73426.5.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: Wojciech Puchar , freebsd-hackers@freebsd.org Subject: Re: wired memory - again! X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Jun 2012 13:15:19 -0000 On Tue, 2012-06-12 at 23:45 +0300, Konstantin Belousov wrote: > On Tue, Jun 12, 2012 at 08:51:34AM -0600, Ian Lepore wrote: > > On Sat, 2012-06-09 at 22:45 +0200, Wojciech Puchar wrote: > > > > > > > > First, all memory allocated by UMA and consequently malloc(9) is > > > > wired. In other words, almost all memory used by kernel is accounted > > > > as wired. > > > > > > > yes i understand this. still i found no way how to find out what allocated > > > that much. > > > > > > > > > > Second, the buffer cache wires the pages which are inserted into VMIO > > > > buffers. So your observation is basically right, cached buffers means > > > > > > what are exactly "VMIO" buffers. i understand that page must be wired WHEN > > > doing I/O. > > > But i have too much wired memory even when doing no I/O at all. > > > > I agree, this is The Big Question for me. Why does the system keep > > wired writable mappings of the buffers in kva after the IO operations > > are completed? > Read about buffer cache, e.g. in the Design and Implementation of > the FreeBSD OS book. > > > > > If it did not do so, it would fix the instruction-cache-disabled bug > > that kills performance on VIVT cache architectures (arm and mips) and it > > would reduce the amount of wired memory (that apparently doesn't need to > > be wired, unless I've missed the implications of a previous reply in > > this thread). > > I have no idea what is the bug you are talking about. If my guess is > right, and it specifically references unability of some processors > to correctly handle several mappings of the same physical page into > different virtual addresses due to cache tagging using virtual address > instead of physical, then this is a hardware bug, not software. > This bug: http://lists.freebsd.org/pipermail/freebsd-arm/2012-January/003288.html The bug isn't the VIVT cache hardware, it's the fact that the way we handle the requirements of the hardware has the side effect of leaving the instruction cache bit disabled on executable pages because the kernel keeps writable mappings of the pages even after the IO is done. > AFAIR, at least HP PA and MIPS have different instantiation of this problem. > Our kernel uses multi-mapping quite often, and buffers is only one example. > > Also, why do you think that the pages entered into buffers shall not be > wired, it is completely beyond my understanding. What's beyond my understanding is why a page has to remain wired after the IO is complete. That question seems to me to be tangentially related to the above question of why the kernel needs to keep a writable mapping of the buffer after it's done writing into the page (either via DMA or via uiomove() depending on the direction of the IO). -- Ian