From owner-freebsd-current@FreeBSD.ORG Sun Jan 29 15:08:22 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CEAB2106566B; Sun, 29 Jan 2012 15:08:22 +0000 (UTC) (envelope-from tijl@coosemans.org) Received: from mailrelay011.isp.belgacom.be (mailrelay011.isp.belgacom.be [195.238.6.178]) by mx1.freebsd.org (Postfix) with ESMTP id 21F3E8FC0C; Sun, 29 Jan 2012 15:08:21 +0000 (UTC) X-Belgacom-Dynamic: yes X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Av4EAMpfJU9bsYWl/2dsb2JhbABErCWCMYEGgXIBAQVWIxALGC45HhCICLdTiD0BKAMECwEIAQUJBgMKgw8NKG0EDwIBBiODHASeRokm Received: from 165.133-177-91.adsl-dyn.isp.belgacom.be (HELO kalimero.tijl.coosemans.org) ([91.177.133.165]) by relay.skynet.be with ESMTP; 29 Jan 2012 16:08:19 +0100 Received: from kalimero.tijl.coosemans.org (kalimero.tijl.coosemans.org [127.0.0.1]) by kalimero.tijl.coosemans.org (8.14.5/8.14.5) with ESMTP id q0TF8J8n001396; Sun, 29 Jan 2012 16:08:19 +0100 (CET) (envelope-from tijl@coosemans.org) From: Tijl Coosemans To: freebsd-current@freebsd.org Date: Sun, 29 Jan 2012 16:08:10 +0100 User-Agent: KMail/1.13.7 (FreeBSD/10.0-CURRENT; KDE/4.7.3; i386; ; ) References: <201201191739.48327.tijl@coosemans.org> <201201201412.13269.jhb@freebsd.org> <201201251129.22368.jhb@freebsd.org> In-Reply-To: <201201251129.22368.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart4901043.djbVi398T5"; protocol="application/pgp-signature"; micalg=pgp-sha256 Content-Transfer-Encoding: 7bit Message-Id: <201201291608.16741.tijl@coosemans.org> Cc: Subject: Re: posix_fadvise noreuse disables file caching X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jan 2012 15:08:23 -0000 --nextPart4901043.djbVi398T5 Content-Type: Text/Plain; charset="iso-8859-15" Content-Transfer-Encoding: quoted-printable On Wednesday 25 January 2012 17:29:22 John Baldwin wrote: > On Friday, January 20, 2012 2:12:13 pm John Baldwin wrote: >> On Thursday, January 19, 2012 11:39:42 am Tijl Coosemans wrote: >>> I recently noticed that multimedia/vlc generates a lot of disk IO when >>> playing media files. For instance, when playing a 320kbps mp3 gstat >>> reports about 1250kBps (=3D10000kbps). That's quite a lot of overhead. >>>=20 >>> It turns out that vlc sets POSIX_FADV_NOREUSE on the entire file and >>> reads in chunks of 1028 bytes. FreeBSD implements NOREUSE as if >>> O_DIRECT was specified during open(2), i.e. it disables all caching. >>> That means every 1028 byte read turns into a 32KiB read (new default >>> block size in 9.0) which explains the above numbers. >>>=20 >>> I've copied the relevant vlc code below (modules/access/file.c:Open()). >>> It's interesting to see that on OSX it sets F_NOCACHE which disables >>> caching too, but combined with F_RDAHEAD there's still read-ahead >>> caching. >>>=20 >>> I don't think POSIX intended for NOREUSE to mean O_DIRECT. It should >>> still cache data (and even do read-ahead if F_RDAHEAD is specified), >>> and once data is fetched from the cache, it can be marked WONTNEED. >>=20 >> POSIX doesn't specify O_DIRECT, so it's not clear what it asks for. >>=20 >>> Is it possible to implement it this way, or if not to just ignore >>> the NOREUSE hint for now? >>=20 >> I think it would be good to improve NOREUSE, though I had sort of >> assumed that applications using NOREUSE would do their own buffering >> and read full blocks. We could perhaps reimplement NOREUSE by doing >> the equivalent of POSIX_FADV_DONTNEED after each read to free buffers >> and pages after the data is copied out to userland. I also have an >> XXX about whether or not NOREUSE should still allow read-ahead as it >> isn't very clear what the right thing to do there is. HP-UX (IIRC) >> has an fadvise() that lets you specify multiple policies, so you >> could specify both NOREUSE and SEQUENTIAL for a single region to >> get read-ahead but still release memory once the data is read once. > > So I've came up with this untested patch. It uses > VOP_ADVISE(FADV_DONTNEED) after read(2) calls to a NOREUSE region, and > leaves read-ahead caching enabled for NOREUSE. FADV_DONTNEED doesn't > do any good really for writes (it only flushes clean buffers), so I've > left write(2) operations as using IO_DIRECT still. Does this sound > reasonable? I've not yet tested this at all: The patch drastically improves vlc, but there's still a tiny overhead. Without NOREUSE the disk is read in chunks of 128KiB (F_RDAHEAD buffer size). With NOREUSE there's an extra transfer of 32KiB (block size). --nextPart4901043.djbVi398T5 Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iF4EABEIAAYFAk8lYOAACgkQfoCS2CCgtivk2AD/RqvjPc4BSLqq/BzerszDDolM cwQBJoh2eMsxfpiJroMA/2pRubGDhtqL3nG7zIYfgm/2fcH2LYFm7W9ZrUCLxTgD =VmXF -----END PGP SIGNATURE----- --nextPart4901043.djbVi398T5--