From owner-freebsd-arch@freebsd.org Mon Nov 7 03:11:08 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9862DC32243 for ; Mon, 7 Nov 2016 03:11:08 +0000 (UTC) (envelope-from alan.l.cox@gmail.com) Received: from mail-it0-x232.google.com (mail-it0-x232.google.com [IPv6:2607:f8b0:4001:c0b::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5E439E3A; Mon, 7 Nov 2016 03:11:08 +0000 (UTC) (envelope-from alan.l.cox@gmail.com) Received: by mail-it0-x232.google.com with SMTP id e187so74606943itc.0; Sun, 06 Nov 2016 19:11:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:in-reply-to:references:from:date:message-id :subject:to:cc; bh=HrAur+984COJR5OIfYdieIozhTYP48GH5E+ZmEdIUVA=; b=m5YTq3gcdNRcjiKafgNULxiCGUDl11vKBpPCO6BhU4SKs0uuQKLyDsJyZYwWKXQYLI fD23GvzICqaqlSgFikjU2Apf9fRZG5qejBpsqfqJyAWFd9KaM/TC7UVEst+RAJ8xS/Ng rDkNNmurYjQh8TQdqvxYg1k9+aKjIE9CBO32pXbww/7p/86YWtcmV8axWhVODi/9Rn8L BzvYddvtkjhvHUgPEzE9vadm0h39XMEXA+uTZLNgrcIYKSClT+zFZwSxt/Ear4pXCBYJ m+WyVOJ3qMWdobavlNyzsVCbyBSQpelQkkVQKL1YG1O+gmKTIHJu1htGxH4f3dc3FG2D Q2QQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:reply-to:in-reply-to:references :from:date:message-id:subject:to:cc; bh=HrAur+984COJR5OIfYdieIozhTYP48GH5E+ZmEdIUVA=; b=epkU7O24qzjSdhrt5w57p4aVEwKHbXYKfpTTxmysB0unttF42wJ0a9L1dRIy2ruKCF a1GDXpsbnk9ct8W8NZixbEbITFChaczuTJ/zmAXeRsFPkJjq+0eHyX6E65rfg/MvAtxJ kzl+K1RnthZN5tG56kQMjMhlYaJeJAKP2YKBDSslSL9TZSXrzZtm/itxwPnhKQO7flb6 h9nArqT/0IioURuBcBZwEmT+ABVM1Tf9AUkGWOo+6VHFe5vnoxwINLrjGaRtB8O4oZoN I5vlQXd9B+Fn0YExCYJmxoDm4y3SgtKfHtwUkRV2crFQmxxtE4SSuv8SDhbvK2g2+Xl4 Ez1g== X-Gm-Message-State: ABUngvcqM650oSdeNd7Clh0CHY+51tvcJ4BDhWd8nzhBcj3UXFWvGToquIFRHV6LRw4GAmxyscdlM9r6xRvKSw== X-Received: by 10.107.157.201 with SMTP id g192mr6535414ioe.70.1478488267591; Sun, 06 Nov 2016 19:11:07 -0800 (PST) MIME-Version: 1.0 Received: by 10.79.126.154 with HTTP; Sun, 6 Nov 2016 19:11:06 -0800 (PST) Reply-To: alc@freebsd.org In-Reply-To: <20161106112326.47238905@ernst.home> References: <20161103182916.GA31178@wkstn-mjohnston.west.isilon.com> <20161105103128.78197d36@ernst.home> <20161105174148.GA75901@raichu> <20161106091230.4e365b55@ernst.home> <20161106112326.47238905@ernst.home> From: Alan Cox Date: Sun, 6 Nov 2016 21:11:06 -0600 Message-ID: Subject: Re: PQ_LAUNDRY To: gljennjohn@gmail.com Cc: Mark Johnston , FreeBSD Arch Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Nov 2016 03:11:08 -0000 On Sun, Nov 6, 2016 at 4:23 AM, Gary Jennejohn wrote: > On Sun, 6 Nov 2016 09:12:30 +0100 > Gary Jennejohn wrote: > > > On Sat, 5 Nov 2016 10:41:48 -0700 > > Mark Johnston wrote: > > > > > On Sat, Nov 05, 2016 at 10:31:28AM +0100, Gary Jennejohn wrote: > > > > On Thu, 3 Nov 2016 11:29:16 -0700 > > > > Mark Johnston wrote: > > > > > Some more details and the diff for PQ_LAUNDRY can be viewed here: > > > > > https://reviews.freebsd.org/D8302 > > > > > > > > > > We would like to commit it next week. Any additional comments, > review, > > > > > or testing would be welcome. > > > > > > > > > > > > > In my use case, which is moving multi-gigabyte video files from > > > > one file system to another, this seems to swap more than the > > > > previous code did. Moving such large files with the previous > > > > code seemed to recycle Inact more quickly and IIRC only a few 10s > > > > of MB were swapped out. In my test this morning 125MB were > > > > swapped out and Inact was not recycled as quickly. The overall > > > > size of the files moved was about the same in the two tests. > > > > > > Are you computing the amount swapped out as the amount of memory > swapped > > > out minus the amount of swapins? Or is 125MB the amount of swap used > > > after the test? Output from "sysctl vm.stats" taken before and after > any > > > test on both HEAD on PQ_LAUNDRY would be most useful. > > > > > > > 125MB was the swap value showed by top after the files had all been > > mv'd. But fairly soon after completion a few MB were swapped back in. > > > > OK, on a level playing field there's no difference between the old and > the new code. In fact, according to top the old code swapped out 272K > and the new code swapped out 220K. An insignificant difference. > > The test scenario was as follows: > 1) boot the box > 2) start X > 3) mount the source directory > 4) start a bash script which copied the same set of files in a for-loop > 5) start top and observe what happens > > Since all the files were either 4.3GB or 2GB cp didn't use mmap, but > rather did read/write in a loop (if the comment in utils.c is still valid). > > My test yesterday did a `mv *`, but since mv used fastcopy(), which > also does read/write in a loop, the pressure on the vm should have > been very similar to cp. > > The major difference between today and yesterday was that I'd been > running firefox and claws-mail for hours when I started the mv, so > there was something to swap out. > > Since I'm not too eager to noodle around for hours before starting > a test, let's just say that the new code appears to be no worse, or > perhaps even better, than the old code. > > The behavior that you describe is most likely a consequence of r254304 (and r254544). You can test this hypothesis by setting the sysctl vm.pageout_update_period to zero.