From owner-freebsd-hackers Fri Jul 21 11:23: 5 2000 Delivered-To: freebsd-hackers@freebsd.org Received: from hq.fsmlabs.com (hq.fsmlabs.com [209.155.42.197]) by hub.freebsd.org (Postfix) with ESMTP id 6B55F37BDDF for ; Fri, 21 Jul 2000 11:23:01 -0700 (PDT) (envelope-from cort@medea.fsmlabs.com) Received: from medea.fsmlabs.com (medea.fsmlabs.com [209.155.42.137]) by hq.fsmlabs.com (8.9.3/8.9.3) with ESMTP id MAA27598; Fri, 21 Jul 2000 12:22:39 -0600 Received: (from cort@localhost) by medea.fsmlabs.com (8.9.3/8.9.3) id MAA02384; Fri, 21 Jul 2000 12:14:57 -0600 Date: Fri, 21 Jul 2000 12:14:57 -0600 From: cort@fsmlabs.com To: Lars Eggert Cc: Alan Cox , hackers@freebsd.org Subject: Re: clearing pages in the idle loop Message-ID: <20000721121457.R26237@medea.fsmlabs.com> References: <20000719234124.H14543@cs.rice.edu> <39788E48.60F8A59F@isi.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0.1us In-Reply-To: <39788E48.60F8A59F@isi.edu>; from larse@ISI.EDU on Fri, Jul 21, 2000 at 10:54:16AM -0700 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG We started losing performance with the idle page clearing so I've disabled it and haven't done much with it in quite a while. The code has fallen into disrepair and some has been removed in the latest versions. I'd suggest looking at the early 2.3.x and 2.2.1[23] series of Linux kernels. That's where it was doing its best. We've also found that using dcbz, which zeros cache lines directly, actually improves performance. That is contradictory to one of the conclusions I drew in that paper. My conjecture was that the zero'd pages would not be used often since a process given a fresh page wouldn't read from it before a write. I still do think the idle page zero-ing can be useful, though. If you do some work with BSD on this please let me know. I'd like to follow along since I've been playing with BSD lately. } Do you still have those FreeBSD patches, Alan? I'd be interested in doing } some more experiments with that code. } } That makes sense. Other factors that may have an impact: } } * if you always have enough zeroed pages remaining over your } benchmark (> ~1/2 free pages), FreeBSD will never do the } idle-time zeroing How do you keep track of what parts of a page are used? In Linux I was just pulling pages off the free-list and zero-ing them. Do you have a bitmap of used regions within pages on BSD? } * it looks to me as if Cort's Linux code will always zero whole } pages, while the FreeBSD code is a little smarter and only zeroes } used regions of a page (less impact on caches?) } } * cache size differences between PPC and i386? Right, those won't be cached if the page is marked non-cacheable. } I'm looking at Cort's code (arch/ppc/kernel/idle.c), and while he turns off } the caching for pages he zeroes, I don't see him disabling the L1/2 caches } explicitly. Is this implicit with setting the non-cacheable flag on the } PPC? Also, idle-time zeroing is commented out in the version I'm looking at } (1.68, 1999/10/15), where problems found after the paper was published? I wanted a few bits for each page to describe its state: zero'd, non-zero'd, busy being zero'd. It would have taken some changes to the non-arch Linux code to do that and at the time we were moving to a more stable tree so I didn't continue with it. It sounds as though you have a framework for doing that sort of thing (and more) with BSD. Can you send me a pointer to the sources you're using? I'd like to look into it. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message