From owner-freebsd-bugs Mon Mar 3 12:10:15 2003 Delivered-To: freebsd-bugs@hub.freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6B05F37B405 for ; Mon, 3 Mar 2003 12:10:12 -0800 (PST) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 754B943FD7 for ; Mon, 3 Mar 2003 12:10:11 -0800 (PST) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.12.6/8.12.6) with ESMTP id h23KABNS006308 for ; Mon, 3 Mar 2003 12:10:11 -0800 (PST) (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.12.6/8.12.6/Submit) id h23KABgR006307; Mon, 3 Mar 2003 12:10:11 -0800 (PST) Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6069437B401; Mon, 3 Mar 2003 12:04:07 -0800 (PST) Received: from mail.eecs.harvard.edu (bowser.eecs.harvard.edu [140.247.60.24]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8D4DF43F75; Mon, 3 Mar 2003 12:04:06 -0800 (PST) (envelope-from ellard@eecs.harvard.edu) Received: from ganon.eecs.harvard.edu (ant.eecs.harvard.edu [140.247.62.111]) by mail.eecs.harvard.edu (Postfix) with ESMTP id 07AEB54C432; Mon, 3 Mar 2003 15:04:03 -0500 (EST) Received: by ganon.eecs.harvard.edu (Postfix, from userid 465) id 6846572CB; Mon, 3 Mar 2003 15:04:02 -0500 (EST) Message-Id: <20030303200402.6846572CB@ganon.eecs.harvard.edu> Date: Mon, 3 Mar 2003 15:04:02 -0500 (EST) From: Daniel Ellard Reply-To: Daniel Ellard To: FreeBSD-gnats-submit@FreeBSD.org Cc: Alfred Perlstein , Matt Dillon X-Send-Pr-Version: 3.113 Subject: kern/48894: Improvements to the NFS read-ahead heuristic code Sender: owner-freebsd-bugs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org >Number: 48894 >Category: kern >Synopsis: Improvements to the NFS read-ahead heuristic code >Confidential: no >Severity: non-critical >Priority: low >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: change-request >Submitter-Id: current-users >Arrival-Date: Mon Mar 03 12:10:10 PST 2003 >Closed-Date: >Last-Modified: >Originator: Daniel Ellard >Release: FreeBSD 4.6.2-RELEASE-p9 i386 >Organization: Harvard University >Environment: System: FreeBSD ant.eecs.harvard.edu 4.6.2-RELEASE-p9 FreeBSD 4.6.2-RELEASE-p9 #12: Mon Feb 24 10:51:51 EST 2003 ellard@ant.eecs.harvard.edu:/usr/obj/usr/freebsd_4.6/src/sys/ANT i386 >Description: There are several potential improvements to the NFS read-ahead heuristics. First, the nfsheur table size is sized too small for large servers. Second, the sequentiality hueristic can be modified to recognize and handle read patterns that have a large sequential component (such as stride reads) that are missed by the current sequentiality metric. Third, the sequentiality metric is fragile in the face of jitter in the request stream (which happens when client nfsiods are stressed). >How-To-Repeat: Beat on your NFS server >Fix: The first problem (nfsheur too small) is easy to fix, and I recommend this be done for all systems: 1. In sys/nfs/nfs_serv.c: change NUM_HEURISTIC from 64 to 1021. The exact value isn't important, but it should be at least a few hundred, and prime. 2. Change all the calculations of the nfsheur hash bucket location from (something) & (NUM_HEURISTIC - 1); to (something) % NUM_HEURISTIC; This is necessary because if NUM_HEURISTIC isn't a power of two, then we can't use bit-fiddling to do the %. 2. In function nfsrv_read, file sys/nfs/nfs_serv.c: change the number of probes from 4 to something larger: /* * Calculate seqcount for heuristic */ { int try = 4; In my experiments, 8 works well, but YMMV. The downside of this fix is that nfsheur uses more memory (approx 20K) and if you have a slow CPU then doing the % instead of & will hurt. But if you have a busy NFS server, the reduction in hash table collisions will be worth the extra computation. Fixing the second and third problems involves much more extensive changes to the code. The ideas are described in the latter half of a paper I am working on for FreeNIX (URL below). I have implemented these changes in a new file, using sysctl to control the parameters and whether or not the new code is used at all. A snapshot of my code and the paper is available at: http://www.eecs.harvard.edu/~ellard/NFS/ ellard-freenix03.pdf - paper snapshot, with description of the new algorithms. nfs_serv.c - modified nfs_serv.c nfs_serv.c.diff - listing of the diffs from the original nfs_serv.c nfs_tweak.c - new source code nfs_tweak.h - new header file Please contact me if you need more information, or have suggestions for how I can improve my code. Thanks, -Dan >Release-Note: >Audit-Trail: >Unformatted: To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-bugs" in the body of the message