From owner-freebsd-current@freebsd.org Fri Nov 25 21:42:55 2016 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2CFA3C550CB for ; Fri, 25 Nov 2016 21:42:55 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-qt0-x229.google.com (mail-qt0-x229.google.com [IPv6:2607:f8b0:400d:c0d::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D8DC4E91 for ; Fri, 25 Nov 2016 21:42:54 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-qt0-x229.google.com with SMTP id w33so75362848qtc.3 for ; Fri, 25 Nov 2016 13:42:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=Iv4XOvndzqgPLsrvvUXPseGdY7ZBwntxYi9UkuJ7Z9o=; b=nVHEodGCj9n7fvplt8eVMwxUpxCk39A+e2olR+8lS70hHVGf6YhffpbKmI64jYswqY ujJNMrfDVKyq13MLsralPOt3+rRbpRtVL3lxl/7Gu7PsE01hoIgTg7Wp6M8WmFx30zHk jh7LswrCkitUk9J60rg0QWRUsIq1zjo0PLzSArfruha3MXd0ajtA5kuvoFaoELZvNHJa eUu9rerHrPlBaf/fE0GJC8ZPmwSuBVQVZFzgH974UuCAmv8NxGalBQYkADOc9CH0pRGI wRMhHhsXLDgWc1aKKzmXntzPMIpfkS3Mv44ep3e+kEX7bgD8XLBDM/QpKIm/t38Rf+VG uBmQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=Iv4XOvndzqgPLsrvvUXPseGdY7ZBwntxYi9UkuJ7Z9o=; b=ZTnKeUZ/h+3d71HHG6sp2Y+r9K0R4/1PR8B9gyUisV02gltdn6XR15ia3nL2UT8Yc/ ot5QQ//OXm30ybuiy1STzCCgd/B2ET1hntcPqSkphIZi2HdRYOi5e6Mo3AOQX0cqYm65 pRO/TqQb/3bPGSeTovrCyc5JoLVa4uc5oDk4TWOSK+Z3QSP30jI6PUn5dYHK8H7ydzlq FqC+EnmJ/CbzqI+I2w4SekliRsXmV3VBAvTcMc9d6gUcjtlGnBeiEq7C34pLhXEx2PEN A4Yg62v11OqLzuO1Dw29kuSMni7uI3/8v+JfhQXrjkgKD+2Iiak2SeGFlfFkvBNENDZG Rx6w== X-Gm-Message-State: AKaTC03pg8JoBaxKi6QhgS2K1OhH58HL785TUqobELeZy2ZNxx3u3zHxbpRR1rYfqObM/n0b/qQL1mw8pZxWdw== X-Received: by 10.200.48.28 with SMTP id f28mr9975891qte.247.1480110173948; Fri, 25 Nov 2016 13:42:53 -0800 (PST) MIME-Version: 1.0 Sender: asomers@gmail.com Received: by 10.12.166.129 with HTTP; Fri, 25 Nov 2016 13:42:53 -0800 (PST) In-Reply-To: <20161125135725.GD54029@kib.kiev.ua> References: <20161124090811.GO54029@kib.kiev.ua> <20161125084106.GX54029@kib.kiev.ua> <20161125135725.GD54029@kib.kiev.ua> From: Alan Somers Date: Fri, 25 Nov 2016 14:42:53 -0700 X-Google-Sender-Auth: G0BdaBcCmb9nLA5osDn-7eOb9sA Message-ID: Subject: Re: NFSv4 performance degradation with 12.0-CURRENT client To: Konstantin Belousov Cc: Rick Macklem , FreeBSD CURRENT Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Nov 2016 21:42:55 -0000 On Fri, Nov 25, 2016 at 6:57 AM, Konstantin Belousov wrote: > On Fri, Nov 25, 2016 at 12:54:07PM +0000, Rick Macklem wrote: >> Well, ideally theer would be a VOP_MMAPDONE() or something like that, which >> would tell the NFSv4 client that I/O is done on the vnode so it can close it. >> If there was some way for the NFSv4 VOP_CLOSE() to be able to tell if the file >> has been mmap'd, that would help since it could close the ones that are not >> mmap'd on the last descriptor close. >> (A counter wouldn't be as useful, since NFSv4 would have to keep checking it to >> see if it can do the close yet, but it might still be doable.) > > I thought that the issue was in tracking any opens and mmaps, but from this > reply it is not that clear. Do you need callback when all opens and mmaps > have ended, or only opens and mmaps for write ? If later, we already have > a suitable mechanism VOP_ADD_WRITECOUNT(). Mounting nullfs with the nocache option, ad kib suggested, fixed the problem. Also, applying kib's patch and then mounting nullfs with default options also fixed the problem. Here is the nfsstat output for "ls -al" when using kib's patch. Notice the client has far fewer opens: nfsstat -s -e -z Server Info: Getattr Setattr Lookup Readlink Read Write Create Remove 494 0 0 0 0 1 0 0 Rename Link Symlink Mkdir Rmdir Readdir RdirPlus Access 0 0 0 0 0 0 0 0 Mknod Fsstat Fsinfo PathConf Commit LookupP SetClId SetClIdCf 0 0 0 0 0 0 0 0 Open OpenAttr OpenDwnGr OpenCfrm DelePurge DeleRet GetFH Lock 0 0 0 0 0 0 0 0 LockT LockU Close Verify NVerify PutFH PutPubFH PutRootFH 0 0 0 0 0 494 0 0 Renew RestoreFH SaveFH Secinfo RelLckOwn V4Create 0 0 0 0 0 0 Server: Retfailed Faults Clients 0 0 0 OpenOwner Opens LockOwner Locks Delegs 0 0 0 0 0 Server Cache Stats: Inprog Idem Non-idem Misses CacheSize TCPPeak 0 0 0 495 17280 17280 nfsstat -c -e -z Client Info: Rpc Counts: Getattr Setattr Lookup Readlink Read Write Create Remove 14 0 0 0 0 0 0 0 Rename Link Symlink Mkdir Rmdir Readdir RdirPlus Access 0 0 0 0 0 0 0 0 Mknod Fsstat Fsinfo PathConf Commit SetClId SetClIdCf Lock 0 0 0 0 0 0 0 0 LockT LockU Open OpenCfr 0 0 0 0 OpenOwner Opens LockOwner Locks Delegs LocalOwn LocalOpen LocalLOwn 592 588 0 0 0 0 0 0 LocalLock 0 Rpc Info: TimedOut Invalid X Replies Retries Requests 0 0 0 0 494 Cache Info: Attr Hits Misses Lkup Hits Misses BioR Hits Misses BioW Hits Misses 1439 12 960 0 0 0 0 0 BioRLHits Misses BioD Hits Misses DirE Hits Misses 1 0 4 0 1 0 Thanks for the help, guys.