From owner-freebsd-fs@FreeBSD.ORG Fri Mar 15 14:25:08 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 4DAD8A52 for ; Fri, 15 Mar 2013 14:25:08 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id DE5373E4 for ; Fri, 15 Mar 2013 14:25:07 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqAEADYuQ1GDaFvO/2dsb2JhbABDiDG6AIJlgXt0gioBAQUjVhsOCgICDRkCWQaIJ7A1knOBI40+NAeCLYETA5ZbiWyHFoMmIIFs X-IronPort-AV: E=Sophos;i="4.84,850,1355115600"; d="scan'208";a="19169934" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-annu.net.uoguelph.ca with ESMTP; 15 Mar 2013 10:25:00 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id B45EBB4034; Fri, 15 Mar 2013 10:25:00 -0400 (EDT) Date: Fri, 15 Mar 2013 10:25:00 -0400 (EDT) From: Rick Macklem To: Konstantin Belousov Message-ID: <2081421885.3937873.1363357500724.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <20130315135950.GU3794@kib.kiev.ua> Subject: Re: should vn_fullpath1() ever return a path with "." in it? MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.201] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - FF3.0 (Win)/6.0.10_GA_2692) Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Mar 2013 14:25:08 -0000 Kostik Belousov wrote: > On Fri, Mar 15, 2013 at 05:21:18PM +0400, Ilia Noskov wrote: > > On 03/14/2013 02:10 PM, Ilia Noskov wrote: > > > On 03/14/2013 01:08 PM, Konstantin Belousov wrote: > > >> On Thu, Mar 14, 2013 at 11:29:11AM +0400, Noskov Ilia wrote: > > >>> Strange behavior on nfs-client after apply this patch: > > >>> > > >>> sysctl debug.disablecwd=0 > > >>> sysctl debug.disablefullpath=0 > > >>> > > >>> # mount -v -t nfs > > >>> 192.168.168.1:/pool on /home (nfs, noatime, nfsv4acls, fsid > > >>> 02ff003a3a000000) You don't mention how your server is configured. Does your V4: line in /etc/exports have "/" as the root or "/pool". See my comment at the bottom for why this might matter. Also, if you have a recent version of nfsstat, you can use "nfsstat -m" to dump out exactly what options the mount is actually using. (Gives a lot more info than the above.) > > >>> # ls /home | wc -l > > >>> 4946 > > >>> # cd /home/user6308/.ro Is user6308 a separate file system than /home on the server? (If so, I would expect the userland getcwd() to stat all the entries in /home to get the st_dev and st_ino fields of them all.) > > >>> # time pwd > > >>> /home/user6308/.ro > > >>> 0.008u 0.269s 0:08.47 3.0% 4+157k 0+0io 0pf+0w > > >>> # ktrace -t+ -i pwd > > >>> > > >>> > > >>> ktrace.out is big (1MB). Attach or not? > > >>> > > >>> > > >>> > > >>> A small piece of trace: > > >>> 19527 pwd CALL > > >>> mmap(0,0x400000,0x3,0x1002,0xffffffff,0) > > >>> > > >>> 19527 pwd RET mmap 34376515584/0x801000000 > > >>> 19527 pwd CALL __getcwd(0x801006400,0x400) > > >>> 19527 pwd NAMI ".." > > >>> 19527 pwd NAMI ".." > > >>> 19527 pwd RET __getcwd -1 errno 2 No such file or directory > > >>> 19527 pwd CALL stat(0x800947a14,0x7fffffffd940) > > >>> 19527 pwd NAMI "/" > > >>> 19527 pwd STRU struct stat {dev=98, ino=2, mode=drwxr-xr-x , > > >>> nlink=19, uid=0, gid=0, rdev=2120, atime=1363244893, > > >>> stime=1362653279, > > >>> ctime=1362653279, birthtime=1200836451, size=1024, > > >>> blksize=16384, > > >>> blocks=4, flags=0x0 } > > >>> 19527 pwd RET stat 0 > > >>> 19527 pwd CALL lstat(0x80094779c,0x7fffffffd940) > > >>> 19527 pwd NAMI "." > > >>> 19527 pwd STRU struct stat {dev=1230702064, ino=145, > > >>> mode=drwxr-xr-x , nlink=2, uid=0, gid=0, rdev=4294967295, > > >>> atime=1363244672.246785874, stime=1363244792.864201338, > > >>> ctime=1363244792.864201338, birthtime=-1, size=3, blksize=4096, > > >>> blocks=3, flags=0x0 } > > >>> 19527 pwd RET lstat 0 > > >>> 19527 pwd CALL openat(0xffffff9c,0x80094779b,0x100000,0x2) > > >>> 19527 pwd NAMI ".." > > >>> 19527 pwd RET openat 3 > > >>> 19527 pwd CALL fstat(0x3,0x7fffffffd880) > > >>> 19527 pwd STRU struct stat {dev=1230702064, ino=4, > > >>> mode=drwxr-xr-x , nlink=9, uid=0, gid=0, rdev=4294967295, > > >>> atime=1363244665.232140704, stime=1363010116.496298252, > > >>> ctime=1363010116.496298252, birthtime=-1, size=14, blksize=4096, > > >>> blocks=3, flags=0x0 } > > >>> 19527 pwd RET fstat 0 > > >>> 19527 pwd CALL fcntl(0x3,F_SETFD,FD_CLOEXEC) > > >>> 19527 pwd RET fcntl 0 > > >>> 19527 pwd CALL fstatfs(0x3,0x7fffffffd660) > > >>> 19527 pwd RET fstatfs 0 > > >>> 19527 pwd CALL fstat(0x3,0x7fffffffd940) > > >>> 19527 pwd STRU struct stat {dev=1230702064, ino=4, > > >>> mode=drwxr-xr-x , nlink=9, uid=0, gid=0, rdev=4294967295, > > >>> atime=1363244665.232140704, stime=1363010116.496298252, > > >>> ctime=1363010116.496298252, birthtime=-1, size=14, blksize=4096, > > >>> blocks=3, flags=0x0 } > > >>> 19527 pwd RET fstat 0 > > >>> 19527 pwd CALL > > >>> getdirentries(0x3,0x801018000,0x1000,0x8010160a8) > > >>> 19527 pwd RET getdirentries 4096/0x1000 > > >>> 19527 pwd CALL fstat(0x3,0x7fffffffd940) > > >>> 19527 pwd STRU struct stat {dev=1230702064, ino=4, > > >>> mode=drwxr-xr-x , nlink=9, uid=0, gid=0, rdev=4294967295, > > >>> atime=1363244665.232140704, stime=1363010116.496298252, > > >>> ctime=1363010116.496298252, birthtime=-1, size=14, blksize=4096, > > >>> blocks=3, flags=0x0 } > > >>> 19527 pwd RET fstat 0 > > >>> 19527 pwd CALL openat(0x3,0x80094779b,0x100000,0) > > >>> 19527 pwd NAMI ".." > > >>> 19527 pwd RET openat 4 > > >>> [..............................] > > >>> 19527 pwd CALL madvise(0x801016000,0x1000,MADV_FREE) > > >>> 19527 pwd RET madvise 0 > > >>> 19527 pwd CALL madvise(0x801018000,0x2000,MADV_FREE) > > >>> 19527 pwd RET madvise 0 > > >>> 19527 pwd CALL close(0x3) > > >>> 19527 pwd RET close 0 > > >>> 19527 pwd CALL fstat(0x4,0x7fffffffd880) > > >>> 19527 pwd STRU struct stat {dev=973143810, ino=4, > > >>> mode=drwxr-xr-x , nlink=4948, uid=0, gid=0, rdev=4294967295, > > >>> atime=1363244767.460164771, stime=1363172100.380266923, > > >>> ctime=1363172100.380266923, birthtime=-1, size=4948, > > >>> blksize=4096, > > >>> blocks=713, flags=0x0 } > > >>> 19527 pwd RET fstat 0 > > >>> 19527 pwd CALL fcntl(0x4,F_SETFD,FD_CLOEXEC) > > >>> 19527 pwd RET fcntl 0 > > >>> 19527 pwd CALL fstatfs(0x4,0x7fffffffd660) > > >>> 19527 pwd RET fstatfs 0 > > >>> 19527 pwd CALL fstat(0x4,0x7fffffffd940) > > >>> 19527 pwd STRU struct stat {dev=973143810, ino=4, > > >>> mode=drwxr-xr-x , nlink=4948, uid=0, gid=0, rdev=4294967295, > > >>> atime=1363244767.460164771, stime=1363172100.380266923, > > >>> ctime=1363172100.380266923, birthtime=-1, size=4948, > > >>> blksize=4096, > > >>> blocks=713, flags=0x0 } > > >>> 19527 pwd RET fstat 0 > > >>> 19527 pwd CALL > > >>> getdirentries(0x4,0x801018000,0x1000,0x8010160a8) > > >>> 19527 pwd RET getdirentries 4096/0x1000 > > >>> 19527 pwd CALL fstatat(0x4,0x801018030,0x7fffffffd940,0x200) > > >>> 19527 pwd NAMI "user6158" > > >>> 19527 pwd STRU struct stat {dev=1774902232, ino=4, > > >>> mode=drwxr-xr-x , nlink=9, uid=0, gid=0, rdev=4294967295, > > >>> atime=1363009687.040357529, stime=1363010116.496298252, > > >>> ctime=1363010116.496298252, birthtime=-1, size=14, blksize=4096, > > >>> blocks=3, flags=0x0 } > > >>> 19527 pwd RET fstatat 0 > > >>> 19527 pwd CALL fstatat(0x4,0x80101804c,0x7fffffffd940,0x200) > > >>> 19527 pwd NAMI "user2289" > > >>> 19527 pwd STRU struct stat {dev=1988229825, ino=4, > > >>> mode=drwxr-xr-x , nlink=9, uid=0, gid=0, rdev=4294967295, > > >>> atime=1363009687.040357529, stime=1363010116.496298252, > > >>> ctime=1363010116.496298252, birthtime=-1, size=14, blksize=4096, > > >>> blocks=3, flags=0x0 } > > >>> 19527 pwd RET fstatat 0 > > >>> 19527 pwd CALL fstatat(0x4,0x801018068,0x7fffffffd940,0x200) > > >>> 19527 pwd NAMI "user4761" > > >>> 19527 pwd STRU struct stat {dev=2438657130, ino=4, > > >>> mode=drwxr-xr-x , nlink=9, uid=0, gid=0, rdev=4294967295, > > >>> atime=1363009687.040357529, stime=1363010116.496298252, > > >>> ctime=1363010116.496298252, birthtime=-1, size=14, blksize=4096, > > >>> blocks=3, flags=0x0 } > > >>> 19527 pwd RET fstatat 0 > > >>> 19527 pwd CALL fstatat(0x4,0x801018084,0x7fffffffd940,0x200) > > >>> 19527 pwd NAMI "user6055" > > >>> [.........................................] > > >>> > > >>> and next get stat of all directories in /home > > >> > > >> Slightly different version of the patch was committed as r247560. > > >> > > >> The situation could only happen if the parent directory contains > > >> the "." > > >> entry with inode number equal to the inode number of the > > >> subdirectory. > > >> Can you confirm that this is your case ? > > >> > > > > > > Yes, it is. > > > I'll try again on the latest snapshot. Thanks! > > > > > > > Yes. > > On latest r248313 similar situation - if path contains "." then > > nfsclient get stat of all directories in /home. > > What path ? > > Did you read the description of the situation when r248313 returns > ENOENT to delegate the resolving to usermode ? Can you confirm that > this is your situation ? I think the patch is doing the correct thing. When __getcwd() returns ENOENT, the userland algorithm in getcwd() must stat() all entries in the directory to figure out if it is a mount point using the st_ino and st_dev fields. (You can look at it in the libc sources, if you'd like. Just look for getcwd.c.) I think this is what Kostik is referring to. If I understand your issue, it is that this takes a long time, since /home is large. There are a couple of things that *might* reduce the time this takes. 1 - If you are NFSv4 mounting "/pool", you could specify /pool as the root in you V4: /etc/exports line. Something like: V4: /pool ... Then you do the mount with 192.168.168.1:/. This will make the "/pool" a root point and might avoid the problem, but I am not sure. 2 - Adding rdirplus to the mount options will make it get attributes for entries in a directory when it does a readdir and cache them. This might speed things up. rick