Date: Fri, 15 Mar 2013 10:25:00 -0400 (EDT) From: Rick Macklem <rmacklem@uoguelph.ca> To: Konstantin Belousov <kostikbel@gmail.com> Cc: freebsd-fs@freebsd.org Subject: Re: should vn_fullpath1() ever return a path with "." in it? Message-ID: <2081421885.3937873.1363357500724.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <20130315135950.GU3794@kib.kiev.ua>
next in thread | previous in thread | raw e-mail | index | archive | help
Kostik Belousov wrote:
> On Fri, Mar 15, 2013 at 05:21:18PM +0400, Ilia Noskov wrote:
> > On 03/14/2013 02:10 PM, Ilia Noskov wrote:
> > > On 03/14/2013 01:08 PM, Konstantin Belousov wrote:
> > >> On Thu, Mar 14, 2013 at 11:29:11AM +0400, Noskov Ilia wrote:
> > >>> Strange behavior on nfs-client after apply this patch:
> > >>>
> > >>> sysctl debug.disablecwd=0
> > >>> sysctl debug.disablefullpath=0
> > >>>
> > >>> # mount -v -t nfs
> > >>> 192.168.168.1:/pool on /home (nfs, noatime, nfsv4acls, fsid
> > >>> 02ff003a3a000000)
You don't mention how your server is configured. Does your V4: line
in /etc/exports have "/" as the root or "/pool". See my comment at
the bottom for why this might matter.
Also, if you have a recent version of nfsstat, you can use "nfsstat -m"
to dump out exactly what options the mount is actually using. (Gives
a lot more info than the above.)
> > >>> # ls /home | wc -l
> > >>> 4946
> > >>> # cd /home/user6308/.ro
Is user6308 a separate file system than /home on the server?
(If so, I would expect the userland getcwd() to stat all the entries in
/home to get the st_dev and st_ino fields of them all.)
> > >>> # time pwd
> > >>> /home/user6308/.ro
> > >>> 0.008u 0.269s 0:08.47 3.0% 4+157k 0+0io 0pf+0w
> > >>> # ktrace -t+ -i pwd
> > >>>
> > >>>
> > >>> ktrace.out is big (1MB). Attach or not?
> > >>>
> > >>>
> > >>>
> > >>> A small piece of trace:
> > >>> 19527 pwd CALL
> > >>> mmap(0,0x400000,0x3<PROT_READ|PROT_WRITE>,0x1002<MAP_PRIVATE|MAP_ANON>,0xffffffff,0)
> > >>>
> > >>> 19527 pwd RET mmap 34376515584/0x801000000
> > >>> 19527 pwd CALL __getcwd(0x801006400,0x400)
> > >>> 19527 pwd NAMI ".."
> > >>> 19527 pwd NAMI ".."
> > >>> 19527 pwd RET __getcwd -1 errno 2 No such file or directory
> > >>> 19527 pwd CALL stat(0x800947a14,0x7fffffffd940)
> > >>> 19527 pwd NAMI "/"
> > >>> 19527 pwd STRU struct stat {dev=98, ino=2, mode=drwxr-xr-x ,
> > >>> nlink=19, uid=0, gid=0, rdev=2120, atime=1363244893,
> > >>> stime=1362653279,
> > >>> ctime=1362653279, birthtime=1200836451, size=1024,
> > >>> blksize=16384,
> > >>> blocks=4, flags=0x0 }
> > >>> 19527 pwd RET stat 0
> > >>> 19527 pwd CALL lstat(0x80094779c,0x7fffffffd940)
> > >>> 19527 pwd NAMI "."
> > >>> 19527 pwd STRU struct stat {dev=1230702064, ino=145,
> > >>> mode=drwxr-xr-x , nlink=2, uid=0, gid=0, rdev=4294967295,
> > >>> atime=1363244672.246785874, stime=1363244792.864201338,
> > >>> ctime=1363244792.864201338, birthtime=-1, size=3, blksize=4096,
> > >>> blocks=3, flags=0x0 }
> > >>> 19527 pwd RET lstat 0
> > >>> 19527 pwd CALL openat(0xffffff9c,0x80094779b,0x100000,0x2)
> > >>> 19527 pwd NAMI ".."
> > >>> 19527 pwd RET openat 3
> > >>> 19527 pwd CALL fstat(0x3,0x7fffffffd880)
> > >>> 19527 pwd STRU struct stat {dev=1230702064, ino=4,
> > >>> mode=drwxr-xr-x , nlink=9, uid=0, gid=0, rdev=4294967295,
> > >>> atime=1363244665.232140704, stime=1363010116.496298252,
> > >>> ctime=1363010116.496298252, birthtime=-1, size=14, blksize=4096,
> > >>> blocks=3, flags=0x0 }
> > >>> 19527 pwd RET fstat 0
> > >>> 19527 pwd CALL fcntl(0x3,F_SETFD,FD_CLOEXEC)
> > >>> 19527 pwd RET fcntl 0
> > >>> 19527 pwd CALL fstatfs(0x3,0x7fffffffd660)
> > >>> 19527 pwd RET fstatfs 0
> > >>> 19527 pwd CALL fstat(0x3,0x7fffffffd940)
> > >>> 19527 pwd STRU struct stat {dev=1230702064, ino=4,
> > >>> mode=drwxr-xr-x , nlink=9, uid=0, gid=0, rdev=4294967295,
> > >>> atime=1363244665.232140704, stime=1363010116.496298252,
> > >>> ctime=1363010116.496298252, birthtime=-1, size=14, blksize=4096,
> > >>> blocks=3, flags=0x0 }
> > >>> 19527 pwd RET fstat 0
> > >>> 19527 pwd CALL
> > >>> getdirentries(0x3,0x801018000,0x1000,0x8010160a8)
> > >>> 19527 pwd RET getdirentries 4096/0x1000
> > >>> 19527 pwd CALL fstat(0x3,0x7fffffffd940)
> > >>> 19527 pwd STRU struct stat {dev=1230702064, ino=4,
> > >>> mode=drwxr-xr-x , nlink=9, uid=0, gid=0, rdev=4294967295,
> > >>> atime=1363244665.232140704, stime=1363010116.496298252,
> > >>> ctime=1363010116.496298252, birthtime=-1, size=14, blksize=4096,
> > >>> blocks=3, flags=0x0 }
> > >>> 19527 pwd RET fstat 0
> > >>> 19527 pwd CALL openat(0x3,0x80094779b,0x100000,0)
> > >>> 19527 pwd NAMI ".."
> > >>> 19527 pwd RET openat 4
> > >>> [..............................]
> > >>> 19527 pwd CALL madvise(0x801016000,0x1000,MADV_FREE)
> > >>> 19527 pwd RET madvise 0
> > >>> 19527 pwd CALL madvise(0x801018000,0x2000,MADV_FREE)
> > >>> 19527 pwd RET madvise 0
> > >>> 19527 pwd CALL close(0x3)
> > >>> 19527 pwd RET close 0
> > >>> 19527 pwd CALL fstat(0x4,0x7fffffffd880)
> > >>> 19527 pwd STRU struct stat {dev=973143810, ino=4,
> > >>> mode=drwxr-xr-x , nlink=4948, uid=0, gid=0, rdev=4294967295,
> > >>> atime=1363244767.460164771, stime=1363172100.380266923,
> > >>> ctime=1363172100.380266923, birthtime=-1, size=4948,
> > >>> blksize=4096,
> > >>> blocks=713, flags=0x0 }
> > >>> 19527 pwd RET fstat 0
> > >>> 19527 pwd CALL fcntl(0x4,F_SETFD,FD_CLOEXEC)
> > >>> 19527 pwd RET fcntl 0
> > >>> 19527 pwd CALL fstatfs(0x4,0x7fffffffd660)
> > >>> 19527 pwd RET fstatfs 0
> > >>> 19527 pwd CALL fstat(0x4,0x7fffffffd940)
> > >>> 19527 pwd STRU struct stat {dev=973143810, ino=4,
> > >>> mode=drwxr-xr-x , nlink=4948, uid=0, gid=0, rdev=4294967295,
> > >>> atime=1363244767.460164771, stime=1363172100.380266923,
> > >>> ctime=1363172100.380266923, birthtime=-1, size=4948,
> > >>> blksize=4096,
> > >>> blocks=713, flags=0x0 }
> > >>> 19527 pwd RET fstat 0
> > >>> 19527 pwd CALL
> > >>> getdirentries(0x4,0x801018000,0x1000,0x8010160a8)
> > >>> 19527 pwd RET getdirentries 4096/0x1000
> > >>> 19527 pwd CALL fstatat(0x4,0x801018030,0x7fffffffd940,0x200)
> > >>> 19527 pwd NAMI "user6158"
> > >>> 19527 pwd STRU struct stat {dev=1774902232, ino=4,
> > >>> mode=drwxr-xr-x , nlink=9, uid=0, gid=0, rdev=4294967295,
> > >>> atime=1363009687.040357529, stime=1363010116.496298252,
> > >>> ctime=1363010116.496298252, birthtime=-1, size=14, blksize=4096,
> > >>> blocks=3, flags=0x0 }
> > >>> 19527 pwd RET fstatat 0
> > >>> 19527 pwd CALL fstatat(0x4,0x80101804c,0x7fffffffd940,0x200)
> > >>> 19527 pwd NAMI "user2289"
> > >>> 19527 pwd STRU struct stat {dev=1988229825, ino=4,
> > >>> mode=drwxr-xr-x , nlink=9, uid=0, gid=0, rdev=4294967295,
> > >>> atime=1363009687.040357529, stime=1363010116.496298252,
> > >>> ctime=1363010116.496298252, birthtime=-1, size=14, blksize=4096,
> > >>> blocks=3, flags=0x0 }
> > >>> 19527 pwd RET fstatat 0
> > >>> 19527 pwd CALL fstatat(0x4,0x801018068,0x7fffffffd940,0x200)
> > >>> 19527 pwd NAMI "user4761"
> > >>> 19527 pwd STRU struct stat {dev=2438657130, ino=4,
> > >>> mode=drwxr-xr-x , nlink=9, uid=0, gid=0, rdev=4294967295,
> > >>> atime=1363009687.040357529, stime=1363010116.496298252,
> > >>> ctime=1363010116.496298252, birthtime=-1, size=14, blksize=4096,
> > >>> blocks=3, flags=0x0 }
> > >>> 19527 pwd RET fstatat 0
> > >>> 19527 pwd CALL fstatat(0x4,0x801018084,0x7fffffffd940,0x200)
> > >>> 19527 pwd NAMI "user6055"
> > >>> [.........................................]
> > >>>
> > >>> and next get stat of all directories in /home
> > >>
> > >> Slightly different version of the patch was committed as r247560.
> > >>
> > >> The situation could only happen if the parent directory contains
> > >> the "."
> > >> entry with inode number equal to the inode number of the
> > >> subdirectory.
> > >> Can you confirm that this is your case ?
> > >>
> > >
> > > Yes, it is.
> > > I'll try again on the latest snapshot. Thanks!
> > >
> >
> > Yes.
> > On latest r248313 similar situation - if path contains "." then
> > nfsclient get stat of all directories in /home.
>
> What path ?
>
> Did you read the description of the situation when r248313 returns
> ENOENT to delegate the resolving to usermode ? Can you confirm that
> this is your situation ?
I think the patch is doing the correct thing. When __getcwd() returns
ENOENT, the userland algorithm in getcwd() must stat() all entries in
the directory to figure out if it is a mount point using the st_ino
and st_dev fields. (You can look at it in the libc sources, if you'd
like. Just look for getcwd.c.) I think this is what Kostik is referring
to.
If I understand your issue, it is that this takes a long time, since
/home is large. There are a couple of things that *might* reduce the
time this takes.
1 - If you are NFSv4 mounting "/pool", you could specify /pool as the
root in you V4: /etc/exports line. Something like:
V4: /pool ...
Then you do the mount with 192.168.168.1:/.
This will make the "/pool" a root point and might avoid the
problem, but I am not sure.
2 - Adding rdirplus to the mount options will make it get attributes
for entries in a directory when it does a readdir and cache them.
This might speed things up.
rick
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?2081421885.3937873.1363357500724.JavaMail.root>