Date: Tue, 31 May 2011 17:21:15 +0200 From: Patrick Proniewski <patpro@patpro.net> To: FreeBSD Filesystems <freebsd-fs@freebsd.org> Subject: No physical znode address Message-ID: <7DA2CB2F-FA87-427D-903E-514882EE6068@patpro.net>
next in thread | raw e-mail | index | archive | help
--Apple-Mail-4--16922292
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
charset=us-ascii
Hi all,
I'm running a FreeBSD 8.2 server, with Apache 2.2 hosting around 260 web =
sites. It's a virtual machine, running on top of ESXi and a SAN storage.
The OS is installed on UFS, and a dedicated ZFS disk holds every web =
sites. Each web site is a ZFS volume created from the zpool "tank".
# zpool list =20
NAME SIZE USED AVAIL CAP HEALTH ALTROOT
tank 149G 53.6G 95.4G 35% ONLINE -
# zpool status =20
pool: tank
state: ONLINE
scrub: scrub completed after 0h19m with 0 errors on Fri May 13 22:57:10 =
2011
config:
NAME STATE READ WRITE CKSUM
tank ONLINE 0 0 0
da1 ONLINE 0 0 0
errors: No known data errors
Today, I've noticed an httpd process, stuck, using 100% CPU for hours. =
It looks like the process has opened non-existing files. Here is a part =
of the output of lsof:
# lsof -p 10453
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
httpd 10453 www cwd No =
physical znode address: 0xffffff0013019c60
httpd 10453 www rtd VDIR 0,87 512 2 /
httpd 10453 www txt VREG 0,87 1321703 406618 =
/usr/local/sbin/httpd
httpd 10453 www txt VREG 0,87 246776 235521 =
/libexec/ld-elf.so.1
httpd 10453 www txt VREG 0,87 154320 659461 =
/lib/libm.so.5
../..
httpd 10453 www 120r No =
physical znode address: 0xffffff00132e2840
httpd 10453 www 121r No =
physical znode address: 0xffffff0013019c60
httpd 10453 www 122r No =
physical znode address: 0xffffff00132e2840
httpd 10453 www 123r No =
physical znode address: 0xffffff0013019c60
httpd 10453 www 124r No =
physical znode address: 0xffffff00132e2840
httpd 10453 www 125r No =
physical znode address: 0xffffff0013019c60
httpd 10453 www 126r No =
physical znode address: 0xffffff00132e2840
httpd 10453 www 127r No =
physical znode address: 0xffffff0013019c60
httpd 10453 www 128r No =
physical znode address: 0xffffff00132e2840
httpd 10453 www 129r No =
physical znode address: 0xffffff0013019c60
httpd 10453 www 130r No =
physical znode address: 0xffffff00132e2840
httpd 10453 www 131r No =
physical znode address: 0xffffff0013019c60
../..
Reading a part of lsof's source code, it seems to relate to ZFS =
(dnode2.c - FreeBSD ZFS node functions for lsof).
Using truss, I've discovered that the process is trying to stat a =
non-existing file, with a way too long path. truss output is a infinite =
repetition of:
=
stat("/Sites/sites//spip-core/sites/spip-core/sites/spip-core/sites/spip-c=
ore/sites/spip-core/sites/spip-core/sites/spip-core/sites/spip-core/sites/=
spip-core/sites/spip-core/sites/spip-core/sites/spip-core/sites/spip-core/=
sites/spip-core/sites/spip-core/sites/spip-core/sites/spip-core/sites/spip=
-core/sites/spip-core/sites/spip-core/sites/spip-core/sites/spip-core/site=
s/spip-core/sites/spip-core/sites/spip-core/sites/spip-core/sites/spip-cor=
e/sites/spip-core/sites/spip-core/sites/spip-core/sites/spip-core/sites/sp=
ip-core-vh/sites/edhum/bd/.Trashes////////////////////////////////////////=
//////////////////////////////////////////////////////////////////////////=
//////////////////////////////////////////////////////////////////////////=
//////////////////////////////////////////////////////////////////////////=
//////////////////////////////////////////////////////////////////////////=
//////////////////////////////////////////////////////////////////////////=
////////////////////////////////////////////////////////////////////",0x7f=
ffffffcc90) ERR#63 'File name too long'
(I had to kill -9 truss process...)
Obviously, there is something wrong with this particular web site. But =
I'm afraid it could come from the file system, or impact the FS.=20
Any idea is welcome.
patpro
--Apple-Mail-4--16922292--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?7DA2CB2F-FA87-427D-903E-514882EE6068>