Skip site navigation (1)Skip section navigation (2)
Date:      Tue, 31 May 2011 17:21:15 +0200
From:      Patrick Proniewski <patpro@patpro.net>
To:        FreeBSD Filesystems <freebsd-fs@freebsd.org>
Subject:   No physical znode address
Message-ID:  <7DA2CB2F-FA87-427D-903E-514882EE6068@patpro.net>

next in thread | raw e-mail | index | archive | help

--Apple-Mail-4--16922292
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii

Hi all,

I'm running a FreeBSD 8.2 server, with Apache 2.2 hosting around 260 web =
sites. It's a virtual machine, running on top of ESXi and a SAN storage.
The OS is installed on UFS, and a dedicated ZFS disk holds every web =
sites. Each web site is a ZFS volume created from the zpool "tank".

# zpool list =20
NAME   SIZE   USED  AVAIL    CAP  HEALTH  ALTROOT
tank   149G  53.6G  95.4G    35%  ONLINE  -

# zpool status         =20
  pool: tank
 state: ONLINE
 scrub: scrub completed after 0h19m with 0 errors on Fri May 13 22:57:10 =
2011
config:

	NAME        STATE     READ WRITE CKSUM
	tank        ONLINE       0     0     0
	  da1       ONLINE       0     0     0

errors: No known data errors

Today, I've noticed an httpd process, stuck, using 100% CPU for hours. =
It looks like the process has opened non-existing files. Here is a part =
of the output of lsof:

# lsof -p 10453
COMMAND   PID USER   FD     TYPE             DEVICE SIZE/OFF   NODE NAME
httpd   10453  www  cwd                                             No =
physical znode address: 0xffffff0013019c60
httpd   10453  www  rtd     VDIR               0,87      512      2 /
httpd   10453  www  txt     VREG               0,87  1321703 406618 =
/usr/local/sbin/httpd
httpd   10453  www  txt     VREG               0,87   246776 235521 =
/libexec/ld-elf.so.1
httpd   10453  www  txt     VREG               0,87   154320 659461 =
/lib/libm.so.5
../..
httpd   10453  www  120r                                            No =
physical znode address: 0xffffff00132e2840
httpd   10453  www  121r                                            No =
physical znode address: 0xffffff0013019c60
httpd   10453  www  122r                                            No =
physical znode address: 0xffffff00132e2840
httpd   10453  www  123r                                            No =
physical znode address: 0xffffff0013019c60
httpd   10453  www  124r                                            No =
physical znode address: 0xffffff00132e2840
httpd   10453  www  125r                                            No =
physical znode address: 0xffffff0013019c60
httpd   10453  www  126r                                            No =
physical znode address: 0xffffff00132e2840
httpd   10453  www  127r                                            No =
physical znode address: 0xffffff0013019c60
httpd   10453  www  128r                                            No =
physical znode address: 0xffffff00132e2840
httpd   10453  www  129r                                            No =
physical znode address: 0xffffff0013019c60
httpd   10453  www  130r                                            No =
physical znode address: 0xffffff00132e2840
httpd   10453  www  131r                                            No =
physical znode address: 0xffffff0013019c60
../..

Reading a part of lsof's source code, it seems to relate to ZFS =
(dnode2.c - FreeBSD ZFS node functions for lsof).
Using truss, I've discovered that the process is trying to stat a =
non-existing file, with a way too long path. truss output is a infinite =
repetition of:

=
stat("/Sites/sites//spip-core/sites/spip-core/sites/spip-core/sites/spip-c=
ore/sites/spip-core/sites/spip-core/sites/spip-core/sites/spip-core/sites/=
spip-core/sites/spip-core/sites/spip-core/sites/spip-core/sites/spip-core/=
sites/spip-core/sites/spip-core/sites/spip-core/sites/spip-core/sites/spip=
-core/sites/spip-core/sites/spip-core/sites/spip-core/sites/spip-core/site=
s/spip-core/sites/spip-core/sites/spip-core/sites/spip-core/sites/spip-cor=
e/sites/spip-core/sites/spip-core/sites/spip-core/sites/spip-core/sites/sp=
ip-core-vh/sites/edhum/bd/.Trashes////////////////////////////////////////=
//////////////////////////////////////////////////////////////////////////=
//////////////////////////////////////////////////////////////////////////=
//////////////////////////////////////////////////////////////////////////=
//////////////////////////////////////////////////////////////////////////=
//////////////////////////////////////////////////////////////////////////=
////////////////////////////////////////////////////////////////////",0x7f=
ffffffcc90) ERR#63 'File name too long'

(I had to kill -9 truss process...)

Obviously, there is something wrong with this particular web site. But =
I'm afraid it could come from the file system, or impact the FS.=20
Any idea is welcome.

patpro


--Apple-Mail-4--16922292--



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?7DA2CB2F-FA87-427D-903E-514882EE6068>