Date: Fri, 26 Dec 2014 10:37:42 +1100 From: Peter Jeremy <peter@rulingia.com> To: freebsd-fs@freebsd.org Subject: "panic: len 0" on NFS read Message-ID: <20141225233742.GA3385@server.rulingia.com>
next in thread | raw e-mail | index | archive | help
--x+6KMIRAuhnl3hBn
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Whilst trying to debug a RPC issue with a NFS tunneling tool, I mounted a
NFS filesystem onto the same host and got a panic when I tried to access it.
I'm running FreeBSD/amd64 10-stable r276177.
I mounted the filesystem with:
# mount -o udp,nfsv3 $(hostname):/tank/src92 /dist
(/tank/src92 and / are ZFS)
And then ran:
$ grep zzzz /dist/*
And got:
panic: len 0
cpuid =3D 3
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0861448=
f30
kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe0861448fe0
vpanic() at vpanic+0x126/frame 0xfffffe0861449020
kassert_panic() at kassert_panic+0x139/frame 0xfffffe0861449090
nfsm_mbufuio() at nfsm_mbufuio+0x9c/frame 0xfffffe08614490f0
nfsrpc_read() at nfsrpc_read+0x584/frame 0xfffffe08614492d0
ncl_readrpc() at ncl_readrpc+0xa5/frame 0xfffffe08614493e0
ncl_doio() at ncl_doio+0x228/frame 0xfffffe0861449480
ncl_bioread() at ncl_bioread+0xb44/frame 0xfffffe08614495f0
VOP_READ_APV() at VOP_READ_APV+0xf1/frame 0xfffffe0861449620
vn_read() at vn_read+0x211/frame 0xfffffe0861449690
vn_io_fault_doio() at vn_io_fault_doio+0x22/frame 0xfffffe08614496d0
vn_io_fault1() at vn_io_fault1+0x7c/frame 0xfffffe0861449830
vn_io_fault() at vn_io_fault+0x18b/frame 0xfffffe08614498b0
dofileread() at dofileread+0x95/frame 0xfffffe0861449900
kern_readv() at kern_readv+0x68/frame 0xfffffe0861449950
sys_read() at sys_read+0x63/frame 0xfffffe08614499a0
amd64_syscall() at amd64_syscall+0x22e/frame 0xfffffe0861449ab0
Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0861449ab0
--- syscall (3, FreeBSD ELF64, sys_read), rip =3D 0x800fd3cba, rsp =3D 0x7f=
ffffffe048, rbp =3D 0x7fffffffe090 ---
I have a crashdump that looks sane and relevant bits around nfsm_mbufuio() =
are:
#4 0xffffffff8041e63c in nfsm_mbufuio (nd=3D0xfffffe08614491b0, uiop=3D0xf=
ffffe0861449420, siz=3D0x4000)
at /usr/src/sys/fs/nfs/nfs_commonsubs.c:222
(kgdb) p mp
$1 =3D 0xfffff80053bab500
(kgdb) p *mp
$2 =3D {
m_hdr =3D {
mh_next =3D 0xfffff8023433dc00,=20
mh_nextpkt =3D 0x0,=20
mh_data =3D 0xfffff80053bab57c "=EF=BF=BD=EF=BF=BD=EF=BF=BD"...,
mh_len =3D 0x0,=20
mh_type =3D 0x1,=20
mh_flags =3D 0x2
},=20
=2E..
(kgdb) p *nd
$4 =3D {
nd_md =3D 0xfffff8005366c500,=20
nd_dpos =3D 0xfffff80562d92068 "=EF=BF=BD=EF=BF=BD=EF=BF=BD"...,
=2E..
(kgdb) p *nd->nd_md
$5 =3D {
m_hdr =3D {
mh_next =3D 0xfffff80486b05b00,=20
mh_nextpkt =3D 0x0,=20
mh_data =3D 0xfffff80562d92000 "",=20
mh_len =3D 0x68,=20
mh_type =3D 0x1,=20
mh_flags =3D 0x1
},=20
=2E..
(kgdb) p *$5.m_hdr.mh_next
$11 =3D {
m_hdr =3D {
mh_next =3D 0xfffff8005325e400,=20
mh_nextpkt =3D 0x0,=20
mh_data =3D 0xfffff80234291800 "=EF=BF=BD",=20
mh_len =3D 0x800,=20
mh_type =3D 0x1,=20
mh_flags =3D 0x1
},=20
=2E..
(kgdb) p *$11.m_hdr.mh_next
$12 =3D {
m_hdr =3D {
mh_next =3D 0xfffff80486b02400,=20
mh_nextpkt =3D 0x0,=20
mh_data =3D 0xfffff8023453c000 "\t",=20
mh_len =3D 0x800,=20
mh_type =3D 0x1,=20
mh_flags =3D 0x1
},=20
=2E..
(kgdb) p *$12.m_hdr.mh_next
$13 =3D {
m_hdr =3D {
mh_next =3D 0xfffff8023433f800,=20
mh_nextpkt =3D 0x0,=20
mh_data =3D 0xfffff80562d92800 "its",=20
mh_len =3D 0x800,=20
mh_type =3D 0x1,=20
mh_flags =3D 0x1
},=20
=2E..
(kgdb) p *$13.m_hdr.mh_next
$14 =3D {
m_hdr =3D {
mh_next =3D 0xfffff80020f36500,=20
mh_nextpkt =3D 0x0,=20
mh_data =3D 0xfffff8058cb1b000 "sbconfig",=20
mh_len =3D 0x800,=20
mh_type =3D 0x1,=20
mh_flags =3D 0x1
},=20
=2E..
(kgdb) p *$14.m_hdr.mh_next
$15 =3D {
m_hdr =3D {
mh_next =3D 0xfffff800533d5e00,=20
mh_nextpkt =3D 0x0,=20
mh_data =3D 0xfffff8041b423800 "",=20
mh_len =3D 0x800,=20
mh_type =3D 0x1,=20
mh_flags =3D 0x1
},=20
=2E..
(kgdb) p *$15.m_hdr.mh_next
$16 =3D {
m_hdr =3D {
mh_next =3D 0xfffff80053182600,=20
mh_nextpkt =3D 0x0,=20
mh_data =3D 0xfffff8023429a800 "ilters",=20
mh_len =3D 0x800,=20
mh_type =3D 0x1,=20
mh_flags =3D 0x1
},=20
=2E..
(kgdb) p *$16.m_hdr.mh_next
$17 =3D {
m_hdr =3D {
mh_next =3D 0xfffff8005379b200,=20
mh_nextpkt =3D 0x0,=20
mh_data =3D 0xfffff8058cb1e000 "",=20
mh_len =3D 0x800,=20
mh_type =3D 0x1,=20
mh_flags =3D 0x1
},=20
=2E..
(kgdb) p *$17.m_hdr.mh_next
$18 =3D {
m_hdr =3D {
mh_next =3D 0xfffff80053bab500,=20
mh_nextpkt =3D 0x0,=20
mh_data =3D 0xfffff8058cb1c800 "\002",=20
mh_len =3D 0x760,=20
mh_type =3D 0x1,=20
mh_flags =3D 0x1
},=20
=2E..
Which is points to mp.
I gather the first mbuf is NFS RPC metadata (since it's skipped). The
remaining mbufs are the start of a 3.9MB binary file (an identifier
database).
Any suggestions as to what has gone wrong?
--=20
Peter Jeremy
--x+6KMIRAuhnl3hBn
Content-Type: application/pgp-signature
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
iQJ8BAEBCgBmBQJUnJ/GXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w
ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRFRUIyOTg2QzMwNjcxRTc0RTY1QzIyN0Ux
NkE1OTdBMEU0QTIwQjM0AAoJEBall6Dkogs0qaUP/RjjDxHDYT2REtWcB79gnEt5
ixElylaIhpbND/uH3rqoJrKRon1WkaSX9TL1hLLT2zaktyWH2UPg3JNTSP5c3LGa
iipm71i0QUeaM60y6IwnkgRNUVLRDBO8t67DN7eLQudhZkdq7ew8VoaPCw23cN94
9QSOBTPC2rnVSE0/dHagw2i1+BOmR4XkynQYp/GtWxlvaRP/7lseDE73Jk5D9f5v
L1QTJabmfoA1LfgHa783T2Dvo+bxnRkeQFjUVwxMahXBbZYjXmRiWMLgC9jTZg1V
+HlbEvLpE9rJDGLPBxnMSf7/SYePg3B+iV81nmxT8/6n2fOsf1SmjWiaL7riQ9OO
gW1V5tDhBzeGqmwvRCjOYoZpn2yPdiVhFbPK4j+a3Bkkh0KbmgNJf0cuoNuQkGQF
qfY+a4wauLSWmWseadnkNXwk2gQjoL6HXiLsN+Z8sxo+lwJM6xn8TYkkt1pBovFp
fC3/CFGzuNTf5HSvVQ9I+yu+MwAQskR83jW5AL9pKs4Sphs6RsDQuZq5Knt1aLVx
eZQjNOTN0G5/HYX/R4eq5E1IISk/C/F5sI5kAGr6X90jnmVuHCDSRh+pvNHQqFqD
5aUfovtZFDGPXbv3y1RMBB+EgQw5ki9xSttHQ33yPHgCpPALZs9H/D0Y6SnLST+w
VIK21pknaW3pWFeT7g3k
=dyix
-----END PGP SIGNATURE-----
--x+6KMIRAuhnl3hBn--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20141225233742.GA3385>