Date: Thu, 25 Dec 2014 20:08:20 -0500 (EST) From: Rick Macklem <rmacklem@uoguelph.ca> To: Peter Jeremy <peter@rulingia.com> Cc: freebsd-fs@freebsd.org, benno@freebsd.org Subject: Re: "panic: len 0" on NFS read Message-ID: <1000783981.2374019.1419556100933.JavaMail.root@uoguelph.ca> In-Reply-To: <20141225233742.GA3385@server.rulingia.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Peter Jeremy wrote:
> Whilst trying to debug a RPC issue with a NFS tunneling tool, I
> mounted a
> NFS filesystem onto the same host and got a panic when I tried to
> access it.
>=20
> I'm running FreeBSD/amd64 10-stable r276177.
>=20
> I mounted the filesystem with:
> # mount -o udp,nfsv3 $(hostname):/tank/src92 /dist
>=20
> (/tank/src92 and / are ZFS)
>=20
> And then ran:
> $ grep zzzz /dist/*
>=20
> And got:
> panic: len 0
r275941 in head changed this KASSERT to allow a 0 length mbuf, so I
don't think the panic is meaningful.
Maybe r275941 should be MFC'd? (I've cc'd benno, who did the commit.)
rick
> cpuid =3D 3
> KDB: stack backtrace:
> db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame
> 0xfffffe0861448f30
> kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe0861448fe0
> vpanic() at vpanic+0x126/frame 0xfffffe0861449020
> kassert_panic() at kassert_panic+0x139/frame 0xfffffe0861449090
> nfsm_mbufuio() at nfsm_mbufuio+0x9c/frame 0xfffffe08614490f0
> nfsrpc_read() at nfsrpc_read+0x584/frame 0xfffffe08614492d0
> ncl_readrpc() at ncl_readrpc+0xa5/frame 0xfffffe08614493e0
> ncl_doio() at ncl_doio+0x228/frame 0xfffffe0861449480
> ncl_bioread() at ncl_bioread+0xb44/frame 0xfffffe08614495f0
> VOP_READ_APV() at VOP_READ_APV+0xf1/frame 0xfffffe0861449620
> vn_read() at vn_read+0x211/frame 0xfffffe0861449690
> vn_io_fault_doio() at vn_io_fault_doio+0x22/frame 0xfffffe08614496d0
> vn_io_fault1() at vn_io_fault1+0x7c/frame 0xfffffe0861449830
> vn_io_fault() at vn_io_fault+0x18b/frame 0xfffffe08614498b0
> dofileread() at dofileread+0x95/frame 0xfffffe0861449900
> kern_readv() at kern_readv+0x68/frame 0xfffffe0861449950
> sys_read() at sys_read+0x63/frame 0xfffffe08614499a0
> amd64_syscall() at amd64_syscall+0x22e/frame 0xfffffe0861449ab0
> Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0861449ab0
> --- syscall (3, FreeBSD ELF64, sys_read), rip =3D 0x800fd3cba, rsp =3D
> 0x7fffffffe048, rbp =3D 0x7fffffffe090 ---
>=20
> I have a crashdump that looks sane and relevant bits around
> nfsm_mbufuio() are:
>=20
> #4 0xffffffff8041e63c in nfsm_mbufuio (nd=3D0xfffffe08614491b0,
> uiop=3D0xfffffe0861449420, siz=3D0x4000)
> at /usr/src/sys/fs/nfs/nfs_commonsubs.c:222
> (kgdb) p mp
> $1 =3D 0xfffff80053bab500
> (kgdb) p *mp
> $2 =3D {
> m_hdr =3D {
> mh_next =3D 0xfffff8023433dc00,
> mh_nextpkt =3D 0x0,
> mh_data =3D 0xfffff80053bab57c "=EF=BF=BD=EF=BF=BD=EF=BF=BD"...,
> mh_len =3D 0x0,
> mh_type =3D 0x1,
> mh_flags =3D 0x2
> },
> ...
> (kgdb) p *nd
> $4 =3D {
> nd_md =3D 0xfffff8005366c500,
> nd_dpos =3D 0xfffff80562d92068 "=EF=BF=BD=EF=BF=BD=EF=BF=BD"...,
> ...
> (kgdb) p *nd->nd_md
> $5 =3D {
> m_hdr =3D {
> mh_next =3D 0xfffff80486b05b00,
> mh_nextpkt =3D 0x0,
> mh_data =3D 0xfffff80562d92000 "",
> mh_len =3D 0x68,
> mh_type =3D 0x1,
> mh_flags =3D 0x1
> },
> ...
> (kgdb) p *$5.m_hdr.mh_next
> $11 =3D {
> m_hdr =3D {
> mh_next =3D 0xfffff8005325e400,
> mh_nextpkt =3D 0x0,
> mh_data =3D 0xfffff80234291800 "=EF=BF=BD",
> mh_len =3D 0x800,
> mh_type =3D 0x1,
> mh_flags =3D 0x1
> },
> ...
> (kgdb) p *$11.m_hdr.mh_next
> $12 =3D {
> m_hdr =3D {
> mh_next =3D 0xfffff80486b02400,
> mh_nextpkt =3D 0x0,
> mh_data =3D 0xfffff8023453c000 "\t",
> mh_len =3D 0x800,
> mh_type =3D 0x1,
> mh_flags =3D 0x1
> },
> ...
> (kgdb) p *$12.m_hdr.mh_next
> $13 =3D {
> m_hdr =3D {
> mh_next =3D 0xfffff8023433f800,
> mh_nextpkt =3D 0x0,
> mh_data =3D 0xfffff80562d92800 "its",
> mh_len =3D 0x800,
> mh_type =3D 0x1,
> mh_flags =3D 0x1
> },
> ...
> (kgdb) p *$13.m_hdr.mh_next
> $14 =3D {
> m_hdr =3D {
> mh_next =3D 0xfffff80020f36500,
> mh_nextpkt =3D 0x0,
> mh_data =3D 0xfffff8058cb1b000 "sbconfig",
> mh_len =3D 0x800,
> mh_type =3D 0x1,
> mh_flags =3D 0x1
> },
> ...
> (kgdb) p *$14.m_hdr.mh_next
> $15 =3D {
> m_hdr =3D {
> mh_next =3D 0xfffff800533d5e00,
> mh_nextpkt =3D 0x0,
> mh_data =3D 0xfffff8041b423800 "",
> mh_len =3D 0x800,
> mh_type =3D 0x1,
> mh_flags =3D 0x1
> },
> ...
> (kgdb) p *$15.m_hdr.mh_next
> $16 =3D {
> m_hdr =3D {
> mh_next =3D 0xfffff80053182600,
> mh_nextpkt =3D 0x0,
> mh_data =3D 0xfffff8023429a800 "ilters",
> mh_len =3D 0x800,
> mh_type =3D 0x1,
> mh_flags =3D 0x1
> },
> ...
> (kgdb) p *$16.m_hdr.mh_next
> $17 =3D {
> m_hdr =3D {
> mh_next =3D 0xfffff8005379b200,
> mh_nextpkt =3D 0x0,
> mh_data =3D 0xfffff8058cb1e000 "",
> mh_len =3D 0x800,
> mh_type =3D 0x1,
> mh_flags =3D 0x1
> },
> ...
> (kgdb) p *$17.m_hdr.mh_next
> $18 =3D {
> m_hdr =3D {
> mh_next =3D 0xfffff80053bab500,
> mh_nextpkt =3D 0x0,
> mh_data =3D 0xfffff8058cb1c800 "\002",
> mh_len =3D 0x760,
> mh_type =3D 0x1,
> mh_flags =3D 0x1
> },
> ...
>=20
> Which is points to mp.
>=20
> I gather the first mbuf is NFS RPC metadata (since it's skipped).
> The
> remaining mbufs are the start of a 3.9MB binary file (an identifier
> database).
>=20
> Any suggestions as to what has gone wrong?
>=20
> --
> Peter Jeremy
>=20
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1000783981.2374019.1419556100933.JavaMail.root>