Date: Tue, 26 Feb 2002 16:16:03 -0800 From: steve@Watt.COM (Steve Watt) To: stable@freebsd.org Subject: NFS/amd weirdness Message-ID: <200202270016.g1R0G3197599@wattres.Watt.COM>
next in thread | raw e-mail | index | archive | help
I'm having some trouble with NFS between a FreeBSD client (4.5-STABLE as
of roughly 0Z 15 Feb) and a Linux (Dell hacked-up 2.2.14-6.1.1smp).
The FreeBSD client is oberon, the server is genova.
Basically, what happens is that quite regularly (but, of course, not
always), amd-mounted nfs points seem to hang for a long time. I don't
see messages about the NFS server not responding in the syslogs, but
I did a tcpdump during one of the failures and got the following
interesting output:
% tcpdump -s1500 -vv host genova
tcpdump: listening on xl0
15:59:50.124361 oberon.314179584 > genova.nfs: 40 null (ttl 64, id 51398, len 68, bad cksum 0!)
15:59:50.124897 genova.nfs > oberon.314179584: reply ok 24 null (ttl 63, id 57434, len 52)
15:59:50.125676 oberon.1022 > genova.sunrpc: [bad udp cksum ea59!] udp 116 (ttl 64, id 51399, len 144, bad cksum 0!)
15:59:50.126360 genova.sunrpc > oberon.1022: [udp sum ok] udp 28 (ttl 63, id 57435, len 56)
15:59:53.144489 oberon.1019 > genova.blackjack: S [bad tcp cksum b880!] 4093252687:4093252687(0) win 65535 <mss 1460> (DF) (ttl 64, id 51407, len 44, bad cksum 0!)
16:00:17.145720 oberon.1019 > genova.blackjack: S [bad tcp cksum b880!] 4093252687:4093252687(0) win 65535 <mss 1460> (DF) (ttl 64, id 51419, len 44, bad cksum 0!)
16:00:20.135911 oberon.851050496 > genova.nfs: 40 null (ttl 64, id 51421, len 68, bad cksum 0!)
16:00:20.137385 genova.nfs > oberon.851050496: reply ok 24 null (ttl 63, id 57561, len 52)
Those bad cksum, bad udp cksum, and bad tcp cksum make me somewhat nervous.
I then ran a tcpdump -X to catch it, and the problem shook itself out
while (or slightly before) that was going on:
% tcpdump -s1500 -X -vv host genova and port nfs
tcpdump: listening on xl0
16:00:35.229138 oberon.teja.com.603827759 > genova.teja.com.nfs: 112 getattr fh Unknown/1 (ttl 64, id 51479, len 140, bad cksum 0!)
0x0000 4500 008c c917 0000 4011 0000 c0a8 0116 E.......@.......
0x0010 c0a8 011e 02da 0801 0078 840e 23fd ae2f .........x..#../
0x0020 0000 0000 0000 0002 0001 86a3 0000 0002 ................
0x0030 0000 0001 0000 0001 0000 0028 0000 0000 ...........(....
0x0040 0000 0000 0000 0000 0000 01f4 0000 0005 ................
0x0050 0000 01f4 0000 0000 0000 000f 0000 01f5 ................
0x0060 0000 01f6 0000 0000 0000 0000 caba ebfe ................
0x0070 dce7 3000 0200 0000 0a08 0000 0a08 0000 ..0.............
0x0080 dce7 3000 f907 5916 0000 0000 ..0...Y.....
16:00:35.230098 genova.teja.com.nfs > oberon.teja.com.603827759: reply ok 96 getattr DIR 40775 ids 539/501 sz 4096 (ttl 63, id 58976, len 124)
0x0000 4500 007c e660 0000 3f11 118c c0a8 011e E..|.`..?.......
0x0010 c0a8 0116 0801 02da 0068 b917 23fd ae2f .........h..#../
0x0020 0000 0001 0000 0000 0000 0000 0000 0000 ................
0x0030 0000 0000 0000 0000 0000 0002 0000 41fd ..............A.
0x0040 0000 0002 0000 021b 0000 01f5 0000 1000 ................
0x0050 0000 1000 0000 0000 0000 0008 0000 080a ................
0x0060 0030 e7dc 3c7b 797b 0000 0000 3bcc b0dc .0..<{y{....;...
0x0070 0000 0000 3bcc b0dc 0000 0000 ....;.......
16:00:35.270632 oberon.teja.com.603827760 > genova.teja.com.nfs: 112 getattr fh Unknown/1 (ttl 64, id 51480, len 140, bad cksum 0!)
0x0000 4500 008c c918 0000 4011 0000 c0a8 0116 E.......@.......
0x0010 c0a8 011e 02d9 0801 0078 840e 23fd ae30 .........x..#..0
0x0020 0000 0000 0000 0002 0001 86a3 0000 0002 ................
0x0030 0000 0001 0000 0001 0000 0028 0000 0000 ...........(....
0x0040 0000 0000 0000 0000 0000 01f4 0000 0005 ................
0x0050 0000 01f4 0000 0000 0000 000f 0000 01f5 ................
0x0060 0000 01f6 0000 0000 0000 0000 caba ebfe ................
0x0070 196f 2400 0200 0000 0a08 0000 0a08 0000 .o$.............
0x0080 196f 2400 1947 dda1 0000 0000 .o$..G......
16:00:35.271486 genova.teja.com.nfs > oberon.teja.com.603827760: reply ok 96 getattr DIR 40777 ids 0/0 sz 4096 (ttl 63, id 58977, len 124)
0x0000 4500 007c e661 0000 3f11 118b c0a8 011e E..|.a..?.......
0x0010 c0a8 0116 0801 02d9 0068 b61b 23fd ae30 .........h..#..0
0x0020 0000 0001 0000 0000 0000 0000 0000 0000 ................
0x0030 0000 0000 0000 0000 0000 0002 0000 41ff ..............A.
0x0040 0000 0027 0000 0000 0000 0000 0000 1000 ...'............
0x0050 0000 1000 0000 0000 0000 0008 0000 080a ................
0x0060 0024 6f19 3c7c 2189 0000 0000 3c75 9c06 .$o.<|!.....<u..
0x0070 0000 0000 3c75 9c06 0000 0000 ....<u......
16:00:35.312283 oberon.teja.com.603827761 > genova.teja.com.nfs: 112 getattr fh Unknown/1 (ttl 64, id 51481, len 140, bad cksum 0!)
0x0000 4500 008c c919 0000 4011 0000 c0a8 0116 E.......@.......
0x0010 c0a8 011e 02d8 0801 0078 840e 23fd ae31 .........x..#..1
0x0020 0000 0000 0000 0002 0001 86a3 0000 0002 ................
0x0030 0000 0001 0000 0001 0000 0028 0000 0000 ...........(....
0x0040 0000 0000 0000 0000 0000 01f4 0000 0005 ................
0x0050 0000 01f4 0000 0000 0000 000f 0000 01f5 ................
0x0060 0000 01f6 0000 0000 0000 0000 caba ebfe ................
0x0070 417c 0700 0200 0000 0a08 0000 0a08 0000 A|..............
0x0080 417c 0700 68e0 b6a5 0000 0000 A|..h.......
16:00:35.313286 genova.teja.com.nfs > oberon.teja.com.603827761: reply ok 96 getattr DIR 40755 ids 544/500 sz 4096 (ttl 63, id 58978, len 124)
0x0000 4500 007c e662 0000 3f11 118a c0a8 011e E..|.b..?.......
0x0010 c0a8 0116 0801 02d8 0068 1b62 23fd ae31 .........h.b#..1
0x0020 0000 0001 0000 0000 0000 0000 0000 0000 ................
0x0030 0000 0000 0000 0000 0000 0002 0000 41ed ..............A.
0x0040 0000 0005 0000 0220 0000 01f4 0000 1000 ................
0x0050 0000 1000 0000 0000 0000 0008 0000 080a ................
0x0060 0007 7c41 3c7c 219b 0000 0000 3c4d e10c ..|A<|!.....<M..
0x0070 0000 0000 3c4d e10c 0000 0000 ....<M......
The bad cksum messages are still there. This feels like a bug somewhere
in the output network path, and buffers are getting handed to bpf before
checksumming is complete.
But NFS still doesn't work correctly all the time. It may just be
an amd vs Linux weirdness. My amd map is quite boring:
/defaults type:=host;fs:=${autodir}/${rhost}/host;rhost:=${key};opts:=rw,grpid,resvport,nosuid,nodev,intr,hard
* opts:=rw.grpid,resvport,nosuid,nodev,intr,hard,version=2
Pointers? Ideas where I should start digging when I see this happen
again?
Thx,
--
Steve Watt KD6GGD PP-ASEL-IA ICBM: 121W 56' 57.8" / 37N 20' 14.9"
Internet: steve @ Watt.COM Whois: SW32
Free time? There's no such thing. It just comes in varying prices...
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-stable" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200202270016.g1R0G3197599>