From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 8 00:05:25 2006 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3085B16A417; Wed, 8 Nov 2006 00:05:25 +0000 (UTC) (envelope-from jhs@flat.berklix.net) Received: from thin.berklix.org (thin.berklix.org [194.246.123.68]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6B20843D45; Wed, 8 Nov 2006 00:05:24 +0000 (GMT) (envelope-from jhs@flat.berklix.net) Received: from js.berklix.net (p549A665E.dip.t-dialin.net [84.154.102.94]) (authenticated bits=128) by thin.berklix.org (8.12.11/8.12.11) with ESMTP id kA805LFG031149; Wed, 8 Nov 2006 01:05:21 +0100 (CET) (envelope-from jhs@flat.berklix.net) Received: from fire.jhs.private (fire.jhs.private [192.168.91.41]) by js.berklix.net (8.13.6/8.13.6) with ESMTP id kA805IOK001566; Wed, 8 Nov 2006 01:05:19 +0100 (CET) (envelope-from jhs@flat.berklix.net) Received: from fire.jhs.private (localhost [127.0.0.1]) by fire.jhs.private (8.13.6/8.13.6) with ESMTP id kA805ICY005742; Wed, 8 Nov 2006 01:05:18 +0100 (CET) (envelope-from jhs@fire.jhs.private) Message-Id: <200611080005.kA805ICY005742@fire.jhs.private> To: John Baldwin In-reply-to: <200611021652.07853.jhb@freebsd.org> References: <200611021050.kA2AoH5S028916@fire.jhs.private> <200611021202.28351.jhb@freebsd.org> <200611022010.kA2KA9O7004585@fire.jhs.private> <200611021652.07853.jhb@freebsd.org> Comments: In-reply-to John Baldwin message dated "Thu, 02 Nov 2006 16:52:07 -0500." Date: Wed, 08 Nov 2006 01:05:18 +0100 From: "Julian H. Stacey" Cc: freebsd-hackers@freebsd.org Subject: Re: NFS on 6.1 limits at 4 Gig X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Nov 2006 00:05:25 -0000 John Baldwin wrote: > On Thursday 02 November 2006 15:10, Julian H. Stacey wrote: > > John Baldwin wrote: > > > On Thursday 02 November 2006 05:50, Julian Stacey wrote: > > > > NFS fails on files >= 4 Gig Can someone confirm please. > > > > > > > > uname -r # 6.1-RELEASE (both hosts) > > > > # echo "1024 1024 * 4 * 1 + p" | dc # 4194305 > > > > dd if=/dev/zero of=junk bs=1k count=4194305 > > > > ls -l junk # 4294968320 bytes > > > > rsh an_nfs_host ls -l /host/`hostname -s`/usr/tmp/junk # 1024 byte size! > > > > # with count=4194304, ls shows 0 bytes. > > > > > > > > It's not AMD failing, but NFS, as with an /etc/amd.map with a > > > > non NFS entry for my host "laps" for efficiency (in case some > > > > shell on host laps mounts itself), the full size 4294968320 is seen. > > > > /etc/amd.map > > > > /defaults type:=host;fs:=${autodir}/${rhost};rhost:=${key} > > > > laps type:=link;fs:=.. > > > > > > > > It's not just ls, cmp fails too, ( as also does my > > > > http://berklix.com/~jhs/src/bsd/jhs/bin/public/cmpd/cmpd.c ) > > > > cmp -z junk /host/laps/usr/tmp/junk # junk /host/laps/usr/tmp/junk differ: size > > > > > > > > Is send-pr appropriate ? > > > > > > Are you using NFS v2 or v3? v2 doesn't support large files. > > > John Baldwin > > > > Thanks, I don't know ! Whatever 6.1-RELEASE comes standard with. > > > > After your mail I did cd /usr/ports ; echo */*nfs* > > net-mgmt/nfsen net/nfsshell net/pcnfsd net/unfs3 > > /usr/ports/net/unfs3 offers a non ernel V3 server > > but I'd still need a v3 client I suppose ? > > Are 6.2-pre or current using V3 NFS then ? > > Hints which way to jump / where to RTFM please :-) > > It should default to v3, the nfs client in the base system can do either > v2 or v3. I'm not sure if amd is going to default to v2 with your map > file though. Ah Yes ! Fresh eyes :-) My /etc/amd.map had bit rot with "vers=2" in * opts:=rw,grpid,resvport,vers=2,proto=udp,nodev I tested with manually mounted mount_nfs -2 & -3, then removed "vers=2" from amd.map & now all OK, comparing 5 gig files over NFS. > You can use tcpdump on the port with NFS traffic to see > if it's v2 or v3 though. I tried tcpdump -v -t -s 192 -i rl0 port 2049 IP (tos 0x0, ttl 64, id 52774, offset 0, flags [none], proto: UDP (17), length: 128) fire.jhs.private.1181802316 > laps.jhs.private.nfs: 100 lookup fh 1041,235166/2 "usr2" IP (tos 0x0, ttl 64, id 35491, offset 0, flags [none], proto: UDP (17), length: 156) laps.jhs.private.nfs > fire.jhs.private.1181802316: reply ok 128 lookup fh 1041,235166/16512 DIR 40755 ids 0/0 sz 512 But didnt know how to read it for NFS 2/3 hence used mount_nfs -2 & -3. Problem solved. Thanks John. -- Julian Stacey. BSD Unix C Net Consultancy, Munich/Muenchen http://berklix.com Mail Ascii, not HTML. Ihr Rauch = mein allergischer Kopfschmerz. http://berklix.org/free-software