From owner-freebsd-questions@freebsd.org Fri Jul 24 20:04:35 2015 Return-Path: Delivered-To: freebsd-questions@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D4C9D9AA50A for ; Fri, 24 Jul 2015 20:04:35 +0000 (UTC) (envelope-from kayasaman@gmail.com) Received: from mail-wi0-x235.google.com (mail-wi0-x235.google.com [IPv6:2a00:1450:400c:c05::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5389B1C89 for ; Fri, 24 Jul 2015 20:04:35 +0000 (UTC) (envelope-from kayasaman@gmail.com) Received: by wicgb10 with SMTP id gb10so42465237wic.1 for ; Fri, 24 Jul 2015 13:04:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=to:from:subject:message-id:date:user-agent:mime-version :content-type; bh=xq7tBWDZDbBDBSz9OEs4GLtUCZarFI+IqSpySs0EMvo=; b=JHHlvfEbynJrRdadA4pX6pvtR3LeECUI2XKFh+QuZsVyi3e9Y+eoGQgV8s+tSFf0jF 4c7h40QaLUlmxVO0L/32xuWX/65ksffOZdCcfpszvJ187gdtTk7pe9+M3woy7tHUsELO 8jiwK4Ywbld+2t4F8UpfFXfmgX+ubSkBZkPE6gz/FcU8I6zuL84WUO4ygDPPaDnefpYG 3fmlZwKbXMiQtr3RvRpplW6sKi9wZepydQRO76OE7senmPUcBYcs4MdfAG/147rClJDd r5ZLXlqJt7uDbnZDN7GtzmuqOwziHv5nIDogfAjxK81a0BRQzImhFl4xWAPx9HUGQGYl EFyA== X-Received: by 10.194.121.100 with SMTP id lj4mr30044766wjb.104.1437768273096; Fri, 24 Jul 2015 13:04:33 -0700 (PDT) Received: from x220.optiplex-networks.com (optiplexnetworks.plus.com. [212.159.80.17]) by smtp.googlemail.com with ESMTPSA id dz4sm5074476wib.17.2015.07.24.13.04.31 for (version=TLSv1/SSLv3 cipher=OTHER); Fri, 24 Jul 2015 13:04:32 -0700 (PDT) To: freebsd-questions From: Kaya Saman Subject: NFS Server keeps timing out on a client? Message-ID: <55B29A4F.4040506@gmail.com> Date: Fri, 24 Jul 2015 21:04:31 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.1.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Jul 2015 20:04:35 -0000 Hi, this is a really difficult one to diagnose :-( The situation is this: I have a server which I'm trying to serve up a home directory for. So when a user logs in they will be taken to /mnt/home/ NFS mount. The home directory is a zfs filesystem stored on a ZPOOL. The server runs FreeBSD 10.1 64bit and is powered by a quad core Intel Xeon with 8GB RAM and is connected to the network using a 4GbE lagg trunk. The client side is Arch Linux, 64 bit connected via gigabit link. They are on different vlans, and the network uses Jumbo MTU's however, I have also tested with regular mtu size of 1500 and same thing seems to be happening. As the firewall/router also runs *BSD I tried to see what was happening using: tcpdump -eni (vlan) host During the timeout periods I keep seeing this message: 2042: truncated-ip - 4 bytes missing!192.168.20.30.981 > 192.168.1.80.2049: xid 0x3b14188c (NFSv3) 1976 write [|nfs] (DF)2042: truncated-ip - 4 bytes missing!192.168.20.30.981 > 192.168.1.80.2049: xid 0x3b14188c (NFSv3) 1976 write [|nfs] (DF) 2042: truncated-ip - 4 bytes missing!.981 > .2049: xid 0x3b14188c (NFSv3) 1976 write [|nfs] (DF) Unfortunately setting the "debug" flag in mountd on FreeBSD doesn't show any thing other then "mount ok". On the client side I have put NFS into debug mode and am seeing this through dmesg: [24012.265800] NFS: nfs_update_inode(0:41/1653343 fh_crc=0x9257f59a ct=1 info=0x27e7f) [24012.266207] NFS: nfs_update_inode(0:41/1653343 fh_crc=0x9257f59a ct=1 info=0x27e7f) [24012.266215] NFS: permission(0:41/1653343), mask=0x26, res=0 [24012.284651] NFS: nfs_update_inode(0:41/1653339 fh_crc=0x998992f7 ct=1 info=0x7feff) [24012.285181] NFS: nfs_update_inode(0:41/1653339 fh_crc=0x998992f7 ct=1 info=0x7feff) [24012.285251] NFS: dentry_delete(entries/ADDBB8F380DD96E0D9B49F9AF600706F6A1431BA, 4808cc) [24012.294851] NFS: nfs_update_inode(0:41/1653196 fh_crc=0x6513c0a6 ct=1 info=0x7feff) [24012.295399] NFS: nfs_update_inode(0:41/1653196 fh_crc=0x6513c0a6 ct=1 info=0x7feff) [24012.295510] NFS: dentry_delete(entries/9AEB965A3D7D6F96E61BD8427D8296215AE6CE13, 4808cc) [24013.731223] NFS: permission(0:41/4), mask=0x1, res=0 [24013.731237] NFS: permission(0:41/1649925), mask=0x1, res=0 [24013.731244] NFS: permission(0:41/1649944), mask=0x1, res=0 [24013.731251] NFS: dentry_delete(.config/menus, 808cc) Has anyone experienced this type of behavior before, and possibly help guide me on how to debug it to get to the root of the problem? Many thanks! Regards, Kaya