From owner-freebsd-stable@FreeBSD.ORG Thu Jul 1 18:01:10 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 42F46106564A for ; Thu, 1 Jul 2010 18:01:10 +0000 (UTC) (envelope-from alan.bryan@yahoo.com) Received: from web50501.mail.re2.yahoo.com (web50501.mail.re2.yahoo.com [206.190.38.77]) by mx1.freebsd.org (Postfix) with SMTP id 02BAF8FC20 for ; Thu, 1 Jul 2010 18:01:09 +0000 (UTC) Received: (qmail 43263 invoked by uid 60001); 1 Jul 2010 18:01:04 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1278007264; bh=jyctgp9elowTKq64lN4Abx34MLD/q9KPog1d+DECw7o=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=vhilqvponJmlzEfcAV0fpHJoijg266FdjMdlgkWVlGtHhmtPrj2K4yN6piH/UZm4aX1TnXGAZNU/S2NuraxIIClWDPF3uKumoPi0ypi1maDlY7+hOmSfEQ4zifUO59Q4CeYmPVgthaVFT6zw5GpObMevXFqECnLbgbcOKk3dWVk= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=bvL7XaV2JPH3kZrqKgMEm++Ea0OwWxOg6bcNzvmfkM3WCKydW0tLZovpUqE10ygRmLL5aUhhqrpdSuYPOClXIc9D2LGk40/XNwkpSsFus5lxKNJnvBMJ7jqFMxC1GrEWXy8j0gqScwVk6b67KRiwN0O+jyqFJXBx6+JgByaWPHw=; Message-ID: <425902.41392.qm@web50501.mail.re2.yahoo.com> X-YMail-OSG: LDbmq7EVM1lkzPsNTgQDAjzOtk1ywG8zS2abJWp8qzJLaNf yBZYrnGM.U_I5HcuVBHTewRPrDbhl3a9fSxRIggfHFaO6.mKZCCjklhy4629 s5xJg8kwT4hYaylVNAMqvwIA3J7v2WGrk5bFI9z_sr.Exg6Y9yiLS1pJaZ30 CuoBZ7LHNWQtSMJJcWJ9CNHbPNhLTxPDd7R7i.D6ik4H2t7XvbbGtPlp8dBD D11BHd6zmMwXa_d0dR_y7ImT6rxnw3fU4lEDU5ntoy9JkisQXBDiMyPPk_qR V5dEieoUK7jmoFZFAh6bd2ackHAZTXBhwWgLPiv6X2YNFSNH5a6WoPFevu5L eldksoadGR4rXFZ1e4BqUwnikpizLRZ4pADH0Wt52L5WIwCiooUXyJ.YuQBg YFQ-- Received: from [99.24.6.121] by web50501.mail.re2.yahoo.com via HTTP; Thu, 01 Jul 2010 11:01:04 PDT X-Mailer: YahooMailClassic/11.1.4 YahooMailWebService/0.8.104.274457 Date: Thu, 1 Jul 2010 11:01:04 -0700 (PDT) From: alan bryan To: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Subject: NFS 75 second stall X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Jul 2010 18:01:10 -0000 Setup:=0A=0Aserver - FreeBSD 8-stable from today.=A0 2 UFS dirs exported vi= a NFS.=0Aclient - FreeBSD 8.0-Release. Running a test php script that copi= es around various files to/from 2 separate NFS mounts.=0A=0ASituation: =0A= =0Ascript is started (forked to do 20 simultaneous runs) and 20 1GB files a= re copied to the NFS dir which works fine.=A0 When it then switches to read= ing those files back and simultaneously writing to the other NFS mount I se= e a hang of 75 seconds.=A0 If I do an "ls -l" on the NFS mount it hangs too= .=A0 After 75 seconds the client has reported:=0A=0Anfs server 192.168.10.1= 33:/usr/local/export1: not responding=0Anfs server 192.168.10.133:/usr/loca= l/export1: is alive again=0Anfs server 192.168.10.133:/usr/local/export1: n= ot responding=0Anfs server 192.168.10.133:/usr/local/export1: is alive agai= n =0A=0Aand then things start working again.=A0 The server was originally F= reeBSD 8.0-Release also but was upgraded to the latest stable to see if thi= s issue could be avoided.=0A=0A# nfsstat -s -W -w 1=0A GtAttr Lookup Rdlink= =A0=A0=A0Read=A0 Write Rename Access=A0 Rddir=0A=A0 =A0 =A0 0=A0 =A0 =A0 0= =A0 =A0 =A0 0=A0 =A0 222=A0 =A0 257=A0 =A0 =A0 0=A0 =A0 =A0 0=A0 =A0 =A0 0= =0A=A0 =A0 =A0 0=A0 =A0 =A0 0=A0 =A0 =A0 0=A0 =A0 178=A0 =A0 135=A0 =A0 =A0= 0=A0 =A0 =A0 0=A0 =A0 =A0 0=0A=A0 =A0 =A0 0=A0 =A0 =A0 0=A0 =A0 =A0 0=A0 = =A0=A0=A085=A0 =A0 127=A0 =A0 =A0 0=A0 =A0 =A0 0=A0 =A0 =A0 0=0A=A0 =A0 =A0= 0=A0 =A0 =A0 0=A0 =A0 =A0 0=A0 =A0 =A0 0=A0 =A0 =A0 0=A0 =A0 =A0 0=A0 =A0 = =A0 0=A0 =A0 =A0 0=0A=A0 =A0 =A0 0=A0 =A0 =A0 0=A0 =A0 =A0 0=A0 =A0 =A0 0= =A0 =A0 =A0 0=A0 =A0 =A0 0=A0 =A0 =A0 0=A0 =A0 =A0 0=0A=A0 =A0 =A0 0=A0 =A0= =A0 0=A0 =A0 =A0 0=A0 =A0 =A0 0=A0 =A0 =A0 0=A0 =A0 =A0 0=A0 =A0 =A0 0=A0 = =A0 =A0 0=0A=A0 =A0 =A0 0=A0 =A0 =A0 0=A0 =A0 =A0 0=A0 =A0 =A0 0=A0 =A0 =A0= 0=A0 =A0 =A0 0=A0 =A0 =A0 0=A0 =A0 =A0 0=0A=A0 =A0 =A0 0=A0 =A0 =A0 0=A0 = =A0 =A0 0=A0 =A0 =A0 0=A0 =A0 =A0 0=A0 =A0 =A0 0=A0 =A0 =A0 0=A0 =A0 =A0 0= =0A=0A... for 75 rows of all zeros=0A=0A=A0 =A0 =A0 0=A0 =A0 =A0 0=A0 =A0 = =A0 0=A0 =A0 272=A0 =A0 266=A0 =A0 =A0 0=A0 =A0 =A0 0=A0 =A0 =A0 0=0A=A0 = =A0 =A0 0=A0 =A0 =A0 0=A0 =A0 =A0 0=A0 =A0 167=A0 =A0 165=A0 =A0 =A0 0=A0 = =A0 =A0 0=A0 =A0 =A0 0=0A=0AI also tried runs with 15 simultaneous processe= s and 25. 15 processes gave only about a 5 second stall but 25 gave again = the same 75 second stall. =0A=0AFurther, I tested with 2 mounts to the sa= me server but from ZFS filesytems with the exact same stall/timeout periods= . So, it doesn't appear to matter what the underlying filesystem is - it's= something in NFS or networking code.=0A=0AAny ideas on what's going on her= e? What's causing the complete stall period of zero NFS activity? Any fl= aws with my testing methods?=0A=0AThanks for any and all help/ideas.=0A=0A-= -Alan=0A=0A=0A=0A=0A=0A