From owner-freebsd-stable@FreeBSD.ORG Wed Mar 22 00:49:21 2006 Return-Path: X-Original-To: stable@freebsd.org Delivered-To: freebsd-stable@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5829A16A426; Wed, 22 Mar 2006 00:49:21 +0000 (UTC) (envelope-from mi+mx@aldan.algebra.com) Received: from zig.murex.com (mail.murex.com [194.98.239.11]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0D5BB43D53; Wed, 22 Mar 2006 00:49:15 +0000 (GMT) (envelope-from mi+mx@aldan.algebra.com) Received: from interscan.fr.murex.com (interscan.fr.murex.com [172.21.17.207] (may be forged)) by zig.murex.com with ESMTP id k2M0owJQ004467; Wed, 22 Mar 2006 01:50:58 +0100 (CET) Received: from mxmail.murex.com (interscan.murex.fr [127.0.0.1]) by interscan.fr.murex.com (8.11.6/8.11.6) with ESMTP id k2M1JGc23011; Wed, 22 Mar 2006 02:19:16 +0100 Received: from mteterin.us.murex.com ([172.21.130.86]) by mxmail.murex.com with Microsoft SMTPSVC(6.0.3790.0); Wed, 22 Mar 2006 01:48:32 +0100 From: Mikhail Teterin Organization: Virtual Estates, Inc. To: Matthew Dillon Date: Tue, 21 Mar 2006 19:48:27 -0500 User-Agent: KMail/1.8.3 References: <200603211607.30372.mi+mx@aldan.algebra.com> <200603211858.02801.mi+mx@aldan.algebra.com> <200603220025.k2M0PmCt007240@apollo.backplane.com> In-Reply-To: <200603220025.k2M0PmCt007240@apollo.backplane.com> MIME-Version: 1.0 Content-Type: text/plain; charset="koi8-u" Content-Transfer-Encoding: 8bit Content-Disposition: inline Message-Id: <200603211948.28178.mi+mx@aldan.algebra.com> X-OriginalArrivalTime: 22 Mar 2006 00:48:32.0876 (UTC) FILETIME=[582ADAC0:01C64D4A] Cc: alc@freebsd.org, stable@freebsd.org Subject: Re: more weird bugs with mmap-ing via NFS X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Mar 2006 00:49:21 -0000 צ×ÔÏÒÏË 21 ÂÅÒÅÚÅÎØ 2006 19:25, Matthew Dillon ÷É ÎÁÐÉÓÁÌÉ: > If the program works over a local > š š filesystem but fails to produce data in the output file on an NFS > š š mount (but completes otherwise), then there is a bug in NFS somewhere. > š š If the problem is simply due to the program stalling, and not > completing due to the stalling, then it could be a problem with dropped > packets in the network stack. The file stops growing, but the network bandwidth remains at 20Mb/s. `Netstat -s' on the client, had the following to say (udp and ip only): udp: 20417616 datagrams received 0 with incomplete header 0 with bad data length field 0 with bad checksum 4 with no checksum 14 dropped due to no socket 89 broadcast/multicast datagrams dropped due to no socket 0 dropped due to full socket buffers 0 not for hashed pcb 20417513 delivered 20409284 datagrams output and ip: 45653354 total packets received 0 bad header checksums 0 with size smaller than minimum 0 with data size < data length 0 with ip length > max ip packet size 0 with header length < data size 0 with data length < header length 0 with bad options 0 with incorrect version number 210876 fragments received 0 fragments dropped (dup or out of space) 0 fragments dropped after timeout 32712 packets reassembled ok 45474799 packets for this host 135 packets for unknown/unsupported protocol 0 packets forwarded (0 packets fast forwarded) 256 packets not forwardable 0 packets received for unknown multicast group 0 redirects sent 43338317 packets sent from this host 0 packets sent with fabricated ip header 0 output packets dropped due to no bufs, etc. 0 output packets discarded due to no route 18604565 output datagrams fragmented 164573586 fragments created 0 datagrams that can't be fragmented 0 tunneling packets that can't find gif 0 datagrams with bad address in header Nothing drastically alarming on the server either. Is this bad: ip: [...] 3241 fragments dropped (dup or out of space) I don't think so -- the client was writing at 20Mb/s since the beginning of this thread... > If the problem is that the program simply runs very inefficiently over > NFS, with excessive network bandwidth for the data being written (as you > also reported), this is probably an artifact of attempting to use mmap() > to write out the data, for reasons previously discussed. The reasons previously discussed could excuse the difference between actual and expected performance of no more than an order of (decimal) magnitude. I'm seeing actual bandwidth use of 20Mb/second. Even if the useful part of that were mere 1% (TWO orders of magnitude), the file would be growing at about 200Kb/s. It is not... The stalling caused by `systat -vm' can not be excused by mmap ineficiencies at all, can it? > I would again caution against using mmap() to populate a file in this > manner. Even with MADV_SEQUENTIAL there is no guarentee that the system > will actually flush the pages to the actual file on the server > sequentially, and you could end up with a very badly fragmented file. > [...] The VM system does its best, but I'd argue, that this does not seem like "best" at all, but I, actually, do not care in this example -- the backups are never read again in normal circumstances. In the infrequent abnormal ones, they will be uncompressed first (a CPU- rather than I/O-intensive process) and only then used for recovery. Yours, -mi