From owner-freebsd-arm@FreeBSD.ORG Sat Mar 14 14:00:22 2015 Return-Path: Delivered-To: freebsd-arm@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 11F7F6C9; Sat, 14 Mar 2015 14:00:22 +0000 (UTC) Received: from raven.bwct.de (raven.bwct.de [85.159.14.73]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "raven.bwct.de", Issuer "BWCT" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id B3D153EB; Sat, 14 Mar 2015 14:00:21 +0000 (UTC) Received: from mail.cicely.de ([10.1.1.37]) by raven.bwct.de (8.13.4/8.13.4) with ESMTP id t2EDxv9m032137 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL); Sat, 14 Mar 2015 14:59:57 +0100 (CET) (envelope-from ticso@cicely7.cicely.de) Received: from cicely7.cicely.de (cicely7.cicely.de [10.1.1.9]) by mail.cicely.de (8.14.5/8.14.4) with ESMTP id t2EDxtue010284 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 14 Mar 2015 14:59:55 +0100 (CET) (envelope-from ticso@cicely7.cicely.de) Received: from cicely7.cicely.de (localhost [127.0.0.1]) by cicely7.cicely.de (8.14.2/8.14.2) with ESMTP id t2EDxtNm041537; Sat, 14 Mar 2015 14:59:55 +0100 (CET) (envelope-from ticso@cicely7.cicely.de) Received: (from ticso@localhost) by cicely7.cicely.de (8.14.2/8.14.2/Submit) id t2EDxsKT041536; Sat, 14 Mar 2015 14:59:54 +0100 (CET) (envelope-from ticso) Date: Sat, 14 Mar 2015 14:59:54 +0100 From: Bernd Walter To: Ian Lepore Subject: Re: BeagleBone slow inbound net I/O Message-ID: <20150314135954.GD40951@cicely7.cicely.de> Reply-To: ticso@cicely.de References: <20150311165115.32327c5a@ivory.wynn.com> <89CEBFCA-6B94-4F48-8DFD-790E4667632D@kientzle.com> <20150314031542.439cdee3@ivory.wynn.com> <1426339400.52318.3.camel@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1426339400.52318.3.camel@freebsd.org> X-Operating-System: FreeBSD cicely7.cicely.de 7.0-STABLE i386 User-Agent: Mutt/1.5.11 X-Spam-Status: No, score=-0.2 required=5.0 tests=ALL_TRUSTED=-1, BAYES_00=-1.9, DNS_FROM_AHBL_RHSBL=2.699, T_RP_MATCHES_RCVD=-0.01 autolearn=no version=3.3.0 X-Spam-Checker-Version: SpamAssassin 3.3.0 (2010-01-18) on spamd.cicely.de Cc: freebsd-arm , Tim Kientzle X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Mar 2015 14:00:22 -0000 On Sat, Mar 14, 2015 at 07:23:20AM -0600, Ian Lepore wrote: > On Sat, 2015-03-14 at 03:15 -0400, Brett Wynkoop wrote: > > On Fri, 13 Mar 2015 23:02:25 -0700 > > Tim Kientzle wrote: > > > > > > > > > On Mar 11, 2015, at 1:51 PM, Brett Wynkoop > > > > wrote: > > > > > > > > Have I managed to find a network driver issue? Any ideas how to > > > > gather more information to help get to the bottom of things? > > > > > > > > > > $ sysctl dev.cpsw > > > > > > This will dump detailed statistics from the Ethernet hardware and > > > driver. > > > > > > Tim > > > > > > > After a short time while doing nfs i/o > > > > > > [wynkoop@beaglebone ~]$ sysctl dev.cpsw | grep -i error > > dev.cpsw.0.stats.RxCrcErrors: 40 > > dev.cpsw.0.stats.RxAlignErrors: 32 > > dev.cpsw.0.stats.CarrierSenseErrors: 0 > [...] > > [wynkoop@beaglebone ~]$ sysctl dev.cpsw | grep -i error > > dev.cpsw.0.stats.RxCrcErrors: 262 > > dev.cpsw.0.stats.RxAlignErrors: 231 > > dev.cpsw.0.stats.CarrierSenseErrors: 0 > > [wynkoop@beaglebone ~]$ > > > > So we can see climbing errors. I am not sure how this compares to the > > results of others. The above was during the first few minutes of a > > buildworld from an nfs share. > > > > At the same time on the console: > > > > Mar 14 03:07:47 beaglebone amd[1163]: mountd rpc failed: RPC: Can't > > decode result Mar 14 03:11:48 beaglebone amd[1399]: mountd rpc failed: > > RPC: Can't decode result > > > > which makes sense with the above errors I think. It doesn't make sense with the ethernet CRC alone, since ethernet CRC failures are basicly dropped packets. If the RPC answer can't be parsed then ethernet packet CRC was Ok, when verified by the MAC and corrupted later. > On mine: > > root@bb:/usr/ports/benchmarks/iperf # sysctl dev.cpsw | grep Err > dev.cpsw.0.stats.RxCrcErrors: 0 > dev.cpsw.0.stats.RxAlignErrors: 0 > dev.cpsw.0.stats.CarrierSenseErrors: 0 > > That's after 3 days of uptime including doing builds over nfs, and all > the iperf testing I was doing yesterday (no errors after megabytes of > transfers). > > I wonder if your power supply is failing and injecting transient > glitches under heavy load or something? Power seems likely. The RPC error and filesystem corruptions make sense with broken RAM too, but the ethernet CRC should be checked during transfer to RAM. The subsystems with failures are very different and mostly isolated from software and HW-logic. About ZFS: It is designed to handle data corruption to some degree and has an extremly different workload, so possible that it works by luck. -- B.Walter http://www.bwct.de Modbus/TCP Ethernet I/O Baugruppen, ARM basierte FreeBSD Rechner uvm.