From owner-freebsd-net@FreeBSD.ORG Thu Mar 31 17:14:19 2011 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C41161065672; Thu, 31 Mar 2011 17:14:19 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 699F78FC14; Thu, 31 Mar 2011 17:14:19 +0000 (UTC) Received: by iwn33 with SMTP id 33so3247556iwn.13 for ; Thu, 31 Mar 2011 10:14:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:from:date:to:cc:subject:message-id:reply-to :references:mime-version:content-type:content-disposition :in-reply-to:user-agent; bh=u13DP9GH/GfVlMoG7Q5+eLJlyoAg27Cv7nvkf1SPHh8=; b=Ofz6TvhR7aEHwjQPK45Gn7hQGIOBRQXzD7fbA9+Mvz1ZXwgqydI5xJ+Nqa/EqjoLb2 ARfQilG0SCqnRxZNkhGyLct/IpI7u7/XndDlVrYgjKCvI+czwA2BNQwtNiVQO9igcuMb tZS5f6KzrxdJZ3nWrcNV4HkM/XjHa0SWCNVj4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; b=I4FSql0ZmbWfG1b3f2ZzknUBtNDk+KpJFWS7TJ3AYyXVi1fvwSNaYIITHr4+czBxBe ibLn04rNDiuqh6vo9EnyYhFh8/ZvNa32EL5Z5yQ41s5NTe+htFAieHoA18moUuxDmAsy dELGEMMDVftp+dQ+7odPCYS1WNeJu9ln2i06Y= Received: by 10.42.161.7 with SMTP id r7mr3665259icx.228.1301591658653; Thu, 31 Mar 2011 10:14:18 -0700 (PDT) Received: from pyunyh@gmail.com ([174.35.1.224]) by mx.google.com with ESMTPS id uk4sm736345icb.9.2011.03.31.10.14.14 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 31 Mar 2011 10:14:16 -0700 (PDT) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Thu, 31 Mar 2011 10:13:03 -0700 From: YongHyeon PYUN Date: Thu, 31 Mar 2011 10:13:03 -0700 To: Yamagi Burmeister Message-ID: <20110331171302.GA11981@michelle.cdnetworks.com> References: <20110330173145.GB8601@michelle.cdnetworks.com> <20110330202858.GC8601@michelle.cdnetworks.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="7AUc2qLy4jB3hD7Z" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i Cc: freebsd-net@freebsd.org, yongari@freebsd.org Subject: Re: Kernel memory corruption(?) with age(4) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 Mar 2011 17:14:19 -0000 --7AUc2qLy4jB3hD7Z Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Thu, Mar 31, 2011 at 09:05:19AM +0200, Yamagi Burmeister wrote: > On Wed, 30 Mar 2011, YongHyeon PYUN wrote: > > >>Okay, I did a test run with RX checksum, TX checksum and both disabled. > >>In all three cases the crash occurs within about 20 minutes. I'm either > >>not sure that age(4) is the problem but it has definedly something to do > >>with the problem, since with another nic driver the same scenario is > >>rock solid... > >> > > > >OK. > > > >>The workload: It's a NFS3 server (FreeBSDs non-experimental > >>implementation), serving and receiving file with about 250 to 500 > >>megabytes at about 20mb/s. The clients are FreeBSD 7 and 8 systems and > >>are mounting the shares via TCP. The connection is 1000mbit/s via a > >>"dumb" gigabit switch. > >> > > > >That's too broad to narrow down the issue. :-( > >I'm not sure but your box seem to have more than 4GB memory. Could > >you limit the available memory to 3GB via loader.conf and test it > >again? > > All boxes are quadcore machines with 8GB RAM, running FreeBSD/amd64. > After limiting the memory via hw.physmem to 3GB the problems are gone. > The box is running crashfree for more than 6 hours and has served over > 300GB of data via age(4). > Thanks for testing. Remove the hw.physmem configuration and try attached patch and let me know how it goes. --7AUc2qLy4jB3hD7Z Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="age.dma.diff" Index: sys/dev/age/if_age.c =================================================================== --- sys/dev/age/if_age.c (revision 220116) +++ sys/dev/age/if_age.c (working copy) @@ -2452,6 +2452,9 @@ /* Update the consumer index. */ sc->age_cdata.age_rr_cons = rr_cons; + bus_dmamap_sync(sc->age_cdata.age_rx_ring_tag, + sc->age_cdata.age_rx_ring_map, + BUS_DMASYNC_PREREAD | BUS_DMASYNC_PREWRITE); /* Sync descriptors. */ bus_dmamap_sync(sc->age_cdata.age_rr_ring_tag, sc->age_cdata.age_rr_ring_map, --7AUc2qLy4jB3hD7Z--