From owner-freebsd-fs@FreeBSD.ORG Tue May 1 22:20:17 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 8E96416A407; Tue, 1 May 2007 22:20:17 +0000 (UTC) (envelope-from kris@obsecurity.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 73F7413C447; Tue, 1 May 2007 22:20:17 +0000 (UTC) (envelope-from kris@obsecurity.org) Received: from obsecurity.dyndns.org (elvis.mu.org [192.203.228.196]) by elvis.mu.org (Postfix) with ESMTP id 533851A4D9E; Tue, 1 May 2007 15:20:50 -0700 (PDT) Received: by obsecurity.dyndns.org (Postfix, from userid 1000) id B013451451; Tue, 1 May 2007 18:20:16 -0400 (EDT) Date: Tue, 1 May 2007 18:20:16 -0400 From: Kris Kennaway To: Rick Macklem Message-ID: <20070501222016.GA6713@xor.obsecurity.org> References: <20070407191517.GN63916@garage.freebsd.pl> <20070407212413.GK8831@cicely12.cicely.de> <20070410003505.GA8189@nowhere> <46365F76.7090708@infidyne.com> <20070430213043.GF67738@garage.freebsd.pl> <463665F2.8090605@infidyne.com> <46373CAD.6000502@infidyne.com> <20070501160213.GA496@xor.obsecurity.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.2i Cc: freebsd-fs@freebsd.org, Craig Boston , freebsd-current@freebsd.org, Pawel Jakub Dawidek , Kris Kennaway Subject: Re: ZFS committed to the FreeBSD base. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 May 2007 22:20:17 -0000 On Tue, May 01, 2007 at 04:39:09PM -0400, Rick Macklem wrote: > > > On Tue, 1 May 2007, Kris Kennaway wrote: > >>I don't know if it relevent, but I've seen "kmem_map: too small" panics > >>when testing my NFSv4 server, ever since about FreeBSD5.4. There is no > >>problem running the same server code on FreeBSD4 (which is what I still > >>run in production mode) or OpenBSD3 or 4. If I increase the size of the > >>map, I can delay the panic for up to about two weeks of hard testing, > >>but it never goes away. I don't see any evidence of a memory leak during > >>the several days of testing leading up to the panic. (NFSv4 uses > >>MALLOC/FREE extensively for state related structures.) > > > >Sounds exactly like a memory leak to me. How did you rule it out? > Well, I had a little program running on the server that grabbed the > mti_stats[] out of the kernel and logged them. I had one client mounted > running thousands of passes of the Connectathon basic tests (one client, > same activity over and over and over again). For a week, the stats don't > show any increase in allocation for any type (alloc - free doesn't get > unreasonably big), then..."panic: kmem_map too small". How many days it > took to happen would vary with the size of the kernel map, but no evidence > of a leak prior to the crash. It seemed to be based on the number of times > MALLOC and FREE were called. Or something else is leaking. Really, if there was a problem with MALLOC/FREE we'd see it. Kris