From owner-freebsd-stable@FreeBSD.ORG Tue May 24 23:01:33 2005 Return-Path: X-Original-To: freebsd-stable@freebsd.org Delivered-To: freebsd-stable@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CF7E516A41C for ; Tue, 24 May 2005 23:01:33 +0000 (GMT) (envelope-from jmg@hydrogen.funkthat.com) Received: from mail27.sea5.speakeasy.net (mail27.sea5.speakeasy.net [69.17.117.29]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7DDA443D1D for ; Tue, 24 May 2005 23:01:33 +0000 (GMT) (envelope-from jmg@hydrogen.funkthat.com) Received: (qmail 25149 invoked from network); 24 May 2005 23:01:33 -0000 Received: from gate.funkthat.com (HELO hydrogen.funkthat.com) ([69.17.45.168]) (envelope-sender ) by mail27.sea5.speakeasy.net (qmail-ldap-1.03) with SMTP for ; 24 May 2005 23:01:32 -0000 Received: from hydrogen.funkthat.com (ezklyv@localhost.funkthat.com [127.0.0.1]) by hydrogen.funkthat.com (8.12.10/8.11.6) with ESMTP id j4ON1W2g022327; Tue, 24 May 2005 16:01:32 -0700 (PDT) (envelope-from jmg@hydrogen.funkthat.com) Received: (from jmg@localhost) by hydrogen.funkthat.com (8.12.10/8.12.10/Submit) id j4ON1Vv0022326; Tue, 24 May 2005 16:01:31 -0700 (PDT) Date: Tue, 24 May 2005 16:01:31 -0700 From: John-Mark Gurney To: Ed Maste Message-ID: <20050524230131.GN959@funkthat.com> Mail-Followup-To: Ed Maste , freebsd-stable@freebsd.org References: <20050524165907.GA20674@sandvine.com> <20050524173648.GA29183@sandvine.com> <20050524215153.GA35718@sandvine.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20050524215153.GA35718@sandvine.com> User-Agent: Mutt/1.4.1i X-Operating-System: FreeBSD 4.2-RELEASE i386 X-PGP-Fingerprint: B7 EC EF F8 AE ED A7 31 96 7A 22 B3 D8 56 36 F4 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html Cc: freebsd-stable@freebsd.org Subject: Re: libc_r kqueue fd leak X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: John-Mark Gurney List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 May 2005 23:01:34 -0000 Ed Maste wrote this message on Tue, May 24, 2005 at 17:51 -0400: > On Tue, May 24, 2005 at 01:36:48PM -0400, Ed Maste wrote: > > > On Tue, May 24, 2005 at 12:59:07PM -0400, Ed Maste wrote: > > > > > We discovered a kqueue leak when running one of our 4.x applications on > > > FreeBSD 5.3 using the compat libc_r. It turns out it's caused by libc_r's > > > close() failing. > > > > I've attached a patch which stops libc_r close() from bailing if fstat() > > returns an error. This fixes the kqueue leak. This logic would also have > > to make its way into the compat library via 4.x to fully resolve the issue. > > After a little more research it seems the kernel fix is already documented > in threads/75795. Version 1.77 of kern_event.c changed kqueue_stat to just > return ENXIO. > > We discovered the problem because each call in our app to getipnodebyname > (and in turn the resolver) leaked one kqueue, which eventually used up all > kernel memory and then caused a panic. > > I noticed that jmg originally did the kqueue locking work which removed > kqueue_stat, hence the CC:. yes, the reason I made _stat return ENXIO is that _read and _write are not supported by kqueue, and so _stat provided useless information. When I added locking, it would only be reading a value that would immediately be able to be changed, making it informational at best.. You'd better spend your syscall calling kevent and getting a few events off the queue than trying to figure out how much work you have to do... (In one of my programs, I have code that dynamicly increases the number of kevent structs I pull off if I get the max..) After being pestered by ps, I have created a patch.. after a quick test that it compiles and runs, I'll commit it... Though libc_r depending upon _stat seems broken to me... Who knows what else doesn't implement _stat and can't be closed.. -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not."