From owner-freebsd-net@FreeBSD.ORG Tue Apr 25 14:33:35 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 70BE116A401 for ; Tue, 25 Apr 2006 14:33:35 +0000 (UTC) (envelope-from pblok@bsd4all.org) Received: from altrade.nijmegen.internl.net (altrade.nijmegen.internl.net [217.149.192.18]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7987143D46 for ; Tue, 25 Apr 2006 14:33:34 +0000 (GMT) (envelope-from pblok@bsd4all.org) Received: from mail.bsd4all.org by altrade.nijmegen.internl.net via 113-9.bbned.dsl.internl.net [82.215.9.113] with ESMTP id k3PEXWwi018850 (8.13.2/2.04); Tue, 25 Apr 2006 16:33:33 +0200 (MET DST) Received: from localhost (localhost.homebrew.bsd4all.org [127.0.0.1]) by mail.bsd4all.org (Postfix) with ESMTP id 4AD055D04; Tue, 25 Apr 2006 16:33:32 +0200 (CEST) X-Virus-Scanned: amavisd-new at bsd4all.org Received: from mail.bsd4all.org ([127.0.0.1]) by localhost (fwgw.homebrew.bsd4all.org [127.0.0.1]) (amavisd-new, port 10024) with LMTP id BbPSy9w+RavW; Tue, 25 Apr 2006 16:33:22 +0200 (CEST) Received: from ntpc (ntpc [192.168.1.138]) by mail.bsd4all.org (Postfix) with ESMTP id D0D6D5C5D; Tue, 25 Apr 2006 16:33:22 +0200 (CEST) From: "Peter Blok" To: "'Vlad GALU'" , Date: Tue, 25 Apr 2006 16:29:23 +0200 Message-ID: <002701c66874$a5fe2870$8a01a8c0@ntpc> MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook 11 In-Reply-To: <79722fad0604240350g602a175al59a926c9ddde9176@mail.gmail.com> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2869 Thread-Index: AcZnt5OfcEQzOIkIRuW0+ynRDRMiYwAvAFNA Cc: Subject: RE: requests for mbufs denied X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Apr 2006 14:33:35 -0000 Hi, I have tried to debug this and it turns out that it is not an allocation failure at all. It happens as part of uma_reclaim, which will eventually call uma_zfree_internal which increments the counters. When I use the following patch, uma_reclaim will skip uma_zfree_internal for both mbuf and mcluster zones. It is my first impression that the size of the two zones are static anyway. I am not sure this is a correct fix, but it works in my case pending further investigation. --- sys/kern/kern_mbuf.c.orig Sun Apr 9 13:32:51 2006 +++ sys/kern/kern_mbuf.c Sun Apr 9 13:33:19 2006 @@ -168,7 +168,7 @@ #else NULL, NULL, #endif - MSIZE - 1, UMA_ZONE_MAXBUCKET); + MSIZE - 1, UMA_ZONE_MAXBUCKET|UMA_ZONE_NOFREE); zone_clust = uma_zcreate(MBUF_CLUSTER_MEM_NAME, MCLBYTES, mb_ctor_clust, mb_dtor_clust, @@ -177,7 +177,7 @@ #else NULL, NULL, #endif - UMA_ALIGN_PTR, UMA_ZONE_REFCNT); + UMA_ALIGN_PTR, UMA_ZONE_REFCNT|UMA_ZONE_NOFREE); if (nmbclusters > 0) uma_zone_set_max(zone_clust, nmbclusters); ~ -----Original Message----- From: owner-freebsd-net@freebsd.org [mailto:owner-freebsd-net@freebsd.org] On Behalf Of Vlad GALU Sent: Monday, April 24, 2006 12:51 PM To: freebsd-net@freebsd.org Subject: requests for mbufs denied The machine in question is a 6.1-RC. It serves a quite big number of clients (the lowest concurrency figures are around 2000, with peaks up to 9000). The in/out buffers for tcp sockets are 8K each. kern.ipc.nmbclusters is set to 327680. The firewall is pf, with the following limits: 131072 states and 262144 src-nodes. So more than generous limits. However, $subj statistics keep growing and growing. The machine has rebooted a few times, without dumping any core upon restart, although debugging support (both symbols and kgdb) is compiled in. Should I worry about the aforementioned stats ? If so, any idea of how to narrow the issue down ? I can't test much on the machine, unfortunately. P.S. I've the feeling that the growth rate of allocation failures went downhill since I removed the pfsync support, but it's just a feeling, I didn't measure it accurately. -- If it's there, and you can see it, it's real. If it's not there, and you can see it, it's virtual. If it's there, and you can't see it, it's transparent. If it's not there, and you can't see it, you erased it. _______________________________________________ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"