From owner-freebsd-ipfw@FreeBSD.ORG Thu Aug 12 19:30:25 2004 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E957616A4CE for ; Thu, 12 Aug 2004 19:30:25 +0000 (GMT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9048D43D48 for ; Thu, 12 Aug 2004 19:30:25 +0000 (GMT) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.12.11/8.12.11) with ESMTP id i7CJUPXt059590 for ; Thu, 12 Aug 2004 19:30:25 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.12.11/8.12.11/Submit) id i7CJUPYE059582; Thu, 12 Aug 2004 19:30:25 GMT (envelope-from gnats) Date: Thu, 12 Aug 2004 19:30:25 GMT Message-Id: <200408121930.i7CJUPYE059582@freefall.freebsd.org> To: ipfw@FreeBSD.org From: Pawel Malachowski Subject: Re: kern/46557: ipfw pipe show fails with lots of queues X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Pawel Malachowski List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Aug 2004 19:30:26 -0000 The following reply was made to PR kern/46557; it has been noted by GNATS. From: Pawel Malachowski To: freebsd-gnats-submit@FreeBSD.org, eugen@grosbein.pp.ru Cc: freebsd-bugs@freebsd.org Subject: Re: kern/46557: ipfw pipe show fails with lots of queues Date: Thu, 12 Aug 2004 16:10:20 +0200 Hello, I've just hit this problem on my 4.10-STABLE dummynet shaper. Very bad, since `ipfw pipe show N' + net.inet.ip.dummynet.expire=0 is quite useful for grabbing some per-user statistics. :/ Here are more details what is going on. I hope someone will look at this and explain why malloc() can fail here and what can be done to prevent this. % ipfw pipe show ipfw: getsockopt(IP_DUMMYNET_GET): No buffer space available This command invokes getsockopt() trying to fetch all pipes data from kernel to userland. Kernel part of this is being done in dummynet_get(), which tries to allocate buf big (previously computed size) enough for all data: buf = malloc(size, M_TEMP, M_NOWAIT); if (buf == 0) { splx(s); return ENOBUFS ; } This malloc() call fails sometimes on loaded system (10k-70k of dynamic pipes + make buildworld;)) causing `ipfw pipe show' command failure. I've registered temporary MALLOC_DEFINE M_YOYO and changed this malloc/free from M_TEMP to M_YOYO, ;) so I can easily track this in `vmstat -m' output: 1. Quite early (no problems): Type InUse MemUse HighUse Limit Requests Limit Limit Size(s) IpFw/IpAcct 113 14K 14K 42107K 113 0 0 64,128,256 yoyo 0 0K 120K 42107K 1 0 0 128K dummynet 1612 311K 313K 42107K 139366 0 0 16,128,256,4K Memory Totals: In Use Free Requests 3921K 32K 173824 2. After some time malloc() in dummynet_get() fails and: Type InUse MemUse HighUse Limit Requests Limit Limit Size(s) IpFw/IpAcct 113 14K 14K 42107K 113 0 0 64,128,256 yoyo 0 0K 2828K 42107K 12913 0 0 128K,512K dummynet 21432 2790K 3354K 42107K 9804136 0 0 16,128,256,512,1K,4K Memory Totals: In Use Free Requests 9932K 3029K 14216860 In /sbin/ipfw2, list(): (do_cmd(ocmd, data, (uintptr_t)&nbytes) < 0) nbytes were 2188288 // In loop, I +1024 numbytes instead of *2+200, ignore this In ip_dummynet.c, dummynet_get(): buf = malloc(size, M_YOYO, M_NOWAIT); // note: M_TEMP => M_YOYO size was 2390416 and malloc() failed. -- Paweł Małachowski