From owner-freebsd-net@FreeBSD.ORG Sun Dec 14 21:22:47 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9D0181065679 for ; Sun, 14 Dec 2008 21:22:47 +0000 (UTC) (envelope-from jason.dicioccio@ods.org) Received: from mail-ew0-f21.google.com (mail-ew0-f21.google.com [209.85.219.21]) by mx1.freebsd.org (Postfix) with ESMTP id 1A6F58FC0C for ; Sun, 14 Dec 2008 21:22:46 +0000 (UTC) (envelope-from jason.dicioccio@ods.org) Received: by ewy14 with SMTP id 14so3138486ewy.19 for ; Sun, 14 Dec 2008 13:22:46 -0800 (PST) Received: by 10.210.121.8 with SMTP id t8mr6846763ebc.180.1229288039564; Sun, 14 Dec 2008 12:53:59 -0800 (PST) Received: by 10.210.131.18 with HTTP; Sun, 14 Dec 2008 12:53:59 -0800 (PST) Message-ID: Date: Sun, 14 Dec 2008 12:53:59 -0800 From: "Jason DiCioccio" To: freebsd-net@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Steady leak of mbufs + panic() X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Dec 2008 21:22:47 -0000 Hey all, I'm running into a strange situation where every ~1.5 days, I will have a kernel panic: Architecture: i386 Architecture Version: 2 Version String: FreeBSD 7.0-RELEASE-p6 #1: Sun Dec 7 17:16:36 EST 2008 geniusj@update.ods.org:/usr/obj/usr/src/sys/UPDATE Panic String: kmem_malloc(16384): kmem_map too small: 536858624 total allocated When tracking down the source of the inflated kmem, I noticed that the mbufs/bytes allocated to network figures in netstat -m were growing steadily up until the crash. Before the crash, bytes allocated to network will be up near 400MB with mbufs reaching nearly 2 million. mbuf clusters will be in the low hundreds, however. This box is running a decent assortment of network protocols and functionality. It's a fairly active DNS server (more active than most), it is running a couple of instances of openvpn (tap), it's using gif for an ipv6 tunnel which is then tunneled over openvpn as well, it's also using BGP over the tunnels with quagga.. The only thing I can think of that changed before this all started happening is that I had added AAAA records for the nameserver (in the registry as well as locally). The inet6 traffic is fairly light, however (with a total of ~1.5MB both in and out in the past 15 hours versus 580MB in and 985MB out for the whole box.). Before I go down the bug report route, I'd like to know if there are accepted conditions where mbufs will grow like this and where I might start looking.. If more information would be helpful, let me know what's needed. Any help would be appreciated. Thanks! Jason DiCioccio