From owner-freebsd-bugs@FreeBSD.ORG Thu Mar 17 22:40:03 2005 Return-Path: Delivered-To: freebsd-bugs@hub.freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8AC4D16A4CE for ; Thu, 17 Mar 2005 22:40:03 +0000 (GMT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 20A1743D5E for ; Thu, 17 Mar 2005 22:40:03 +0000 (GMT) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.13.3/8.13.3) with ESMTP id j2HMe2CP012493 for ; Thu, 17 Mar 2005 22:40:02 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.13.3/8.13.1/Submit) id j2HMe212012492; Thu, 17 Mar 2005 22:40:02 GMT (envelope-from gnats) Resent-Date: Thu, 17 Mar 2005 22:40:02 GMT Resent-Message-Id: <200503172240.j2HMe212012492@freefall.freebsd.org> Resent-From: FreeBSD-gnats-submit@FreeBSD.org (GNATS Filer) Resent-To: freebsd-bugs@FreeBSD.org Resent-Reply-To: FreeBSD-gnats-submit@FreeBSD.org, Jakub Kruszona-Zawadzki Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 81AA616A4CE for ; Thu, 17 Mar 2005 22:38:40 +0000 (GMT) Received: from www.freebsd.org (www.freebsd.org [216.136.204.117]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3A62F43D49 for ; Thu, 17 Mar 2005 22:38:40 +0000 (GMT) (envelope-from nobody@FreeBSD.org) Received: from www.freebsd.org (localhost [127.0.0.1]) by www.freebsd.org (8.13.1/8.13.1) with ESMTP id j2HMceFf082606 for ; Thu, 17 Mar 2005 22:38:40 GMT (envelope-from nobody@www.freebsd.org) Received: (from nobody@localhost) by www.freebsd.org (8.13.1/8.13.1/Submit) id j2HMce15082605; Thu, 17 Mar 2005 22:38:40 GMT (envelope-from nobody) Message-Id: <200503172238.j2HMce15082605@www.freebsd.org> Date: Thu, 17 Mar 2005 22:38:40 GMT From: Jakub Kruszona-Zawadzki To: freebsd-gnats-submit@FreeBSD.org X-Send-Pr-Version: www-2.3 Subject: kern/78968: FreeBSD freezes on mbufs exhaustion (network interface independent) X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Mar 2005 22:40:03 -0000 >Number: 78968 >Category: kern >Synopsis: FreeBSD freezes on mbufs exhaustion (network interface independent) >Confidential: no >Severity: critical >Priority: high >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Thu Mar 17 22:40:02 GMT 2005 >Closed-Date: >Last-Modified: >Originator: Jakub Kruszona-Zawadzki >Release: FreeBSD 5.3 >Organization: Gemius SA >Environment: FreeBSD router.gucio.pl 5.3-STABLE FreeBSD 5.3-STABLE #4: Sun Dec 5 04:53:26 CET 2004 root@router.gucio.pl:/usr/obj/usr/src/sys/GENERIC i386 >Description: We are using all of our output bandwidth. Tcp stack allocates mbufs for output queues. When mbufs space is exhausted system freezes. Even 'numlock' on keyboard don't work. It looks like kernel is going into an infinite loop or something - system is dead and the only thing you can do is reboot. >How-To-Repeat: Set max mbufs to a small number (only for testing). Generate a file about 300kB (i.e. dd if=/dev/urandom of=/tmp/file bs=1024 count=300). Add to inetd.conf line like this: telnet stream tcp nowait root /bin/cat cat /tmp/file Restart inetd with these options (maybe not all are necessary): inetd -c 0 -C 0 -R 0 -s 0 On remote machine run many processes that will open many connections to your serwer on telent port, but WILL NOT download the file. You can use these programs: In python: #!/usr/bin/env python from socket import socket from time import sleep socks = [] for i in xrange(240): try: s = socket() s.connect(('192.168.0.1',23)) socks.append(s) except Exception: pass print "opened" sleep(100) for i in socks: i.close() In php: FreeBSD 5.3 with default number of mbufs (about 17000) freezes after a few minutes, when both these programs work in several (6 pythons an 15 phps) copies. While these programs are active, "netstat -n" should be showing many "ESTABLISHED" connections with full output queue. In "netstat -m" output number of "mbufs in use" and "mbuf clusters in use" should increase. When current number of "mbuf clusters in use" reaches max value, system freezes in a few seconds. During normal work "netstat -m" looks like this: 334 mbufs in use 198/17088 mbuf clusters in use (current/max) 0/3/4528 sfbufs in use (current/peak/max) 479 KBytes allocated to network 0 requests for sfbufs denied 0 requests for sfbufs delayed 0 requests for I/O initiated by sendfile 104 calls to protocol drain routines >Fix: This problem doesn't exist in FreeBSD 4.x >Release-Note: >Audit-Trail: >Unformatted: