From owner-freebsd-stable@FreeBSD.ORG Wed Jul 5 16:41:55 2006 Return-Path: X-Original-To: freebsd-stable@freebsd.org Delivered-To: freebsd-stable@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B53E116A4DF for ; Wed, 5 Jul 2006 16:41:55 +0000 (UTC) (envelope-from lists@niamodnikufesin.de) Received: from mx.does.not-exist.de (mx.does.not-exist.de [62.159.88.14]) by mx1.FreeBSD.org (Postfix) with ESMTP id B3F6343D69 for ; Wed, 5 Jul 2006 16:41:49 +0000 (GMT) (envelope-from lists@niamodnikufesin.de) Received: from rw.does.not-exist.de ([10.42.23.2]) by mx.does.not-exist.de with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.52 (FreeBSD)) id 1FyAQq-000AsB-MJ; Wed, 05 Jul 2006 18:40:57 +0200 Received: from hank by rw.does.not-exist.de with local (Exim 4.52 (FreeBSD)) id 1FyAQs-0001N1-ET; Wed, 05 Jul 2006 18:40:58 +0200 Date: Wed, 5 Jul 2006 18:40:58 +0200 From: Hank Hampel To: freebsd-stable@freebsd.org Message-ID: <20060705164058.GA5194@warning.this.domain.does.not-exist.de> Mail-Followup-To: freebsd-stable@freebsd.org Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="jI8keyz6grp/JLjh" Content-Disposition: inline User-Agent: Mutt/1.4.2.1i X-fcc-folder: freebsd-stable-list Subject: 5.5-stable network interface rl0 stops working X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jul 2006 16:41:55 -0000 --jI8keyz6grp/JLjh Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hello everybody, I have a very disturbing problem with one of our FreeBSD 5.5-stable machines. It is a box on which ~10 jail systems run, each with small to moderate network traffic. Now from time to time - sometimes after a few days, sometimes after a couple of weeks - the network interface rl0 (which is the main interface on the maschine, rl1 is for backups/internal use only) stops working. Each jailed system has its own firewall ruleset, permitting only traffic for the services in that specific jail. The packet filter used is ipfw. Some of the rules are stateful (keep-state). When rl0 stops working ipfw loggs lots of denied packets so that it seems that the dynamic (keep-state) rules don't work any longer. We checked and increased the buffers for the dynamic rules to no avail - I doubt they are part of the problem. I'm not even sure ipfw is part of the problem. After the stop on the interface occurs there is no other way to get the interface up and running again than rebooting the whole machine. Restarting /etc/rc.d/netif, the jails or ipfw doesn't help anything. The bad thing is I haven't found any way to trigger this problem so that I can only check and change things and wait if the situation improves or not. For example I've already set debug.mpsafenet="0" but this doesn't help, in contrast it seems to worsen the problem a little bit. Find attached the dmesg output of the machine. If any other information is needed to hunt down the cause of this problem please let me know. I checked various list archives but haven't found a clue yet. -------------------------[ dmesg ]------------------------- Copyright (c) 1992-2006 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD 5.5-STABLE #5: Tue May 30 13:51:55 CEST 2006 root@shawshank.nr-city.net:/usr/obj/usr/src/sys/SHAWSHANK WARNING: MPSAFE network stack disabled, expect reduced performance. Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Intel(R) Pentium(R) 4 CPU 2.40GHz (2411.60-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0xf34 Stepping = 4 Features=0xbfebfbff real memory = 2147418112 (2047 MB) avail memory = 2096037888 (1998 MB) ACPI APIC Table: ioapic0 irqs 0-23 on motherboard npx0: on motherboard npx0: INT 16 interface acpi0: on motherboard acpi0: Power Button (fixed) Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0 cpu0: on acpi0 acpi_button0: on acpi0 pcib0: port 0x1000-0x10bf,0xcf8-0xcff on acpi0 pci0: on pcib0 agp0: mem 0xe8000000-0xefffffff at device 0.0 on pci0 pcib1: at device 1.0 on pci0 pci1: on pcib1 pcib2: at device 30.0 on pci0 pci2: on pcib2 pci2: at device 0.0 (no driver attached) rl0: port 0x9000-0x90ff mem 0xf5000000-0xf50000ff irq 21 at device 1.0 on pci2 miibus0: on rl0 rlphy0: on miibus0 rlphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto rl0: Ethernet address: 00:02:2a:d5:39:74 rl1: port 0x9400-0x94ff mem 0xf5001000-0xf50010ff irq 22 at device 2.0 on pci2 miibus1: on rl1 rlphy1: on miibus1 rlphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto rl1: Ethernet address: 00:02:2a:d5:39:53 isab0: at device 31.0 on pci0 isa0: on isab0 atapci0: port 0xf000-0xf00f,0x376,0x170-0x177,0x3f6,0x1f0-0x1f7 at device 31.1 on pci0 ata0: channel #0 on atapci0 ata1: channel #1 on atapci0 pci0: at device 31.3 (no driver attached) acpi_tz0: on acpi0 sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 sio0: type 16550A, console sio1: <16550A-compatible COM port> port 0x2f8-0x2ff irq 3 on acpi0 sio1: type 16550A pmtimer0 on isa0 orm0: at iomem 0xc0000-0xc7fff on isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x100> vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 atkbdc0: at port 0x64,0x60 on isa0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 ppc0: parallel port not found. Timecounter "TSC" frequency 2411601876 Hz quality 800 Timecounters tick every 10.000 msec ipfw2 initialized, divert disabled, rule-based forwarding disabled, default to deny, logging disabled ad0: 114497MB [232629/16/63] at ata0-master UDMA100 acd0: DVDROM at ata1-master PIO4 Mounting root from ufs:/dev/ad0s1a -------------------------[ dmesg ]------------------------- Best regards, Hank --jI8keyz6grp/JLjh Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (FreeBSD) iD8DBQFEq+uaXSKk5/a79toRAj8sAJ94aRlBwuxaWHIadN52uhULfom3EQCgmzYJ S1lUdr6+ZSyF2pBJe50bSio= =iZKr -----END PGP SIGNATURE----- --jI8keyz6grp/JLjh--