From owner-freebsd-net Thu Apr 4 14:10:22 2002 Delivered-To: freebsd-net@freebsd.org Received: from skippy.eecs.umich.edu (skippy.eecs.umich.edu [141.213.8.138]) by hub.freebsd.org (Postfix) with ESMTP id C74CE37B417 for ; Thu, 4 Apr 2002 14:10:10 -0800 (PST) Received: from skippy.eecs.umich.edu (localhost [127.0.0.1]) by skippy.eecs.umich.edu (8.11.6/8.11.6) with ESMTP id g34MAAR42207 for ; Thu, 4 Apr 2002 17:10:10 -0500 (EST) (envelope-from dwatson@skippy.eecs.umich.edu) Message-Id: <200204042210.g34MAAR42207@skippy.eecs.umich.edu> X-Mailer: exmh version 2.3.1 01/18/2001 with nmh-1.0.4 To: freebsd-net@freebsd.org Subject: Failure to set promiscuous correctly Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Thu, 04 Apr 2002 17:10:10 -0500 From: David Watson Sender: owner-freebsd-net@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org I'm experience a really weird condition with the bridging code. It looks like there is some race condition that causes an interface to look like it's in promiscuous mode when it really isn't. My setup has two Intel Gigabit cards with the Intel em driver. (The gx driver causes auto-negotiation to fail with the switch I'm using.) I'm running 4.5-RELEASE, with some custom modifications to the kernel. I have been able to verify the condition with a 4.5 GENERIC kernel which uses the wx drivers. Immediately after booting the first card (em0) is up and assigned an ip address (10.5.5.2), the second card is unmodified (no ip address assigned) and has the flags: em1: flags=0xffff8802 I then issue the following commands (from a file): sysctl -w net.link.ether.bridge=1 sysctl -w net.link.ether.bridge_ipfw=1 sysctl -w net.link.ether.bridge_cfg=em0:1,em1:1 and get the following output from my my instrumented ifpromisc(): em0: promiscuous mode enabled (dw) em0: ioctl succeeded! >> now em0 promisc ON if_flags 0xffff8943 bdg_flags 0x5 em1: promiscuous mode enabled (dw) em1: ioctl succeeded! em1: promiscuous counter = 2, promiscuous mode is already enabled >> now em1 promisc ON if_flags 0xffff8943 bdg_flags 0x5 >> now em1 promisc ON if_flags 0xffff8943 bdg_flags 0x5 em0: promiscuous mode disabled em0: ioctl succeeded! em0: promiscuous mode enabled (dw) em0: ioctl succeeded! >> now em0 promisc ON if_flags 0xffff8943 bdg_flags 0x5 em1: promiscuous counter = 2, promiscuous mode is already enabled >> now em1 promisc ON if_flags 0xffff8943 bdg_flags 0x5 em1 now has the flags: em1: flags=0xffff8943 but is _not_ really in promisc mode (i.e. bridging doesn't work, packets that I can verify are hitting the card never hit the interface statistics...). I can manually take the card out of promisc and back in using ioctl and things work just fine. It looks like one attempt to turn off promisc mode manages to turn it off on the card, but not in the flags and the promisc counter. Note that this doesn't always happen. Sometimes the interface truly ends up in promisc and bridging works just fine. Enabling the DEB macro in bridge.c seems to ensure that it works correctly. I thought I would fix the problem while I had the code torn apart but I can't think of the next place to look. dmesg and my instrumentation follows. David Watson dmesg: Copyright (c) 1992-2002 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD 4.5-RELEASE #26: Thu Apr 4 11:30:33 EST 2002 dwatson@skippy.eecs.umich.edu:/usr/home/dwatson/work/scrubber/new_scrub/fbsd_4.5/sys/compile/SCRUB Timecounter "i8254" frequency 1193182 Hz Timecounter "TSC" frequency 731023035 Hz CPU: Pentium III/Pentium III Xeon/Celeron (731.02-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0x683 Stepping = 3 Features=0x383fbff real memory = 1744814080 (1703920K bytes) Kernel map size: 2145390592, (2046 M) sio0: gdb debugging port avail memory = 382730240 (373760K bytes) Preloaded elf kernel "kernel" at 0x802ff000. Pentium Pro MTRR support enabled md0: Malloc disk npx0: on motherboard npx0: INT 16 interface pcib0: on motherboard pci0: on pcib0 pci0: at 1.0 pci0: (vendor=0x1022, dev=0x2000) at 2.0 irq 11 pci0: (vendor=0x8086, dev=0x1229) at 9.0 irq 15 isab0: at device 15.0 on pci0 isa0: on isab0 atapci0: port 0x840-0x84f at device 15.1 on pci0 ata0: at 0x1f0 irq 14 on atapci0 ata1: at 0x170 irq 15 on atapci0 pci0: at 15.2 irq 9 pcib1: on motherboard pci1: on pcib1 ahc0: port 0x4b00-0x4bff mem 0xeffff000-0xefffffff irq 10 at device 3.0 on pci1 aic7899: Ultra160 Wide Channel A, SCSI Id=7, 32/255 SCBs ahc1: port 0x4c00-0x4cff mem 0xefffe000-0xefffefff irq 10 at device 3.1 on pci1 aic7899: Ultra160 Wide Channel B, SCSI Id=7, 32/255 SCBs em0: mem 0xeffc0000-0xeffdffff irq 11 at device 5.0 on pci1 em0: Speed:1000 Mbps Duplex:Full em1: mem 0xeffa0000-0xeffbffff irq 15 at device 6.0 on pci1 em1: Speed:1000 Mbps Duplex:Full orm0: