Skip site navigation (1)Skip section navigation (2)
Date:      Wed, 18 Nov 2009 16:30:27 -0500
From:      Mike Tancsa <mike@sentex.net>
To:        freebsd-stable@freebsd.org
Cc:        jfvogel@gmail.com
Subject:   bug with some em nics on RELENG_7
Message-ID:  <200911182130.nAILUJCR089766@lava.sentex.ca>

next in thread | raw e-mail | index | archive | help

On two Intel chipset Supermicro boards (X8STi and X8STE-0) using the 
onboard em nics (dmesg info below), I seem to have run into an issue 
where if I boot the box up with the cables unplugged, I cannot get 
the NICS to properly work post boot up.  This is quite repeatable for 
me. So at boot time, I have

# ifconfig em5
em5: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
         options=19b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4>
         ether 00:30:48:d6:ef:13
         inet 3.3.3.3 netmask 0xffffff00 broadcast 3.3.3.255
         media: Ethernet autoselect
         status: no carrier


I then ping something that would be across the wire while the nic is 
down. e.g. ping 3.3.3.1

I then plug in the cable so that the other side has 3.3.3.1

ifconfig shows all looks good

# ifconfig em5
em5: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
         options=19b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4>
         ether 00:30:48:d6:ef:13
         inet 3.3.3.3 netmask 0xffffff00 broadcast 3.3.3.255
         media: Ethernet autoselect (1000baseTX <full-duplex>)
         status: active

I try and ping 3.3.3.1 which is on xover (via a switch shows the same 
behaviour), and no response to the pings.... BUT, I do see the MAC addr show up
# ping -c 2 -S 3.3.3.3 3.3.3.1
PING 3.3.3.1 (3.3.3.1) from 3.3.3.3: 56 data bytes

--- 3.3.3.1 ping statistics ---
2 packets transmitted, 0 packets received, 100.0% packet loss
# arp -na
? (3.3.3.1) at 00:30:48:94:88:20 on em5 [ethernet]
? (3.3.3.3) at 00:30:48:d6:ef:13 on em5 permanent [ethernet]

I can see its mac addr ?!?

Furthermore, if I do

# ifconfig em5 3.3.3.55/32 alias

On the other side, I see

0(ich10)# tcpdump -nei igb0
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on igb0, link-type EN10MB (Ethernet), capture size 96 bytes
16:16:03.380886 00:30:48:d6:ef:13 > ff:ff:ff:ff:ff:ff, ethertype ARP 
(0x0806), length 60: Request who-has 3.3.3.55 tell 3.3.3.55, length 46


and I can ping if I specify the alias as the source IP

# ping -S 3.3.3.55 3.3.3.1
PING 3.3.3.1 (3.3.3.1) from 3.3.3.55: 56 data bytes
64 bytes from 3.3.3.1: icmp_seq=0 ttl=64 time=0.184 ms
64 bytes from 3.3.3.1: icmp_seq=1 ttl=64 time=0.051 ms
64 bytes from 3.3.3.1: icmp_seq=2 ttl=64 time=0.055 ms



16:17:01.603345 00:30:48:d6:ef:13 > 00:30:48:94:88:20, ethertype ARP 
(0x0806), length 60: Reply 3.3.3.55 is-at 00:30:48:d6:ef:13, length 46
16:17:01.603349 00:30:48:94:88:20 > 00:30:48:d6:ef:13, ethertype IPv4 
(0x0800), length 98: 3.3.3.1 > 3.3.3.55: ICMP echo reply, id 7946, 
seq 0, length 64
16:17:02.603497 00:30:48:d6:ef:13 > 00:30:48:94:88:20, ethertype IPv4 
(0x0800), length 98: 3.3.3.55 > 3.3.3.1: ICMP echo request, id 7946, 
seq 1, length 64
16:17:02.603502 00:30:48:94:88:20 > 00:30:48:d6:ef:13, ethertype IPv4 
(0x0800), length 98: 3.3.3.1 > 3.3.3.55: ICMP echo reply, id 7946, 
seq 1, length 64
16:17:03.604510 00:30:48:d6:ef:13 > 00:30:48:94:88:20, ethertype IPv4 
(0x0800), length 98: 3.3.3.55 > 3.3.3.1: ICMP echo request, id 7946, 
seq 2, length 64
16:17:03.604516 00:30:48:94:88:20 > 00:30:48:d6:ef:13, ethertype IPv4 
(0x0800), length 98: 3.3.3.1 > 3.3.3.55: ICMP echo reply, id 7946, 
seq 2, length 64



but not using the initial IP addr

0[iolite3A]# ping -S 3.3.3.3 3.3.3.1
PING 3.3.3.1 (3.3.3.1) from 3.3.3.3: 56 data bytes
^C
--- 3.3.3.1 ping statistics ---
2 packets transmitted, 0 packets received, 100.0% packet loss
#

Yet,

# ping -S 3.3.3.3 3.3.3.1
PING 3.3.3.1 (3.3.3.1) from 3.3.3.3: 56 data bytes
^C
--- 3.3.3.1 ping statistics ---
2 packets transmitted, 0 packets received, 100.0% packet loss
# ping -S 3.3.3.4 3.3.3.1
PING 3.3.3.1 (3.3.3.1) from 3.3.3.4: 56 data bytes
64 bytes from 3.3.3.1: icmp_seq=0 ttl=64 time=0.176 ms
64 bytes from 3.3.3.1: icmp_seq=1 ttl=64 time=0.050 ms
^C
--- 3.3.3.1 ping statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.050/0.113/0.176/0.063 ms


Strange, eh ?


em4@pci0:6:0:0: class=0x020000 card=0x10d315d9 chip=0x10d38086 
rev=0x00 hdr=0x00
     vendor     = 'Intel Corporation'
     class      = network
     subclass   = ethernet
     cap 01[c8] = powerspec 2  supports D0 D3  current D0
     cap 05[d0] = MSI supports 1 message, 64 bit
     cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1)
     cap 11[a0] = MSI-X supports 5 messages in map 0x1c enabled
em5@pci0:7:0:0: class=0x020000 card=0x10d315d9 chip=0x10d38086 
rev=0x00 hdr=0x00
     vendor     = 'Intel Corporation'
     class      = network
     subclass   = ethernet
     cap 01[c8] = powerspec 2  supports D0 D3  current D0
     cap 05[d0] = MSI supports 1 message, 64 bit
     cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1)
     cap 11[a0] = MSI-X supports 5 messages in map 0x1c enabled


em4: <Intel(R) PRO/1000 Network Connection 6.9.6> port 0xdc00-0xdc1f 
mem 0xfaee0000-0xfaefffff,0xfaedc000-0xfaedffff irq 16 at device 0.0 on pci6
em4: Using MSIX interrupts
em4: [ITHREAD]
em4: [ITHREAD]
em4: [ITHREAD]
em4: Ethernet address: 00:30:48:d6:ef:12
pcib7: <ACPI PCI-PCI bridge> irq 16 at device 28.1 on pci0
pci7: <ACPI PCI bus> on pcib7
em5: <Intel(R) PRO/1000 Network Connection 6.9.6> port 0xec00-0xec1f 
mem 0xfafe0000-0xfaffffff,0xfafdc000-0xfafdffff irq 17 at device 0.0 on pci7
em5: Using MSIX interrupts
em5: [ITHREAD]
em5: [ITHREAD]
em5: [ITHREAD]
em5: Ethernet address: 00:30:48:d6:ef:13

The same does NOT happen with my external 2 port pcie nics (it says 
HP, but they are intel branded)
eg
em0@pci0:1:0:0: class=0x020000 card=0x115e8086 chip=0x105e8086 
rev=0x06 hdr=0x00
     vendor     = 'Intel Corporation'
     device     = 'HP NC360T PCIe DP Gigabit Server Adapter (n1e5132)'
     class      = network
     subclass   = ethernet
     cap 01[c8] = powerspec 2  supports D0 D3  current D0
     cap 05[d0] = MSI supports 1 message, 64 bit enabled with 1 message
     cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x4(x4)
em1@pci0:1:0:1: class=0x020000 card=0x115e8086 chip=0x105e8086 
rev=0x06 hdr=0x00
     vendor     = 'Intel Corporation'
     device     = 'HP NC360T PCIe DP Gigabit Server Adapter (n1e5132)'
     class      = network
     subclass   = ethernet
     cap 01[c8] = powerspec 2  supports D0 D3  current D0
     cap 05[d0] = MSI supports 1 message, 64 bit enabled with 1 message
     cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x4(x4)

         ---Mike

--------------------------------------------------------------------
Mike Tancsa,                                      tel +1 519 651 3400
Sentex Communications,                            mike@sentex.net
Providing Internet since 1994                    www.sentex.net
Cambridge, Ontario Canada                         www.sentex.net/mike




Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200911182130.nAILUJCR089766>