Skip site navigation (1)Skip section navigation (2)
Date:      Tue, 17 Dec 2013 17:03:42 -0800
From:      John-Mark Gurney <jmg@funkthat.com>
To:        Claude Marinier <claude.marinier@cae.com>
Cc:        Michael Tuexen <Michael.Tuexen@lurchi.franken.de>, "freebsd-net@freebsd.org" <freebsd-net@freebsd.org>
Subject:   Re: em3 no carrier
Message-ID:  <20131218010342.GG99167@funkthat.com>
In-Reply-To: <34E38D657499684D9EBAA6D73CD2D1E21958EE4AD3@CAEMEX81.caecorp.cae.com>
References:  <34E38D657499684D9EBAA6D73CD2D1E21958EE4A62@CAEMEX81.caecorp.cae.com> <D711A5E9-82F8-4495-9C5F-BF72BF0BD4D3@lurchi.franken.de> <34E38D657499684D9EBAA6D73CD2D1E21958EE4AD3@CAEMEX81.caecorp.cae.com>

next in thread | previous in thread | raw e-mail | index | archive | help
Claude Marinier wrote this message on Tue, Dec 17, 2013 at 19:43 -0500:
> -----Original Message-----
> From: Michael Tuexen [mailto:Michael.Tuexen@lurchi.franken.de] 
> Sent: Tuesday, December 17, 2013 5:24 PM
> To: Claude Marinier
> Cc: freebsd-net@freebsd.org
> Subject: Re: em3 no carrier
> 
> On Dec 17, 2013, at 10:51 PM, Claude Marinier <claude.marinier@cae.com> wrote:
> 
> > 
> >>> -----Original Message-----
> >>> From: Rodrigo Osorio [mailto:rodrigo at bebik.net]
> >>> Sent: Tuesday, December 17, 2013 11:40 AM
> >>> To: Claude Marinier
> >>> Cc: freebsd-net at freebsd.org
> >>> Subject: Re: em3 no carrier
> >>> 
> >>> Hi,
> >>> 
> >>> Can you give us more informations about the network card / chips ?
> >>> Did you found any error in /var/log/ ?
> >>> 
> >>> - rodrigo
> >> 
> >> My apologies, I missed that bit of information.
> >> 
> >> Yes, there is an error in /var/log/messages.
> >> 
> >> Dec 17 11:33:07 WANemu bird: OSPF: Socket error on em3: No buffer 
> >> space available Dec 17 11:33:47 WANemu last message repeated 4 times 
> >> Dec 17 11:35:57 WANemu last message repeated 13 times Dec 17 11:45:57 
> >> WANemu last message repeated 60 times Dec 17 11:55:47 WANemu last 
> >> message repeated 59 times
> >> 
> >> 
> >> Also note that the HP NC364T uses an Intel 82571EB chipset.
> >> 
> >>> On 17/12/13 11:27 -0500, Claude Marinier wrote:
> >>>> Hi,
> >>>> 
> >>>> This is a WAN emulator (ipfw, DummyNet, and BIRD). FreeBSD 9.2 
> >>>> Release (amd64) is running on DL360 G5 with a new NC364T quad-port 
> >>>> Ethernet NIC. I installed the new NIC yesterday. The four ports are 
> >>>> connected to routers in a lab.
> >>>> 
> >>>> FreeBSD WAMemu 9.2-RELEASE FreeBSD 9.2-RELEASE #0 r255898: Thu Sep 
> >>>> 26 22:50:31 UTC 2013 root at 
> >>>> bake.isc.freebsd.org:/usr/obj/usr/src/sys/GENERIC  amd64
> >>>> 
> >>>> Dec 16 17:13:19 WANemu kernel: em3: <Intel(R) PRO/1000 Network 
> >>>> Connection 7.3.8> port 0x6020-0x603f mem 
> >>>> 0xfdee0000-0xfdefffff,0xfde00000-0xfde7ffff irq 17 at device 0.1 on 
> >>>> pci22 Dec 16 17:13:19 WANemu kernel: em3: Using an MSI interrupt 
> >>>> Dec 16 17:13:19 WANemu kernel: em3: Ethernet address: 
> >>>> e8:39:35:13:21:6e
> >>>> 
> >>>> The four em interfaces are configured the same way (lines in 
> >>>> rc.conf are identical except for IP address.
> >>>> 
> >>>> ifconfig_em3="inet x.y.113.197/29 media 100baseTX mediaopt full-duplex"
> >>>> ifconfig_em0="inet x.y.113.14/29 media 100baseTX mediaopt full-duplex"
> >>>> ifconfig_em2="inet x.y.113.109/29 media 100baseTX mediaopt full-duplex"
> >>>> ifconfig_em1="inet x.y.113.189/29 media 100baseTX mediaopt full-duplex"
> >>>> 
> >>>> The GE 0/1 port on the Cisco 2821 shows no link and ifconfig shows 
> >>>> "no carrier" for em3. Oddly, it also shows autoselect.
> >>>> 
> >>>> em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
> >>>>    options=4019b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,VLAN_HWTSO>
> >>>>    ether e8:39:35:13:21:6d
> >>>>    inet x.y.113.14 netmask 0xfffffff8 broadcast x.y.113.15
> >>>>    inet6 fe80::ea39:35ff:fe13:216d%em0 prefixlen 64 scopeid 0x1
> >>>>    nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
> >>>>    media: Ethernet 100baseTX <full-duplex>
> >>>>    status: active
> >>>> em1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
> >>>>    options=4019b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,VLAN_HWTSO>
> >>>>    ether e8:39:35:13:21:6c
> >>>>    inet x.y.113.189 netmask 0xfffffff8 broadcast x.y.113.191
> >>>>    inet6 fe80::ea39:35ff:fe13:216c%em1 prefixlen 64 scopeid 0x2
> >>>>    nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
> >>>>    media: Ethernet 100baseTX <full-duplex>
> >>>>    status: active
> >>>> em2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
> >>>>    options=4019b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,VLAN_HWTSO>
> >>>>    ether e8:39:35:13:21:6f
> >>>>    inet 131.140.113.109 netmask 0xfffffff8 broadcast 131.140.113.111
> >>>>    inet6 fe80::ea39:35ff:fe13:216f%em2 prefixlen 64 scopeid 0x3
> >>>>    nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
> >>>>    media: Ethernet 100baseTX <full-duplex>
> >>>>    status: active
> >>>> em3: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
> >>>>    options=4019b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,VLAN_HWTSO>
> >>>>    ether e8:39:35:13:21:6e
> >>>>    inet 131.140.113.197 netmask 0xfffffff8 broadcast 131.140.113.199
> >>>>    inet6 fe80::ea39:35ff:fe13:216e%em3 prefixlen 64 scopeid 0x4
> >>>>    nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
> >>>>    media: Ethernet 100baseTX <full-duplex> (autoselect)
> >>>>    status: no carrier
> >>>> 
> >>>> I have tried three Ethernet cables with no difference. This 
> >>>> morning, I briefly booted Puppy Linux from USB and manually 
> >>>> configured the four Ethernet interfaces. They all work (ping good 
> >>>> to all four routers); the Cisco on em3 shows happy lights. I then 
> >>>> booted back into FreeBSD, the problem with em3 remains. I have not 
> >>>> had any success searching with Google; I may not have used suitable search terms.
> >>>> 
> >>>> One more odd symptom: bmon has trouble with these interfaces.
> >>>> 
> >>>> em0è95^S!m on WANemu
> >>>>          Name
> >>>> qqqqqqqqqqqqqqqqqqqqqqqq
> >>>> WANemu (local)
> >>>>  0   em0è95^S!m
> >>>>  1   em1è95^S!l
> >>>>  2   em2è95^S!o
> >>>>  3   em3è95^S!n
> >>>>  4   usbus0
> >>>>  5   usbus1
> >>>>  6   usbus2
> >>>>  7   usbus3
> >>>>  8   usbus4
> >>>>  9   usbus5
> >>>>  10  ipfw0
> >>>>  11  lo0
> >>>> 
> >>>> It looks like FreeBSD is somehow mishandling these interfaces. 
> >>>> Before purchasing it, I searched with Google and found reports of 
> >>>> people using this device without problems. I do not know how to proceed.
> >>>> 
> >>>> Thank you for your help.
> > 
> > Oddly, replacing em3 with bce1 left the system with only two 
> > functionning interfaces. The order of the interface definitions in 
> > rc.local makes a difference. The worst order is bce1, em0, em2, em1 
> > which left only
> > em0 and em2 working. Changing the order to bce1, em0, em1, em2 allows 
> > the three 'em' interfaces to work. In both, bce1 is not working (no 
> > carrier). Changing the order to em0, em1, em2, bce1 did not improve 
> > the situation.
> > 
> > This is a WAN emulator and the rc.conf definitions are generated by 
> > the configuration program from site names, hence the odd order. I am 
> > changing the order manually.
> > 
> > I have reconfigured the Cisco router to use G0/0 instead of G0/1. This 
> > has made no noticeable difference.
> 
> Did you configure it to 100MBit, full-duplex?
> 
> -------
> 
> Yes, I configured everything (all eight interfaces: four on FreeBSD and one on each of the routers) to be 100baseTX and full-duplex.
> 
> As a test, I left out the 'media' and 'mediaopt' portions of the interface definitions in rc.conf and all the interfaces came up but three were in half-duplex mode, I think em0 was full-duplex (I am not in the lab to check).
> 
> I have heard horror stories about Ethernet auto sensing, so I make sure the Cisco routers and FreeBSD specify 100 Mbps and full-duplex. I begin to wonder if even that is safe.  :-(
> 
> [ why does Outlook not quote properly ]

It could be that there is a race condition when the media opts are
set at the same time the interface is brought up... I have you tried
to down the interface, set the media opts, wait a second or two, and
then bring up the interface?

I would go back to autoneg...  Even though the 2821 is old, the autoneg
issue was mostly when both sides were playing bad, and your nics are
modern...

At a previous company, we had an issue w/ our box at a client site...
The box would work fine for months, and then for some reason it would
suffer packet loss...  After a long bit of puzzling, (I don't even
remeber how we figured it out, but I think we got a dump of the switch
configuration), it turned out that someone had configured our box to
be 100mbit/full duplex, but not the switch...  It turned out that this
was "ok" for fxp as it would autoneg when brought up, and then it would
switch to 100mbit/full...  Both sides would be happy till the switch
rebooted and then tried to autoneg again, but now the fxp had disabled
it, and the switch would fall back to 100mbit/half...

I fixed fxp to down/up the phy when setting fixed modes so that the
switch would loose it's autoneg and then the customer would immediately
see the issue instead of weeks/months/years later when the switch was
power cycled...

after the above, I considered anything not autoneg to be bad...

-- 
  John-Mark Gurney				Voice: +1 415 225 5579

     "All that I will do, has been done, All that I have, has not."



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20131218010342.GG99167>