Date: Sat, 5 Sep 2015 19:13:09 -0700 From: Artem Belevich <artemb@gmail.com> To: "M. V." <bored_to_death85@yahoo.com> Cc: "freebsd-net@freebsd.org" <freebsd-net@freebsd.org>, Luigi Rizzo <rizzo@iet.unipi.it> Subject: Re: Problem with receiving packets right after remote-interface is up Message-ID: <CAFqOu6ihCTrOiTppYA%2B%2BGkTD_n1e%2BUTiV6x2TznVUp65mno2rA@mail.gmail.com> In-Reply-To: <CA%2BhQ2%2BhgiTJSSmdWSuHTWqUP4Gj2EEKxAHWxONC8xuw8e-emRg@mail.gmail.com> References: <1441485132.75285.YahooMailBasic@web165005.mail.bf1.yahoo.com> <CA%2BhQ2%2BhgiTJSSmdWSuHTWqUP4Gj2EEKxAHWxONC8xuw8e-emRg@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sat, Sep 5, 2015 at 2:44 PM, Luigi Rizzo <rizzo@iet.unipi.it> wrote: > You may be able to avoid the issue if you avoid autonegotiation and > manually set the media type, speed and duplex. > Technically, 1G ethernet *requires* autonegotiation. http://standards.ieee.org/findstds/interps/IEEE802.3af-2003interp-6.pdf While you can indeed set speed/duplex manually, you will also need to make sure both have have proper clock master/slave selection which is normally done via autonegotiation. On Freebsd some interfaces support "mediaopt master" option -- check if that's the case for your interface with ifconfig -m. You will also need to make sure that the other end of the link is also configured to set matching link parameters manually. Even with all of that, copper PHYs take a while to bring link up as PHYs need to tune their parameters. Ethernet MAC will not be able to send or receive any packets *on the wire* until PHYs are done which may take relatively long time (~hundreds of milliseconds). > > On Saturday, September 5, 2015, M. V. via freebsd-net < > freebsd-net@freebsd.org> wrote: > > > Hi, > > > > Our product is being tested with Spirent TestCenter, and we're facing an > > unusual problem with the tests. > > We use NICs with intel 82574 and 82576 on FreeBSD 9.2 with latest em and > > igb drivers (we also tested this on FreeBSD-10.1) It seems what Spirent > > TestCenter does to start any individual test is, it disables its own > > interface, and at the beginning of the new test, it suddenly "up"s its > > interface and sends test packet right after that without any delay. > You may want to go over spirent box settings and look for something along the lines of "wait for the link to come up". It may be somewhere in per-interface settings. > This is > > where we have problem, and we don't receive this first packets most of the > > time (result is vary, in 100 tests, we lose about 60~70% of this "first" > > packets on each test, so we FAIL most of tests because apparently we need > > about 0.5~1 seconds after setup and renegotiation before we can receive > > packets) > That's in the ballpark of time it takes for a copper link to come up. > > we checked both em and igb drivers and both have this problem. > > this is how we reproduce the problem: > > - connect sender-system's first port to receiver-system's first port. > > - In sender-system: > > -- ifconfig em0 down > > -- sleep 5 > > -- ifconfig em0 up && tcpreplay -l 1 -i em0 one_packet.pcap > > > > We checked, and even though sender sends packet successfully (tx led > > blinks), > It's not a conclusive proof that the packet made it to the wire. Depending on where the LED is connected, it may mean different things -- if it's set up to indicate that MAC is sending a packet to PHY, for instance, it would happily blink if the link is down. > in receiver even driver doesn't receive it (rx led doesn't blink, > > and "rxeof" function in "if_em" tells us no new packet is received) > > we removed optional interface options like msix, tso, rxcsum, ... but no > > luck. > Which is another point that the link is not up and the packet did not get received by the PHY. > > > > So, does anyone know any solution for this problem? or any suggestions on > > where we should look or what we should do that might solve it and prevent > > our tests from being failed? > It sounds to me like the problem is with your test setup. If your test requires no packet drops, then it implicitly assumes having established link and sending the packets right after the interface (and not necessarily the physical link) is up is really pushing it. You will either need to configure your test gear to wait some time after the interface is up, or redesign your test so it can tolerate some packet loss while the physical link is being established. To make things more interesting, consider that ethernet does not guarantee packet delivery at all. --Artem
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAFqOu6ihCTrOiTppYA%2B%2BGkTD_n1e%2BUTiV6x2TznVUp65mno2rA>