Date: Sun, 20 Jan 2019 09:18:29 +0100 From: Martin Birgmeier <d8zNeCFG@aon.at> To: Bruce Evans <brde@optusnet.com.au> Cc: Eugene Grosbein <eugen@grosbein.net>, net@freebsd.org Subject: Re: [Bug 235031] [em] em0: poor NFS performance, strange behavior Message-ID: <fd67eca6-7c1d-687d-91ae-e09138732ed1@aon.at> In-Reply-To: <20190120145627.X1077@besplex.bde.org> References: <bug-235031-7501@https.bugs.freebsd.org/bugzilla/> <bug-235031-7501-goXNmp3zVl@https.bugs.freebsd.org/bugzilla/> <20190119204156.D929@besplex.bde.org> <3e407ee7-54e3-a6ac-5535-d11aceca9558@grosbein.net> <20190120061258.X3312@besplex.bde.org> <16ce1832-13da-d7bb-cce2-6682e058b5a6@aon.at> <20190120145627.X1077@besplex.bde.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Regarding duplex, ifconfig shows the following: [0]# ifconfig em0 em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=81249b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,LRO,WOL_MAGIC,VLAN_HWFILTER> ether f0:de:f1:98:86:a9 inet 192.168.1.19 netmask 0xffffff00 broadcast 192.168.1.255 inet6 fe80::f2de:f1ff:fe98:86a9%em0 prefixlen 64 scopeid 0x1 inet6 fec0:0:0:4d42::13 prefixlen 64 inet6 fec0::4d42:f2de:f1ff:fe98:86a9 prefixlen 64 autoconf inet6 2002:bc17:f381:4d42:f2de:f1ff:fe98:86a9 prefixlen 64 autoconf media: Ethernet autoselect (1000baseT <full-duplex>) status: active nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL> [0]# This seems to be o.k. -- Martin On 20.01.19 06:28, Bruce Evans wrote: > On Sat, 19 Jan 2019, Martin Birgmeier wrote: > >> I just tried the patch by Bruce (from the mail sent 10 hours ago), but >> it makes no difference. >> >> Also, it does not seem like bad frames or too high an interrupt rate are >> the problem (the machine should easily handle what is coming from its >> NFS client which only has a 100 Mbps interface). >> >> I believe that the simplifications introduced to sys/dev/e1000 between >> 11.2 and 12.0 have broken something. > > They aren't exactly simplifications :-). > > Did you check for the common problex of a duplex mismatch? ISR that some > versions if iflib'ed em didn't negotiate right for your speed of 100 > Mbps. > > Here I can break nfs using "ifconfig em0 media 100baseTX mediaopt > full-duplex" and forgetting the mediaopt part. This gives half-duplex. > ipv4 ping still works, but its latency increases from ~125 usec to ~76 > msec. The latter latency destroys nfs performance. After the media > change, there are a lot of DUP packets with an initial latency of ~43 > second and the latency decreasing by the ping interval of 1 second for > the next 42 or 43 DUPs until the backlog is cleared; the latency is > then between 71 and 80 msec. Changing the media and mediaopt back > to 1000baseT[X] full-duplex restores low latency but causes 1 DUP with > delay ~19 seconds > > Suspend/resume used to give much the same misbehaviour, by not stopping > the NIC when reinitializing it in resume. This was fixed in r342855. > This might be the bug! iflib_media_change() calls iflib_init_locked() > liked resume used to, so seems to be missing stopping. Changing this > should fix at least the DUPs. > > The function names or layering are confusing. iflib_init_locked() > doesn't initialize the if. iflib_if_init_locked() does that. All > iflib_init_locked() does is call iflib_stop(), then iflib_init_locked(). > and iflib. Grep shows the following related iflib*init*() calls: > - iflib_netmap_register manually inlines iflib_if_init_locked(). This > is a style bug > - iflib_media_change() only calls iflib_init_locked(). This seems to be > a bug > - _task_fn_admin() calls iflib_if_init_locked() for resetting. This > seems > to be correctly obfuscated > - iflib_if_init_locked() calls iflib_init_locked(). This is part of > implementing the obfuscation - iflib_if_init() calls > iflib_if_init_locked(). This is correct > - iflib_if_ioctl(): SIOCSIFMTU calls iflib_stop(), then does some > locking, > then sets the mtu in software, then calls iflib_init_locked(). This > seems to be correct, and shows that the iflib_if_init_locked() is not > even generally useful. This gives down/up for non-null changes. This > works correctly (some ping packets are lost, but there are no DUPs. > - iflib_if_ioctl(): SIOCSIFCAP is like SIOCSIFMTU, except I didn't test > it and its splitting of stopping and init'ing is a bit messier because > both operations are under a more complicated conditional. > - iflib_if_ioctl():<reinit case at the end> calls iflib_if_init(). This > is correct. > - iflib_vlan_[un]register() call iflib_if_init_locked(). This seems > to be > correctly obfuscated > - iflib_device_resume() calls iflib_if_init_locked(). This is correctly > obfuscated > - if_setinitfn() is called to set iflib_if_init as the init function. > This > is correct. > > Summary: only media change seems to be broken, but there are some > style bugs. > > The bug apparently btoke resume by reinitializing an active state > (even locking doesn't help much, but I now remember than resume > succeeded every 10-100 tries in the buggy versions -- there were always > a lot of DUPs, but sometimes to low latency came back). My tests > usually used zzz and my zzz and other utilities are on nfs, so nfs was > fairly active just before suspend. > > I don't know if iflib_media_change() is called at boot time, especially > if the media is autoselect. At boot time, the state might be less > active or closer to the reset state, so that even a manual media change > that surely calls iflib_media_change() has more chance of working than > at resume time with zzz and other utilities on nfs. > > I don't know what the media was after the broken resume. Its reported > result can't be trusted anyway. To recover from the broken resume, it > usually worked to repeat down/up a few times. This is consistent with > bug -- eventually, previous down/up's change the state to close enough > to stopped. But using the interface in any way (including pinging it > to see if it is still broken) makes it not so close to being stopped. > > Bruce
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?fd67eca6-7c1d-687d-91ae-e09138732ed1>