Skip site navigation (1)Skip section navigation (2)
Date:      Sun, 20 Jan 2019 09:18:29 +0100
From:      Martin Birgmeier <d8zNeCFG@aon.at>
To:        Bruce Evans <brde@optusnet.com.au>
Cc:        Eugene Grosbein <eugen@grosbein.net>, net@freebsd.org
Subject:   Re: [Bug 235031] [em] em0: poor NFS performance, strange behavior
Message-ID:  <fd67eca6-7c1d-687d-91ae-e09138732ed1@aon.at>
In-Reply-To: <20190120145627.X1077@besplex.bde.org>
References:  <bug-235031-7501@https.bugs.freebsd.org/bugzilla/> <bug-235031-7501-goXNmp3zVl@https.bugs.freebsd.org/bugzilla/> <20190119204156.D929@besplex.bde.org> <3e407ee7-54e3-a6ac-5535-d11aceca9558@grosbein.net> <20190120061258.X3312@besplex.bde.org> <16ce1832-13da-d7bb-cce2-6682e058b5a6@aon.at> <20190120145627.X1077@besplex.bde.org>

next in thread | previous in thread | raw e-mail | index | archive | help
Regarding duplex, ifconfig shows the following:

[0]# ifconfig em0
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
       
options=81249b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,LRO,WOL_MAGIC,VLAN_HWFILTER>
        ether f0:de:f1:98:86:a9
        inet 192.168.1.19 netmask 0xffffff00 broadcast 192.168.1.255
        inet6 fe80::f2de:f1ff:fe98:86a9%em0 prefixlen 64 scopeid 0x1
        inet6 fec0:0:0:4d42::13 prefixlen 64
        inet6 fec0::4d42:f2de:f1ff:fe98:86a9 prefixlen 64 autoconf
        inet6 2002:bc17:f381:4d42:f2de:f1ff:fe98:86a9 prefixlen 64 autoconf
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
[0]#

This seems to be o.k.

-- Martin

On 20.01.19 06:28, Bruce Evans wrote:
> On Sat, 19 Jan 2019, Martin Birgmeier wrote:
>
>> I just tried the patch by Bruce (from the mail sent 10 hours ago), but
>> it makes no difference.
>>
>> Also, it does not seem like bad frames or too high an interrupt rate are
>> the problem (the machine should easily handle what is coming from its
>> NFS client which only has a 100 Mbps interface).
>>
>> I believe that the simplifications introduced to sys/dev/e1000 between
>> 11.2 and 12.0 have broken something.
>
> They aren't exactly simplifications :-).
>
> Did you check for the common problex of a duplex mismatch?  ISR that some
> versions if iflib'ed em didn't negotiate right for your speed of 100
> Mbps.
>
> Here I can break nfs using "ifconfig em0 media 100baseTX mediaopt
> full-duplex" and forgetting the mediaopt part.  This gives half-duplex.
> ipv4 ping still works, but its latency increases from ~125 usec to ~76
> msec.  The latter latency destroys nfs performance.  After the media
> change, there are a lot of DUP packets with an initial latency of ~43
> second and the latency decreasing by the ping interval of 1 second for
> the next 42 or 43 DUPs until the backlog is cleared; the latency is
> then between 71 and 80 msec.  Changing the media and mediaopt back
> to 1000baseT[X] full-duplex restores low latency but causes 1 DUP with
> delay ~19 seconds
>
> Suspend/resume used to give much the same misbehaviour, by not stopping
> the NIC when reinitializing it in resume.  This was fixed in r342855.
> This might be the bug!  iflib_media_change() calls iflib_init_locked()
> liked resume used to, so seems to be missing stopping.  Changing this
> should fix at least the DUPs.
>
> The function names or layering are confusing.  iflib_init_locked()
> doesn't initialize the if.  iflib_if_init_locked() does that.  All
> iflib_init_locked() does is call iflib_stop(), then iflib_init_locked().
> and iflib.  Grep shows the following related iflib*init*() calls:
> - iflib_netmap_register manually inlines iflib_if_init_locked().  This
>   is a style bug
> - iflib_media_change() only calls iflib_init_locked().  This seems to be
>   a bug
> - _task_fn_admin() calls iflib_if_init_locked() for resetting.  This
> seems
>   to be correctly obfuscated
> - iflib_if_init_locked() calls iflib_init_locked().  This is part of
>   implementing the obfuscation - iflib_if_init() calls
> iflib_if_init_locked().  This is correct
> - iflib_if_ioctl(): SIOCSIFMTU calls iflib_stop(), then does some
> locking,
>   then sets the mtu in software, then calls iflib_init_locked().  This
>   seems to be correct, and shows that the iflib_if_init_locked() is not
>   even generally useful.  This gives down/up for non-null changes.  This
>   works correctly (some ping packets are lost, but there are no DUPs.
> - iflib_if_ioctl(): SIOCSIFCAP is like SIOCSIFMTU, except I didn't test
>   it and its splitting of stopping and init'ing is a bit messier because
>   both operations are under a more complicated conditional.
> - iflib_if_ioctl():<reinit case at the end> calls iflib_if_init().  This
>   is correct.
> - iflib_vlan_[un]register() call iflib_if_init_locked().  This seems
> to be
>   correctly obfuscated
> - iflib_device_resume() calls iflib_if_init_locked().  This is correctly
>   obfuscated
> - if_setinitfn() is called to set iflib_if_init as the init function. 
> This
>   is correct.
>
> Summary: only media change seems to be broken, but there are some
> style bugs.
>
> The bug apparently btoke resume by reinitializing an active state
> (even locking doesn't help much, but I now remember than resume
> succeeded every 10-100 tries in the buggy versions -- there were always
> a lot of DUPs, but sometimes to low latency came back).  My tests
> usually used zzz and my zzz and other utilities are on nfs, so nfs was
> fairly active just before suspend.
>
> I don't know if iflib_media_change() is called at boot time, especially
> if the media is autoselect.  At boot time, the state might be less
> active or closer to the reset state, so that even a manual media change
> that surely calls iflib_media_change() has more chance of working than
> at resume time with zzz and other utilities on nfs.
>
> I don't know what the media was after the broken resume.  Its reported
> result can't be trusted anyway.  To recover from the broken resume, it
> usually worked to repeat down/up a few times.  This is consistent with
> bug -- eventually, previous down/up's change the state to close enough
> to stopped.  But using the interface in any way (including pinging it
> to see if it is still broken) makes it not so close to being stopped.
>
> Bruce



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?fd67eca6-7c1d-687d-91ae-e09138732ed1>