Date: Wed, 3 Nov 2010 19:27:20 -0400 (EDT) From: Rick Macklem <rmacklem@uoguelph.ca> To: pyunyh@gmail.com Cc: freebsd-current@freebsd.org Subject: Re: re(4) driver dropping packets when reading NFS files Message-ID: <1320759482.60041.1288826840754.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <20101101234001.GD1433@michelle.cdnetworks.com>
next in thread | previous in thread | raw e-mail | index | archive | help
------=_Part_60040_1528688248.1288826840752 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit > > I'm more interested in number of dropped frames. See below how to > extract that information. > I've attached the stats. I'm guessing that the Rx missed frames : 14792 is the culprit. This was for a read of a fairly large file via NFS over TCP, getting a read rate of about 450Kbytes/sec. (No DEVICE_POLLING option.) (with your patch applied) > > > (It almost looks like it only handles the first received packet, > > although > > it appears to be using a receive ring of 64 buffers.) > > > > No, re(4) uses 256 TX/RX buffers for RTL810xE controllers. > Oops, my mistake. At a quick glance, I had thought rl_type was set to 8139, but I now see it's 8169. Btw, I printed out the hwrev and its a RL_HWREV_8102EL_SPIN1, if that is of any use to you. > > Ok, here is patch. > http://people.freebsd.org/~yongari/re/re.intr.patch > > The patch has the following changes. > o 64bit DMA support for PCIe controllers. > o Hardware MAC statistics counter support. You can extract these > counters with "sysctl dev.re.0.stats=1". You can check the > output on console or dmesg. It seems extracting these counters > take a lot of time so I didn't try to accumulate the counters. > You can see how many frames are dropped from the output. I saw a > lot FAE(frame alignment errors) under high RX load and I can't > explain how this can happen. This may indicate PHY hardware is > poor or it may need DSP fixups. Realtek seems to maintain large > set of DSP fixups for each PHY revisions and re(4) does not > have the magic code at this moment. > o Overhaul MSI interrupt handler such that make it give fairness > to TX as well as serving RX. Because re(4) controllers do not > have interrupt moderation mechanism, naive interrupt handler can > generate more than 125k intrs/sec under high load. Fortunately, > Bill implemented TX interrupt moderation with a timer register > and it seems to work well on TX path. One drawback of the > approach is it will require extra timer register accesses in > fast path. There is no second timer register to use in RX path > so no RX interrupt moderation is done in driver such that it can > generate about 25k intrs/sec under high RX load. However, I > think most systems can handle that interrupt load. Note, this > feature is activated only when MSI is in use and DEVICE_POLLING > is not defined. > > >From my limited testing, it seems it works as expected. Would you > give it try and let me know how well it behaves with NFS? > Without DEVICE_POLLING it behaves just like the unpatched one. I'm going to look at the driver tomorrow and try some hacks on it, rick ------=_Part_60040_1528688248.1288826840752 Content-Type: text/plain; name=re.stats Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename=re.stats cmUwIHN0YXRpc3RpY3M6DQpUcmFuc21pdCBnb29kIGZyYW1lcyA6IDEwMDk2Ng0KUmVjZWl2ZSBn b29kIGZyYW1lcyA6IDEzMzQ3MA0KVHggZXJyb3JzIDogMA0KUnggZXJyb3JzIDogMA0KUnggbWlz c2VkIGZyYW1lcyA6IDE0NzkyDQpSeCBmcmFtZSBhbGlnbm1lbnQgZXJycyA6IDANClR4IHNpbmds ZSBjb2xsaXNpb25zIDogMA0KVHggbXVsdGlwbGUgY29sbGlzaW9ucyA6IDANClJ4IHVuaWNhc3Qg ZnJhbWVzIDogMTMzNDYzDQpSeCBicm9hZGNhc3QgZnJhbWVzIDogMA0KUnggbXVsdGljYXN0IGZy YW1lcyA6IDcNClR4IGFib3J0cyA6IDANClR4IHVuZGVycnVucyA6IDANCg== ------=_Part_60040_1528688248.1288826840752--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1320759482.60041.1288826840754.JavaMail.root>