Date: Wed, 3 Nov 2010 19:27:20 -0400 (EDT) From: Rick Macklem <rmacklem@uoguelph.ca> To: pyunyh@gmail.com Cc: freebsd-current@freebsd.org Subject: Re: re(4) driver dropping packets when reading NFS files Message-ID: <1320759482.60041.1288826840754.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <20101101234001.GD1433@michelle.cdnetworks.com>
next in thread | previous in thread | raw e-mail | index | archive | help
[-- Attachment #1 --] > > I'm more interested in number of dropped frames. See below how to > extract that information. > I've attached the stats. I'm guessing that the Rx missed frames : 14792 is the culprit. This was for a read of a fairly large file via NFS over TCP, getting a read rate of about 450Kbytes/sec. (No DEVICE_POLLING option.) (with your patch applied) > > > (It almost looks like it only handles the first received packet, > > although > > it appears to be using a receive ring of 64 buffers.) > > > > No, re(4) uses 256 TX/RX buffers for RTL810xE controllers. > Oops, my mistake. At a quick glance, I had thought rl_type was set to 8139, but I now see it's 8169. Btw, I printed out the hwrev and its a RL_HWREV_8102EL_SPIN1, if that is of any use to you. > > Ok, here is patch. > http://people.freebsd.org/~yongari/re/re.intr.patch > > The patch has the following changes. > o 64bit DMA support for PCIe controllers. > o Hardware MAC statistics counter support. You can extract these > counters with "sysctl dev.re.0.stats=1". You can check the > output on console or dmesg. It seems extracting these counters > take a lot of time so I didn't try to accumulate the counters. > You can see how many frames are dropped from the output. I saw a > lot FAE(frame alignment errors) under high RX load and I can't > explain how this can happen. This may indicate PHY hardware is > poor or it may need DSP fixups. Realtek seems to maintain large > set of DSP fixups for each PHY revisions and re(4) does not > have the magic code at this moment. > o Overhaul MSI interrupt handler such that make it give fairness > to TX as well as serving RX. Because re(4) controllers do not > have interrupt moderation mechanism, naive interrupt handler can > generate more than 125k intrs/sec under high load. Fortunately, > Bill implemented TX interrupt moderation with a timer register > and it seems to work well on TX path. One drawback of the > approach is it will require extra timer register accesses in > fast path. There is no second timer register to use in RX path > so no RX interrupt moderation is done in driver such that it can > generate about 25k intrs/sec under high RX load. However, I > think most systems can handle that interrupt load. Note, this > feature is activated only when MSI is in use and DEVICE_POLLING > is not defined. > > >From my limited testing, it seems it works as expected. Would you > give it try and let me know how well it behaves with NFS? > Without DEVICE_POLLING it behaves just like the unpatched one. I'm going to look at the driver tomorrow and try some hacks on it, rick [-- Attachment #2 --] re0 statistics: Transmit good frames : 100966 Receive good frames : 133470 Tx errors : 0 Rx errors : 0 Rx missed frames : 14792 Rx frame alignment errs : 0 Tx single collisions : 0 Tx multiple collisions : 0 Rx unicast frames : 133463 Rx broadcast frames : 0 Rx multicast frames : 7 Tx aborts : 0 Tx underruns : 0
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1320759482.60041.1288826840754.JavaMail.root>