Date: Tue, 19 Oct 2004 01:12:02 -0700 From: John-Mark Gurney <gurney_j@resnet.uoregon.edu> To: Sean McNeil <sean@mcneil.com> Cc: freebsd-current@freebsd.org Subject: Re: re0 fix that works with polling Message-ID: <20041019081202.GI22681@funkthat.com> In-Reply-To: <1098165611.77189.16.camel@server.mcneil.com> References: <1097896460.1123.2.camel@server.mcneil.com> <20041018011235.GB22681@funkthat.com> <1098135068.2861.6.camel@server.mcneil.com> <20041018230208.GE22681@funkthat.com> <1098141908.3424.6.camel@server.mcneil.com> <20041019053319.GG22681@funkthat.com> <1098165611.77189.16.camel@server.mcneil.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--pyE8wggRBhVBcj8z Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Sean McNeil wrote this message on Mon, Oct 18, 2004 at 23:00 -0700: > On Mon, 2004-10-18 at 22:33, John-Mark Gurney wrote: > > Sean McNeil wrote this message on Mon, Oct 18, 2004 at 16:25 -0700: > > > > Do you have a video that you could send me, that I could do some testing > > > > with this? (and vls config)? I've been doing my testing on an i386, > > > > but that shouldn't be different enough to cause problems... (if it is, > > > > then we need to think about what the re is behaving badly)... > > > > > > I have placed 11 megs of the stream in > > > > > > http://mcneil.com/~sean/freebsd/stream.mpg > > > > damn, is that hdtv quality?? :) looks pretty nice... > > Yes, that is low-quality HD at 15mbps. It isn't too bad, though, until > it gets to a part I don't think was in the short sample I provided. > Gandalf is attacking the fire demon as they fall. Very blocky. well, I guess I'm mostly impressed by the high res.. the window at 1x res doesn't even fit on my laptop's display... :) > > > You can install vls from ports (net/vls) and I run it with > > > > > > vls -d udp:224.1.1.1:1234 file:stream.mpg > > > > > > I agree that your assesment appears to be accurate in identifying no > > > packet loss. Are you going through a switch or is this a cross from > > > machine-machine? > > > > The bad news is that it appears that I'm not dropping any packets at > > all... I'm using vlc on my mac (which is also at gige), and it appears > > to be getting full data rate.. though things are gittery,but the logs > > are kinda wierd... > > The mac probably doesn't have enough horse power. Typically you need an > Intel 3GHz processor to get it to play smoothly with software. yeh, not surprised... my laptop is only 867mhz... > > now the bad part of the news... if I drop my re0 card over to my 100mbit > > switch: > > media: Ethernet autoselect (100baseTX <full-duplex>) > > > > Ok, here's a more complete run of what happened: > > re0 <-> SMC gige switch <-> NetGear gige switch <-> MacOSX laptop (gige) > > no packet loss observed.. ~1.9megs/sec or 1400pps received at MacOSX > > laptop > > > > re0 <-> summit48 (100mbit) <-> SMC gige switch <-> MacOSX laptop (gige) > > w/ or w/o netgear switch between SMC and laptop, packet loss is > > observed, only about 700-900kbyte/sec or 500-600pps > > This is very interesting. This is completely opposite to my > experience. I have > > re0 <-> linksys gige <-> hardware decoder or freebsd box just doing > capture both at 100bt > > packet loss. With mods, no loss. > > re0 at 100bt <-> linksys 100bt <-> same as above > > no packet loss. > > Could this be some sort of n-way negotiation issue? hmmm.. well, just make sure that the machines and the switches all agree... I couldn't hardcode re0 to 100baset w/ it plugged into the gige switch (though I didn't try hard coding it, and pulling the plug).. it always autodetected fine for me... > > now a bit more information, I did use snmpnetstat on the switch to verify > > that the switch was receiving (but I forgot to verify that it was sending > > all packets to the gige port) all packets even w/ the final packet loss.. > > > > re0 <-> summit48 (100mbit) <-> MacOSX laptop > > no packet loss observed... similar stats as to the straight gige above.. > > > > now, I also tried receiving on win2k box, and I received a bit better > > frame rate.. I don't think my macosx laptop can handle rendering the > > data stream (or at least vlc doesn't know how to make use of the proper > > acceleration).. the frame rate was really choppy, and sound not to good.. > > Yes, that would be expected. Fastest laptop being a 1.5GHz and vlc may > not be taking advantage of altivec. But then again maybe it is. Never > tried it on mine. > > > Do you have an application that gets good stats from vlc? I did see > > a few messages in the message log about dropped packets... but I coudln't > > really figure out how many/much they were, and considering it was udp, > > I expected some loss... > > The best thing I can recommend is to just capture the stream. I have a > program at > > http://www.mcneil.com/~sean/freebsd/stream.c > > just compile as cc -o stream stream.c and run as > > ./stream 224.1.1.1:1234 >capture.mpg > > assuming address:port as before. > > I have another program that will compare the streams and report missing > packets if you find that they appear to be dropping for you. Yeh, it appears I'm getting the drops here using your stream capture program... around 16%-20% of the file.. w/ your fix of the timer down to 0x200 I still see about 5% data drop.. and increasing the ring buffer size to 512 (w/o your timer fix) does see as low as 3% data drop, but I've still seen a 13% drop with it.. with both a ring increase and your fix, I normally saw about 4% data loss, but I did get one run with 18% loss... now, I didn't run hard numbers, so these numbers aren't statisticly proven... The biggest problem I see with your patch is that it appears to completely disable any type of interrupt moderation... we'd probably be better off w/ enable tx interrupts than using the timer... with my tests with your timer fix (just setting it to 0x200), I always saw double interrupts, one for transmit, and a second for receive... When I ran the netsend program at 14kpps, I was seeing 27kints!! this was using like (note netsend pkt size of 1316 to try to emulate your side) just for kicks I decided to double the timer w/ a ring buffer of 512, at 14kpps to 0x1000, and when doing 14kpps I only see 64int/sec w/o any errors returned (and apparent 100% receive rate) by netsend... after further examination, I believe I did find a slight logic flaw with the timer... Every time a packet was delivered to the interface, the timer was reset... which means it might of been possible to start the buffers, and with a shorter timer, this avoided the problem... attached is a patch (with a new sysctl that lets you adjust the timer on the fly for quicker/easier testing of different values)... It also prevents prematurely clearing the OACTIVE flag... see if you get better results w/o having to adjust the timer down.. > > again, most of the stats was gathered with netstat, since this is the > > easiest way I know to see things happening.. > > I really do appreciate the attention you are giving this, Jean-Mark. :) I'm not French.. it's John-Mark.. Considering your name is Sean, I can see how'd your fingers might make a mistake... -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." --pyE8wggRBhVBcj8z Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="if_re.diff" Index: if_re.c =================================================================== RCS file: /home/ncvs/src/sys/dev/re/if_re.c,v retrieving revision 1.35 diff -u -r1.35 if_re.c --- if_re.c 2004/09/28 18:22:24 1.35 +++ if_re.c 2004/10/19 08:09:16 @@ -111,6 +111,7 @@ #include <sys/param.h> #include <sys/endian.h> #include <sys/systm.h> +#include <sys/sysctl.h> #include <sys/sockio.h> #include <sys/mbuf.h> #include <sys/malloc.h> @@ -156,6 +157,9 @@ #define RE_CSUM_FEATURES (CSUM_IP | CSUM_TCP | CSUM_UDP) +static int retimerval = -1; +SYSCTL_INT(_hw, OID_AUTO, re_timer, CTLFLAG_RW, &retimerval, 0, "re tx mod timer value"); + /* * Various supported device vendors/types and their names. */ @@ -1712,11 +1716,8 @@ /* No changes made to the TX ring, so no flush needed */ - if (idx != sc->rl_ldata.rl_tx_considx) { + if (idx != sc->rl_ldata.rl_tx_considx) sc->rl_ldata.rl_tx_considx = idx; - ifp->if_flags &= ~IFF_OACTIVE; - ifp->if_timer = 0; - } /* * If not all descriptors have been released reaped yet, @@ -1726,6 +1727,12 @@ */ if (sc->rl_ldata.rl_tx_free != RL_TX_DESC_CNT) CSR_WRITE_4(sc, RL_TIMERCNT, 1); + + /* only when we are completely done clear the OACTIVE and timer */ + if (sc->rl_ldata.rl_tx_free == RL_TX_DESC_CNT) { + ifp->if_flags &= ~IFF_OACTIVE; + ifp->if_timer = 0; + } } static void @@ -2008,7 +2015,9 @@ struct rl_softc *sc; struct mbuf *m_head = NULL; int idx; + int oactive; + oactive = ifp->if_flags & IFF_OACTIVE; sc = ifp->if_softc; RL_LOCK_ASSERT(sc); @@ -2058,8 +2067,11 @@ * the value in the TIMERINT register, and then trigger an * interrupt. Each time we write to the TIMERCNT register, * the timer count is reset to 0. + * + * Only reset the timer if we are not currently sending pkts */ - CSR_WRITE_4(sc, RL_TIMERCNT, 1); + if (!oactive) + CSR_WRITE_4(sc, RL_TIMERCNT, 1); /* * Set a timeout in case the chip goes out to lunch. @@ -2216,10 +2228,17 @@ * reloaded on each transmit. This gives us TX interrupt * moderation, which dramatically improves TX frame rate. */ + if (retimerval == -1) { + if (sc->rl_type == RL_8169) + retimerval = 0x800; + else + retimerval = 0x400; + } if (sc->rl_type == RL_8169) - CSR_WRITE_4(sc, RL_TIMERINT_8169, 0x800); + CSR_WRITE_4(sc, RL_TIMERINT_8169, retimerval); else - CSR_WRITE_4(sc, RL_TIMERINT, 0x400); + CSR_WRITE_4(sc, RL_TIMERINT, retimerval); + printf("re timer set to: 0x%x\n", retimerval); /* * For 8169 gigE NICs, set the max allowed RX packet --pyE8wggRBhVBcj8z--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20041019081202.GI22681>