Date: Tue, 4 Aug 2009 12:26:23 +0200 From: Invernizzi Fabrizio <fabrizio.invernizzi@telecomitalia.it> To: "freebsd-performance@freebsd.org" <freebsd-performance@freebsd.org> Subject: RE: Test on 10GBE Intel based network card Message-ID: <36A93B31228D3B49B691AD31652BCAE9A4569672AD@GRFMBX702BA020.griffon.local> In-Reply-To: <36A93B31228D3B49B691AD31652BCAE9A45696721F@GRFMBX702BA020.griffon.local> References: <36A93B31228D3B49B691AD31652BCAE9A4560DF911@GRFMBX702BA020.griffon.local> <0E567C7E-4EAA-4B89-9A8D-FD0450D32ED7@moneybookers.com> <36A93B31228D3B49B691AD31652BCAE9A4560DF947@GRFMBX702BA020.griffon.local> <4A77094C.8030308@elischer.org> <36A93B31228D3B49B691AD31652BCAE9A45696721F@GRFMBX702BA020.griffon.local>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi all. Going on with my tests. I noticed that I have always a (received) pps/interrupt ratio(per second) t= hat is always about 14 (a very low ratio in my opinion). If I disable aim via sysctl sysctl -w dev.ix.0.enable_aim=3D0 dev.ix.0.enable_aim: 1 -> 0 this ration remain the same. Something going wrong with interrupt coalescin= g, but I can't find out what. Any idea? Fabrizio > -----Original Message----- > From: owner-freebsd-performance@freebsd.org [mailto:owner-freebsd- > performance@freebsd.org] On Behalf Of Invernizzi Fabrizio > Sent: marted=EC 4 agosto 2009 9.55 > To: Julian Elischer > Cc: freebsd-performance@freebsd.org; Stefan Lambrev > Subject: RE: Test on 10GBE Intel based network card > > > >> The limitation that you see is about the max number of packets that > > >> FreeBSD can handle - it looks like your best performance is reached > at > > >> 64 byte packets? > > > > > > If you are meaning in term of Packet per second, you are right. These > > are the packet per second measured during tests: > > > > > > 64 byte: 610119 Pps > > > 512 byte: 516917 Pps > > > 1492 byte: 464962 Pps > > > > > > > > >> Am I correct that the maximum you can reach is around 639,000 packet= s > > >> per second? > > > > > > Yes, as you can see the maximum is 610119 Pps. > > > Where does this limit come from? > > > > ah that's the whole point of tuning :-) > > there are severalpossibities: > > 1/ the card's interrupts are probably attache dto aonly 1 cpu, > > so that cpu can do no more work > > This seems not to be the problem. See below a top snapshot during a > 64byte-long packet storm > > last pid: 8552; load averages: 0.40, 0.09, 0.03 > up 0+20:36:58 09:40:29 > 124 processes: 13 running, 73 sleeping, 38 waiting > CPU: 0.0% user, 0.0% nice, 86.3% system, 12.3% interrupt, 1.5% idle > Mem: 13M Active, 329M Inact, 372M Wired, 68K Cache, 399M Buf, 7207M Free > Swap: 2048M Total, 2048M Free > > PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU > COMMAND > 11 root 1 171 ki31 0K 16K RUN 3 20.2H 51.17% idle: > cpu3 > 14 root 1 171 ki31 0K 16K RUN 0 20.2H 50.88% idle: > cpu0 > 12 root 1 171 ki31 0K 16K RUN 2 20.2H 50.49% idle: > cpu2 > 13 root 1 171 ki31 0K 16K RUN 1 20.2H 50.10% idle: > cpu1 > 42 root 1 -68 - 0K 16K RUN 1 14:20 36.47% ix0 > rxq > 38 root 1 -68 - 0K 16K CPU0 0 14:15 36.08% ix0 > rxq > 44 root 1 -68 - 0K 16K CPU2 2 14:08 34.47% ix0 > rxq > 40 root 1 -68 - 0K 16K CPU3 3 13:42 32.37% ix0 > rxq > .... > > It looks like the 4 rxq processes are bound to the 4 available cores with > equal distribution. > > > > > 2/ if more than 1 cpu is working, it may be that there is a lock in > > heavy contention somewhere. > > This I think is the problem. I am trying to understand how to > 1- see where the heavy contention is (context switching? Some limiting > setting?) > 2- mitigate it > > > > > is the machine still responsive to other networks while > > running at maximum capacity on this network? (make sure that > > the other networks are on a differnet CPU (hmm I can't remember how to > > do that :-). > > > > Questo messaggio e i suoi allegati sono indirizzati esclusivamente alle > persone indicate. La diffusione, copia o qualsiasi altra azione derivante > dalla conoscenza di queste informazioni sono rigorosamente vietate. > Qualora abbiate ricevuto questo documento per errore siete cortesemente > pregati di darne immediata comunicazione al mittente e di provvedere alla > sua distruzione, Grazie. > > This e-mail and any attachments is confidential and may contain privilege= d > information intended for the addressee(s) only. Dissemination, copying, > printing or use by anybody else is unauthorised. If you are not the > intended recipient, please delete this message and any attachments and > advise the sender by return e-mail, Thanks. > > _______________________________________________ > freebsd-performance@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-performance > To unsubscribe, send any mail to "freebsd-performance- > unsubscribe@freebsd.org" Questo messaggio e i suoi allegati sono indirizzati esclusivamente alle per= sone indicate. La diffusione, copia o qualsiasi altra azione derivante dall= a conoscenza di queste informazioni sono rigorosamente vietate. Qualora abb= iate ricevuto questo documento per errore siete cortesemente pregati di dar= ne immediata comunicazione al mittente e di provvedere alla sua distruzione= , Grazie. This e-mail and any attachments is confidential and may contain privileged = information intended for the addressee(s) only. Dissemination, copying, pri= nting or use by anybody else is unauthorised. If you are not the intended r= ecipient, please delete this message and any attachments and advise the sen= der by return e-mail, Thanks.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?36A93B31228D3B49B691AD31652BCAE9A4569672AD>