Date: Wed, 05 Dec 2007 12:31:12 -0500 From: Mike Tancsa <mike@sentex.net> To: Philipp Wuensche <cryx-freebsd@h3q.com> Cc: freebsd-performance@freebsd.org Subject: Re: tuning for high connection rates Message-ID: <200712051729.lB5HTAkn029896@lava.sentex.ca> In-Reply-To: <4756DE7E.1000104@h3q.com> References: <4755ED57.6030603@h3q.com> <200712051616.lB5GGGvb029587@lava.sentex.ca> <4756DE7E.1000104@h3q.com>
next in thread | previous in thread | raw e-mail | index | archive | help
At 12:23 PM 12/5/2007, Philipp Wuensche wrote:
>Mike Tancsa wrote:
> > At 07:14 PM 12/4/2007, Philipp Wuensche wrote:
> >
> >> The debug output of em0 looks like this:
> >>
> >> em0: CTRL = 0x40140248 RCTL = 0x8002
> >> em0: Packet buffer = Tx=20k Rx=12k
> >> em0: Flow control watermarks high = 10240 low = 8740
> >> em0: tx_int_delay = 66, tx_abs_int_delay = 66
> >> em0: rx_int_delay = 32, rx_abs_int_delay = 66
> >> em0: fifo workaround = 0, fifo_reset_count = 0
> >> em0: hw tdh = 183, hw tdt = 183
> >> em0: hw rdh = 139, hw rdt = 139
> >> em0: Num Tx descriptors avail = 223
> >> em0: Tx Descriptors not avail1 = 6225
> >> em0: Tx Descriptors not avail2 = 3
> >> em0: Std mbuf failed = 0
> >> em0: Std mbuf cluster failed = 0
> >> em0: Driver dropped packets = 0
> >> em0: Driver tx dma failure in encap = 0
> >
> > If you do a
> > sysctl -w dev.em.0.stats=1
> >
> > It will spit the nic stats to syslog. What are the results ?
>
>em0: Excessive collisions = 0
>em0: Sequence errors = 0
>em0: Defer count = 0
>em0: Missed Packets = 12876719
>em0: Receive No Buffers = 5950326
>em0: Receive Length Errors = 0
>em0: Receive errors = 0
>em0: Crc errors = 0
>em0: Alignment errors = 0
>em0: Collision/Carrier extension errors = 0
>em0: RX overruns = 56256
>em0: watchdog timeouts = 0
>em0: XON Rcvd = 0
>em0: XON Xmtd = 0
>em0: XOFF Rcvd = 0
>em0: XOFF Xmtd = 0
>em0: Good Packets Rcvd = 3384408375
>em0: Good Packets Xmtd = 2657550034
>em0: TSO Contexts Xmtd = 6925441
>em0: TSO Contexts Failed = 0
>
>
> > Also,
> > what does ifconfig em0 look like (i.e. what options do you have set,
> > speed etc)
>
>Its on a 100Mbit/s switch, we haven't changed the options:
>
>em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
> options=19b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4>
> media: Ethernet autoselect (100baseTX <full-duplex>)
Some people have reported that TSO is a "bad thing" on 100Mb. Can
you try disabling that ? Also, you seem to have a lot of RX overruns
and missed packets such that the nic cannot process things fast
enough. I havent done any benchmarks yet, but the Yandex people
claim their modified EM driver can handle higher PPS rates than the
stock em driver. Not sure if they have a RELENG_7 port or not but
they might have some insight.
---Mike
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200712051729.lB5HTAkn029896>