From owner-freebsd-hackers@FreeBSD.ORG Sun Apr 13 11:28:11 2003 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 57E3D37B417; Sun, 13 Apr 2003 11:28:11 -0700 (PDT) Received: from samson.dc.luth.se (samson.dc.luth.se [130.240.112.30]) by mx1.FreeBSD.org (Postfix) with ESMTP id C109D43F75; Sun, 13 Apr 2003 11:28:09 -0700 (PDT) (envelope-from bj@dc.luth.se) Received: from dc.luth.se (root@bompe.dc.luth.se [130.240.60.42]) by samson.dc.luth.se (8.12.5/8.12.5) with ESMTP id h3DIRxLG014071; Sun, 13 Apr 2003 20:27:59 +0200 (MET DST) Received: from bompe.dc.luth.se (bj@localhost.dc.luth.se [127.0.0.1]) by dc.luth.se (8.12.6/8.11.3) with ESMTP id h3DIRv2F004468; Sun, 13 Apr 2003 20:27:57 +0200 (CEST) (envelope-from bj@bompe.dc.luth.se) Message-Id: <200304131827.h3DIRv2F004468@dc.luth.se> X-Mailer: exmh version 2.5 07/13/2001 with nmh-1.0.4 To: Terry Lambert In-reply-to: Your message of Fri, 11 Apr 2003 09:32:51 PDT. <3E96EE33.FAF4FABB@mindspring.com> Dcc: X-Disposition-notification-to: Borje.Josefsson@dc.luth.se X-uri: http://www.dc.luth.se/~bj/index.html Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Date: Sun, 13 Apr 2003 20:27:57 +0200 From: Borje Josefsson cc: freebsd-hackers@freebsd.org cc: freebsd-performance@freebsd.org cc: "Jin Guojun \[DSD\]" cc: Eric Anderson cc: David Gilbert Subject: Re: tcp_output starving -- is due to mbuf get delay? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: bj@dc.luth.se List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Apr 2003 18:28:11 -0000 On Fri, 11 Apr 2003 09:32:51 PDT Terry Lambert wrote: > See other posting; I listed a bunch of OIDs to play with. > = > One other, if you are running -CURRENT, would be: > = > net.isr.netisr_enable -> 1 I found "net.isr.enable", I hope that is what You mean. > This basically implements part 1 of 3 of LRP, which should > reduce your per packet latency by about 50ms +/- 50ms. > = > Note: The logic here is inverted; you'd expect "0=3DNo NETISR", > but it's just the opposite. I tried to install -current on a third system (just a PIII-933, with 33MH= z = bus) today: root@stinky 4# ttcp -s -t -f m -l 61440 -n 20345 dino ttcp-t: buflen=3D61440, nbuf=3D20345, align=3D16384/0, port=3D5001 tcp -= > dino ttcp-t: socket ttcp-t: connect ttcp-t: 1249996800 bytes in 27.04 real seconds =3D 352.65 Mbit/sec +++ ttcp-t: 20345 I/O calls, msec/call =3D 1.36, calls/sec =3D 752.31 ttcp-t: 0.0user 15.6sys 0:27real 57% 15i+359d 420maxrss 0+0pf 16+44950csw= root@stinky 5# = root@stinky 5# sysctl net.isr.enable=3D1 net.isr.enable: 0 -> 1 root@stinky 6# ttcp -s -t -f m -l 61440 -n 20345 dino ttcp-t: buflen=3D61440, nbuf=3D20345, align=3D16384/0, port=3D5001 tcp -= > dino ttcp-t: socket ttcp-t: connect ttcp-t: 1249996800 bytes in 27.27 real seconds =3D 349.70 Mbit/sec +++ ttcp-t: 20345 I/O calls, msec/call =3D 1.37, calls/sec =3D 746.03 ttcp-t: 0.0user 16.4sys 0:27real 60% 15i+355d 420maxrss 0+0pf 15+70547csw= I.e. no change. The same host with NetBSD gives me at least 525 Mbit/sec.= The symptoms with -current are the same as before. 100% CPU load. Note the strange behaviour on netstat below. I start at 50kpps (output), = and then fall back to 27,5k (steady). input (Total) output packets errs bytes packets errs bytes colls 16177 0 1067840 28767 0 43555599 0 28563 0 1885158 50583 0 76573756 0 28566 0 1885356 50582 0 76575270 0 28492 0 1880472 50451 0 76381478 0 28538 0 1883508 50534 0 76511682 0 27723 0 1829718 49096 0 74325466 0 15173 0 1001418 27498 0 41628940 0 15268 0 1007688 27581 0 41755578 0 15230 0 1005180 27498 0 41628732 0 15280 0 1008480 27498 0 41627892 0 15290 0 1009140 27624 0 41820384 0 15244 0 1006104 27498 0 41628172 0 15229 0 1005114 27498 0 41628108 0 15287 0 1008942 27540 0 41692056 0 15200 0 1003194 27413 0 41500866 0 15292 0 1009272 27625 0 41820522 0 15221 0 1004586 27411 0 41498918 0 15239 0 1005774 27540 0 41692552 0 15206 0 1003596 27540 0 41692752 0 15265 0 1007490 27584 0 41756928 0 15222 0 1004652 27539 0 41691102 0 15209 0 1003794 27540 0 41692944 0 15185 0 1002210 27496 0 41627448 0 15253 0 1006698 27540 0 41693344 0 15191 0 1002606 27496 0 41626656 0 15176 0 1001616 27540 0 41692880 0 15232 0 1005312 27627 0 41821342 0 2876 0 189816 5183 0 7838606 0 stinky# netstat -ss tcp: 863330 packets sent 863320 data packets (1250011712 bytes) 1 data packet (1448 bytes) retransmitted 9 ack-only packets (0 delayed) 1 control packet 483425 packets received 446021 acks (for 1250011665 bytes) 29 packets (1344 bytes) received in-sequence 37376 window update packets 1 connection request 1 connection established (including accepts) 1 connection updated cached RTT on close 1 connection updated cached RTT variance on close 446021 segments updated rtt (of 21215 attempts) 11864 correct ACK header predictions 26 correct data packet header predictions If time permits tomorrow, I'll install -current on one of the faster = hosts, but I don't think it will make any significant difference. --B=F6rje