Date: Sat, 12 Apr 2003 15:57:11 -0400 From: Hiten Pandya <hiten@unixdaemons.com> To: Mike Silbersack <silby@silby.com> Cc: freebsd-hackers@freebsd.org Subject: Re: tcp_output starving -- is due to mbuf get delay? Message-ID: <20030412195711.GA30459@unixdaemons.com> In-Reply-To: <20030410114227.A472@odysseus.silby.com> References: <200304101311.h3ADBgjY022790@samson.dc.luth.se> <20030410114227.A472@odysseus.silby.com>
next in thread | previous in thread | raw e-mail | index | archive | help
[ cross post to hackers removed, thanks ]
Mike Silbersack (Thu, Apr 10, 2003 at 11:44:42AM -0500) wrote:
>
> On Thu, 10 Apr 2003, Borje Josefsson wrote:
>
> > What we did in NetBSD (-current) was to increase IFQ_MAXLEN in (their)
> > sys/net/if.h, apart from that it's only "traditional" TCP tuning.
> >
> > My hosts are connected directly to core routers in a 10Gbps nationwide
> > network, so if anybody is interested in some testing I am more than
> > willing to participate. If anybody produces a patch, I have a third system
> > that I can use for piloting of that too.
> >
> > --B?rje
>
> This brings up something I've been wondering about, which you might want
> to investigate:
>
> >From tcp_output:
>
> if (error == ENOBUFS) {
> if (!callout_active(tp->tt_rexmt) &&
> !callout_active(tp->tt_persist))
> callout_reset(tp->tt_rexmt, tp->t_rxtcur,
> tcp_timer_rexmt, tp);
> tcp_quench(tp->t_inpcb, 0);
> return (0);
> }
>
> That tcp_quench knocks the window size back to one packet, if I'm not
> mistaken. You might want to put a counter there and see if that's
> happening frequently to you; if so, it might explain some loss of
> performance.
Maybe something like this:
%%%
Index: sys/netinet/tcp_output.c
===================================================================
RCS file: /home/ncvs/src/sys/netinet/tcp_output.c,v
retrieving revision 1.78
diff -u -r1.78 tcp_output.c
--- sys/netinet/tcp_output.c 19 Feb 2003 22:18:05 -0000 1.78
+++ sys/netinet/tcp_output.c 12 Apr 2003 19:52:31 -0000
@@ -930,6 +930,7 @@
!callout_active(tp->tt_persist))
callout_reset(tp->tt_rexmt, tp->t_rxtcur,
tcp_timer_rexmt, tp);
+ tcpstat.tcps_selfquench++;
tcp_quench(tp->t_inpcb, 0);
return (0);
}
Index: sys/netinet/tcp_var.h
===================================================================
RCS file: /home/ncvs/src/sys/netinet/tcp_var.h,v
retrieving revision 1.88
diff -u -r1.88 tcp_var.h
--- sys/netinet/tcp_var.h 1 Apr 2003 21:16:46 -0000 1.88
+++ sys/netinet/tcp_var.h 12 Apr 2003 19:52:31 -0000
@@ -394,6 +394,8 @@
u_long tcps_sc_zonefail; /* zalloc() failed */
u_long tcps_sc_sendcookie; /* SYN cookie sent */
u_long tcps_sc_recvcookie; /* SYN cookie received */
+
+ u_long tcps_selfquench; /* self-quench count */
};
/*
Index: usr.bin/netstat/inet.c
===================================================================
RCS file: /home/ncvs/src/usr.bin/netstat/inet.c,v
retrieving revision 1.58
diff -u -r1.58 inet.c
--- usr.bin/netstat/inet.c 2 Apr 2003 20:14:44 -0000 1.58
+++ usr.bin/netstat/inet.c 12 Apr 2003 19:52:32 -0000
@@ -389,6 +389,7 @@
p(tcps_sndprobe, "\t\t%lu window probe packet%s\n");
p(tcps_sndwinup, "\t\t%lu window update packet%s\n");
p(tcps_sndctrl, "\t\t%lu control packet%s\n");
+ p(tcps_selfquench, "\t\t%lu send%s resulting in self-quench\n");
p(tcps_rcvtotal, "\t%lu packet%s received\n");
p2(tcps_rcvackpack, tcps_rcvackbyte, "\t\t%lu ack%s (for %lu byte%s)\n");
p(tcps_rcvdupack, "\t\t%lu duplicate ack%s\n");
%%%
Cheers.
-- Hiten
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20030412195711.GA30459>