Date: Tue, 1 Apr 2008 18:05:29 +0400 From: Anthony Pankov <ap00@mail.ru> To: Rui Paulo <rpaulo@FreeBSD.org> Cc: freebsd-net@freebsd.org, performance@freebsd.org Subject: Re[2]: packet delay because of blackhole Message-ID: <1493139437.20080401180529@mail.ru> In-Reply-To: <20080401104128.GA1194@fnop.net> References: <1333421734.20080328201458@mail.ru> <20080401104128.GA1194@fnop.net>
next in thread | previous in thread | raw e-mail | index | archive | help
Hello Rui, Tuesday, April 01, 2008, 2:41:29 PM, you wrote: RP> On Fri, Mar 28, 2008 at 08:14:58PM +0300, Anthony Pankov wrote: >> Just for somebody convince. >> >> While analyzing client<->server HTTPS conversation one second delay in >> packet exchange was discovered (strongly reproducible): >> >> Sample: >> N time >> 6 0.002303 10.28.4.14 10.28.4.50 SSL Client Hello >> 7 0.106710 10.28.4.50 10.28.4.14 TCP 443 > 1447 [ACK] Seq=1 Ack=103 Win=65535 Len=0 >> 8 1.045712 10.28.4.50 10.28.4.14 TLSv1 Server Hello, Certificate, Server Hello Done >> >> Another sample: >> 10 0.011722 10.28.4.14 10.28.4.50 TLSv1 Application Data >> 11 0.115933 10.28.4.50 10.28.4.14 TCP 443 > 1442 [ACK] Seq=839 Ack=519 Win=65466 Len=0 >> 12 1.054037 10.28.4.50 10.28.4.14 TLSv1 Application Data >> >> The reason for delay is sysctl tcp.blackhole value grater than 0, much to surprise. >> >> So, turning tcp.blackhole to 0 eliminate any delay (strongly reproducible). >> >> System: FreeBSD 6_2_stable RP> I'm not sure how performance penalty can induce a cache miss and I RP> it's very processor specific. So, you're best guess is to profile the RP> kernel. RP> Regards, I'm not fully understand what cache miss do you mean. I'll try to be more clear. During client<->server HTTPS conversation there is a packet delay (see "sample" and "Another sample") about 900 ms. Delay appear one per conversation in random place (between 7-8 packet in "sample", 11-12 in "another sample"). Of course, it's not depending from SSL session cache, SSL negotiation or any other apache/mod_ssl/OpenSSL setting/performance, otherwise i should write to another maillist. I have disabled all my sysctl tuning, one by one. No effect has achieved. But when i turn tcp.blackhole to zero, all things became fine. Maximum delay between packet is 6 ms. It is strange, so i've reported to all. I suggest to keep tcp.blackhole=0 and use firewall for protection. If one would raise tcp.blackhole value, than he should dump packets and make sure that there is no strange delay between packets. It most likely FreeBSD net issue. P.S. "Another sample" is not a sequel of "Sample", it is a dump of different transaction. -- Best regards, Anthony mailto:ap00@mail.ru
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1493139437.20080401180529>