From owner-freebsd-performance@FreeBSD.ORG Tue Feb 1 17:04:42 2011 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BF7F2106566B; Tue, 1 Feb 2011 17:04:42 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail06.syd.optusnet.com.au (mail06.syd.optusnet.com.au [211.29.132.187]) by mx1.freebsd.org (Postfix) with ESMTP id 26FC88FC16; Tue, 1 Feb 2011 17:04:41 +0000 (UTC) Received: from c122-106-165-206.carlnfd1.nsw.optusnet.com.au (c122-106-165-206.carlnfd1.nsw.optusnet.com.au [122.106.165.206]) by mail06.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id p11H4aAR017659 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 2 Feb 2011 04:04:38 +1100 Date: Wed, 2 Feb 2011 04:04:36 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Slawa Olhovchenkov In-Reply-To: <20110201113724.GS18170@zxy.spb.ru> Message-ID: <20110202034337.E1550@besplex.bde.org> References: <20110128143355.GD18170@zxy.spb.ru> <22E77EED-6455-4164-9115-BBD359EC8CA6@moneybookers.com> <20110128161035.GF18170@zxy.spb.ru> <4D42F87C.7020909@freebsd.org> <20110128172516.GG18170@zxy.spb.ru> <20110129070205.Q7034@besplex.bde.org> <20110201113724.GS18170@zxy.spb.ru> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-performance@freebsd.org, Julian Elischer , Bruce Evans , Stefan Lambrev Subject: Re: Interrupt performance X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Feb 2011 17:04:42 -0000 On Tue, 1 Feb 2011, Slawa Olhovchenkov wrote: > I do some more test and build kernel with KTR. > Now I don't think that inetrrupt overhead on FreeBSD weight: I try > polling and don't see any difference. > > I see many reported by netperf send errors. I found this > http://docs.freebsd.org/cgi/mid.cgi?E1Aice9-0002by-00. > > After insert into src/nettest_bsd.c usleep(1000) if ENOBUF I see 53% > idle and ./loop 2000000000 "Elapsed 15188006 us" -- this near to linux > (Elapsed 14107670 us). This partly works around the problem that it is impossible to select() on the ENOBUFS condition in FreeBSD at least, and thus impossible to write ttcp or nettest correctly. The userland sender either has to sleep for a while it gets an ENOBUFS error, and thus let the hardware sender go idle in the interval between the condition becoming clear and the sleep finishing, or it has to retry immediately and thus consume 100% CPU getting ENOBUFS errors until the condition clears. I use HZ = 100. Thus usleep(1000) would actually sleep for an average of 15000 us, and the system would be idle (doing nothing) for about 10 times as long as with HZ = 1000. I uses an old version of ttcp which tries to sleep for 18000 us. This ensures that the the sleep is too long even with HZ = 1000 (except I changed 1 line in the old ttcp to either not sleep at all or to try to sleep for only 1000 us). Not sleeping at all uses 100% CPU, but since I mostly use this for testing the maximum packet rate I don't care much about that unless the CPU being used by ttcp interferes with kernel and/or hardware activity. Another reply said that Linux blocks on ENOBUFS instead of returning it. That seems better, provided it doesn't block in the O_NOBLOCK case. This should involve select() working so that you can avoid the block even in the !O_NONBLOCK case. Correct versions of ttcp and maybe nettest can then be written very easily -- at least ttcp would prefer to just block in sendto(). Bruce