From owner-freebsd-hackers@FreeBSD.ORG Mon Mar 9 18:27:03 2009 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E8220106564A for ; Mon, 9 Mar 2009 18:27:03 +0000 (UTC) (envelope-from cliftonr@lava.net) Received: from outgoing01.lava.net (cake.lava.net [IPv6:2001:1888:0:1:230:48ff:fe5b:3b50]) by mx1.freebsd.org (Postfix) with ESMTP id 352228FC12 for ; Mon, 9 Mar 2009 18:27:03 +0000 (UTC) (envelope-from cliftonr@lava.net) Received: from malasada.lava.net (malasada.lava.net [64.65.64.17]) by outgoing01.lava.net (Postfix) with ESMTP id 5184ED0069; Mon, 9 Mar 2009 08:27:02 -1000 (HST) Received: by malasada.lava.net (Postfix, from userid 102) id 04804153882; Mon, 9 Mar 2009 08:27:01 -1000 (HST) Date: Mon, 9 Mar 2009 08:27:01 -1000 From: Clifton Royston To: freebsd-hackers@freebsd.org Message-ID: <20090309182700.GA20062@lava.net> Mail-Followup-To: freebsd-hackers@freebsd.org, Martin Badie References: <20090308120022.523F81065672@hub.freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20090308120022.523F81065672@hub.freebsd.org> User-Agent: Mutt/1.4.2.2i Cc: Martin Badie Subject: Re: freebsd-hackers Digest, Vol 310, Issue 6 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Mar 2009 18:27:04 -0000 On Sun, Mar 08, 2009 at 12:00:22PM +0000, freebsd-hackers-request@freebsd.org wrote: > Date: Sat, 7 Mar 2009 13:08:56 -0800 (PST) > From: Martin Badie > Subject: select.h FD_SETSIZE and Qmail-Postfix test > To: freebsd-hackers@freebsd.org > Message-ID: <67469.69113.qm@web59906.mail.ac4.yahoo.com> > Content-Type: text/plain; charset=us-ascii > > Hi, > > There is a test that I am doing with FreeBSD and Linux. This test > involves qmail and postfix comparison. Both FreeBSD and Linux seems > to have 1024 File Descriptor limit. (FD_SETSIZE in select.h in > FreeBSD) . > > To have a better concurrency in qmail on smtp level. I have used a > patch named big-todo patch also used big-concurrency patch. These > patches helps me to increase concurrency in operating system. I set > concurrent connection to 500(tcpserver -c 500). There is no problem > until around 400-500 active smtp connection. But if the total smtp > connection exceeds 500, load average increases to ~40-50 but cpu > system time arises to %50-60. The strange issue is that, this load > increases when the connection is limited to accept 500 connections > but the tool I use is configured to 700 (more than 500) connections. > Normally ucspi-tcp software limits connection to 500 ( -c 500) I > suspect it is something to do with Operating system level. > > Additionally I have also patched FreeBSD kernel with 4096 FD_SETSIZE > in select.h in kernel and booted with that kernel. I have also > compiled qmail from scratch to accept 2040 connections (in > conf-spawn) but there is no change I mean I still can't get more than > decent 500 connections with acceptable load average. > > I have also used postfix on both Linux FreeBSD: > > default_process_limit = 500 > smtpd_client_connection_count_limit = 500 > > but I still get strange load when connection raises more than 500 > > I suspect something is missing or need to be configured on the operating system level (both Linux and FreeBSD) One point which you might be missing is that both FreeBSD and Linux (and I think most other modern OSes) have long since deprecated the select interface for high performance/high concurrency software. On FreeBSD the preferred mechanism is kqueue, and IIRC Postfix prefers to build with the kqueue interface on FreeBSD. Linux uses something else which escapes me at the moment; perhaps epoll? This makes benchmarks on select() primarily of historic interest. -- Clifton -- Clifton Royston -- cliftonr@iandicomputing.com / cliftonr@lava.net President - I and I Computing * http://www.iandicomputing.com/ Custom programming, network design, systems and network consulting services