From owner-freebsd-stable Thu Jul 5 19:24:41 2001 Delivered-To: freebsd-stable@freebsd.org Received: from peter3.wemm.org (c1315225-a.plstn1.sfba.home.com [65.0.135.147]) by hub.freebsd.org (Postfix) with ESMTP id 70CE537B405; Thu, 5 Jul 2001 19:24:34 -0700 (PDT) (envelope-from peter@wemm.org) Received: from overcee.netplex.com.au (overcee.wemm.org [10.0.0.3]) by peter3.wemm.org (8.11.0/8.11.0) with ESMTP id f662OYM65792; Thu, 5 Jul 2001 19:24:34 -0700 (PDT) (envelope-from peter@wemm.org) Received: from wemm.org (localhost [127.0.0.1]) by overcee.netplex.com.au (Postfix) with ESMTP id 43DC93809; Thu, 5 Jul 2001 19:24:34 -0700 (PDT) (envelope-from peter@wemm.org) X-Mailer: exmh version 2.3.1 01/18/2001 with nmh-1.0.4 To: Greg Lehey Cc: Len Conrad , freebsd-hackers@FreeBSD.ORG, freebsd-stable@FreeBSD.ORG Subject: Re: helping Wietse help postfix on FreeBSD In-Reply-To: <20010705175538.C455@sydney.worldwide.lemis.com> Content-Transfer-Encoding: 8bit Date: Thu, 05 Jul 2001 19:24:34 -0700 From: Peter Wemm Message-Id: <20010706022434.43DC93809@overcee.netplex.com.au> Sender: owner-freebsd-stable@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Greg Lehey wrote: > On Tuesday, 3 July 2001 at 23:08:57 +0200, Len Conrad wrote: > > I´m trying to gather tuning information for Wietse Venema who says: > > > > "I'm writing a document that describes how to crank up FreeBSD so > > that it can run lots of processes, and so that it can handle lots > > of connections. > > > > Right now, these guidelines vary from sysctl, loader.conf, to > > recompiling a kernel. This is confusing." > > > > Of course, what he means "lots of connections" required by 100´s and > > 100´s of smtp and smtpd processes in postfix. > > What system does Wietse normally use? I've just discovered how very > badly Linux handles a large number of processes, and it's possible > that Wietse thinks that FreeBSD is as bad. Hundreds of processes are > not an issue; ftp.FreeBSD.org used to handle thousands. Before we fix > a problem, we should know what it is, and if it even exists. The other interesting thing to throw into the mix is that mx2.freebsd.org really causes some pain for postfix. We needed to make quite a few tweaks to get it to run well. postfix's biggest problem is that it uses select() and is extremely vulnerable to select collisions (as in: millions per day on mx2.freebsd.org at one point). FreeBSD's scheduler is not up to the task. We run 512 parallel delivery smtp senders on mx2, and this causes severe pain for the kernel. It thrashes priorities and does generally really unhappy things. We had to do things like run certain processes in realtime priority mode to avoid the starvation when the scheduler drastically overcompensates and moves the key processes out of the top priority band slot (slot 0 in the 32 run queues). We also had to make the context switching happen a lot faster. Running sched_cpu() at 1 second intervals just SUCKS because it takes a full second to correct drastic miscalculations. (we run HZ faster, reduce kern.quantum, tweak the sched_cpu run rate and algorithms, run stuff in rtprio mode (qmgr), etc) In spite of this, we do get the majority of the mailing list recipients delivered in a few seconds. Cheers, -Peter -- Peter Wemm - peter@FreeBSD.org; peter@yahoo-inc.com; peter@netplex.com.au "All of this is for nothing if we don't go to the stars" - JMS/B5 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-stable" in the body of the message