From owner-freebsd-hackers Mon Apr 24 15:10:58 2000 Delivered-To: freebsd-hackers@freebsd.org Received: from canonware.com (canonware.com [199.181.107.39]) by hub.freebsd.org (Postfix) with SMTP id 58E5937B5F2 for ; Mon, 24 Apr 2000 15:10:56 -0700 (PDT) (envelope-from jasone@canonware.com) Received: (qmail 39107 invoked by uid 1001); 24 Apr 2000 22:00:52 -0000 Date: Mon, 24 Apr 2000 15:00:52 -0700 From: Jason Evans To: "Richard Seaman, Jr." Cc: freebsd-hackers@FreeBSD.ORG Subject: Re: Multithreaded server performance Message-ID: <20000424150052.J31925@sturm.canonware.com> References: <3903AEA6.FA7CBBAB@freenet.co.uk> <20000423212115.E31925@sturm.canonware.com> <20000424164405.G370@seaman.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0.1i In-Reply-To: <20000424164405.G370@seaman.org>; from dick@seaman.org on Mon, Apr 24, 2000 at 04:44:05PM -0500 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Mon, Apr 24, 2000 at 04:44:05PM -0500, Richard Seaman, Jr. wrote: > On Sun, Apr 23, 2000 at 09:21:15PM -0700, Jason Evans wrote: > > FreeBSD's threads should perform adequately given the design of your program > > and the hardware you listed. Actually trying it on the various operating > > systems would be a good exercise, but I have found that FreeBSD's threads > > perform at least as well as Solaris's for such an application. > > Interesting. I would have thought Solaris would be much superior. > > > LinuxThreads will definitely bog down with so many threads because the > > kernel scheduler has to deal with so many clone()d processes. > > Somewhat. But its nothing compared to the slowdown in the libc_r scheduler, > in my experience. Each context switch (or each time the libc_r scheduler > examines its state to consider a context switch), it executes a poll call > to examine the state of *each* file descriptor that is considered "blocked" > (of course its not really blocked in the kernel -- libc_r converts all > i/o calls to non-blocking and polls their status to determined when i/o > can be done). If you have hundreds/thousands of threads and each has an > open file descriptor that has to be polled, linuxthreads is much faster, > at least for the tests I've run. However, linuxthreads will consume a > lot more kernel resources, so this only holds if you can tolerate the load > on your resources. Interesting. Unfortunately, when I was developing the server I referred to, I was never able to get past about 100 connections on FreeBSD without crashing the kernel, because of the static buffer limits in the kernel (yes, I manually specified very large mbuf limits). As a result, I never experienced the scaling problems you mention. =) Linux runs into problems at less than 4000 threads because of a limit on the total number of processes, even if the thread stack size is decreased. I have run threaded tests on Solaris with over 30,000 connections without problems, other than kernel deadlock due to resource starvation when stuffing too many kernel-side socket buffers with data. > > FreeBSD's libc_r does not use clone() or anything similar. Instead, it is > > a userland call conversion library that multiplexes threads in a single > > process. This style of threads library should perform well for the type of > > application you are dealing with. > > Perhaps. The proposed "new" threads model that I understand you are working > on should be *much* superior in this kind of application to either libc_r or > linuxthreads, I would think. I hope so, because it's going to be a bit of work. =) Jason To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message