From owner-freebsd-arch@FreeBSD.ORG Wed Aug 10 12:03:03 2005 Return-Path: X-Original-To: arch@freebsd.org Delivered-To: freebsd-arch@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9087616A41F for ; Wed, 10 Aug 2005 12:03:03 +0000 (GMT) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4A03643D70 for ; Wed, 10 Aug 2005 12:03:00 +0000 (GMT) (envelope-from andre@freebsd.org) Received: (qmail 14984 invoked from network); 10 Aug 2005 11:44:39 -0000 Received: from dotat.atdotat.at (HELO [62.48.0.47]) ([62.48.0.47]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 10 Aug 2005 11:44:39 -0000 Message-ID: <42F9ECF2.8080809@freebsd.org> Date: Wed, 10 Aug 2005 14:02:58 +0200 From: Andre Oppermann User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8b) Gecko/20050217 MIME-Version: 1.0 To: arch@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Subject: Special schedulers, one CPU only kernel, one only userland X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Aug 2005 12:03:03 -0000 When using FreeBSD as a high performance router there are some desirable changes to the way multiple CPUs are handled. Normally a second CPU doesn't add much (if any) performance to routing because of locking overhead and packets randomly being processed by the CPUs wasting cache efficiency. On the other hand having just one CPU is not optimal in running the routing daemon in userland. When there are large changes to the table (eg. BGP full feed flap) userland sucks time away from the packet forwarding in the kernel. The idea is to combine both worlds by designating CPU0 exclusively for all kernel processing (thus avoiding the expensive mutex synchronization and bus locking instructions) and CPU1 exclusively for all userland processing. Whenever a userland program does a syscall the kernel CPU will take over. When it's done, the process get run by the userland CPU again. That way we get a very good scalability out of two CPUs for this particular task. Hence my question to the SMP and scheduler gurus: How well does the current SMP and scheduler architecture lend itself to this kind of special handling? Is it just a matter of modifying (or plugging in) the schedule or are there more involved things to consider? -- Andre