From owner-freebsd-questions@FreeBSD.ORG Sat Jun 3 16:28:43 2006 Return-Path: X-Original-To: freebsd-questions@freebsd.org Delivered-To: freebsd-questions@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7F36E16A478 for ; Sat, 3 Jun 2006 16:28:43 +0000 (UTC) (envelope-from danial_thom@yahoo.com) Received: from web33307.mail.mud.yahoo.com (web33307.mail.mud.yahoo.com [68.142.206.122]) by mx1.FreeBSD.org (Postfix) with SMTP id A6CB843D45 for ; Sat, 3 Jun 2006 16:28:42 +0000 (GMT) (envelope-from danial_thom@yahoo.com) Received: (qmail 24829 invoked by uid 60001); 3 Jun 2006 16:28:42 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:Received:Date:From:Reply-To:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=HQ4pQH9vpqerX5MT6n6WJiqtSVsETKpEtAGo2oGp3mCyVS8Y5XVgAoQ/dTOytnJcsBHSxGu5+/JHvjaoCUUROjoLK02/q1YJv0Fzchwi3K2F3aPjmfXKj6iTQXtAjAldhoCVd4Tr2VW5dKnZJU6ff5Xb8U+WkdCwtIWi1oCyov0= ; Message-ID: <20060603162842.24827.qmail@web33307.mail.mud.yahoo.com> Received: from [65.34.182.15] by web33307.mail.mud.yahoo.com via HTTP; Sat, 03 Jun 2006 09:28:42 PDT Date: Sat, 3 Jun 2006 09:28:42 -0700 (PDT) From: Danial Thom To: Steve Kargl , freebsd-questions@freebsd.org In-Reply-To: <20060603153847.GA36267@troutmask.apl.washington.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Cc: Subject: Re: Tuning GigE network for cluster computing? X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: danial_thom@yahoo.com List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Jun 2006 16:28:44 -0000 --- Steve Kargl wrote: > First, I'm running FreeBSD 6.1-stable on a > small cluster > with 6 nodes that contain Tyan motheriboards. > These broads > have Broadcom GigE NICs that use the bge > device. > > The cluster will be using MPI to possibly shove > large data > sets through a GigE switch, so I'm trying to > determine how > best to optimize the transfer. > > Would DEVICE_POLLING and/or jumbo frames likely > methods for > enhanced speeds? > > -- > Steve --- Steve Kargl wrote: > First, I'm running FreeBSD 6.1-stable on a > small cluster > with 6 nodes that contain Tyan motheriboards. > These broads > have Broadcom GigE NICs that use the bge > device. > > The cluster will be using MPI to possibly shove > large data > sets through a GigE switch, so I'm trying to > determine how > best to optimize the transfer. > > Would DEVICE_POLLING and/or jumbo frames likely > methods for > enhanced speeds? Firstly we've been discussing how bad and inefficient the broadcom controllers are. If you have a tyan MB with onboard controllers they are incredibly slow (ie inefficient), as well as being quirky. If you have a PCI-X slot put in an intel card. They have built-in interrupt moderation so you don't have dick around with polling. A 133Mhz intel controller (make sure they have the GB chips on them and not the EB) will use half the cpu of the on-board broadcoms. If you don't believe try it with one box and measure the cpu usage before and after. Its worth the $50. investment in the card, believe me. While polling may marginally decrease the cpu load (depending on how bad FreeBSD 6.1 is on interrupt overhead), its also going to add latency to the processing of packets, which is the opposite of what you want to do. DT __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com