From owner-freebsd-questions@FreeBSD.ORG Thu Sep 19 19:05:23 2013 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id BBA187AD for ; Thu, 19 Sep 2013 19:05:23 +0000 (UTC) (envelope-from dg@pki2.com) Received: from btw.pki2.com (btw.pki2.com [IPv6:2001:470:a:6fd::2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 69D2C221B for ; Thu, 19 Sep 2013 19:05:23 +0000 (UTC) Received: from btw.pki2.com (btw.pki2.com [192.168.23.1]) by btw.pki2.com (8.14.7/8.14.5) with ESMTP id r8JJ5ENh061299 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Thu, 19 Sep 2013 12:05:15 -0700 (PDT) (envelope-from dg@pki2.com) Date: Thu, 19 Sep 2013 12:05:14 -0700 (PDT) From: Dennis Glatting X-X-Sender: dennisg@btw.pki2.com To: Vincent Schut Subject: Re: this 48-core box... In-Reply-To: <20130919155327.115e7344@sarvision.nl> Message-ID: References: <52388C9B.6030205@foxbatcapital.com> <20130919155327.115e7344@sarvision.nl> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-yoursite-MailScanner-Information: Dennis Glatting X-yoursite-MailScanner-ID: r8JJ5ENh061299 X-yoursite-MailScanner: Found to be clean X-MailScanner-From: dg@pki2.com Cc: freebsd-questions@freebsd.org X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Sep 2013 19:05:23 -0000 On Thu, 19 Sep 2013, Vincent Schut wrote: > On Tue, 17 Sep 2013 12:08:43 -0500 > Michael Chen wrote: > >> I'm considering bidding on this 48-core box: >> >> http://www.ebay.com/itm/Supermicro-A-Server-1042G-TF-1U-H8QG6-4-CPUS-48-cores-2-2Ghz-128GB-RAM-/151119828428?pt=COMP_EN_Servers&hash=item232f7195cc >> >> Does anyone have experience with it and can I use all the cores? >> >> Thanks! >> _______________________________________________ >> freebsd-questions@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-questions >> To unsubscribe, send any mail to >> "freebsd-questions-unsubscribe@freebsd.org" >> > > I recently bought one like that (48 cores but 'only' 96 Gb ram). It was > meant to play a double role as both zfs file server and data processing > server (we do lots of satellite image processing), running FreeBSD 9.1. > It connects with a SAN and we'll use it to process about 36TB of > satellite data in the next months. (In a couple of weeks we will > probably have budget to split those roles, and buy a dedicated file > server.) After several weeks of tweaking and testing, I can say that: > - the zfs/file server part runs without problems > - the satellite data processing had problems scaling to all 48 cores, I > got max performance when running about 18 processes in parallel, > scaling up more would lower the overall performance. However, this > (sorry guys) appeared to be a FreeBSD problem, and not a hardware > problem. As a test I switched to linux with ZoL (ZFS on Linux), and, > though zfs performance is less compared to freebsd, data processing > is much much better, like a factor 12 or so. > I've noticed this same scaling problem on 32+ core servers but haven't had a chance to look into the detail. From the performance graphs I am confused whether my problems are processing problems or a data I/O problem. > Conclusion: the hardware is alright, however when needed to do lots of > heavy calculations on terabytes of data, the combination with FreeBSD > appears not ideal. > > Of course it is you get what you pay for. Decent, OK working hardware, > but none of the special handy-dandy features expensive brands will give > you. If you don't need them, in my experience it is decent hardware for > a good price. > > regards, > Vincent. > > _______________________________________________ > freebsd-questions@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-questions > To unsubscribe, send any mail to "freebsd-questions-unsubscribe@freebsd.org" >