From owner-freebsd-arch@FreeBSD.ORG Fri Aug 19 09:29:16 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6860F1065672; Fri, 19 Aug 2011 09:29:16 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) by mx1.freebsd.org (Postfix) with ESMTP id BBFEE8FC16; Fri, 19 Aug 2011 09:29:15 +0000 (UTC) Received: from slw by zxy.spb.ru with local (Exim 4.69 (FreeBSD)) (envelope-from ) id 1QuL1E-000OJq-Ih; Fri, 19 Aug 2011 13:05:36 +0400 Date: Fri, 19 Aug 2011 13:05:36 +0400 From: Slawa Olhovchenkov To: Robert Watson Message-ID: <20110819090536.GA92576@zxy.spb.ru> References: <810527321.20110819123700@serebryakov.spb.ru> <319607032.20110819125005@serebryakov.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.20 (2009-06-14) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false Cc: Lev Serebryakov , freebsd-arch@freebsd.org Subject: Re: FreeBSD problems and preliminary ways to solve X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Aug 2011 09:29:16 -0000 On Fri, Aug 19, 2011 at 09:56:02AM +0100, Robert Watson wrote: > > On Fri, 19 Aug 2011, Lev Serebryakov wrote: > > >> Our network stack is actually pretty parallel as such things go, and there are > >> a number of changes in FreeBSD 9.x that extend this work. Most of the > >> performance work is being done on edge nodes rather than middle nodes -- i.e., > >> maxing out multiple 10gbps links serving content, etc, rather than in routing > >> configurations, though. We also have a strong and growing collection of > >> 10gbps drivers. You'll find our drivers lifted for many other systems, > >> including Solaris :-). > > > > I need to bribe our admins (OPs) on my paid work to try FreeBSD instead of > > Solaris on new servers :) We are processing huge amount of multicast streams > > (up to 2.5-3Gbit/s with 500-1000 bytes packets) and have difficulties not to > > lose any packets on Solaris :) They tried Linux without success, but FreeBSD > > is unknown to them. > > > > One problem for FreeBSD is that our applications are Java-based... Problems > > are not in Java, but in Intel drivers (igb / ixgb in FreeBSD terms), which > > sometimes lose packets with "buffer is not available" diagnostics when > > consumer is heavily-multithreaded. > > Is the issue here that FreeBSD is dropping more packes, or just that FreeBSD > is reporting that it drops packets? Historically, we've returned ENOBUFS from > datagram sockets when the interface queue is overflowed, but some other > systems (most noticeably Linux) simply return success when they drop a packet > on an outgoing interface queue. You can debate which is the better model, but > one impact is that sometimes people report errors on FreeBSD that they don't > see on Linux -- when actually, the same failure is present, we just allow the > application to learn about it. Historically, Linux on datagram (UDP) socket allow use select, FreeBSD -- don't allow. FreeBSD always report 'UDP socket ready to transmit'. And after try to send packet -- 'oops, ENOBUFS'. -- Slawa Olhovchenkov