From owner-freebsd-virtualization@FreeBSD.ORG Sun Feb 8 14:42:03 2009 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 09C6E1065673 for ; Sun, 8 Feb 2009 14:42:03 +0000 (UTC) (envelope-from ticso@cicely7.cicely.de) Received: from raven.bwct.de (raven.bwct.de [85.159.14.73]) by mx1.freebsd.org (Postfix) with ESMTP id 6B4408FC12 for ; Sun, 8 Feb 2009 14:42:02 +0000 (UTC) (envelope-from ticso@cicely7.cicely.de) Received: from cicely5.cicely.de ([10.1.1.7]) by raven.bwct.de (8.13.4/8.13.4) with ESMTP id n18Efwta047095 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Sun, 8 Feb 2009 15:41:59 +0100 (CET) (envelope-from ticso@cicely7.cicely.de) Received: from cicely7.cicely.de (cicely7.cicely.de [10.1.1.9]) by cicely5.cicely.de (8.14.2/8.14.2) with ESMTP id n18Efueu018202 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sun, 8 Feb 2009 15:41:56 +0100 (CET) (envelope-from ticso@cicely7.cicely.de) Received: from cicely7.cicely.de (localhost [127.0.0.1]) by cicely7.cicely.de (8.14.2/8.14.2) with ESMTP id n18Efuxl038391; Sun, 8 Feb 2009 15:41:56 +0100 (CET) (envelope-from ticso@cicely7.cicely.de) Received: (from ticso@localhost) by cicely7.cicely.de (8.14.2/8.14.2/Submit) id n18EftUk038390; Sun, 8 Feb 2009 15:41:55 +0100 (CET) (envelope-from ticso) Date: Sun, 8 Feb 2009 15:41:55 +0100 From: Bernd Walter To: Ragnar Lonn Message-ID: <20090208144155.GN32126@cicely7.cicely.de> References: <498DF945.3000702@gatorhole.com> <498E0797.4040002@elischer.org> <498EC554.4020905@gatorhole.com> <20090208130435.GL32126@cicely7.cicely.de> <498EE22E.7020005@gatorhole.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <498EE22E.7020005@gatorhole.com> X-Operating-System: FreeBSD cicely7.cicely.de 7.0-STABLE i386 User-Agent: Mutt/1.5.11 X-Spam-Status: No, score=-4.3 required=5.0 tests=ALL_TRUSTED=-1.8, AWL=0.052, BAYES_00=-2.599 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on spamd.cicely.de Cc: ticso@cicely.de, freebsd-virtualization@freebsd.org, Julian Elischer Subject: Re: More open sockets with vimages? X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: ticso@cicely.de List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 Feb 2009 14:42:03 -0000 On Sun, Feb 08, 2009 at 02:46:22PM +0100, Ragnar Lonn wrote: > Bernd Walter wrote: > >This is simple maths: > >100k Sockets with 32k TX and 64k RX buffer take 9G Memory. > >Just buffer space, not to mention socket state, ... > >On i386 this is limited by kmem, which defaults to IIRC 512MB and > >is limited by 32bit virtual address space on i386. > >On amd64 depending on the OS version you can have a kmem of slighty > >less than 2G max or several GB. > >Nevertheless you are still limited with physical RAM. > >Smaller buffers are possible, but usually people want larger buffers > >to keep up with recent line speeds. > >Today buffer sizes can be dynamic - don't know the exact details, but > >you should keep in mind that 32k/96k is already quite small for > >many purposes. > > > > But physical memory is cheap, and most low-end machines can have 16G or > more today. Is it just a matter of having enough RAM and a 64-bit OS > then? How much is "several GB [kmem]" that you mention above? AFAIK it is the only limitation - people are using 100k+ sockets since at least FreeBSD-4, but with several restrictions because of memory. It mostly depends on your application and network topology to your peers. Don't know where the current kmem limits exactly are - AFAIK kmem is hold within KVA and KVA is limited by a static map size. It has been widely discussed recently, because ZFS loves a large kmem. -- B.Walter http://www.bwct.de Modbus/TCP Ethernet I/O Baugruppen, ARM basierte FreeBSD Rechner uvm.