From owner-freebsd-arch@FreeBSD.ORG Sat Jan 15 10:49:36 2005 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CF6A716A4CE for ; Sat, 15 Jan 2005 10:49:36 +0000 (GMT) Received: from xorpc.icir.org (xorpc.icir.org [192.150.187.68]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8637D43D41 for ; Sat, 15 Jan 2005 10:49:36 +0000 (GMT) (envelope-from rizzo@icir.org) Received: from xorpc.icir.org (localhost [127.0.0.1]) by xorpc.icir.org (8.12.11/8.12.8) with ESMTP id j0FAnUjM032025; Sat, 15 Jan 2005 02:49:30 -0800 (PST) (envelope-from rizzo@xorpc.icir.org) Received: (from rizzo@localhost) by xorpc.icir.org (8.12.11/8.12.3/Submit) id j0FAnU3V032024; Sat, 15 Jan 2005 02:49:30 -0800 (PST) (envelope-from rizzo) Date: Sat, 15 Jan 2005 02:49:30 -0800 From: Luigi Rizzo To: Yar Tikhiy Message-ID: <20050115024930.A31992@xorpc.icir.org> References: <18962.1104749259@critter.freebsd.dk> <20050103065715.A67451@xorpc.icir.org> <20050115103607.GA81277@comp.chem.msu.su> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: <20050115103607.GA81277@comp.chem.msu.su>; from yar@comp.chem.msu.su on Sat, Jan 15, 2005 at 01:36:07PM +0300 cc: arch@freebsd.org cc: Poul-Henning Kamp Subject: Re: Socket rate limiting (was: Re: making nmdm(4) emulate actual speed.) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Jan 2005 10:49:36 -0000 two comments: the reasons why apps implement throttling on their own are 1) portability - they cannot assume the required services are available on the platforms they are going to run on; 2) scheduling and resource management - apps have their own requirement on how to schedule things and it is often unlikely that they can express them in term of altq/dummynet "classes". This said, there is no problem in having thousands of dummynet pipes -- the algorithms used in dummynet have O(log N) cost where N is the number of active pipes. cheers luigi On Sat, Jan 15, 2005 at 01:36:07PM +0300, Yar Tikhiy wrote: ... > A lot of network daemons implement rate limits by their own. For > instance, Apache httpd has mod_throttle, Squid has delay pools, and > even lukemftpd has some sort of rate limiting code in it. With the > demand for rate limiting being so high, it could be natural for an > OS to provide such service to applications through, e.g., a per-socket > option. > > However, I've got an impression from the first glance at the issue > that it would be next to impossible to implement such service in a > fashion independent of lower layers of network abstraction. I > suppose that we have such elaborate and IP-centric subsystems as > DUMMYNET and ALTQ partly because of the complexity of the task. > Therefore a possible approach is to implement the rate-limit socket > option using one of those existing subsystems. Such solution would > scale poorly though--imagine a server with thousands of rate-limited > sockets open, each of them requiring a separate DUMMYNET pipe. Did > anybody have other thoughts or see publications regarding the > problem? > > -- > Yar