From owner-freebsd-current@FreeBSD.ORG Tue Mar 30 12:22:38 2004 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 966F016A4CE for ; Tue, 30 Mar 2004 12:22:38 -0800 (PST) Received: from anduin.net (anduin.net [212.12.46.226]) by mx1.FreeBSD.org (Postfix) with SMTP id DC0A643D1F for ; Tue, 30 Mar 2004 12:22:37 -0800 (PST) (envelope-from ltning@anduin.net) Received: (qmail 598 invoked by uid 6759); 30 Mar 2004 20:22:37 -0000 Received: from ltning@anduin.net by anduin.net by uid 82 with qmail-scanner-1.20 (clamscan: 0.60. spamassassin: 2.60. Clear:RC:1(217.8.136.185):. Processed in 0.026971 secs); 30 Mar 2004 20:22:37 -0000 X-Qmail-Scanner-Mail-From: ltning@anduin.net via anduin.net X-Qmail-Scanner: 1.20 (Clear:RC:1(217.8.136.185):. Processed in 0.026971 secs) Received: from gatekeeper.in-space.org (HELO anduin.net) (217.8.136.185) by anduin.net with SMTP; 30 Mar 2004 20:22:36 -0000 Message-ID: <4069D708.9040609@anduin.net> Date: Tue, 30 Mar 2004 22:22:32 +0200 From: Eirik Oeverby User-Agent: Mozilla Thunderbird 0.5 (X11/20040228) X-Accept-Language: en-us, en MIME-Version: 1.0 To: =?ISO-8859-1?Q?Dag-Erling_Sm=F8rgrav?= References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit cc: current@freebsd.org Subject: Re: performance of jailed processes X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Mar 2004 20:22:38 -0000 Hi there, and I thought I was using jails on a 'big' scale... 400 jails on one single box, that's pretty amazing! What kind of jails are these, i.e. what are they used for? Encapsulating single processes/tasks only, or more complex things too? And what hardware are you on, CPU and memory-wise? /Eirik Dag-Erling Smørgrav wrote: > Robert Watson writes: > >>- DNS -- I know you mentioned it, but I'd check anyway. Especially if >> resolv.conf has bad DNS servers in it in the jails, etc. You might try >> writing a trivial gethostbyname() test app and timing it in and out of >> the jail. Also look at the reverse lookup done by the MySQL server. >> The impact of the source IP address might be particularly interesting. > > > Packet traces already show that there is no delay between query and > reply, the reply just takes a long time to transmit. > > >>- It would be interesting to know if applications outside the jail bound >> to various IP addresses see performance differences depending on the IP >> used. We have hashed IP address lookup, but there are some operations >> in the stack that require walking the list of addresses, etc. If the >> non-jailed software always uses the first address because they're all in >> the same subnet, that might conceivably make a difference. Taking jail >> out of the picture in some basic micro-benchmarks might help here also. > > > Non-jailed software always uses the first IP address, which is in its > own subnet. The jails draw from a pool of ~1000 IP addresses on the > same interface, but in a different subnet. The jail I've been testing > in is about a quarter of the way down the list. > > >>Can you identify any micro-benchmarks rather than macro-benchmarks that >>reflect a significant difference? > > > haven't had much luck with that... fetch, for instance, doesn't seem > to suffer, but with mysql the difference is dramatic: > > (outside jail) > 1 row in set (0.01 sec) > > (inside jail) > 1 row in set (13.20 sec) > > note that 13 seconds is far too short for a DNS issue, and that the > time reported is measured *after* login (i.e. after any DNS lookup) > > DES