From owner-freebsd-jail@FreeBSD.ORG Sun Jan 6 20:20:57 2008 Return-Path: Delivered-To: freebsd-jail@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E2F2C16A417; Sun, 6 Jan 2008 20:20:57 +0000 (UTC) (envelope-from freebsd@hub.org) Received: from hub.org (hub.org [200.46.204.220]) by mx1.freebsd.org (Postfix) with ESMTP id B963213C469; Sun, 6 Jan 2008 20:20:57 +0000 (UTC) (envelope-from freebsd@hub.org) Received: from localhost (unknown [200.46.204.187]) by hub.org (Postfix) with ESMTP id D035011FDD31; Sun, 6 Jan 2008 16:20:56 -0400 (AST) Received: from hub.org ([200.46.204.220]) by localhost (mx1.hub.org [200.46.204.187]) (amavisd-maia, port 10024) with ESMTP id 62671-10; Sun, 6 Jan 2008 16:20:54 -0400 (AST) Received: from fserv.hub.org (blk-7-245-234.eastlink.ca [71.7.245.234]) by hub.org (Postfix) with ESMTP id C019311FDD2B; Sun, 6 Jan 2008 16:20:55 -0400 (AST) Received: from [192.168.1.2] (unknown [192.168.1.2]) by fserv.hub.org (Postfix) with ESMTP id 457798BC01; Sun, 6 Jan 2008 16:20:57 -0400 (AST) Date: Sun, 06 Jan 2008 16:20:50 -0400 From: "Marc G. Fournier" To: Oliver Peter , freebsd-questions@freebsd.org, freebsd-jail@freebsd.org Message-ID: <82C327FC3C9E75F1458EF3B0@ganymede.hub.org> In-Reply-To: <20071023180719.GA22904@nemesis.frida.mouhaha.de> References: <20071023180719.GA22904@nemesis.frida.mouhaha.de> X-Mailer: Mulberry/4.0.8 (Linux/x86) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Disposition: inline Cc: Subject: Re: multiple postgresql servers in multiple jails? X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jan 2008 20:20:58 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Talk about 'old email', but better then 'use different ports', which means you have to modify your clients for this ... after you install postgresql within the jail, vipw the password file and change the uid of the pgsql user (and chown the appropriate files) ... then you can run all instances on port 5432, which clients will expect, but without overrunning shared memory ... - --On Tuesday, October 23, 2007 20:07:19 +0200 Oliver Peter wrote: > Does anybody have a running system with more than one jail hosting > more than one postgres server? > > I can only have one pgsql database on one host at all. I already > tried to increase the shared memory off my machine with additional > kernel [1] and sysctl parameters [2] and I also tried to change > the numeric UID directly in the jails into a seperate one. Same > errors. > > Of course I already have defined jail_sysvipc_allow="YES" in > rc.conf. I have this issue on 6.2-RELEASE-p8 and 8.0-CURRENT with > postgresql-server-8.2.5_1. > > ------------------------------------------------------------------------ > % psql > psql: FATAL: semctl(458753, 15, SETVAL, 0) failed: Invalid argument > ------------------------------------------------------------------------ > > Or some fun with perl/DBD > ------------------------------------------------------------------------ > Out of memory during request for 108 bytes, total sbrk() is 534585344 bytes! > Out of memory during request for 288 bytes, total sbrk() is 534585344 bytes! > Out of memory during request for 288 bytes, total sbrk() is 534585344 bytes! > ------------------------------------------------------------------------ > > [1] > options SYSVSHM > options SYSVSEM > options SYSVMSG > options SHMMAXPGS=65536 > options SEMMNI=40 > options SEMMNS=240 > options SEMUME=40 > options SEMMNU=120 > > [2] http://www.freebsddiary.org/jail-multiple.php > > -- > Oliver PETER, eMail: hoschi@mouhaha.de, ICQ# 113969174 > "Worker bees can leave. Even drones can fly away. > The Queen is their slave." - ---- Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . scrappy@hub.org MSN . scrappy@hub.org Yahoo . yscrappy Skype: hub.org ICQ . 7615664 -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFHgTgi4QvfyHIvDvMRArJ8AJ983aj4+QqbEbs/cFM6UIrby1DUJwCgwZfz rg+dUi5ExXey99rpL/NLb/I= =Z3xL -----END PGP SIGNATURE----- From owner-freebsd-jail@FreeBSD.ORG Mon Jan 7 16:55:39 2008 Return-Path: Delivered-To: freebsd-jail@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 17C1D16A419 for ; Mon, 7 Jan 2008 16:55:39 +0000 (UTC) (envelope-from jrc@rednetgroup.com) Received: from rednetgroup.com (60.175.73.200-static.serversur.net [200.73.175.60]) by mx1.freebsd.org (Postfix) with SMTP id 8358713C457 for ; Mon, 7 Jan 2008 16:55:38 +0000 (UTC) (envelope-from jrc@rednetgroup.com) Received: (qmail 25187 invoked from network); 7 Jan 2008 14:28:56 -0200 Received: from unknown (HELO ?192.168.0.92?) (200.73.175.230) by web.rednetgroup.com with SMTP; 7 Jan 2008 14:28:56 -0200 Message-ID: <4782534C.5010604@rednetgroup.com> Date: Mon, 07 Jan 2008 14:29:00 -0200 From: "Jorge R. Constenla" Organization: Red Net Group SA User-Agent: Thunderbird 2.0.0.9 (Windows/20071031) MIME-Version: 1.0 To: freebsd-jail@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit Subject: How many jails X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jan 2008 16:55:39 -0000 Hi, We planning to move all linux servers to freebsd in jail, and I need to know, How many jails for webhosting can run on a server? Hosting platform: Apache2.2, Perl, PHP5.0, MySQL 5.0, qmail, dovecot, proftpd Servers: - MOTHER INTEL S3000AH - 1 QUAD XEON X3220 2.4 1066 8M INTEL - 4G RAM - DISK 500 GB SATA 16MB W.DIGITAL Thanks in advance -- Jorge R. Constenla jrc@rednetgroup.com Director General ----------------------------- RED NET GROUP SA Internet Services Provider ----------------------------- Av. Corrientes 1189 Piso 8 C1043AAL - Capital Federal Buenos Aires - Argentina ----------------------------- Teléfono: (54 11) 4119.2000 Fax : (54 11) 4119.2005 http://www.rednetgroup.com ----------------------------- From owner-freebsd-jail@FreeBSD.ORG Mon Jan 7 18:12:01 2008 Return-Path: Delivered-To: freebsd-jail@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8D5CB16A41B for ; Mon, 7 Jan 2008 18:12:01 +0000 (UTC) (envelope-from wmoran@collaborativefusion.com) Received: from mx00.pub.collaborativefusion.com (mx00.pub.collaborativefusion.com [206.210.89.199]) by mx1.freebsd.org (Postfix) with ESMTP id 267F013C44B for ; Mon, 7 Jan 2008 18:12:01 +0000 (UTC) (envelope-from wmoran@collaborativefusion.com) Received: from vanquish.pitbpa0.priv.collaborativefusion.com (vanquish.ws.pitbpa0.priv.collaborativefusion.com [192.168.2.162]) (SSL: TLSv1/SSLv3,256bits,AES256-SHA) by wingspan with esmtp; Mon, 07 Jan 2008 13:01:45 -0500 id 00056403.47826909.0000D418 Date: Mon, 7 Jan 2008 13:01:43 -0500 From: Bill Moran To: "Jorge R. Constenla" Message-Id: <20080107130143.7a640678.wmoran@collaborativefusion.com> In-Reply-To: <4782534C.5010604@rednetgroup.com> References: <4782534C.5010604@rednetgroup.com> Organization: Collaborative Fusion X-Mailer: Sylpheed 2.4.7 (GTK+ 2.12.1; i386-portbld-freebsd6.2) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: freebsd-jail@freebsd.org Subject: Re: How many jails X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jan 2008 18:12:01 -0000 In response to "Jorge R. Constenla" : > Hi, > > We planning to move all linux servers to freebsd in jail, and I need to > know, How many jails for webhosting can run on a server? Use the ezjail port, it will make everything about jails easier and more efficient. > Hosting platform: Apache2.2, Perl, PHP5.0, MySQL 5.0, qmail, dovecot, > proftpd > > Servers: > - MOTHER INTEL S3000AH > - 1 QUAD XEON X3220 2.4 1066 8M INTEL > - 4G RAM > - DISK 500 GB SATA 16MB W.DIGITAL Based on those hardware specs, I'm guessing you could run 50 - 60 jails easily. However, this is going to be highly dependent on how much RAM/Disk your individual PHP applications use, and how much CPU load they generate. YMMV. -- Bill Moran Collaborative Fusion Inc. http://people.collaborativefusion.com/~wmoran/ wmoran@collaborativefusion.com Phone: 412-422-3463x4023 From owner-freebsd-jail@FreeBSD.ORG Wed Jan 9 01:20:35 2008 Return-Path: Delivered-To: freebsd-jail@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2F85E16A417 for ; Wed, 9 Jan 2008 01:20:35 +0000 (UTC) (envelope-from andrew@modulus.org) Received: from email.octopus.com.au (host-122-100-2-232.octopus.com.au [122.100.2.232]) by mx1.freebsd.org (Postfix) with ESMTP id EC79A13C461 for ; Wed, 9 Jan 2008 01:20:34 +0000 (UTC) (envelope-from andrew@modulus.org) Received: by email.octopus.com.au (Postfix, from userid 1002) id BD29E1146B; Wed, 9 Jan 2008 12:02:03 +1100 (EST) X-Spam-Checker-Version: SpamAssassin 3.2.3 (2007-08-08) on email.octopus.com.au X-Spam-Level: X-Spam-Status: No, score=-1.4 required=5.0 tests=ALL_TRUSTED autolearn=failed version=3.2.3 Received: from anzac.hos (132.169.233.220.exetel.com.au [220.233.169.132]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: admin@email.octopus.com.au) by email.octopus.com.au (Postfix) with ESMTP id A8BE21143D for ; Wed, 9 Jan 2008 12:01:59 +1100 (EST) Message-ID: <47841D07.20902@modulus.org> Date: Wed, 09 Jan 2008 12:01:59 +1100 From: Andrew Snow User-Agent: Thunderbird 2.0.0.0 (X11/20070426) MIME-Version: 1.0 To: freebsd-jail@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Jails as a VPS X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jan 2008 01:20:35 -0000 Hi Guys, I am running a hoster providing "VPS" using FreeBSD Jails on 6.2 FYI, I have patched my kernel in several places to make it work for me: * jails have their own SYSV shared memory and semaphores * per-jail number of processes limit * jail ability to be bound to a given CPU core * jails have a limited range of nice values (10 to -10) compared to the host environment and last but not least: * memory usage measurement and limiting. It is this last one that is causing me the most problems. I modified obreak() to deny requests for more memory when memory limit is exceeded, and that works OK. But measuring the jail memory usage in the first place is proving to be a pain, and I wonder if you guys have any ideas. I am doing something similar to the Google SoC, by measuring the resident page count of every VM map held by every process in the jail. This does not measure memory fairly - it counts shared memory too many times. To see this in action, I can allocate a jail with 500mb memory limit then try to start 10 or 20 large apache HTTPD processes. While using only a small amount of actual system ram (under 100mb probably), it measures it to be much larger. I am now looking at adding fields to VM memory maps and tagging them so I can ensure I don't count them twice, but this is starting to get non-trivial. Anyone else been able to solve this problem or have any better knowledge? Thanks, - Andrew From owner-freebsd-jail@FreeBSD.ORG Wed Jan 9 12:45:01 2008 Return-Path: Delivered-To: freebsd-jail@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DEFA016A41A for ; Wed, 9 Jan 2008 12:45:01 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from relay02.kiev.sovam.com (relay02.kiev.sovam.com [62.64.120.197]) by mx1.freebsd.org (Postfix) with ESMTP id 6E9C113C44B for ; Wed, 9 Jan 2008 12:45:01 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from [212.82.216.226] (helo=deviant.kiev.zoral.com.ua) by relay02.kiev.sovam.com with esmtps (TLSv1:AES256-SHA:256) (Exim 4.67) (envelope-from ) id 1JCZbK-000OZQ-4I for freebsd-jail@freebsd.org; Wed, 09 Jan 2008 14:00:08 +0200 Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.2/8.14.2) with ESMTP id m09C05lo039418; Wed, 9 Jan 2008 14:00:05 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.2/8.14.2/Submit) id m09C04PG039417; Wed, 9 Jan 2008 14:00:04 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 9 Jan 2008 14:00:04 +0200 From: Kostik Belousov To: Andrew Snow Message-ID: <20080109120004.GV57756@deviant.kiev.zoral.com.ua> References: <47841D07.20902@modulus.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="Bs30vi9tNOOGUgTv" Content-Disposition: inline In-Reply-To: <47841D07.20902@modulus.org> User-Agent: Mutt/1.4.2.3i X-Scanner-Signature: 962db39f35b00980c0b011075312dce6 X-DrWeb-checked: yes X-SpamTest-Envelope-From: kostikbel@gmail.com X-SpamTest-Group-ID: 00000000 X-SpamTest-Info: Profiles 1976 [Dec 29 2007] X-SpamTest-Info: helo_type=3 X-SpamTest-Info: {TO: local part of email appears in body} X-SpamTest-Info: {received from trusted relay: not dialup} X-SpamTest-Method: none X-SpamTest-Method: Local Lists X-SpamTest-Rate: 9 X-SpamTest-Status: Not detected X-SpamTest-Status-Extended: not_detected X-SpamTest-Version: SMTP-Filter Version 3.0.0 [0255], KAS30/Release Cc: freebsd-jail@freebsd.org Subject: Re: Jails as a VPS X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jan 2008 12:45:02 -0000 --Bs30vi9tNOOGUgTv Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Jan 09, 2008 at 12:01:59PM +1100, Andrew Snow wrote: >=20 > Hi Guys, >=20 > I am running a hoster providing "VPS" using FreeBSD Jails on 6.2 >=20 > FYI, I have patched my kernel in several places to make it work for me: > * jails have their own SYSV shared memory and semaphores > * per-jail number of processes limit > * jail ability to be bound to a given CPU core > * jails have a limited range of nice values (10 to -10) compared to=20 > the host environment >=20 > and last but not least: > * memory usage measurement and limiting. >=20 > It is this last one that is causing me the most problems. I modified=20 > obreak() to deny requests for more memory when memory limit is exceeded,= =20 > and that works OK. >=20 > But measuring the jail memory usage in the first place is proving to be= =20 > a pain, and I wonder if you guys have any ideas. >=20 > I am doing something similar to the Google SoC, by measuring the=20 > resident page count of every VM map held by every process in the jail. >=20 > This does not measure memory fairly - it counts shared memory too many=20 > times. To see this in action, I can allocate a jail with 500mb memory=20 > limit then try to start 10 or 20 large apache HTTPD processes. While=20 > using only a small amount of actual system ram (under 100mb probably),=20 > it measures it to be much larger. >=20 > I am now looking at adding fields to VM memory maps and tagging them so= =20 > I can ensure I don't count them twice, but this is starting to get=20 > non-trivial. >=20 > Anyone else been able to solve this problem or have any better knowledge? You may look at the http://people.freebsd.org/~kib/overcommit The text describes the changes more or less accurate, just ignore the status part and the patches itself. The current patch is at http://people.freebsd.org/~kib/overcommit/vm_overcommit.4.patch It does not account nor enforce any per-jail limits, only per uid (as all FreeBSD accounting does at the moment). But, having this done, per-jail and per-uid-in-jail would be much easier. --Bs30vi9tNOOGUgTv Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (FreeBSD) iD8DBQFHhLdDC3+MBN1Mb4gRAn07AKCe4j6m7Enk8Zan2UFDsHqmSPGBxACfdCXq LZKmzSd/zJaOZgF+Wa402jE= =dgcC -----END PGP SIGNATURE----- --Bs30vi9tNOOGUgTv--