From owner-freebsd-jail@FreeBSD.ORG Mon Feb 6 11:07:05 2012 Return-Path: Delivered-To: freebsd-jail@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 237EF1065678 for ; Mon, 6 Feb 2012 11:07:05 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 1158F8FC23 for ; Mon, 6 Feb 2012 11:07:05 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q16B748R007852 for ; Mon, 6 Feb 2012 11:07:04 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q16B73dc007850 for freebsd-jail@FreeBSD.org; Mon, 6 Feb 2012 11:07:04 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 6 Feb 2012 11:07:04 GMT Message-Id: <201202061107.q16B73dc007850@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-jail@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-jail@FreeBSD.org X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Feb 2012 11:07:05 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- p bin/161957 jail jls(8): jls -v doesn't show anything if system compile o kern/159918 jail [jail] inter-jail communication failure o kern/156111 jail [jail] procstat -b not supported in jail o misc/155765 jail [patch] `buildworld' does not honors WITHOUT_JAIL o conf/154246 jail [jail] [patch] Bad symlink created if devfs mount poin o conf/149050 jail [jail] rcorder ``nojail'' too coarse for Jail+VNET s conf/142972 jail [jail] [patch] Support JAILv2 and vnet in rc.d/jail o conf/141317 jail [patch] uncorrect jail stop in /etc/rc.d/jail o kern/133265 jail [jail] is there a solution how to run nfs client in ja o kern/119842 jail [smbfs] [jail] "Bad address" with smbfs inside a jail o bin/99566 jail [jail] [patch] fstat(1) according to specified jid o bin/32828 jail [jail] w(1) incorrectly handles stale utmp slots with 12 problems total. From owner-freebsd-jail@FreeBSD.ORG Mon Feb 6 20:29:20 2012 Return-Path: Delivered-To: freebsd-jail@FreeBSD.org Received: from mx2.freebsd.org (mx2.freebsd.org [IPv6:2001:4f8:fff6::35]) by hub.freebsd.org (Postfix) with ESMTP id 535F0106566B for ; Mon, 6 Feb 2012 20:29:20 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from 172-17-150-251.globalsuite.net (hub.freebsd.org [IPv6:2001:4f8:fff6::36]) by mx2.freebsd.org (Postfix) with ESMTP id 5130314E518; Mon, 6 Feb 2012 20:29:19 +0000 (UTC) Message-ID: <4F30381E.2020100@FreeBSD.org> Date: Mon, 06 Feb 2012 12:29:18 -0800 From: Doug Barton Organization: http://SupersetSolutions.com/ User-Agent: Mozilla/5.0 (X11; FreeBSD i386; rv:10.0) Gecko/20120201 Thunderbird/10.0 MIME-Version: 1.0 To: freebsd-jail@FreeBSD.org X-Enigmail-Version: 1.3.5 OpenPGP: id=1A1ABC84 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Subject: Practical limit to number of jails on a given host? X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Feb 2012 20:29:20 -0000 Howdy, Thinking about implementing a poor-man's virtualization solution with lots'o'jails, and wondering what people think about the practical limits of such a system. I realize that part of the answer is going to depend on CPU and RAM, so let's assume for the sake of argument that the answer to that bit is, "Lots of both." So first question is, is there some sort of hard-coded limit somewhere? If not, what is the largest number of jails that you've created successfully/reliably on a system, and what are the specs for that system? On a related note, what are the limits in terms of mount points on the system and/or jails? I'm thinking of a fairly typical "nullfs mount the system, devfs, and 2 or 3 NFS mount points" per jail type of situation. And finally, has anyone run into trouble with a large number of IP addresses for the jails? ISTR that way back when, the IP addresses associated with a particular interface were stored in a linked list, so as you added more you would start seeing O(N) slowdown on a lot of network stuff in the kernel. Any thoughts or advice along these lines will be greatly appreciated. :) Doug -- It's always a long day; 86400 doesn't fit into a short. Breadth of IT experience, and depth of knowledge in the DNS. Yours for the right price. :) http://SupersetSolutions.com/ From owner-freebsd-jail@FreeBSD.ORG Mon Feb 6 20:53:04 2012 Return-Path: Delivered-To: freebsd-jail@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E3B99106564A; Mon, 6 Feb 2012 20:53:04 +0000 (UTC) (envelope-from prvs=1383ec7016=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) by mx1.freebsd.org (Postfix) with ESMTP id 402168FC16; Mon, 6 Feb 2012 20:53:03 +0000 (UTC) X-Spam-Processed: mail1.multiplay.co.uk, Mon, 06 Feb 2012 20:41:39 +0000 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on mail1.multiplay.co.uk X-Spam-Level: X-Spam-Status: No, score=-5.0 required=6.0 tests=USER_IN_WHITELIST shortcircuit=ham autolearn=disabled version=3.2.5 Received: from r2d2 ([188.220.16.49]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50017912421.msg; Mon, 06 Feb 2012 20:41:37 +0000 X-MDRemoteIP: 188.220.16.49 X-Return-Path: prvs=1383ec7016=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk Message-ID: From: "Steven Hartland" To: "Doug Barton" , References: <4F30381E.2020100@FreeBSD.org> Date: Mon, 6 Feb 2012 20:41:40 -0000 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: Subject: Re: Practical limit to number of jails on a given host? X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Feb 2012 20:53:05 -0000 ----- Original Message ----- From: "Doug Barton" > So first question is, is there some sort of hard-coded limit somewhere? > If not, what is the largest number of jails that you've created > successfully/reliably on a system, and what are the specs for that system? We happilly run up ~80 single process jails on 24 core machines without issue. One thing to be aware of is the issue with prison0->uref becoming 0 and panicing the machine if a fix for this hasnt been commited yet. If you need a working patch for this I can provide one. Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From owner-freebsd-jail@FreeBSD.ORG Mon Feb 6 20:58:59 2012 Return-Path: Delivered-To: freebsd-jail@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 176FA106566C; Mon, 6 Feb 2012 20:58:59 +0000 (UTC) (envelope-from wmoran@collaborativefusion.com) Received: from mx00.pub.collaborativefusion.com (mx00.pub.collaborativefusion.com [206.210.89.199]) by mx1.freebsd.org (Postfix) with ESMTP id B5C0A8FC15; Mon, 6 Feb 2012 20:58:58 +0000 (UTC) Received: from soundwave.ws.pitbpa0.priv.collaborativefusion.com (soundwave.ws.pitbpa0.priv.collaborativefusion.com [192.168.2.119]) (AUTH: LOGIN wmoran, TLS: TLSv1/SSLv3,256bits,AES256-SHA) by wingspan with esmtp; Mon, 06 Feb 2012 15:48:54 -0500 id 0003F403.000000004F303CB6.0000918F Date: Mon, 6 Feb 2012 15:48:54 -0500 From: Bill Moran To: Doug Barton Message-Id: <20120206154854.8b6ff961.wmoran@collaborativefusion.com> In-Reply-To: <4F30381E.2020100@FreeBSD.org> References: <4F30381E.2020100@FreeBSD.org> Organization: Bill Moran X-Mailer: Sylpheed 3.1.0 (GTK+ 2.24.4; x86_64-pc-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: freebsd-jail@FreeBSD.org Subject: Re: Practical limit to number of jails on a given host? X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Feb 2012 20:58:59 -0000 In response to Doug Barton : > > Any thoughts or advice along these lines will be greatly appreciated. :) I don't know of any hardcoded limits, but I can give you some specifics of what I know works. We have production servers running 40 jails. They perform a variety of functions from database jails that are constantly busy to batch job jails that sit idle 99% of the time. These systems have 16G of RAM, 16 CPUs, and about 200 nullfs mounts. They run like production-quality stuff should, long uptimes (only rebooted for OS upgrades) and very little maintenance or babysitting required. HTH -- http://www.intermedix.com Bill Moran Senior Vice President of Databases Phone: 412-422-3463x4023 The information contained in this message is confidential and may be privileged and/or protected under law. If you received this message in error, please notify us immediately by forwarding a copy to karen.collier@intermedix.com and then deleting the original message and any attachments. From owner-freebsd-jail@FreeBSD.ORG Mon Feb 6 21:14:51 2012 Return-Path: Delivered-To: freebsd-jail@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6AC1A106566B; Mon, 6 Feb 2012 21:14:51 +0000 (UTC) (envelope-from freebsd@hub.org) Received: from hub.org (hub.org [200.46.204.220]) by mx1.freebsd.org (Postfix) with ESMTP id 2DBD88FC15; Mon, 6 Feb 2012 21:14:50 +0000 (UTC) Received: from maia.hub.org (unknown [200.46.151.189]) by hub.org (Postfix) with ESMTP id B974F1EA602A; Mon, 6 Feb 2012 16:59:08 -0400 (AST) Received: from hub.org ([200.46.204.220]) by maia.hub.org (mx1.hub.org [200.46.151.189]) (amavisd-maia, port 10024) with ESMTP id 46573-02; Mon, 6 Feb 2012 20:59:08 +0000 (UTC) Received: from [192.168.1.4] (24-246-4-43.cable.teksavvy.com [24.246.4.43]) by hub.org (Postfix) with ESMTPA id 958211EA6028; Mon, 6 Feb 2012 16:59:06 -0400 (AST) Mime-Version: 1.0 (Apple Message framework v1257) Content-Type: text/plain; charset=windows-1252 From: Hub- FreeBSD In-Reply-To: <4F30381E.2020100@FreeBSD.org> Date: Mon, 6 Feb 2012 15:59:05 -0500 Content-Transfer-Encoding: quoted-printable Message-Id: References: <4F30381E.2020100@FreeBSD.org> To: Doug Barton X-Mailer: Apple Mail (2.1257) Cc: freebsd-jail@FreeBSD.org Subject: Re: Practical limit to number of jails on a given host? X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Feb 2012 21:14:51 -0000 On 2012-02-06, at 3:29 PM, Doug Barton wrote: >=20 > So first question is, is there some sort of hard-coded limit = somewhere? > If not, what is the largest number of jails that you've created > successfully/reliably on a system, and what are the specs for that = system? ~150 full jails (postfix+cyrus-imapd+apache) on a Quad Core Xeon with = 32G of RAM ... they weren't high load sites, obviously, but loading was generally <1, = and the machine was perfectly responsive =85= From owner-freebsd-jail@FreeBSD.ORG Mon Feb 6 21:18:07 2012 Return-Path: Delivered-To: freebsd-jail@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 62602106564A for ; Mon, 6 Feb 2012 21:18:07 +0000 (UTC) (envelope-from ltning@anduin.net) Received: from mail.modirum.com (mail.modirum.com [31.185.27.10]) by mx1.freebsd.org (Postfix) with ESMTP id 02F728FC08 for ; Mon, 6 Feb 2012 21:18:06 +0000 (UTC) Received: from [84.38.152.7] (helo=ranger.home.anduin.net) by mail.modirum.com with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.77 (FreeBSD)) (envelope-from ) id 1RuVws-000NPl-Ol; Mon, 06 Feb 2012 21:18:06 +0000 Mime-Version: 1.0 (Apple Message framework v1257) Content-Type: text/plain; charset=iso-8859-1 From: =?iso-8859-1?Q?Eirik_=D8verby?= In-Reply-To: <4F30381E.2020100@FreeBSD.org> Date: Mon, 6 Feb 2012 22:18:04 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <744BFFD8-23A6-4583-A266-B4976F494CC1@anduin.net> References: <4F30381E.2020100@FreeBSD.org> To: Doug Barton X-Mailer: Apple Mail (2.1257) X-SA-Authenticated: Yes X-SA-Exim-Connect-IP: 84.38.152.7 X-SA-Exim-Mail-From: ltning@anduin.net X-SA-Exim-Scanned: No (on mail.modirum.com); SAEximRunCond expanded to false Cc: freebsd-jail@FreeBSD.org Subject: Re: Practical limit to number of jails on a given host? X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Feb 2012 21:18:07 -0000 On Feb 6, 2012, at 21:29, Doug Barton wrote: > Howdy, >=20 > Thinking about implementing a poor-man's virtualization solution with > lots'o'jails, and wondering what people think about the practical = limits > of such a system. I realize that part of the answer is going to depend > on CPU and RAM, so let's assume for the sake of argument that the = answer > to that bit is, "Lots of both." Worry more about disk I/O.=20 ZFS with fast spindles in raid-Z combined with SSD L2ARC and ZIL got me = much, much further than only spindles, but in the end I caved and did = SSD across the board on the most busy jail hosts. They have anywhere = between 40 and 70 jails running, many of them very busy, all of them = different. The process count seen from the host is in the low four = digits. > So first question is, is there some sort of hard-coded limit = somewhere? > If not, what is the largest number of jails that you've created > successfully/reliably on a system, and what are the specs for that = system? I've - for the sake of testing - had about 350 jails on one system, each = with a mysql, a java/tomcat, and an nginx. They all worked and responded = fine to queries. I have no reason to think it would be a problem to add = more. The system in question was a 12-core (2 CPU), 48GB system. > On a related note, what are the limits in terms of mount points on the > system and/or jails? I'm thinking of a fairly typical "nullfs mount = the > system, devfs, and 2 or 3 NFS mount points" per jail type of = situation. I have no idea about NFS in such a setting; I use nullfs (ro) for all = the system stuff (6 per jail iirc), and use zfs datasets for /, /tmp, = /var, /etc and /usr/local inside the jails. Devfs of course. I implement = filesystem quotas and the likes using zfs, along with compression for = datasets that generally benefit from that. Make sure you allow for enough open files. Also make sure any postgreses = you allow are on different UIDs (unless 9.x has a new way of "fixing" = that sysv limitation). If you use ZFS, it might be an idea to limit the = ARC size (loader.conf) to avoid ZFS gobbling up all the free memory = after booting but before processes in the jails have ballooned). And make sure you have plenty of swap. You don't want to swap, but if = things get hot it's better to have a slowdown from swapping than having = random processes being killed off ;) > And finally, has anyone run into trouble with a large number of IP > addresses for the jails? ISTR that way back when, the IP addresses > associated with a particular interface were stored in a linked list, = so > as you added more you would start seeing O(N) slowdown on a lot of > network stuff in the kernel. I remember DES complained about his 1-something ghz athlon getting slow = with 1500 jails due to this. That was back around ..5-BETA? I remember = laughing long and hard at the insanity of 1500 jails on one box, and = even more at him being surprised that "something" would barf .. But I am = pretty sure it was fixed soon after. > Any thoughts or advice along these lines will be greatly appreciated. = :) >=20 >=20 > Doug >=20 > --=20 >=20 > It's always a long day; 86400 doesn't fit into a short. >=20 > Breadth of IT experience, and depth of knowledge in the DNS. > Yours for the right price. :) http://SupersetSolutions.com/ >=20 > _______________________________________________ > freebsd-jail@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-jail > To unsubscribe, send any mail to = "freebsd-jail-unsubscribe@freebsd.org" >=20 From owner-freebsd-jail@FreeBSD.ORG Tue Feb 7 09:49:37 2012 Return-Path: Delivered-To: freebsd-jail@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3337A106566C; Tue, 7 Feb 2012 09:49:37 +0000 (UTC) (envelope-from bzeeb-lists@lists.zabbadoz.net) Received: from mx1.sbone.de (mx1.sbone.de [IPv6:2a01:4f8:130:3ffc::401:25]) by mx1.freebsd.org (Postfix) with ESMTP id DAA4A8FC0C; Tue, 7 Feb 2012 09:49:36 +0000 (UTC) Received: from mail.sbone.de (mail.sbone.de [IPv6:fde9:577b:c1a9:31::2013:587]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.sbone.de (Postfix) with ESMTPS id 2C78325D3899; Tue, 7 Feb 2012 09:49:36 +0000 (UTC) Received: from content-filter.sbone.de (content-filter.sbone.de [IPv6:fde9:577b:c1a9:31::2013:2742]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPS id 6D844BDAD61; Tue, 7 Feb 2012 09:49:35 +0000 (UTC) X-Virus-Scanned: amavisd-new at sbone.de Received: from mail.sbone.de ([IPv6:fde9:577b:c1a9:31::2013:587]) by content-filter.sbone.de (content-filter.sbone.de [fde9:577b:c1a9:31::2013:2742]) (amavisd-new, port 10024) with ESMTP id UWBcLgKrafTI; Tue, 7 Feb 2012 09:49:34 +0000 (UTC) Received: from orange-en1.sbone.de (orange-en1.sbone.de [IPv6:fde9:577b:c1a9:31:cabc:c8ff:fecf:e8e3]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPSA id 195E2BDAD60; Tue, 7 Feb 2012 09:49:34 +0000 (UTC) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: "Bjoern A. Zeeb" In-Reply-To: <4F30381E.2020100@FreeBSD.org> Date: Tue, 7 Feb 2012 09:49:33 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: <5F01F653-A001-4EA1-9A99-F5C14CC39755@lists.zabbadoz.net> References: <4F30381E.2020100@FreeBSD.org> To: Doug Barton X-Mailer: Apple Mail (2.1084) Cc: freebsd-jail@FreeBSD.org Subject: Re: Practical limit to number of jails on a given host? X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Feb 2012 09:49:37 -0000 On 6. Feb 2012, at 20:29 , Doug Barton wrote: > Howdy, >=20 > Thinking about implementing a poor-man's virtualization solution with > lots'o'jails, and wondering what people think about the practical = limits > of such a system. I realize that part of the answer is going to depend > on CPU and RAM, so let's assume for the sake of argument that the = answer > to that bit is, "Lots of both." >=20 > So first question is, is there some sort of hard-coded limit = somewhere? > If not, what is the largest number of jails that you've created > successfully/reliably on a system, and what are the specs for that = system? Yes, jails provide you 6 9s ... though that's not 99.9999% but 999999 is the maximum number of jails. And yes, I have started this many before = -- without processes or anything. It took a couple of days, due to some list handling, which could be optimized. You will find that once you get there, you'll have a syscall which never returns... You notice once the start loop slows down if you print a . every 100 or = 1000. The machine was a 4 or 8 core amd64 with 8G of memory. I think I had a slide in there: = http://www.bsdcan.org/2010/schedule/attachments/130_2010-bz-the-new-vvorld= .pdf I know if using vnets; you can get the 4k (or more) but I also have = reports that it may not scale. The other limit you'll run into is the number of PIDs. And eventually scheduling depending on what you want to do. > And finally, has anyone run into trouble with a large number of IP > addresses for the jails? ISTR that way back when, the IP addresses > associated with a particular interface were stored in a linked list, = so > as you added more you would start seeing O(N) slowdown on a lot of > network stuff in the kernel. Yeah, we still do list walks here and there. /bz --=20 Bjoern A. Zeeb You have to have visions! It does not matter how good you are. It matters what good you do! From owner-freebsd-jail@FreeBSD.ORG Wed Feb 8 03:05:59 2012 Return-Path: Delivered-To: freebsd-jail@FreeBSD.org Received: from mx2.freebsd.org (mx2.freebsd.org [IPv6:2001:4f8:fff6::35]) by hub.freebsd.org (Postfix) with ESMTP id B8C2F1065686 for ; Wed, 8 Feb 2012 03:05:59 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from 172-17-150-251.globalsuite.net (hub.freebsd.org [IPv6:2001:4f8:fff6::36]) by mx2.freebsd.org (Postfix) with ESMTP id EC1CB158E35; Wed, 8 Feb 2012 03:05:58 +0000 (UTC) Message-ID: <4F31E696.6070400@FreeBSD.org> Date: Tue, 07 Feb 2012 19:05:58 -0800 From: Doug Barton Organization: http://SupersetSolutions.com/ User-Agent: Mozilla/5.0 (X11; FreeBSD i386; rv:10.0) Gecko/20120201 Thunderbird/10.0 MIME-Version: 1.0 To: freebsd-jail@FreeBSD.org References: <4F30381E.2020100@FreeBSD.org> <5F01F653-A001-4EA1-9A99-F5C14CC39755@lists.zabbadoz.net> In-Reply-To: <5F01F653-A001-4EA1-9A99-F5C14CC39755@lists.zabbadoz.net> X-Enigmail-Version: 1.3.5 OpenPGP: id=1A1ABC84 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Subject: Re: Practical limit to number of jails on a given host? X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Feb 2012 03:05:59 -0000 Thanks everyone for the very helpful responses. :) -- It's always a long day; 86400 doesn't fit into a short. Breadth of IT experience, and depth of knowledge in the DNS. Yours for the right price. :) http://SupersetSolutions.com/ From owner-freebsd-jail@FreeBSD.ORG Thu Feb 9 15:06:46 2012 Return-Path: Delivered-To: freebsd-jail@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 796971065673; Thu, 9 Feb 2012 15:06:46 +0000 (UTC) (envelope-from chris_bender@cellularatsea.com) Received: from wireless.icgws.com (wireless.icgws.com [198.211.94.23]) by mx1.freebsd.org (Postfix) with ESMTP id 381938FC16; Thu, 9 Feb 2012 15:06:46 +0000 (UTC) Received: by wireless.icgws.com (Postfix, from userid 1003) id 7B775180DE8; Thu, 9 Feb 2012 10:05:35 -0500 (EST) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on wireless.icgws.com X-Spam-Level: X-Spam-Status: No, score=-2.9 required=3.5 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.3.1 Received: from wmstp.corp.cellularatsea.com (unknown [10.200.104.195]) by wireless.icgws.com (Postfix) with SMTP id B9860180CD5; Thu, 9 Feb 2012 10:05:34 -0500 (EST) Received: from wmstp.corp.wms.cellularatsea.com (localhost [127.0.0.1]) by wmstp.corp.cellularatsea.com (Postfix) with SMTP id 55CC8B1885D; Thu, 9 Feb 2012 10:06:25 -0500 (EST) Received: from wmsexg01.corp.cellularatsea.com ([10.200.104.15] helo=wmsexg01.corp.cellularatsea.com) by wmstp.corp.wms.cellularatsea.com with SMTP (ASSP 1.9.1.1); 9 Feb 2012 10:06:25 -0500 X-Ninja-PIM: Scanned by Ninja X-MimeOLE: Produced By Microsoft Exchange V6.5 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Content-class: urn:content-classes:message x-vipre-scanned: 005903C8002CFE00590515 Date: Thu, 9 Feb 2012 10:06:23 -0500 Message-ID: In-Reply-To: <4F2AAFCA.4060804@FreeBSD.org> Thread-Topic: jails Thread-Index: AczhwdvAjUMVmOBHR1+90UOih1dHJgFegbuA References: <863259E16B6C464DAD1E9DD10BB311540582EC53@wmsexg01.corp.cellularatsea.com> <4F27FD77.30409@FreeBSD.org> <863259E16B6C464DAD1E9DD10BB311540582ED0E@wmsexg01.corp.cellularatsea.com> <4F28273A.1070905@FreeBSD.org> <863259E16B6C464DAD1E9DD10BB311540582ED21@wmsexg01.corp.cellularatsea.com> <4F285235.1010407@FreeBSD.org> <863259E16B6C464DAD1E9DD10BB311540582EE18@wmsexg01.corp.cellularatsea.com> <4F2897F5.90900@FreeBSD.org> <863259E16B6C464DAD1E9DD10BB311540582F0A6@wmsexg01.corp.cellularatsea.com> <4F2AAFCA.4060804@FreeBSD.org> From: "Bender, Chris" To: X-Assp-Whitelisted: Yes () X-Assp-Envelope-From: chris_bender@cellularatsea.com X-Assp-Intended-For: glarkin@FreeBSD.org X-Assp-Passing: 10.200.104.15 in acceptAllMail X-Assp-ID: wmstp.corp.wms.cellularatsea.com (32879-30819) X-Assp-Version: 1.9.1.1(1.0.00) Cc: freebsd-jail@freebsd.org Subject: RE: jails X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Feb 2012 15:06:46 -0000 Hi Greg, I am having an issue with one of my jailed systems. It has run out of space. I have identified many files to delete but I can not=20 Delete the files as the system comes back with "No Space available". = I tried to delete them from the host system as well but I get The same system issue. How does one delete files or free up space? Thanks for your help in advance! From owner-freebsd-jail@FreeBSD.ORG Thu Feb 9 16:05:11 2012 Return-Path: Delivered-To: freebsd-jail@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id ADB76106566B for ; Thu, 9 Feb 2012 16:05:11 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) by mx1.freebsd.org (Postfix) with ESMTP id 6AF828FC12 for ; Thu, 9 Feb 2012 16:05:10 +0000 (UTC) Received: from elsa.codelab.cz (localhost [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id C125728433; Thu, 9 Feb 2012 16:45:44 +0100 (CET) Received: from [192.168.1.2] (ip-86-49-61-235.net.upcbroadband.cz [86.49.61.235]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id 0665F28431; Thu, 9 Feb 2012 16:45:44 +0100 (CET) Message-ID: <4F33EA27.8090405@quip.cz> Date: Thu, 09 Feb 2012 16:45:43 +0100 From: Miroslav Lachman <000.fbsd@quip.cz> User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.9.1.19) Gecko/20110420 Lightning/1.0b1 SeaMonkey/2.0.14 MIME-Version: 1.0 To: chris_bender@cellularatsea.com Content-Type: text/plain; charset=ISO-8859-2; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-jail@freebsd.org Subject: File system issue [was Re: jails] X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Feb 2012 16:05:11 -0000 > Hi Greg, > > I am having an issue with one of my jailed systems. It has run out of > space. I have identified many files to delete but I can not > Delete the files as the system comes back with "No Space available". I > tried to delete them from the host system as well but I get > The same system issue. How does one delete files or free up space? What version you are running? (uname -a) Are you using ZFS or UFS? If ZFS, do you have some snapshots of given filesystem? If yes, then you must firstly delete some snapshots to get some free space. With snapshot, the deleted file needs additional space to alocate in last snapshot. Miroslav Lachman From owner-freebsd-jail@FreeBSD.ORG Thu Feb 9 16:13:37 2012 Return-Path: Delivered-To: freebsd-jail@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DEC5C106567A for ; Thu, 9 Feb 2012 16:13:37 +0000 (UTC) (envelope-from chris_bender@cellularatsea.com) Received: from wireless.icgws.com (wireless.icgws.com [198.211.94.23]) by mx1.freebsd.org (Postfix) with ESMTP id A37F48FC12 for ; Thu, 9 Feb 2012 16:13:37 +0000 (UTC) Received: by wireless.icgws.com (Postfix, from userid 1003) id A3F8D180DEA; Thu, 9 Feb 2012 11:12:24 -0500 (EST) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on wireless.icgws.com X-Spam-Level: X-Spam-Status: No, score=-1.3 required=3.5 tests=ALL_TRUSTED,BAYES_00, URIBL_SBL autolearn=no version=3.3.1 Received: from wmstp.corp.cellularatsea.com (unknown [10.200.104.195]) by wireless.icgws.com (Postfix) with SMTP id B11E7180DEA; Thu, 9 Feb 2012 11:12:16 -0500 (EST) Received: from wmstp.corp.wms.cellularatsea.com (localhost [127.0.0.1]) by wmstp.corp.cellularatsea.com (Postfix) with SMTP id 5181DB1885D; Thu, 9 Feb 2012 11:13:07 -0500 (EST) Received: from wmsexg01.corp.cellularatsea.com ([10.200.104.15] helo=wmsexg01.corp.cellularatsea.com) by wmstp.corp.wms.cellularatsea.com with SMTP (ASSP 1.9.1.1); 9 Feb 2012 11:13:07 -0500 X-Ninja-PIM: Scanned by Ninja Received: from 10.200.104.30 ([10.200.104.30]) by wmsexg01.corp.cellularatsea.com ([10.200.104.15]) with Microsoft Exchange Server HTTP-DAV ; Thu, 9 Feb 2012 16:13:06 +0000 References: <4F33EA27.8090405@quip.cz> Content-Transfer-Encoding: quoted-printable From: "Bender, Chris" Content-Type: text/plain; charset="us-ascii" Thread-Topic: File system issue [was Re: jails] Thread-Index: AcznRbTxaVv1REmTRbGB+aLCpTpuDA== In-Reply-To: <4F33EA27.8090405@quip.cz> Message-ID: Date: Thu, 9 Feb 2012 11:11:04 -0500 To: "Miroslav Lachman" <000.fbsd@quip.cz> MIME-Version: 1.0 X-Assp-Whitelisted: Yes () X-Assp-Envelope-From: chris_bender@cellularatsea.com X-Assp-Intended-For: freebsd-jail@freebsd.org X-Assp-Passing: 10.200.104.15 in acceptAllMail X-Assp-ID: wmstp.corp.wms.cellularatsea.com (32880-30909) X-Assp-Version: 1.9.1.1(1.0.00) Cc: freebsd-jail@freebsd.org Subject: Re: File system issue [was Re: jails] X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Feb 2012 16:13:37 -0000 I am running 8.2 and I am also running ZFS. I have no snapshots as I hav= e deleted them all. Still need more space. Thanks Sent from my iPhone On Feb 9, 2012, at 10:46 AM, "Miroslav Lachman" <000.fbsd@quip.cz> wrote: >> Hi Greg, >>=20 >> I am having an issue with one of my jailed systems. It has run out= of >> space. I have identified many files to delete but I can not >> Delete the files as the system comes back with "No Space available".= I >> tried to delete them from the host system as well but I get >> The same system issue. How does one delete files or free up space? >=20 > What version you are running? (uname -a) > Are you using ZFS or UFS? > If ZFS, do you have some snapshots of given filesystem? If yes, then= you must firstly delete some snapshots to get some free space. With sna= pshot, the deleted file needs additional space to alocate in last snapsh= ot. >=20 > Miroslav Lachman From owner-freebsd-jail@FreeBSD.ORG Thu Feb 9 18:40:43 2012 Return-Path: Delivered-To: freebsd-jail@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D5E9E1065672 for ; Thu, 9 Feb 2012 18:40:43 +0000 (UTC) (envelope-from fbsdq@peterk.org) Received: from poshta.pknet.net (poshta.pknet.net [216.241.167.213]) by mx1.freebsd.org (Postfix) with ESMTP id A8A438FC16 for ; Thu, 9 Feb 2012 18:40:43 +0000 (UTC) Received: from poshta.pknet.net (localhost [127.0.0.1]) by poshta.pknet.net (Postfix) with ESMTP id A099216466; Thu, 9 Feb 2012 11:20:44 -0700 (MST) Received: from poshta.pknet.net ([127.0.0.1]) by poshta.pknet.net (poshta.pknet.net [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Jto_lFPEjFlC; Thu, 9 Feb 2012 11:20:39 -0700 (MST) Received: from pop.pknet.net (localhost [127.0.0.1]) by poshta.pknet.net (Postfix) with ESMTP id 0CA4616006; Thu, 9 Feb 2012 11:20:39 -0700 (MST) Received: from 74.63.162.21 (SquirrelMail authenticated user fbsdq@peterk.org) by pop.pknet.net with HTTP; Thu, 9 Feb 2012 11:20:39 -0700 Message-ID: <17d39c292fb307c3e07a01dbe3992bf3.squirrel@pop.pknet.net> In-Reply-To: References: <4F33EA27.8090405@quip.cz> Date: Thu, 9 Feb 2012 11:20:39 -0700 From: "Peter" To: "Bender, Chris" User-Agent: SquirrelMail/1.4.22 MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal Cc: freebsd-jail@freebsd.org Subject: Re: File system issue [was Re: jails] X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Feb 2012 18:40:43 -0000 > > On Feb 9, 2012, at 10:46 AM, "Miroslav Lachman" <000.fbsd@quip.cz> wrote: > >>> Hi Greg, >>> >>> I am having an issue with one of my jailed systems. It has run out of >>> space. I have identified many files to delete but I can not >>> Delete the files as the system comes back with "No Space available". I >>> tried to delete them from the host system as well but I get >>> The same system issue. How does one delete files or free up space? >> >> What version you are running? (uname -a) >> Are you using ZFS or UFS? >> If ZFS, do you have some snapshots of given filesystem? If yes, then you >> must firstly delete some snapshots to get some free space. With >> snapshot, the deleted file needs additional space to alocate in last >> snapshot. >> >> Miroslav Lachman > I am running 8.2 and I am also running ZFS. I have no snapshots as I have > deleted them all. Still need more space. > > Thanks > > Sent from my iPhone What if you null out the files? ":> file.to.delete" or "echo > file.to.delete" ? If that works, can you then 'rm' them? ]Peter[ From owner-freebsd-jail@FreeBSD.ORG Fri Feb 10 10:57:21 2012 Return-Path: Delivered-To: freebsd-jail@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 80EC4106564A for ; Fri, 10 Feb 2012 10:57:21 +0000 (UTC) (envelope-from joris.dedieu@gmail.com) Received: from mail-qy0-f182.google.com (mail-qy0-f182.google.com [209.85.216.182]) by mx1.freebsd.org (Postfix) with ESMTP id 409F68FC08 for ; Fri, 10 Feb 2012 10:57:20 +0000 (UTC) Received: by qcmt40 with SMTP id t40so2011803qcm.13 for ; Fri, 10 Feb 2012 02:57:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; bh=CtQLRHwYKpH/2VT/4ZPv0AvDV3OrlMrCUEQumNZXkqw=; b=u3VPVMxRBHF636IczGmkvp2Fw9FAXxbRmnPIDoqJI799s3x+tDr5Xq7BJSnsb9QagB CUTY4I8BANLqZyIq8F1dBbcVZA2yPn5PIWNAXtyLS8ZtFyUE9m1sOckZqrbQf5+Jg0c6 3K11/34MPfWe9f+sJNK9s6JNUuOqb1gnWPXTU= MIME-Version: 1.0 Received: by 10.229.137.21 with SMTP id u21mr3956048qct.23.1328869832072; Fri, 10 Feb 2012 02:30:32 -0800 (PST) Received: by 10.229.131.204 with HTTP; Fri, 10 Feb 2012 02:30:32 -0800 (PST) In-Reply-To: <4F33EA27.8090405@quip.cz> References: <4F33EA27.8090405@quip.cz> Date: Fri, 10 Feb 2012 11:30:32 +0100 Message-ID: From: joris dedieu To: freebsd-jail@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Subject: Re: File system issue [was Re: jails] X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Feb 2012 10:57:21 -0000 2012/2/9 Miroslav Lachman <000.fbsd@quip.cz>: >> Hi Greg, >> >> I am having an issue with one of my jailed systems. It has run out of >> space. I have identified many files to delete but I can =A0not >> Delete the files as the system comes back with "No Space available". =A0= I I had the same issue some times ago on 8.1. Just temporally changing the refquota on the host solved it. Joris >> tried to delete them from the host system as well but I get >> The same system issue. How does one delete files or free up space? > > > What version you are running? (uname -a) > Are you using ZFS or UFS? > If ZFS, do you have some snapshots of given filesystem? If yes, then you > must firstly delete some snapshots to get some free space. With snapshot, > the deleted file needs additional space to alocate in last snapshot. > > Miroslav Lachman > _______________________________________________ > freebsd-jail@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-jail > To unsubscribe, send any mail to "freebsd-jail-unsubscribe@freebsd.org" From owner-freebsd-jail@FreeBSD.ORG Fri Feb 10 16:33:41 2012 Return-Path: Delivered-To: freebsd-jail@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9546E106564A for ; Fri, 10 Feb 2012 16:33:41 +0000 (UTC) (envelope-from marquis@roble.com) Received: from mx5.roble.com (mx5.roble.com [206.40.34.5]) by mx1.freebsd.org (Postfix) with ESMTP id 66BDB8FC19 for ; Fri, 10 Feb 2012 16:33:41 +0000 (UTC) Received: from mx5.roble.com (mx5.roble.com [206.40.34.5]) by mx5.roble.com (Postfix) with ESMTP id DEE7C67BE3 for ; Fri, 10 Feb 2012 08:16:44 -0800 (PST) Date: Fri, 10 Feb 2012 08:16:44 -0800 (PST) From: Roger Marquis To: freebsd-jail@freebsd.org In-Reply-To: <20120210120038.84725106587A@hub.freebsd.org> References: <20120210120038.84725106587A@hub.freebsd.org> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Message-Id: <20120210163341.9546E106564A@hub.freebsd.org> Subject: Re: File system issue [was Re: jails] X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Feb 2012 16:33:41 -0000 > I am having an issue with one of my jailed systems. It has run out of > space. I have identified many files to delete but I can not > Delete the files as the system comes back with "No Space available". I > tried to delete them from the host system as well but I get > The same system issue. How does one delete files or free up space? Are you using FreeBSD's default partitioning, with inter-disk partitions for /usr, /var, ...? If so you should reinstall and _don't_ create partitions within a disk without a specific requirement that precludes use of another disk. The fact that FreeBSD installs still recommend legacy partitioning is one of the reasons so many sites have switched to Linux. It's not that the server designers know better than to avoid unnecessary partitions but that Linux's single partition defaults (other than swap) are so much more robust. As a result problems with partitioning are chalked-up to FreeBSD, and rightly so, by Linux advocates. Next time an OS choice has to be made ops and other managers will remember all the diskfull outages and symlink hacks and choose Linux over FreeBSD. Is there any Unix or Linux distribution other than FreeBSD which still defaults to partitions for /usr et al? IME, Roger Marquis From owner-freebsd-jail@FreeBSD.ORG Fri Feb 10 17:01:07 2012 Return-Path: Delivered-To: freebsd-jail@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C2B361065676 for ; Fri, 10 Feb 2012 17:01:07 +0000 (UTC) (envelope-from galtsev@kicp.uchicago.edu) Received: from cosmo.uchicago.edu (cosmo.uchicago.edu [128.135.52.97]) by mx1.freebsd.org (Postfix) with ESMTP id 8C19A8FC0C for ; Fri, 10 Feb 2012 17:01:07 +0000 (UTC) Received: by cosmo.uchicago.edu (Postfix, from userid 48) id E1E47CB8C71; Fri, 10 Feb 2012 11:01:06 -0600 (CST) Received: from 128.135.70.2 (SquirrelMail authenticated user valeri) by cosmo.uchicago.edu with HTTP; Fri, 10 Feb 2012 11:01:06 -0600 (CST) Message-ID: <63654.128.135.70.2.1328893266.squirrel@cosmo.uchicago.edu> In-Reply-To: <20120210163341.9546E106564A@hub.freebsd.org> References: <20120210120038.84725106587A@hub.freebsd.org> <20120210163341.9546E106564A@hub.freebsd.org> Date: Fri, 10 Feb 2012 11:01:06 -0600 (CST) From: "Valeri Galtsev" To: freebsd-jail@freebsd.org User-Agent: SquirrelMail/1.4.8-5.el5.centos.7 MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal Cc: Roger Marquis Subject: Re: File system issue [was Re: jails] X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: galtsev@kicp.uchicago.edu List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Feb 2012 17:01:07 -0000 Wow, this did impress me (backwards ;-) ! server with single partition / (plus swap partition) ! It was - I forgot how long ago - when I start feeling myself as admin building more robust systems when I started meticulously creating different partitions for all things that shouldn't affect each other. And still I will keep doing that because of following. Do you want some unprivileged user's script writing into /tmp to fill up (or run filesystem out of file handlers) / partition holding other things like mail spool, or database storage? BTW: on mail servers where my users can log in I always mount their home directories, and spool with "noexec, nosuid, nodev" options (the same goes about /tmp, and wherever web server stores uploaded stuff...). I do have a feeling that the trend is opposite the one you mentioned: people are switching not to linux, but away from it. As linux lately (say, last 3 years or so) became more like windows: once every 45 days on average: kernel update ==> reboot. I remember way back (2.4 kernels) we had a bunch of linux machines with uptime 2 - 3 years... Not any more. Sorry, I just couldn't hold myself ;-( Valeri On Fri, February 10, 2012 10:16 am, Roger Marquis wrote: >> I am having an issue with one of my jailed systems. It has run out of >> space. I have identified many files to delete but I can not >> Delete the files as the system comes back with "No Space available". I >> tried to delete them from the host system as well but I get >> The same system issue. How does one delete files or free up space? > > Are you using FreeBSD's default partitioning, with inter-disk partitions > for /usr, /var, ...? If so you should reinstall and _don't_ create > partitions within a disk without a specific requirement that precludes > use of another disk. > > The fact that FreeBSD installs still recommend legacy partitioning is one > of the reasons so many sites have switched to Linux. It's not that the > server designers know better than to avoid unnecessary partitions but > that Linux's single partition defaults (other than swap) are so much more > robust. As a result problems with partitioning are chalked-up to > FreeBSD, and rightly so, by Linux advocates. Next time an OS choice has > to be made ops and other managers will remember all the diskfull outages > and symlink hacks and choose Linux over FreeBSD. > > Is there any Unix or Linux distribution other than FreeBSD which still > defaults to partitions for /usr et al? > > IME, > Roger Marquis > > > _______________________________________________ > freebsd-jail@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-jail > To unsubscribe, send any mail to "freebsd-jail-unsubscribe@freebsd.org" > ++++++++++++++++++++++++++++++++++++++++ Valeri Galtsev Sr System Administrator Department of Astronomy and Astrophysics Kavli Institute for Cosmological Physics University of Chicago Phone: 773-702-4247 ++++++++++++++++++++++++++++++++++++++++ From owner-freebsd-jail@FreeBSD.ORG Fri Feb 10 17:59:07 2012 Return-Path: Delivered-To: freebsd-jail@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 88385106566C for ; Fri, 10 Feb 2012 17:59:07 +0000 (UTC) (envelope-from marquis@roble.com) Received: from mx5.roble.com (mx5.roble.com [206.40.34.5]) by mx1.freebsd.org (Postfix) with ESMTP id 722FB8FC0A for ; Fri, 10 Feb 2012 17:59:07 +0000 (UTC) Received: from mx5.roble.com (mx5.roble.com [206.40.34.5]) by mx5.roble.com (Postfix) with ESMTP id 0F393688FA; Fri, 10 Feb 2012 09:59:07 -0800 (PST) Date: Fri, 10 Feb 2012 09:59:07 -0800 (PST) From: Roger Marquis To: Valeri Galtsev In-Reply-To: <63654.128.135.70.2.1328893266.squirrel@cosmo.uchicago.edu> References: <20120210120038.84725106587A@hub.freebsd.org> <20120210163341.9546E106564A@hub.freebsd.org> <63654.128.135.70.2.1328893266.squirrel@cosmo.uchicago.edu> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Message-Id: <20120210175907.88385106566C@hub.freebsd.org> Cc: freebsd-jail@freebsd.org Subject: Re: File system issue [was Re: jails] X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Feb 2012 17:59:07 -0000 > Do you want some unprivileged user's script writing into /tmp to fill up > (or run filesystem out of file handlers) / partition holding other things > like mail spool, or database storage? Has never been an issue on any of our systems. The reasons for that may be twofold: 1) we don't partition without an actual use case where we give the app its own disk to allocate free blocks from, and 2) we always spec alert scripts to alert ops when any partition is over X% full. OTOH we do have systems installed with partitions, not by us, that constantly have diskfull issues. Most of them are due to /var/ and /tmp/ and printer or other temp files. Most importantly, none of those systems would have issues had they originally been installed with a single root disk partition. > BTW: on mail servers where my users can log in I always mount their home > directories, and spool with "noexec, nosuid, nodev" options (the same goes > about /tmp, and wherever web server stores uploaded stuff...). Never had a need to do that but OMMV, question is why would you carve these partitions out of the root disk instead of putting them on a disk of their own? There are lots of good reasons for creating partitions. It's just that the vast majority of partitioned systems we come across have no reason to be so partitioned. A look at Unix history shows that partitions were originally created before raid to deal with root disks that were too small for the OS. The overwhelming majority of Unix and Linux systems today, both server and desktop, are single-partition. Roger Marquis