From owner-freebsd-jail@FreeBSD.ORG Fri Aug 23 18:49:20 2013 Return-Path: Delivered-To: freebsd-jail@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 86576ED1 for ; Fri, 23 Aug 2013 18:49:20 +0000 (UTC) (envelope-from galtsev@kicp.uchicago.edu) Received: from cosmo.uchicago.edu (cosmo.uchicago.edu [128.135.52.97]) by mx1.freebsd.org (Postfix) with ESMTP id 4F33E2B49 for ; Fri, 23 Aug 2013 18:49:19 +0000 (UTC) Received: by cosmo.uchicago.edu (Postfix, from userid 48) id 9ED52CB8C90; Fri, 23 Aug 2013 13:49:19 -0500 (CDT) Received: from 128.135.70.2 (SquirrelMail authenticated user valeri) by cosmo.uchicago.edu with HTTP; Fri, 23 Aug 2013 13:49:19 -0500 (CDT) Message-ID: <37112.128.135.70.2.1377283759.squirrel@cosmo.uchicago.edu> In-Reply-To: <20130823182356.GH4972@kib.kiev.ua> References: <19176.128.135.70.2.1377267872.squirrel@cosmo.uchicago.edu> <20130823160549.GD4972@kib.kiev.ua> <17536.128.135.70.2.1377281124.squirrel@cosmo.uchicago.edu> <20130823182356.GH4972@kib.kiev.ua> Date: Fri, 23 Aug 2013 13:49:19 -0500 (CDT) Subject: Re: per user quotas inside jail? From: "Valeri Galtsev" To: "Konstantin Belousov" User-Agent: SquirrelMail/1.4.8-5.el5.centos.7 MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal Cc: freebsd-jail@freebsd.org X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: galtsev@kicp.uchicago.edu List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Aug 2013 18:49:20 -0000 On Fri, August 23, 2013 1:23 pm, Konstantin Belousov wrote: > On Fri, Aug 23, 2013 at 01:05:24PM -0500, Valeri Galtsev wrote: >> On Fri, August 23, 2013 11:05 am, Konstantin Belousov wrote: >> > On Fri, Aug 23, 2013 at 09:24:32AM -0500, Valeri Galtsev wrote: >> >> Dear Experts, >> >> After searching the web, reading FreeBSD Docs, trying some hacks >> found on >> >> some discussion boards... I feel it is not easily possible. Yet, as >> always >> >> there may be some expert who knows how to do it: >> >> How can one have per user quotas inside jail? >> >> Basically, I would like to give users shell access to some server, >> but >> that I prefer to have in jail, where I will mount all filesystems they >> need access to... and the only question is: how do I restrict them so >> one >> >> (or few) user doesn't fill up the whole filesystem. My mind is not >> married >> >> to any particular filesystem, UFS2, XFS, ZFS... - the only thing I >> would >> >> stay away from is NFS exporting on host and then NFS mounting in jail >> (which may be easiest if not the only way quota wise). >> > >> > UFS quotas work regardless of jailed/non-jailed user. The only >> confusing >> > issue is that quotas are per host uid. In other words, if host and >> jail >> user, or two users from different jails has the same uid, you get one >> quota setting applied and accounted for them. >> > >> > Usual mitigation is to ensure that user uids are globally unique. >> > >> >> Thanks, Konstantin. >> >> Still it doesn't work for me. My system is: >> >> 9.1-RELEASE-p5 amd64 >> >> Kernel: the same as GENERIC, with one option added: >> >> options QUOTA # Add disk quota support >> >> filesystem with quota enabled is directly mounted (UFS; rw,userquota) >> into >> directory inside jail. User (with the same username and UID) exists on >> the >> host system and in jail. Quotas work on the host system. Quotas don't >> work >> inside jail, so this user can fill up the whole filesystem when logged >> into jail (jail accepts ssh connections with different hostname...) >> >> Apart from that I tried a hack which I lifted from someone's FreeBSD 7 >> hack (only the variable name changed since then), namely: >> >> in kernel, in: >> >> /usr/src/sys/kern/vfs_syscalls.c >> >> I kicked out two lines: >> >> if (!prison_allow(td->td_ucred, PR_ALLOW_QUOTAS)) >> return (EPERM); >> >> (which basically obliterate that if done from inside jail as far as I >> understand), >> >> rebuilt and installed this kernel; in file >> >> /etc/rc.d/quota >> >> removed line >> >> # KEYWORD: nojail >> >> Yet, I'm still where I was: quotas work outside jail, not inside jail... >> >> So, I'm at loss. I guess I will have to dive into zfs following Aaron >> Kaufman's suggestion... Sigh. > > UFS quotas work per mount. So if jail root is on a filesystem which > has no quotas configured, obviously the thing cannot work. > > You did not provided any details of your configuration, which makes > a diagnostic impossible. > Hi Konstantin, Thanks a lot for helping me! Sorry, my usual fault, not sufficient details... Jails are set up pretty much as in: http://www.freebsd.org/doc/handbook/jails-application.html (directory names and locations are slightly different). Someone mentioned, ezjail does virtually the same too - if that helps. In /jail/mroot there is the structure resembling real system (binaries, libraries,...) except for a few things that have to be writable inside jail; those are replaced with symlinks pointing to these inside subdirectory s: ls -l /jail/mroot total 48 drwxr-xr-x 2 root wheel 1024 Aug 19 13:02 bin drwxr-xr-x 7 root wheel 1024 Aug 19 13:03 boot dr-xr-xr-x 2 root wheel 512 Aug 19 13:02 dev lrwxr-xr-x 1 root wheel 5 Aug 19 13:11 etc -> s/etc lrwxr-xr-x 1 root wheel 6 Aug 19 13:11 home -> s/home drwxr-xr-x 3 root wheel 1536 Aug 19 13:03 lib drwxr-xr-x 3 root wheel 512 Aug 19 13:03 libexec drwxr-xr-x 2 root wheel 512 Aug 19 13:02 media drwxr-xr-x 2 root wheel 512 Aug 19 13:02 mnt dr-xr-xr-x 2 root wheel 512 Aug 19 13:02 proc drwxr-xr-x 2 root wheel 2560 Aug 19 13:03 rescue lrwxr-xr-x 1 root wheel 6 Aug 19 13:11 root -> s/root drwxr-xr-x 2 root wheel 512 Aug 19 13:11 s drwxr-xr-x 2 root wheel 2560 Aug 19 13:03 sbin lrwxr-xr-x 1 root wheel 11 Aug 19 13:02 sys -> usr/src/sys lrwxr-xr-x 1 root wheel 5 Aug 19 13:11 tmp -> s/tmp drwxr-xr-x 14 root wheel 512 Aug 19 13:11 usr lrwxr-xr-x 1 root wheel 5 Aug 19 13:11 var -> s/var particular jail lives in its root directory: /jail/shell /jail/mroot is nullfs readonly mounted onto /jail/shell, rw unique for each shell filesystem is mounted into /jail/shell/s (and populated with appropriate /etc, /var ....), filesystem that has to be with quotas is mounted (UFS; rw,userquota) into /jail/shell/s/home This last one is the one in question: quotas on this work when user will ssh to host system and will write to /jail/shell/s/home; quota does not work if user will ssh into jail (which is accessible from network with different hostname). When the user writes into /home in jail (into /s/home actually, symlink points there which on host system is our /jail/shell/s/home), the quotas do not work. I don't quite understand what quota on jail root filesystem (enabled or not enabled) has to do with quota on different filesystem that is mounted inside that filesystem. Outside jail / has no quotas, different filesystem mounted somewhere inside (/jail/shell/s/home or just /home or /var) with quotas and it does honor quotas. Am I missing something trivial or fundamental? Thanks again for helping me! What other details could help? Sincerely yours, Valeri ++++++++++++++++++++++++++++++++++++++++ Valeri Galtsev Sr System Administrator Department of Astronomy and Astrophysics Kavli Institute for Cosmological Physics University of Chicago Phone: 773-702-4247 ++++++++++++++++++++++++++++++++++++++++