Date: Sat, 24 Aug 2013 18:08:31 +0300 From: Konstantin Belousov <kostikbel@gmail.com> To: Valeri Galtsev <galtsev@kicp.uchicago.edu> Cc: freebsd-jail@freebsd.org Subject: Re: per user quotas inside jail? Message-ID: <20130824150831.GO4972@kib.kiev.ua> In-Reply-To: <37112.128.135.70.2.1377283759.squirrel@cosmo.uchicago.edu> References: <19176.128.135.70.2.1377267872.squirrel@cosmo.uchicago.edu> <20130823160549.GD4972@kib.kiev.ua> <17536.128.135.70.2.1377281124.squirrel@cosmo.uchicago.edu> <20130823182356.GH4972@kib.kiev.ua> <37112.128.135.70.2.1377283759.squirrel@cosmo.uchicago.edu>
next in thread | previous in thread | raw e-mail | index | archive | help
--wyheH+OylYa5POrl Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Aug 23, 2013 at 01:49:19PM -0500, Valeri Galtsev wrote: >=20 > On Fri, August 23, 2013 1:23 pm, Konstantin Belousov wrote: > > On Fri, Aug 23, 2013 at 01:05:24PM -0500, Valeri Galtsev wrote: > >> On Fri, August 23, 2013 11:05 am, Konstantin Belousov wrote: > >> > On Fri, Aug 23, 2013 at 09:24:32AM -0500, Valeri Galtsev wrote: > >> >> Dear Experts, > >> >> After searching the web, reading FreeBSD Docs, trying some hacks > >> found on > >> >> some discussion boards... I feel it is not easily possible. Yet, as > >> always > >> >> there may be some expert who knows how to do it: > >> >> How can one have per user quotas inside jail? > >> >> Basically, I would like to give users shell access to some server, > >> but > >> that I prefer to have in jail, where I will mount all filesystems they > >> need access to... and the only question is: how do I restrict them so > >> one > >> >> (or few) user doesn't fill up the whole filesystem. My mind is not > >> married > >> >> to any particular filesystem, UFS2, XFS, ZFS... - the only thing I > >> would > >> >> stay away from is NFS exporting on host and then NFS mounting in ja= il > >> (which may be easiest if not the only way quota wise). > >> > > >> > UFS quotas work regardless of jailed/non-jailed user. The only > >> confusing > >> > issue is that quotas are per host uid. In other words, if host and > >> jail > >> user, or two users from different jails has the same uid, you get one > >> quota setting applied and accounted for them. > >> > > >> > Usual mitigation is to ensure that user uids are globally unique. > >> > > >> > >> Thanks, Konstantin. > >> > >> Still it doesn't work for me. My system is: > >> > >> 9.1-RELEASE-p5 amd64 > >> > >> Kernel: the same as GENERIC, with one option added: > >> > >> options QUOTA # Add disk quota support > >> > >> filesystem with quota enabled is directly mounted (UFS; rw,userquota) > >> into > >> directory inside jail. User (with the same username and UID) exists on > >> the > >> host system and in jail. Quotas work on the host system. Quotas don't > >> work > >> inside jail, so this user can fill up the whole filesystem when logged > >> into jail (jail accepts ssh connections with different hostname...) > >> > >> Apart from that I tried a hack which I lifted from someone's FreeBSD 7 > >> hack (only the variable name changed since then), namely: > >> > >> in kernel, in: > >> > >> /usr/src/sys/kern/vfs_syscalls.c > >> > >> I kicked out two lines: > >> > >> if (!prison_allow(td->td_ucred, PR_ALLOW_QUOTAS)) > >> return (EPERM); > >> > >> (which basically obliterate that if done from inside jail as far as I > >> understand), > >> > >> rebuilt and installed this kernel; in file > >> > >> /etc/rc.d/quota > >> > >> removed line > >> > >> # KEYWORD: nojail > >> > >> Yet, I'm still where I was: quotas work outside jail, not inside jail.= =2E. > >> > >> So, I'm at loss. I guess I will have to dive into zfs following Aaron > >> Kaufman's suggestion... Sigh. > > > > UFS quotas work per mount. So if jail root is on a filesystem which > > has no quotas configured, obviously the thing cannot work. > > > > You did not provided any details of your configuration, which makes > > a diagnostic impossible. > > >=20 > Hi Konstantin, >=20 > Thanks a lot for helping me! Sorry, my usual fault, not sufficient detail= s... >=20 > Jails are set up pretty much as in: >=20 > http://www.freebsd.org/doc/handbook/jails-application.html >=20 > (directory names and locations are slightly different). Someone mentioned, > ezjail does virtually the same too - if that helps. >=20 > In /jail/mroot there is the structure resembling real system (binaries, > libraries,...) except for a few things that have to be writable inside > jail; those are replaced with symlinks pointing to these inside > subdirectory s: >=20 > ls -l /jail/mroot > total 48 > drwxr-xr-x 2 root wheel 1024 Aug 19 13:02 bin > drwxr-xr-x 7 root wheel 1024 Aug 19 13:03 boot > dr-xr-xr-x 2 root wheel 512 Aug 19 13:02 dev > lrwxr-xr-x 1 root wheel 5 Aug 19 13:11 etc -> s/etc > lrwxr-xr-x 1 root wheel 6 Aug 19 13:11 home -> s/home > drwxr-xr-x 3 root wheel 1536 Aug 19 13:03 lib > drwxr-xr-x 3 root wheel 512 Aug 19 13:03 libexec > drwxr-xr-x 2 root wheel 512 Aug 19 13:02 media > drwxr-xr-x 2 root wheel 512 Aug 19 13:02 mnt > dr-xr-xr-x 2 root wheel 512 Aug 19 13:02 proc > drwxr-xr-x 2 root wheel 2560 Aug 19 13:03 rescue > lrwxr-xr-x 1 root wheel 6 Aug 19 13:11 root -> s/root > drwxr-xr-x 2 root wheel 512 Aug 19 13:11 s > drwxr-xr-x 2 root wheel 2560 Aug 19 13:03 sbin > lrwxr-xr-x 1 root wheel 11 Aug 19 13:02 sys -> usr/src/sys > lrwxr-xr-x 1 root wheel 5 Aug 19 13:11 tmp -> s/tmp > drwxr-xr-x 14 root wheel 512 Aug 19 13:11 usr > lrwxr-xr-x 1 root wheel 5 Aug 19 13:11 var -> s/var >=20 > particular jail lives in its root directory: >=20 > /jail/shell >=20 > /jail/mroot is nullfs readonly mounted onto /jail/shell, rw unique for > each shell filesystem is mounted into /jail/shell/s (and populated with > appropriate /etc, /var ....), filesystem that has to be with quotas is > mounted (UFS; rw,userquota) into >=20 > /jail/shell/s/home >=20 > This last one is the one in question: quotas on this work when user will > ssh to host system and will write to /jail/shell/s/home; quota does not > work if user will ssh into jail (which is accessible from network with > different hostname). When the user writes into /home in jail (into /s/home > actually, symlink points there which on host system is our > /jail/shell/s/home), the quotas do not work. >=20 > I don't quite understand what quota on jail root filesystem (enabled or > not enabled) has to do with quota on different filesystem that is mounted > inside that filesystem. Outside jail / has no quotas, different filesystem > mounted somewhere inside (/jail/shell/s/home or just /home or /var) with > quotas and it does honor quotas. Am I missing something trivial or > fundamental? >=20 > Thanks again for helping me! What other details could help? I decided that I have no desire to try to understand all the layers of indirections which are only relevant to you anyway. Instead, I demostrate you what I mean by working quotas. Below is the transcript of the simple test. sandy% mount -v /mnt = ~ mount: /dev/ada1p4: Operation not permitted /dev/ada1p4 on /mnt (ufs, local, with quotas, soft-updates, writes: sync 2 = async 37, reads: sync 7 async 0) sandy% sudo repquota -uah | grep kostik = ~ kostik -- 14G 0 0 - 461057 = 0 0 - sandy% sudo jail -u kostik / test1 127.0.0.1 /bin/sh = ~ $ dd if=3D/dev/zero bs=3D1m of=3D/mnt/1/dddd count=3D1024 1024+0 records in 1024+0 records out 1073741824 bytes transferred in 10.765265 secs (99741328 bytes/sec) $ ^D% = sandy% sudo repquota -uah | grep kostik = ~ kostik -- 15G 0 0 - 461058 = 0 0 - You could see that the accounted space and inodes are properly increased after the dd. IMO, you should make sure that the users operate on the filesystem which has quotas enabled. Or, you should provide a simple to reproduce test case, among the lines of the script I pasted above, for me to recreate the issue locally. --wyheH+OylYa5POrl Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.21 (FreeBSD) iQIcBAEBAgAGBQJSGMxuAAoJEJDCuSvBvK1B74kQAJOEuuli0OBlx3Zrslxy7WUg tmTDAZl06sGgKhDotEIZLNOpg+6eJWYWZm/9q51MVHV43u+nmzcWq5Ll+HoEoO7M lDR0Whda17jMZcqKnz+vdZ0ZdMDQU/yMxah2njn11s2od4CvPRb8CgJxH5y0RGej NwaB5cAf/slrF4DMtLYhI7/zcVFzgsVFdeFBugWNESJxi0yfTm5MLf8B+uKO8V3N VpR8gbEyRkub05EhmJ14GiPS/vissCFKMOr46obsy9/w6wOLGTvRwU3/E989tdbQ loNh5xPDiMFFaih0WIEImrgnfW3SC5uxAJiLbRlERfrTOZqi1b0n3zUzvFx/qW9S nPr+JC9lltTz7Kc3KHRAj5Yj8jukfp6HJNshJe2xd1Fd8J1UY6c3X1rYxhz2C/dP xbj+sW4TFzkDB+7+qmXThNREla0kk7f2Kck8/SPGMrDxFjmC7We1EqnhjF19vkdl +A5XZycR9qQAwgSVHVrky0WYUHUGWmjwKjbFdoGoOpdrYKmk4gDYftVm2Q/KzuH1 5IU8MttCqKORKeFpaaBc1gadvJ0d8Nd8rUX82yLE+iVVys/+8lhSkcVPz5BhWhzD mPwroboY46EE2r1QFBNGZnbOkCCJYL5B4dJ5rrmj+aXagwtnMPWl/76yWBOyVO5j fkNntAgkqPH/OlWtzerc =pd8g -----END PGP SIGNATURE----- --wyheH+OylYa5POrl--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20130824150831.GO4972>