From owner-freebsd-jail@FreeBSD.ORG Wed Aug 28 16:57:49 2013 Return-Path: Delivered-To: freebsd-jail@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 56CA6C8B for ; Wed, 28 Aug 2013 16:57:49 +0000 (UTC) (envelope-from galtsev@kicp.uchicago.edu) Received: from cosmo.uchicago.edu (cosmo.uchicago.edu [128.135.52.97]) by mx1.freebsd.org (Postfix) with ESMTP id 1F98A27C4 for ; Wed, 28 Aug 2013 16:57:48 +0000 (UTC) Received: by cosmo.uchicago.edu (Postfix, from userid 48) id AFE76CB8CBD; Wed, 28 Aug 2013 11:57:42 -0500 (CDT) Received: from 128.135.70.2 (SquirrelMail authenticated user valeri) by cosmo.uchicago.edu with HTTP; Wed, 28 Aug 2013 11:57:42 -0500 (CDT) Message-ID: <65400.128.135.70.2.1377709062.squirrel@cosmo.uchicago.edu> In-Reply-To: <20130824211734.GT4972@kib.kiev.ua> References: <19176.128.135.70.2.1377267872.squirrel@cosmo.uchicago.edu> <20130823160549.GD4972@kib.kiev.ua> <17536.128.135.70.2.1377281124.squirrel@cosmo.uchicago.edu> <20130823182356.GH4972@kib.kiev.ua> <37112.128.135.70.2.1377283759.squirrel@cosmo.uchicago.edu> <20130824150831.GO4972@kib.kiev.ua> <55726.68.255.103.36.1377376501.squirrel@cosmo.uchicago.edu> <20130824211734.GT4972@kib.kiev.ua> Date: Wed, 28 Aug 2013 11:57:42 -0500 (CDT) Subject: Re: per user quotas inside jail? From: "Valeri Galtsev" To: "Konstantin Belousov" User-Agent: SquirrelMail/1.4.8-5.el5.centos.7 MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal Cc: freebsd-jail@freebsd.org X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: galtsev@kicp.uchicago.edu List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Aug 2013 16:57:49 -0000 On Sat, August 24, 2013 4:17 pm, Konstantin Belousov wrote: > On Sat, Aug 24, 2013 at 03:35:01PM -0500, Valeri Galtsev wrote: >> >> On Sat, August 24, 2013 10:08 am, Konstantin Belousov wrote: >> > >> > I decided that I have no desire to try to understand all the layers of >> > indirections which are only relevant to you anyway. Instead, I >> demostrate >> > you what I mean by working quotas. Below is the transcript of the >> simple >> > test. >> > >> > sandy% mount -v /mnt >> > ~ >> > mount: /dev/ada1p4: Operation not permitted >> > /dev/ada1p4 on /mnt (ufs, local, with quotas, soft-updates, writes: >> sync 2 >> > async 37, reads: sync 7 async 0) >> > sandy% sudo repquota -uah | grep kostik >> > ~ >> > kostik -- 14G 0 0 - >> 461057 >> > 0 0 - >> > sandy% sudo jail -u kostik / test1 127.0.0.1 /bin/sh >> > ~ >> > $ dd if=/dev/zero bs=1m of=/mnt/1/dddd count=1024 >> > 1024+0 records in >> > 1024+0 records out >> > 1073741824 bytes transferred in 10.765265 secs (99741328 bytes/sec) >> > $ ^D% >> > sandy% sudo repquota -uah | grep kostik >> > ~ >> > kostik -- 15G 0 0 - >> 461058 >> > 0 0 - >> > >> > You could see that the accounted space and inodes are properly >> increased >> > after the dd. >> > >> > IMO, you should make sure that the users operate on the filesystem >> which >> > has quotas enabled. Or, you should provide a simple to reproduce test >> > case, among the lines of the script I pasted above, for me to recreate >> > the issue locally. >> > >> >> Thanks again for helping me! I guess, I understand now what the >> difference >> is. Apparently, you are much better expert, so correct me if I'm wrong. >> >> You run your jail with root of jail filesystems (/) the same as root >> filesystem of host (/). Therefore, inside your jail you have access to >> all >> host's /etc/fstab; /dev, ... I'll try to run jail the same way and will >> see if in that case quotas will work for me. If yes, then I at least I >> will know that my problem is not on the kernel level, but in the >> environment accessible inside jail. > After the quotas are configured and running, it is purely kernel-side > code which handles the limits and accounting. You do not need usermode > access to fstab or quota files. > > The same experiment as was done above, but now I copied /bin/dd and > ld-elf.so+libc.so into jail root, to convince you that access to the > full host environment does not matter: > > sandy% ls -la /mnt/1/fsx > ~ > -rw-r--r-- 1 kostik kostik 1032128299 Dec 21 2012 /mnt/1/fsx > sandy% sudo repquota -uah | grep kostik > ~ > kostik -- 15G 0 0 - 461064 > 0 0 - > sandy% sudo jail -u kostik /mnt/1 test1 127.0.0.1 ./dd if=fsx of=xsf bs=1m > ~ > 984+1 records in > 984+1 records out > 1032128299 bytes transferred in 10.262390 secs (100573871 bytes/sec) > sandy% sudo repquota -uah | grep kostik > ~ > kostik -- 16G 0 0 - 461065 > 0 0 - > >> >> I have all jails set up so that one when in jail is not able to access >> filesystem outside jail's own root, which is something like >> /jail/{$jailname}... therefore host's /etc /dev are not visible for one >> inside jail; what they see inside jail as / is /jail/{$jailname} on >> host. > > Let me repeat, verify that the actions which are supposed to be limited > by quotas happen on the filesystem which has quotas configured. > > Or provide me with the minimal example in style I posted so that I can > reproduce the issue locally (I very much doubt that this is the case, and > not a misconfiguration). > Hi Konstantin, as you said, my problem is in misconfiguration. The main trouble came from the configuration not done "by the book": http://www.freebsd.org/doc/en/books/handbook/quotas.html which says to add into /etc/rc.conf the line: quota_enable="YES" but for whatever reason I stupidly had: enable_quotas="YES" (which I must have lifted from some text relevant to older branch...) Thanks again for all your help! Sincerely yours, Valeri ++++++++++++++++++++++++++++++++++++++++ Valeri Galtsev Sr System Administrator Department of Astronomy and Astrophysics Kavli Institute for Cosmological Physics University of Chicago Phone: 773-702-4247 ++++++++++++++++++++++++++++++++++++++++