From owner-freebsd-stable@FreeBSD.ORG Wed Dec 29 04:22:58 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D9420106566B for ; Wed, 29 Dec 2010 04:22:58 +0000 (UTC) (envelope-from mdf356@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id 9D5EE8FC12 for ; Wed, 29 Dec 2010 04:22:58 +0000 (UTC) Received: by iyb26 with SMTP id 26so8972362iyb.13 for ; Tue, 28 Dec 2010 20:22:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=yGSfrwIiCAEHLMyCfP72Ao6OYshDF68kqtaMKuW10GU=; b=Qml+ZYeEmhR/9yzCHp3y/qj57sCWBdxmipIxQ/qdiTV+pLjdPOu1JJgX1rw/eXw7lM TJP0+Jg553c+0JG4ZjqcQ5RfWPJ378dvvm7l2Kl1tjVSbdBlXynMXU9zJmvtPaZp0ClS Vfif8dFOzyHCDzvweNS0ErjY33+T3CFJcGaBE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=Sn1uBF3xt3j4hRy7JmshYLx2krt3B6sLdgHMIPF0ok8USL5sjey9iypVbkdtMIIr/7 zxfzRveyG6U4P9+UakZW1ScliYBuPA5H30NTKqUgg9JiOj+vTRZCEFlA9VkbbjPNfmYy Am+FwcOgmfIprpmXB0oytbZ3dzmcjZtF16In4= MIME-Version: 1.0 Received: by 10.231.169.74 with SMTP id x10mr14045627iby.26.1293595228111; Tue, 28 Dec 2010 20:00:28 -0800 (PST) Received: by 10.231.172.69 with HTTP; Tue, 28 Dec 2010 20:00:28 -0800 (PST) In-Reply-To: <201012281123.53669.jhb@freebsd.org> References: <4BCE4D0F.2020807@quip.cz> <201012130824.33968.jhb@freebsd.org> <4D15D8DD.9010900@quip.cz> <201012281123.53669.jhb@freebsd.org> Date: Tue, 28 Dec 2010 20:00:28 -0800 Message-ID: From: Matthew Fleming To: John Baldwin Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Garrett Cooper , freebsd-stable@freebsd.org, Miroslav Lachman <000.fbsd@quip.cz> Subject: Re: /libexec/ld-elf.so.1: Cannot execute objects on / X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Dec 2010 04:22:58 -0000 On Tue, Dec 28, 2010 at 8:23 AM, John Baldwin wrote: > On Saturday, December 25, 2010 6:43:25 am Miroslav Lachman wrote: >> John Baldwin wrote: >> > On Saturday, December 11, 2010 11:51:41 am Miroslav Lachman wrote: >> >> Miroslav Lachman wrote: >> >>> Garrett Cooper wrote: >> >>>> 2010/4/20 Miroslav Lachman<000.fbsd@quip.cz>: >> >>>>> I have large storage partition (/vol0) mounted as noexec and nosui= d. >> >>>>> Then >> >>>>> one directory from this partition is mounted by nullfs as "exec an= d >> >>>>> suid" so >> >>>>> anything on it can be executed. >> >>>>> >> >>>>> The directory contains full installation of jail. Jail is running >> >>>>> fine, but >> >>>>> some ports (PHP for example) cannot be compiled inside the jail wi= th >> >>>>> message: >> >>>>> >> >>>>> /libexec/ld-elf.so.1: Cannot execute objects on / >> >>>>> >> >>>>> The same apply to executing of apxs >> >>>>> >> >>>>> root@rainnew ~/# /usr/local/sbin/apxs -q MPM_NAME >> >>>>> /libexec/ld-elf.so.1: Cannot execute objects on / >> >>>>> >> >>>>> apxs:Error: Sorry, no shared object support for Apache. >> >>>>> apxs:Error: available under your platform. Make sure. >> >>>>> apxs:Error: the Apache module mod_so is compiled into. >> >>>>> apxs:Error: your server binary '/usr/local/sbin/httpd'.. >> >>>>> >> >>>>> (it should return "prefork") >> >>>>> >> >>>>> So I think there is some bug in checking the mountpoint options, >> >>>>> where the >> >>>>> check is made on "parent" of the nullfs instead of the nullfs targ= et >> >>>>> mountpoint. >> >>>>> >> >>>>> It is on 6.4-RELEASE i386 GENERIC. I did not test it on another re= lease. >> >>>>> >> >>>>> This is list of related mount points: >> >>>>> >> >>>>> /dev/mirror/gm0s2d on /vol0 (ufs, local, noexec, nosuid, soft-upda= tes) >> >>>>> /vol0/jail/.nullfs/rain on /vol0/jail/rain_new (nullfs, local) >> >>>>> /usr/ports on /vol0/jail/rain_new/usr/ports (nullfs, local) >> >>>>> devfs on /vol0/jail/rain_new/dev (devfs, local) >> >>>>> >> >>>>> If I changed /vol0 options to (ufs, local, soft-updates) the above >> >>>>> error is >> >>>>> gone and apxs / compilation works fine. >> >>>>> >> >>>>> Can somebody look at this problem? >> >>>> >> >>>> Can you please provide output from ktrace / truss for the issue? >> >>> >> >>> I did >> >>> # ktrace /usr/local/sbin/apxs -q MPM_NAME >> >>> >> >>> The output is here http://freebsd.quip.cz/ld-elf/ktrace.out >> >>> >> >>> Let me know if you need something else. >> >>> >> >>> Thank you for your interest! >> >> >> >> The problem is still there in FreeBSD 8.1-RELEASE amd64 GENERIC (and = in >> >> 7.x). >> >> >> >> Can somebody say if this is a bug or an expected "feature"? >> > >> > I think this is the expected behavior as nullfs is simply re-exposing = /vol0 >> > and it shouldn't be able to create a more privileged mount than the un= derlying >> > mount I think (e.g. a read/write nullfs mount of a read-only filesyste= m would >> > not make the underlying files read/write). =A0It can be used to provid= e less >> > privilege (e.g. a readonly nullfs mount of a read/write filesystem doe= s not >> > allow writes via the nullfs layer). >> > >> > That said, I'm not sure exactly where the permission check is failing. >> > execve() only checks MNT_NOEXEC on the "upper" vnode's mountpoint (i.e= . the >> > nullfs mountpoint) and the VOP_ACCESS(.., V_EXEC) check does not look = at >> > MNT_NOEXEC either. >> > >> > I do think there might be bugs in that a nullfs mount that specifies n= oexec or >> > nosuid might not enforce the noexec or nosuid bits if the underlying m= ount >> > point does not have them set (from what I can see). >> >> Thank you for your explanation. Then it is strange, that there is bug, >> that allows execution on originally non executable mountpoint. >> It should be mentioned in the bugs section of the mount_nullfs man page. >> >> It would be useful, if 'mount' output shows inherited options for nullfs= . >> >> If parent is: >> /dev/mirror/gm0s2d on /vol0 (ufs, local, noexec, nosuid, soft-updates) >> >> Then nullfs line will be: >> /vol0/jail/.nullfs/rain on /vol0/jail/rain_new (nullfs, local, noexec, >> nosuid) >> >> instead of just >> /vol0/jail/.nullfs/rain on /vol0/jail/rain_new (nullfs, local) >> >> >> Then I can understand what is expected behavior, but our current state >> is half working, if I can execute scripts and binaries, run jail on it, >> but can't execute "apxs -q MPM_NAME" and few others. > > Hmm, so I was a bit mistaken. =A0The kernel is not failing to exec the bi= nary. > Instead, rtld is reporting the error here: > > static Obj_Entry * > do_load_object(int fd, const char *name, char *path, struct stat *sbp, > =A0int flags) > { > =A0 =A0Obj_Entry *obj; > =A0 =A0struct statfs fs; > > =A0 =A0/* > =A0 =A0 * but first, make sure that environment variables haven't been > =A0 =A0 * used to circumvent the noexec flag on a filesystem. > =A0 =A0 */ > =A0 =A0if (dangerous_ld_env) { > =A0 =A0 =A0 =A0if (fstatfs(fd, &fs) !=3D 0) { > =A0 =A0 =A0 =A0 =A0 =A0_rtld_error("Cannot fstatfs \"%s\"", path); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0return NULL; > =A0 =A0 =A0 =A0} > =A0 =A0 =A0 =A0if (fs.f_flags & MNT_NOEXEC) { > =A0 =A0 =A0 =A0 =A0 =A0_rtld_error("Cannot execute objects on %s\n", fs.f= _mntonname); > =A0 =A0 =A0 =A0 =A0 =A0return NULL; > =A0 =A0 =A0 =A0} > =A0 =A0} > > I wonder if the fstatfs is falling down to the original mount rather than > being caught by nullfs. > > Hmm, nullfs' statfs method returns the flags for the underlying mount, no= t > the flags for the nullfs mount. =A0This is possibly broken, but it is the > behavior nullfs has always had and the behavior it still has on other BSD= s. Not sure if it's relevant here but we fixed a bug locally that I still need to upstream, where NFS is putting another flags word here and incorrectly reporting MNT_NOEXEC. Unfortunately I'm on vacation so I can't easily look it up but there are very few lines in NFS that set f_flags. Thanks, matthew