From owner-freebsd-jail@freebsd.org Sun Feb 19 14:22:18 2017 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 652E0CE5D59 for ; Sun, 19 Feb 2017 14:22:18 +0000 (UTC) (envelope-from nvass@gmx.com) Received: from mout.gmx.net (mout.gmx.net [212.227.15.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B446698F for ; Sun, 19 Feb 2017 14:22:17 +0000 (UTC) (envelope-from nvass@gmx.com) Received: from iris.berlin.strato.de ([192.166.200.216]) by mail.gmx.com (mrgmx001 [212.227.17.184]) with ESMTPSA (Nemesis) id 0MKHMk-1ch73R30TO-001kkf; Sun, 19 Feb 2017 15:21:19 +0100 Subject: Re: unionfs and nullfs combination To: Kurt Jaeger References: <72a56f7e-8e71-2b98-0978-6de863013ce5@gmx.com> <20170219115930.GX13006@home.opsec.eu> Cc: freebsd-jail@freebsd.org From: Nikos Vassiliadis Message-ID: Date: Sun, 19 Feb 2017 15:21:19 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.6.0 MIME-Version: 1.0 In-Reply-To: <20170219115930.GX13006@home.opsec.eu> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:WD019ql4TXbw0ltkRN0bI54i864Qtpo/VvkpWT/WOzB/JDZRS32 6DEDDhpShVawUF/XQ9rie6OTO7NRvrJbxcbRk7ETkyoarS2/f+Q1OMuRvX7I1cJQW3lX864 /PnYfTzhoeHiSkMXtnuXuNI/79ppr6uYTTMszVtWACzCYZdNFFjdyYzIR5oLr5EPuU8LvIh o9dNvicldg++aQ5RmXT8Q== X-UI-Out-Filterresults: notjunk:1;V01:K0:M+nSS1HRZ/Y=:29auel9J8Vp+i4e0vWuw+G Vu+QlvwUnuROQh4q9K8y1scxnyOZYMyNEOhW+2Cr/yF75j+GsyqvAHhqWr3tZUUNfgEKstyV5 TiYCO9ACBQ01hs/KPuO/1E/yx9jKTlAKbnW7UUVVkZsQ3laNNv16PbjTGLa426BjUV2l6azyu Rs9OZXnE8IXHzO+DQdTWD7f1PrcxK0DVfGGg82sEaQikVAtpMQmEPQV6da1AQFFC2BH/E0R8r Tk0Offoi2PzufxEiBCAQ0WNjVXoGy+rY4evNGYnVRAW6zdImWDK643+5pFiVwtbNBNIZ08/vh YDK2YVPYE3+YJQxPzEZmz1OiKVH6uxVPUJ63K3fou7lpEg4/VaZss1U6wQUeT1LR0oIvAXUEH JZ1k1NMBVnk2wHht0rCqxTIMIdf2JHRCY+jewbdUathuZHXtFRWunhcCnoukqPgY1O1q8zvfj 4/A7LK24cKJTvDmYDcS3sejdoVUbE5F5whPxEBdcKxUXbgr2yItfjvD3hn1rw6WK5Ho7aS9lk bMcafR/Zboaq00BEXZY8yE/XlTrLJNTjwpj0eFidkSAVuHuMtaSprUl5WK91112+YDA6G69kl Gdo9ljsD2Ef1KygMpxhLz9g+iFkA/KBDA9o5WQtrHkuH04S72IOcceEqDF1epg/IWR7Kx0TGV 9lzK6KPX0RUVY/7GYSykrkco51v5y0wlmm5jgNS23IrXfd3hv81WTTPJQ2Oxzu51rO3RweirE yAgmtFotPIeG85gHwciOvHGvR9l1HJmw5XIFw8LU+Z8rD/aYyWqHSSBNaYOXQc9BCaB4WUpAB 60Ksqt4 X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 Feb 2017 14:22:18 -0000 Hi Kurt, On 02/19/2017 12:59 PM, Kurt Jaeger wrote: > Ah, to correct myself here: We only used unionfs, not in combination > with nullfs. Can you describe why nullfs with unionfs does not > work in 11 ? > It panics easily. I use the following shell script to create a working environment: > PREFIX=/jails > BASEJAIL=${PREFIX}/base-jail > JAILS="mongo-1 mongo-2 mongo-3 mongo-4 mongo-5 mongo-6" > > mkdir -p $BASEJAIL > for jail in $JAILS > do > mkdir -p ${PREFIX}/$jail > mkdir -p ${PREFIX}/upper/$jail > mount -t nullfs -o ro $BASEJAIL ${PREFIX}/$jail > mount -t unionfs -o noatime ${PREFIX}/upper/$jail ${PREFIX}/$jail > # mount -t devfs none ${PREFIX}/$jail/dev > # cp /etc/resolv.conf ${PREFIX}/$jail/etc/resolv.conf > done > > #chroot $PREFIX/mongo-1 rm -rv /var > #chroot $PREFIX/mongo-2 rm -rv /var Then I can trigger a panic if I run this: rm -rf /jails/mongo-*/* And the panic: > Unread portion of the kernel message buffer: > panic: Memory modified after free 0xfffff80004d0e000(1024) val=adc0dede @ 0xfffff80004d0e000 > > cpuid = 0 > KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe012058ce90 > vpanic() at vpanic+0x186/frame 0xfffffe012058cf10 > panic() at panic+0x43/frame 0xfffffe012058cf70 > trash_ctor() at trash_ctor+0x4b/frame 0xfffffe012058cf80 > uma_zalloc_arg() at uma_zalloc_arg+0x514/frame 0xfffffe012058cfe0 > unionfs_relookup() at unionfs_relookup+0x41/frame 0xfffffe012058d040 > unionfs_mkshadowdir() at unionfs_mkshadowdir+0x120/frame 0xfffffe012058d270 > unionfs_lookup() at unionfs_lookup+0x883/frame 0xfffffe012058d3c0 > VOP_CACHEDLOOKUP_APV() at VOP_CACHEDLOOKUP_APV+0xda/frame 0xfffffe012058d3f0 > vfs_cache_lookup() at vfs_cache_lookup+0xd6/frame 0xfffffe012058d450 > VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0xda/frame 0xfffffe012058d480 > lookup() at lookup+0x6d2/frame 0xfffffe012058d520 > namei() at namei+0x504/frame 0xfffffe012058d5e0 > kern_statat() at kern_statat+0x98/frame 0xfffffe012058d790 > sys_fstatat() at sys_fstatat+0x2c/frame 0xfffffe012058d830 > amd64_syscall() at amd64_syscall+0x2f9/frame 0xfffffe012058d9b0 > Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe012058d9b0 > --- syscall (493, FreeBSD ELF64, sys_fstatat), rip = 0x8008ba62a, rsp = 0x7fffffffe728, rbp = 0x7fffffffe7e0 --- > KDB: enter: panic Thanks in advance for any ideas, Nikos