From owner-freebsd-jail@freebsd.org Thu Mar 9 02:02:00 2017 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 62B06D036BE for ; Thu, 9 Mar 2017 02:02:00 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from venus.codepro.be (venus.codepro.be [IPv6:2a01:4f8:162:1127::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.codepro.be", Issuer "Gandi Standard SSL CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 14F5E11FB for ; Thu, 9 Mar 2017 02:02:00 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from [192.168.34.129] (unknown [153.150.77.170]) (Authenticated sender: kp) by venus.codepro.be (Postfix) with ESMTPSA id 2D99C1E5B4 for ; Thu, 9 Mar 2017 03:01:56 +0100 (CET) From: "Kristof Provost" To: "freebsd-jail@freebsd.org" Subject: Jail stuck in dying Date: Thu, 09 Mar 2017 11:01:48 +0900 Message-ID: MIME-Version: 1.0 X-Mailer: MailMate (2.0BETAr6080) Content-Type: text/plain; charset=utf-8; format=flowed; markup=markdown Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Mar 2017 02:02:00 -0000 Hi, On a current box (r314933) I can’t seem to stop my jails. It’s started like this: jail -c name=test0 host.hostname=test vnet persist vnet.interface=epair0b \ path=/usr/jails/jail1 exec.start="/bin/sh /etc/rc" I terminate it with `jail -R test0`, yet the jail stays stuck in dying state: $ jls -a JID IP Address Hostname Path 1 test /usr/jails/jail1 $ jls -na devfs_ruleset=0 dying enforce_statfs=2 host=new ip4=inherit ip6=inherit jid=1 name=test0 osreldate=1200023 osrelease=12.0-CURRENT parent=0 path=/usr/jails/jail1 nopersist securelevel=-1 sysvmsg=disable sysvsem=disable sysvshm=disable vnet=new allow.nochflags allow.nomount allow.mount.nodevfs allow.mount.nofdescfs allow.mount.nolinprocfs allow.mount.nolinsysfs allow.mount.nonullfs allow.mount.noprocfs allow.mount.notmpfs allow.mount.nozfs allow.noquotas allow.noraw_sockets allow.set_hostname allow.nosocket_af allow.nosysvipc children.cur=0 children.max=0 cpuset.id=2 host.domainname="" host.hostid=0 host.hostname=test host.hostuuid=00000000-0000-0000-0000-000000000000 ip4.addr= ip4.saddrsel ip6.addr= ip6.saddrsel I’ve tried debugging this, but the most I can say is that there appears to be something wrong with the reference counting. prison_deref() returns because pr->pr_ref is 3. There are no more jailed processes running, so I have no idea why this happens. The problem doesn’t appear to be related to vnet. I can reproduce it without setting the vnet flag when creating the jail as well. Regards, Kristof From owner-freebsd-jail@freebsd.org Thu Mar 9 03:22:24 2017 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 23E80CD4680 for ; Thu, 9 Mar 2017 03:22:24 +0000 (UTC) (envelope-from dewaynegeraghty@gmail.com) Received: from mail-io0-x232.google.com (mail-io0-x232.google.com [IPv6:2607:f8b0:4001:c06::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E26FA12C1; Thu, 9 Mar 2017 03:22:23 +0000 (UTC) (envelope-from dewaynegeraghty@gmail.com) Received: by mail-io0-x232.google.com with SMTP id g6so20567381ioj.1; Wed, 08 Mar 2017 19:22:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=b+e9jAk2MWhG9ieeY9obazc+NQplfRd//cn1fJGi7+o=; b=YE9OBV7vBtMqOCab/s90DxpZt4DDqiwGKaNBxbBe3TbmEoJ6HEadExOw1REJ1Gk2kY 5UxOyiJpcelFcwPtBl1L4BzGf3FxZE+9gDg4+YEgVLnOyJcLwaLstls2hQZCdh/8zZfu 90UpCPEd674HBuyIn0WOqjPlOZHuhX9wyA+Hzh2clzlPN7HrmJB1gMQzNbIUDaRwDYme G9HPo0aCRZUO/j+Ok3crUNQKElpv9BIofN58v+PYr9awRbIAvbuevLR/LT9bKI1hgqeL h9Fuaddosm0fdFFhD1oYIeVvxBK/HZ87bRnnm5ba2MKIlvQTB7xKmR0lASl8+8Uz+TuB kylA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=b+e9jAk2MWhG9ieeY9obazc+NQplfRd//cn1fJGi7+o=; b=dQJ0XmjRv+fugq87v3M+zzHBGY/qAJ4vTQRkvfKYCjYeoTMaDSQOvkp5Dj9QWCDdwW UUVhMzbJkbqqIFBO1i/jdho3hksoAmgAHywHsEorn+3ikgkoN8Dt1DTaMK2Kyh1Sl4GS jKf0FAjwpl9QB5EHpoJPs8ZmrCGw6a+YoEzNf9WdESN17Ri0ZE2gVjR+QPg1EIddCWte oDTgnTzOe5jpFldimwFIE8aVP/4NV7uGAX3cXlYHQ/FpdhsyulBUDRU+wXzBLW8zCL5W LkAsAsDRhTKsiCY3EbiIHSAylrCY/WCK4lVznij9QopfbNLA2eF5+V1da+B8XyMNzefq 6ADA== X-Gm-Message-State: AMke39m6f9hH2LuHSuVpZ2K2AuX2dQx1E1YIUoKJ0gqnNJaSp4iP7Jt2QBFYj8rcAh19JBDE+TwfoZKXELdRfQ== X-Received: by 10.107.33.8 with SMTP id h8mr10334698ioh.144.1489029743133; Wed, 08 Mar 2017 19:22:23 -0800 (PST) MIME-Version: 1.0 Received: by 10.79.13.142 with HTTP; Wed, 8 Mar 2017 19:21:52 -0800 (PST) In-Reply-To: References: From: Dewayne Geraghty Date: Thu, 9 Mar 2017 14:21:52 +1100 Message-ID: Subject: Re: Jail stuck in dying To: Kristof Provost Cc: "freebsd-jail@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Mar 2017 03:22:24 -0000 Kristof, would you share your devfs.rules settings for the jail that remains in a dying state. I have a similar situation and similar jail.conf settings, in that I use: allow.sysvipc; allow.socket_af; allow.raw_sockets; However I have other jails that stop and can be restarted with the same attributes; but different device rules. So what's different? For the jail that gets stuck in state "dying" devfs.rules: add include $devfsrules_hide_all add include $devfsrules_unhide_basic add include $devfsrules_unhide_login add path 'bpf' unhide add path 'bpf0' unhide As a side note: in my jail.conf I also have jid=3, ie b3 { jid=3; ip4.addr = "10.0.7.91,10.0.5.91,127.1.5.91"; devfs_ruleset = "7"; allow.sysvipc; allow.socket_af; allow.raw_sockets; } (Yes its an application testing jail) Regards, Dewayne.