From owner-freebsd-stable@freebsd.org Wed Dec 9 11:18:37 2020 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 70B26478B99 for ; Wed, 9 Dec 2020 11:18:37 +0000 (UTC) (envelope-from pmc@citylink.dinoex.sub.org) Received: from uucp.dinoex.sub.de (uucp.dinoex.sub.de [IPv6:2001:1440:5001:1::2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "uucp.dinoex.sub.de", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CrZL44ckLz4mPP for ; Wed, 9 Dec 2020 11:18:35 +0000 (UTC) (envelope-from pmc@citylink.dinoex.sub.org) Received: from uucp.dinoex.sub.de (uucp.dinoex.org [185.220.148.12]) by uucp.dinoex.org (8.16.0.50/8.16.0.50) with ESMTPS id 0B9BI4jQ077658 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Wed, 9 Dec 2020 12:18:04 +0100 (CET) (envelope-from pmc@citylink.dinoex.sub.org) X-Authentication-Warning: uucp.dinoex.sub.de: Host uucp.dinoex.org [185.220.148.12] claimed to be uucp.dinoex.sub.de Received: (from uucp@localhost) by uucp.dinoex.sub.de (8.16.0.50/8.16.0.50/Submit) with UUCP id 0B9BI4UB077657; Wed, 9 Dec 2020 12:18:04 +0100 (CET) (envelope-from pmc@citylink.dinoex.sub.org) Received: from gate.oper.dinoex.org (gate-e [192.168.98.2]) by citylink.dinoex.sub.de (8.16.1/8.16.1) with ESMTP id 0B9B7Kn1005466; Wed, 9 Dec 2020 12:07:20 +0100 (CET) (envelope-from peter@gate.oper.dinoex.org) Received: from gate.oper.dinoex.org (gate-e [192.168.98.2]) by gate.oper.dinoex.org (8.16.1/8.16.1) with ESMTP id 0B9B5NWp005088; Wed, 9 Dec 2020 12:05:23 +0100 (CET) (envelope-from peter@gate.oper.dinoex.org) Received: (from peter@localhost) by gate.oper.dinoex.org (8.16.1/8.16.1/Submit) id 0B9B5NuG005087; Wed, 9 Dec 2020 12:05:23 +0100 (CET) (envelope-from peter) Date: Wed, 9 Dec 2020 12:05:23 +0100 From: Peter Sender: li-fbsd@citylink.dinoex.sub.org To: Dewayne Geraghty Cc: freebsd-stable@freebsd.org Subject: Re: Panic: 12.2 fails to use VIMAGE jails Message-ID: References: <20201207125451.GA11406@gate.oper.dinoex.org> <39DBEA53-960F-4D70-86D7-847E6DFA437D@FreeBSD.org> <20201207233449.GA11025@gate.oper.dinoex.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Milter: Spamilter (Reciever: uucp.dinoex.sub.de; Sender-ip: 185.220.148.12; Sender-helo: uucp.dinoex.sub.de; ) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (uucp.dinoex.org [185.220.148.12]); Wed, 09 Dec 2020 12:18:07 +0100 (CET) X-Rspamd-Queue-Id: 4CrZL44ckLz4mPP X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of pmc@citylink.dinoex.sub.org has no SPF policy when checking 2001:1440:5001:1::2) smtp.mailfrom=pmc@citylink.dinoex.sub.org X-Spamd-Result: default: False [-2.10 / 15.00]; ARC_NA(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[2001:1440:5001:1::2:from]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; HAS_XAW(0.00)[]; DMARC_NA(0.00)[sub.org]; AUTH_NA(1.00)[]; SPAMHAUS_ZRD(0.00)[2001:1440:5001:1::2:from:127.0.2.255]; RCVD_COUNT_THREE(0.00)[4]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCPT_COUNT_TWO(0.00)[2]; R_SPF_NA(0.00)[no SPF record]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:8469, ipnet:2001:1440::/32, country:DE]; RCVD_TLS_LAST(0.00)[]; MAILMAN_DEST(0.00)[freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Dec 2020 11:18:37 -0000 On Wed, Dec 09, 2020 at 02:00:37PM +1100, Dewayne Geraghty wrote: ! On a jail with config: ! exec.start = "/bin/sh -x /etc/rc"; ! exec.stop = "/bin/sh /etc/rc.shutdown"; ! exec.clean; ! ! test_prod { jid=7; persist; ip4.addr = ! "10.0.7.96,10.0.5.96,127.0.5.96"; devfs_ruleset = "6"; ! host.hostuuid=00000000-0000-0000-0001-000000000302; host.hostid=000302; } ! ! I successfully performed ! for i in `seq 10`; do jail -vc test_prod; sleep 3; jail -vr test_prod; done But, this is not a VIMAGE jail, is it? Old-style jails are unaffected by this issue. Only VIMAGE jails, using epair or netgraph, might be affected. (In that case, you would not have an "ip4.addr" configured, and rather a "vnet.interface".) ! I think the normal use of jail.conf is to NOT explicitly use a jid in ! the definition, which may be why this may not have been picked up? ! (Maybe a clue). This is an interesting point. When you stop a jail, it may stay for a more or less long time in a "dying" state (visible with "jls -d"), keeping the jid occupied. During that time, the jail cannot be restarted with that same jid. Once ago, I read people complaining about this, and the advice was to just not define the jid in the definition, so that the jail can be restarted immediately (and will probably grab another jid). I did not find a solid explanation for what is happening in that "dying" state (and why it does take more or less long), even less an approach to fix that. I found some theories circling the net, but these don't really figure. So I would need to look into the source myself - and I did postpone that indefinitely. ;) But what I found out, with the VIMAGE jails (those that can carry their own network interfaces), when you make a slight mistake with managing and handling the interfaces, then the jail will stay in the dying state forever. If you don't make a mistake, then it will finally die within some time. So I decided to keep the jid, so that rightaway nothing is allowed to linger from misconfigured unnoticed. (The tradeoff is obviousely that one might have to wait before restarting.) cheerio, PMc P.S. 41 celsius is phantastic! I envy You! :)