From owner-freebsd-questions@freebsd.org Sun Aug 2 19:19:01 2020 Return-Path: Delivered-To: freebsd-questions@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 345383A518E; Sun, 2 Aug 2020 19:19:01 +0000 (UTC) (envelope-from dan@langille.org) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4BKW5w3FVfz3ZlF; Sun, 2 Aug 2020 19:19:00 +0000 (UTC) (envelope-from dan@langille.org) Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.nyi.internal (Postfix) with ESMTP id C54F55C0113; Sun, 2 Aug 2020 15:18:59 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute2.internal (MEProxy); Sun, 02 Aug 2020 15:18:59 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=langille.org; h= content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; s=fm1; bh=6 0tPHlhGipLqhlriBCG12YoArhP0+M3aQnPe0rKBmFU=; b=n6VoBWM6hEq4j6HXX ZrXEnlzQ0xn6utnuthaRNkPQDZzrn7kJ2OfDN9Vgj3mxcxnhhNEnQp4SSXz2vE7T x256XlUFJ0JgHz5t25YOhwxt5gb9W7sJwmgkBvwCcLHwEJZZu1oU+EczdEJgnAH/ ZC/fWgkc9lNMvUAIrM9pO7k9BtwDrmJ05LO4mdAhPPSprT2WMbOWLVX2wpQsIl3N 2IM3TS1CLH8/xYjElTsfhgJoIG+IduDDIesNK6rgQq3LK4+SW4QEw9nWdA/Ll5m9 fKc6tBUBefpYDpZfz3EfjNdDFmZL0v43Y4qq1mSL06KF99OBIF3pVjtvAxY2ELtr +mxHA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; bh=60tPHlhGipLqhlriBCG12YoArhP0+M3aQnPe0rKBm FU=; b=QlEiGkan5Qh9n+Mf4DBoTJbAY6/7yLmvMZrRwrxH0x0WxtZfUnuCYqTWg 4TTzunvmCmC6wKk2jREw3o7UvIe0+DKIr6ZMdJx+WVxBLDdMXZYO8jYITrmiJZh+ WxMd2fjVplzEJMhg/Ut/hX9/6lI1EpDqosh8X8RROPrcjMsW7E2PgnMMeY2AtBs0 +pN2xo1RycvftUK5kYWZ4aiPKggf/WtE5PGDYE9Q+8EskZYDvgkd5OtkBlEMp1jy 9ltN3KDTXGVHh+5yYsgQdJR5/WY3MdM3eNOYZkcEIRnLjqa0B+i8XFYSCZ5qFot4 +KfhB+Lqf8oVrDLpH2oJKVFU4loYg== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduiedrjedvgddufeelucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurheptggguffhjgffgffkfhfvofesthhqmhdthhdtjeenucfhrhhomhepffgrnhcu nfgrnhhgihhllhgvuceouggrnheslhgrnhhgihhllhgvrdhorhhgqeenucggtffrrghtth gvrhhnpeevvdfgveeugeffieduhfetffduvdffkedtleejteegleduffevhfefkeehgedv veenucffohhmrghinhepthifihhtthgvrhdrtghomhenucfkphepuddtkedrfeeirdelhe druddtnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhep uggrnheslhgrnhhgihhllhgvrdhorhhg X-ME-Proxy: Received: from air01.wifi.int.unixathome.org (pool-108-36-95-10.phlapa.fios.verizon.net [108.36.95.10]) by mail.messagingengine.com (Postfix) with ESMTPA id 73EE13060067; Sun, 2 Aug 2020 15:18:59 -0400 (EDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.1\)) Subject: Re: jail(8) bug with vnet & non-vnet jails running at same time? From: Dan Langille In-Reply-To: <5F270AD4.8080001@gmail.com> Date: Sun, 2 Aug 2020 15:18:58 -0400 Cc: "freebsd-questions@freebsd.org" , "freebsd-jail@freebsd.org" Content-Transfer-Encoding: quoted-printable Message-Id: <01D7BB67-FCC8-4896-8E02-0C26CF6036CC@langille.org> References: <5F26FC5B.6030706@gmail.com> <5F270AD4.8080001@gmail.com> To: Ernie Luzar X-Mailer: Apple Mail (2.3608.120.23.2.1) X-Rspamd-Queue-Id: 4BKW5w3FVfz3ZlF X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=langille.org header.s=fm1 header.b=n6VoBWM6; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=QlEiGkan; dmarc=pass (policy=none) header.from=langille.org; spf=pass (mx1.freebsd.org: domain of dan@langille.org designates 66.111.4.25 as permitted sender) smtp.mailfrom=dan@langille.org X-Spamd-Result: default: False [-2.87 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.25]; RWL_MAILSPIKE_GOOD(0.00)[66.111.4.25:from]; RCVD_COUNT_THREE(0.00)[4]; DKIM_TRACE(0.00)[langille.org:+,messagingengine.com:+]; DMARC_POLICY_ALLOW(-0.50)[langille.org,none]; NEURAL_HAM_SHORT(-1.35)[-1.350]; FREEMAIL_TO(0.00)[gmail.com]; RECEIVED_SPAMHAUS_PBL(0.00)[108.36.95.10:received]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:11403, ipnet:66.111.4.0/24, country:US]; SUBJECT_ENDS_QUESTION(1.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[66.111.4.25:from]; RCVD_TLS_LAST(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[langille.org:s=fm1,messagingengine.com:s=fm3]; NEURAL_HAM_MEDIUM(-1.00)[-1.002]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_LONG(-0.92)[-0.922]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_SOME(0.00)[] X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Aug 2020 19:19:01 -0000 > On Aug 2, 2020, at 2:49 PM, Ernie Luzar wrote: >=20 > Dan Langille wrote: >>> On Aug 2, 2020, at 1:48 PM, Ernie Luzar wrote: >>>=20 >>> Hello list; >>> Please review configuration looking for something I may have missed. = Hopping someone can suggest something that will change the behavior = eliminating the problem. >>>=20 >>>=20 >>> Equipment. Real hardware, 12.1 release, amd64 dual cpu. >>>=20 >>> Description; >>> non-vnet jails and vnet jails using the bridge/epair method can ping = the public internet when only non-vnet jails are started at a time or = when only vnet jails are started at a time. But when both non-vnet jails = and vnet jails are started together then neither one can ping the public = internet. The order of the jails definitions in the jail.conf file has = no effect on changing what is happening. >>>=20 >>> Bug description: >>> When non-vnet jails are started their ip addresses are added to the = NIC facing the public AFTER the public ip address and the non-vnet jail = has access to the public internet. But when both non-vnet jails and vnet = jails are started at the same time then the non-vnet jails ip addresses = gets added before the public ip address of the NIC facing the public = internet causing the host to lose all access to the public internet. = This seems to be a jail(8) bug. >>>=20 >>> It makes no difference which command method is used to start and = stop the jails. >>> Service jail onestart jailname or jail =C3=A2=E2=82=AC=E2=80=9Ccv = jailname >> This may be related to my twitter rant about vnet problems in my own = jails: >> https://twitter.com/DLangille/status/1289944047763693569 >> The symptoms you describe to similar to my own. I cannot access = ports on jails on the same host, but I can access ports on other hosts. >=20 > Your twitter posts are all pf firewall related. =46rom what I can = tell you are using local only vnet jails and want to talk between them. >=20 > Do you have any non-vnet jails running on the host where the 2 vnet = jails are running? >=20 > Do you have any local only vnet jails working on any other systems? One of those two jails in question is vnet, the other is not. There are = many non-vnet jails on this host, only one vnet. > To me knowledge there is only 1 way to have local only vnet jails to = talk to each other. Do not assign ip address to epairXa or to the = bridge. Only assign an ip address to epairXb the interface in the vnet = jail. All the vnet jails you want to be local only have to be members on = the same bridge. I will look at that for this jail. Thank you. --=20 Dan Langille - BSDCan / PGCon dan@langille.org