From owner-freebsd-current@freebsd.org Wed Jan 13 11:47:32 2021 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3F3CD4D9E03 for ; Wed, 13 Jan 2021 11:47:32 +0000 (UTC) (envelope-from qroxana@protonmail.com) Received: from mail-40130.protonmail.ch (mail-40130.protonmail.ch [185.70.40.130]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "protonmail.com", Issuer "SwissSign Server Gold CA 2014 - G22" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DG5KH4HnRz4Y7D for ; Wed, 13 Jan 2021 11:47:31 +0000 (UTC) (envelope-from qroxana@protonmail.com) Date: Wed, 13 Jan 2021 11:47:14 +0000 To: "Alexander V. Chernikov" From: qroxana Cc: "freebsd-current@freebsd.org" Reply-To: qroxana Subject: Re: jail fib no longer works after net.add_addr_allfibs=0 Message-ID: In-Reply-To: <4325361610393748@mail.yandex.ru> References: <20210111155022.71549b54@bsd64.grem.de> <4325361610393748@mail.yandex.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Status: No, score=-1.2 required=10.0 tests=ALL_TRUSTED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM shortcircuit=no autolearn=disabled version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on mailout.protonmail.ch X-Rspamd-Queue-Id: 4DG5KH4HnRz4Y7D X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.89 / 15.00]; HAS_REPLYTO(0.00)[qroxana@protonmail.com]; TO_DN_EQ_ADDR_SOME(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:185.70.40.0/24:c]; FREEMAIL_FROM(0.00)[protonmail.com]; DKIM_TRACE(0.00)[protonmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[protonmail.com,quarantine]; NEURAL_HAM_SHORT(-0.89)[-0.889]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[protonmail.com]; ASN(0.00)[asn:62371, ipnet:185.70.40.0/24, country:CH]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[protonmail.com:s=protonmail]; REPLYTO_EQ_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; FREEMAIL_REPLYTO(0.00)[protonmail.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RWL_MAILSPIKE_VERYGOOD(0.00)[185.70.40.130:from]; MAILMAN_DEST(0.00)[freebsd-current] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Jan 2021 11:47:32 -0000 =E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90 Original Me= ssage =E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90 On Monday, January 11, 2021 7:37 PM, Alexander V. Chernikov wrote: > 11.01.2021, 14:59, "qroxana" qroxana@protonmail.com: > > > On Mon, 11 Jan 2021 13:25:51 +0000, Alexander V. Chernikov melifaro@ipf= w.ru wrote: > > > > > Could you please consider clarifying the end result you want to achie= ve? > > > =C2=A0If you could include some more details of how it was configured= earlier, it would help as well. > > > > Thank you for the quick reply. > > Let's say there are two jails defined in /etc/jail.conf > > jail1 { > > =C2=A0=C2=A0=C2=A0=C2=A0... > > =C2=A0=C2=A0=C2=A0=C2=A0ip4.addr =3D 192.168.1.101; > > =C2=A0=C2=A0=C2=A0=C2=A0exec.fib =3D 1; > > =C2=A0=C2=A0=C2=A0=C2=A0... > > } > > jail2 { > > =C2=A0=C2=A0=C2=A0=C2=A0... > > =C2=A0=C2=A0=C2=A0=C2=A0ip4.addr =3D 192.168.1.102; > > =C2=A0=C2=A0=C2=A0=C2=A0exec.fib =3D 2; > > =C2=A0=C2=A0=C2=A0=C2=A0... > > } > > Got it, thank you for the clarification. > > > All the traffic in jail1 goes to the default router defined in fib 1, > > and traffic in jail2 goes to the default router defined in fib 2. > > Could you describe interface&routing setup as well? > In particular, I'm looking into details of setting up # of fibs, interfac= e configuration and default route setup. Sure, the interface is em0 for both host and jails: /etc/rc.conf ipv4_addrs_em0=3D"192.168.1.100/24" static_routes=3D"jail1 jail2" route_jail1=3D"default 192.168.1.10 -fib 1" route_jail2=3D"default 192.168.1.20 -fib 2" /etc/jail.conf jail1 { ... interface =3D em0; ip4.addr =3D 192.168.1.101; exec.fib =3D 1; ... } jail2 { ... interface =3D em0; ip4.addr =3D 192.168.1.102; exec.fib =3D 2; ... } I noticed net.add_addr_allfibs defaults to 0 after the commit 2d39824195933c173bbfc9b31773070202d2e30e svn path=3D/head/; revision=3D367491 I also noted that net.add_addr_allfibs=3D1 needs to be added into /etc/sysctl.conf so it can be set before running /etc/rc.d/netif. # setfib -F 2 route add default 192.168.1.20 route: writing to routing socket: Network is unreachable add net default: gateway 192.168.1.20 fib 2: Network is unreachable # sysctl net.add_addr_allfibs=3D1 net.add_addr_allfibs: 0 -> 1 # setfib -F 2 route add default 192.168.1.20 route: writing to routing socket: Network is unreachable add net default: gateway 192.168.1.20 fib 2: Network is unreachable # /etc/rc.d/netif restart # setfib -F 2 route add default 192.168.1.20 add net default: gateway 192.168.1.20 fib 2 I'm just wondering what's the best practice for using jails with fib when net.add_addr_allfibs=3D0? Thanks.