Date: Tue, 24 May 2022 12:00:37 +0200 From: Ole Lemke <ol@dbconn.net> To: FreeBSD User <freebsd@walstatt-de.de> Cc: freebsd-jail@freebsd.org, freebsd-net@freebsd.org Subject: Re: FreeBSD 12.3-p5: problems vnet on if_bridge Message-ID: <20220524120037.46b49baa@lenp43s> In-Reply-To: <20220511204755.2028dce9@hermann> References: <20220510212129.35041f02@hermann> <20220511204755.2028dce9@hermann>
next in thread | previous in thread | raw e-mail | index | archive | help
--Sig_//J.G76yaS6QF16AnR+droUK Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable Hello, could you solve the problem? I think I ran into the same problem. I opened a Ticket. https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D264198 I seems to be related to IPFW and effects vnet-Jails and also bhyve VMs. regards Ole Wed, 11 May 2022 20:47:55 +0200 - FreeBSD User <freebsd@walstatt-de.de>: > On Tue, 10 May 2022 21:21:29 +0200 > FreeBSD User <freebsd@walstatt-de.de> wrote: >=20 > > Hello, > >=20 > > I ran into serious trouble setting up a FreeBSD 12.3-RELEASE-p5 > > host having a second NIC and vnt jails attached to that second NIC > > (basically, the host is a recent Xigmanas with Bastille jails, but > > the issue also occurs on a vanilla FreeBSD 12.3). > >=20 > > The host is compromised of two NICs, em0 (management only) and igb0 > > (service/jails). Both, the server and the jails as well as the igb0 > > interface are residing on the same network, but both NICs are > > connected to two different ports on a switch, to which we do not > > have access (part of the campus infrastructure). > >=20 > > Both NICs are attached with a IPv4 of the same network, the host is > > listening on both NICs for services, i.e. port 22 for ssh. No > > problem to connect to both(!) addresses via ssh. igb0 is member of > > an if_bridge. The box also hosts a bunch of vnet jails, each jail > > does have an if_epair created via "jib" and these vnet epairs are > > members of the bridge, to which ifb0 is also member. > >=20 > > Problem: while any service bound to NIC igb0/IPv4 residing on igb0 > > is accessible flawlessly, accessing an jail is almost impossible. > > Pinging a jail does work after a while the ping initiating host has > > been waiting, in ery rare situations someone can access the sshd of > > the jail, but any access of that kind is highly erratic. From 5 > > jails, at most two are responding to pings, the other don't and it > > is non-deterministic which host will respond.=20 > >=20 > > Following some advices found on the web, the following sysctl > > settings are provided to if_bridge:=20 > >=20 > > device if_bridge > > net.link.bridge.ipfw: 0 > > net.link.bridge.allow_llz_overlap: 0 > > net.link.bridge.inherit_mac: 0 > > net.link.bridge.log_stp: 0 > > net.link.bridge.pfil_local_phys: 0 > > net.link.bridge.pfil_member: 0 > > net.link.bridge.ipfw_arp: 0 > > net.link.bridge.pfil_bridge: 0 > > net.link.bridge.pfil_onlyip: 0 > >=20 > > We do not have access to the switch the box is connected to, so I > > don't have access to any logs revealing a problem either to a > > conceptual misunderstanding of networking of mine and so a > > misconfiguration or a probelm with Layer 2 or the switches > > themselfes. > >=20 > > I'd like to ask whether someone has a similar setup up and running > > and could report this > > - or give a hint of the problem I possibly made (igb0 is attached > > to an IPv4 AND is member of an if_brige on which IPv4 attached vnet > > jails are residing). > >=20 > > We have also already setup another "similar" scenarion with the > > same FreeBSD 12.3-p5 version and also two NICs, but our > > "service/jail" NIC is part of a different IPv4 network and the NIC > > is attached to a different switch (to which we have full access). > >=20 > > Thanks in advance, > >=20 > > O. Hartmann > >=20 >=20 > On FreeBSD 12.3-p5, em0 seems to suffer from a bug regarding hardware > chesum support, I see a lot of : > [...] > Flags [.], cksum 0xe826 (incorrect -> 0x606b), seq > 101269476:101270000, ack 5077, win 257, options [nop,nop,TS val > 2618589801 ecr 3610923914], length 524 >=20 > Disabling TXCSUM via "ifconfig em0 -txcsum" renders incorrect -> > correct. >=20 > em0 is: >=20 > em0@pci0:0:25:0: class=3D0x020000 card=3D0x20528086 > chip=3D0x153b8086 rev=3D0x04 hdr=3D0x00 vendor =3D 'Intel Corporation' > device =3D 'Ethernet Connection I217-V' > class =3D network > subclass =3D ethernet > bar [10] =3D type Memory, range 32, base 0xf7d00000, size 131072, > enabled bar [14] =3D type Memory, range 32, base 0xf7d35000, size > 4096, enabled bar [18] =3D type I/O Port, range 32, base 0xf080, size > 32, enabled cap 01[c8] =3D powerspec 2 supports D0 D3 current D0 > cap 05[d0] =3D MSI supports 1 message, 64 bit enabled with 1 message > cap 13[e0] =3D PCI Advanced Features: FLR TP >=20 >=20 > I remember faintly that there was an issue when I used to use FBSD 12 >=20 >=20 >=20 >=20 --Sig_//J.G76yaS6QF16AnR+droUK Content-Type: application/pgp-signature Content-Description: Digitale Signatur von OpenPGP -----BEGIN PGP SIGNATURE----- iQIzBAEBCgAdFiEEcfiBcfJXFWtoGXjffPhD3vCz2EwFAmKMrMUACgkQfPhD3vCz 2EzuvQ//R896neYnAL0V0sAXHkXspYDulmKBD1vS4POImoc10ATP9nOfE55jIiOr Uvdwq9cImshcH8FHC3Ozf8QYfrewmXHMX6eZTjSt99qwJggDVDXJXtLOQxMZbL7w 5+suO4gxbmFCcXjeqHen8oxKNgu9wcoFxC2e2p2E/z41oN69iviF8tV+WE9qwWlI 4OsyN0KpttgA5L12dk81pOMnu46nkpZ7fH0QSW77Q/qR58bpjMFCPx5OBvMySdN/ 25en208W/TAOwvTl1W7qFDX6RmEUbSp16CrijlgeRU8kSny9ttIe/sMzxDMo3dCZ 9FD8DpsBkRQnQYKg98aNmC/gzzklQvkoAXi+O/KRUrNkCzJGObfvF6cykusjLYgH vYD7yjUpMqtiSY7OmyWWrNJJvPLX9zQ3qhhWinZw9RetGhe6eWfGJkLJYwRkBaIv eNjaqL+2ru/o7aHZNlioMfLPko5xCoAQr4NfT4eF0GXR433851ngxbsNMsRbzx7g TDEpOYtlUxX/jJEQeurjQc5Ymay8/rFDMIeOM9nuBlBeZkmxxy/wqhhF0qUHPYfH L5Y8SZeaw9QZHZ7zsuMT4bJj/TbokyeHnQ2S4aQjCFCCq56CKFhgyHV4WSxJvwsV 7Ft6UAUOQnhvBB+K7DqHHXXiRWBKo7afkGWtwoHYDf4XJbkpvUo= =EyM7 -----END PGP SIGNATURE----- --Sig_//J.G76yaS6QF16AnR+droUK--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20220524120037.46b49baa>