Date: Wed, 2 Oct 2019 12:41:59 -0500 From: Dan Lists <lists.dan@gmail.com> To: Michael Sierchio <kudzu@tenebras.com> Cc: "freebsd-net@freebsd.org" <freebsd-net@freebsd.org> Subject: Re: Bridge Not Forwarding ARP Message-ID: <CAPW8bZ3xcK9UZibk7o2vqNJcP67FUzS5_t7sUBifEm0DuoG8ag@mail.gmail.com> In-Reply-To: <CAPW8bZ27of-L2rc--x6oRK0TiBzOPPtcUeTKA8CW62W7--pesQ@mail.gmail.com> References: <CAPW8bZ2NaXB24p1mtH=A2f8ZukTPn7%2BPKXwUN2F0Osrn0exYNw@mail.gmail.com> <CAHu1Y72BjAgrM6=gFAJK6D9drAqda_oKz1V=cA4Ex18=fdFAQQ@mail.gmail.com> <CAPW8bZ3PE20dCaeddfBGA1FOobCa%2BHAxLVeHgvjKp9%2BB_TapkQ@mail.gmail.com> <CAPW8bZ27of-L2rc--x6oRK0TiBzOPPtcUeTKA8CW62W7--pesQ@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
I finally figured out what is happening. I could use some help fixing the problem. The client (behind the virtual Firewall) sends an ARP Request and the Firewall sends the ARP Request on. ESX is echoing the Request back to the Firewall, which is causing the Firewall to drop the ARP Reply when it arrives. On rare occasions the server sends the ARP Reply fast enough that it arrives before the echoed Request and the Reply is passed. How do I get the firewall to pass the ARP Reply even though it has received the same Request back? On Wed, Jul 24, 2019 at 1:17 PM Dan Lists <lists.dan@gmail.com> wrote: > > Does anyone have any ideas on how to debug this issue? I can build a > custom kernel or use dtrace if necessary. I'm not familiar with the kern= el > source so I don't really know where to look. > > Thanks for any help you can provide. > > > On Mon, Jul 8, 2019 at 12:19 PM Dan Lists <lists.dan@gmail.com> wrote: > >> >> On Mon, Jul 8, 2019 at 11:55 AM Michael Sierchio <kudzu@tenebras.com> >> wrote: >> >>> What's your firewall ruleset look like? (show, don't tell) >>> >> >> The firewall is off for testing (the machine is only on a private >> network). >> >> # ipfw list >> 65535 allow ip from any to any >> >> >>> What does sysctl report on the interfaces and on arp? >>> >>> >> I have not changed any settings. >> >> net.link.ether.inet.log_arp_permanent_modify: 1 >> >> net.link.ether.inet.log_arp_movements: 1 >> >> net.link.ether.inet.log_arp_wrong_iface: 1 >> >> net.link.ether.inet.garp_rexmit_count: 0 >> >> net.link.bridge.allow_llz_overlap: 0 >> >> net.link.bridge.inherit_mac: 0 >> >> net.link.bridge.log_stp: 0 >> >> net.link.bridge.pfil_local_phys: 0 >> >> net.link.bridge.pfil_member: 1 >> >> net.link.bridge.pfil_bridge: 1 >> >> net.link.bridge.ipfw_arp: 0 >> net.link.bridge.pfil_onlyip: 1 >> >> hw.em.eee_setting: 1 >> >> hw.em.rx_process_limit: 100 >> >> hw.em.enable_msix: 1 >> >> hw.em.sbp: 0 >> >> hw.em.smart_pwr_down: 0 >> >> hw.em.txd: 1024 >> >> hw.em.rxd: 1024 >> >> hw.em.rx_abs_int_delay: 66 >> >> hw.em.tx_abs_int_delay: 66 >> >> hw.em.rx_int_delay: 0 >> >> hw.em.tx_int_delay: 66 >> >> hw.em.disable_crc_stripping: 0 >> >> dev.em.2.wake: 0 >> >> dev.em.2.mac_stats.tso_ctx_fail: 0 >> >> dev.em.2.mac_stats.tso_txd: 0 >> >> dev.em.2.mac_stats.tx_frames_1024_1522: 3 >> >> dev.em.2.mac_stats.tx_frames_512_1023: 2 >> >> dev.em.2.mac_stats.tx_frames_256_511: 5675 >> >> dev.em.2.mac_stats.tx_frames_128_255: 4255 >> >> dev.em.2.mac_stats.tx_frames_65_127: 35563 >> >> dev.em.2.mac_stats.tx_frames_64: 918055 >> >> dev.em.2.mac_stats.mcast_pkts_txd: 0 >> >> dev.em.2.mac_stats.bcast_pkts_txd: 0 >> >> dev.em.2.mac_stats.good_pkts_txd: 963553 >> >> dev.em.2.mac_stats.total_pkts_txd: 963553 >> >> dev.em.2.mac_stats.good_octets_txd: 60423175 >> >> dev.em.2.mac_stats.good_octets_recvd: 16906 >> >> dev.em.2.mac_stats.rx_frames_1024_1522: 1 >> >> dev.em.2.mac_stats.rx_frames_512_1023: 0 >> >> dev.em.2.mac_stats.rx_frames_256_511: 0 >> >> dev.em.2.mac_stats.rx_frames_128_255: 0 >> >> dev.em.2.mac_stats.rx_frames_65_127: 235 >> >> dev.em.2.mac_stats.rx_frames_64: 0 >> >> dev.em.2.mac_stats.mcast_pkts_recvd: 0 >> >> dev.em.2.mac_stats.bcast_pkts_recvd: 0 >> >> dev.em.2.mac_stats.good_pkts_recvd: 236 >> >> dev.em.2.mac_stats.total_pkts_recvd: 236 >> >> dev.em.2.mac_stats.xoff_txd: 0 >> >> dev.em.2.mac_stats.xoff_recvd: 0 >> >> dev.em.2.mac_stats.xon_txd: 0 >> >> dev.em.2.mac_stats.xon_recvd: 0 >> >> dev.em.2.mac_stats.coll_ext_errs: 0 >> >> dev.em.2.mac_stats.alignment_errs: 0 >> >> dev.em.2.mac_stats.crc_errs: 0 >> >> dev.em.2.mac_stats.recv_errs: 0 >> >> dev.em.2.mac_stats.recv_jabber: 0 >> >> dev.em.2.mac_stats.recv_oversize: 0 >> >> dev.em.2.mac_stats.recv_fragmented: 0 >> >> dev.em.2.mac_stats.recv_undersize: 0 >> >> dev.em.2.mac_stats.recv_no_buff: 0 >> >> dev.em.2.mac_stats.missed_packets: 0 >> >> dev.em.2.mac_stats.defer_count: 0 >> >> dev.em.2.mac_stats.sequence_errors: 0 >> >> dev.em.2.mac_stats.symbol_errors: 0 >> >> dev.em.2.mac_stats.collision_count: 0 >> >> dev.em.2.mac_stats.late_coll: 0 >> >> dev.em.2.mac_stats.multiple_coll: 0 >> >> dev.em.2.mac_stats.single_coll: 0 >> >> dev.em.2.mac_stats.excess_coll: 0 >> >> dev.em.2.rxd_tail: 235 >> >> dev.em.2.rxd_head: 236 >> >> dev.em.2.txd_tail: 127 >> >> dev.em.2.txd_head: 127 >> >> dev.em.2.fifo_reset: 0 >> >> dev.em.2.fifo_workaround: 0 >> >> dev.em.2.fc_low_water: 45604 >> >> dev.em.2.fc_high_water: 47104 >> >> dev.em.2.rx_control: 32794 >> >> dev.em.2.device_control: 1086325321 >> >> dev.em.2.watchdog_timeouts: 0 >> >> dev.em.2.rx_overruns: 0 >> >> dev.em.2.tx_desc_fail2: 0 >> >> dev.em.2.tx_desc_fail1: 0 >> >> dev.em.2.tx_dma_fail: 0 >> >> dev.em.2.dropped: 0 >> >> dev.em.2.mbuf_defrag_fail: 0 >> >> dev.em.2.cluster_alloc_fail: 0 >> >> dev.em.2.flow_control: 3 >> >> dev.em.2.rx_processing_limit: 100 >> >> dev.em.2.itr: 488 >> >> dev.em.2.tx_abs_int_delay: 66 >> >> dev.em.2.rx_abs_int_delay: 66 >> >> dev.em.2.tx_int_delay: 66 >> >> dev.em.2.rx_int_delay: 0 >> >> dev.em.2.nvm: -1 >> >> dev.em.2.%parent: pci2 >> >> dev.em.2.%pnpinfo: vendor=3D0x8086 device=3D0x100f subvendor=3D0x15ad >> subdevice=3D0x0750 class=3D0x020000 >> >> dev.em.2.%location: slot=3D3 function=3D0 dbsf=3Dpci0:2:3:0 >> handle=3D\_SB_.PCI0.P2P0.S4F0 >> >> dev.em.2.%driver: em >> >> dev.em.2.%desc: Intel(R) PRO/1000 Legacy Network Connection 1.1.0 >> >> dev.em.1.wake: 0 >> >> dev.em.1.mac_stats.tso_ctx_fail: 0 >> >> dev.em.1.mac_stats.tso_txd: 0 >> >> dev.em.1.mac_stats.tx_frames_1024_1522: 1 >> >> dev.em.1.mac_stats.tx_frames_512_1023: 0 >> >> dev.em.1.mac_stats.tx_frames_256_511: 0 >> >> dev.em.1.mac_stats.tx_frames_128_255: 0 >> >> dev.em.1.mac_stats.tx_frames_65_127: 13 >> >> dev.em.1.mac_stats.tx_frames_64: 222 >> >> dev.em.1.mac_stats.mcast_pkts_txd: 0 >> >> dev.em.1.mac_stats.bcast_pkts_txd: 0 >> >> dev.em.1.mac_stats.good_pkts_txd: 236 >> >> dev.em.1.mac_stats.total_pkts_txd: 236 >> >> dev.em.1.mac_stats.good_octets_txd: 15962 >> >> dev.em.1.mac_stats.good_octets_recvd: 5286574431 >> >> dev.em.1.mac_stats.rx_frames_1024_1522: 2876152 >> >> dev.em.1.mac_stats.rx_frames_512_1023: 269810 >> >> dev.em.1.mac_stats.rx_frames_256_511: 605978 >> >> dev.em.1.mac_stats.rx_frames_128_255: 1879331 >> >> dev.em.1.mac_stats.rx_frames_65_127: 3179015 >> >> dev.em.1.mac_stats.rx_frames_64: 6 >> >> dev.em.1.mac_stats.mcast_pkts_recvd: 0 >> >> dev.em.1.mac_stats.bcast_pkts_recvd: 0 >> >> dev.em.1.mac_stats.good_pkts_recvd: 8809664 >> >> dev.em.1.mac_stats.total_pkts_recvd: 8816562 >> >> dev.em.1.mac_stats.xoff_txd: 0 >> >> dev.em.1.mac_stats.xoff_recvd: 0 >> >> dev.em.1.mac_stats.xon_txd: 0 >> >> dev.em.1.mac_stats.xon_recvd: 0 >> >> dev.em.1.mac_stats.coll_ext_errs: 0 >> >> dev.em.1.mac_stats.alignment_errs: 0 >> >> dev.em.1.mac_stats.crc_errs: 0 >> >> dev.em.1.mac_stats.recv_errs: 0 >> >> dev.em.1.mac_stats.recv_jabber: 0 >> >> dev.em.1.mac_stats.recv_oversize: 0 >> >> dev.em.1.mac_stats.recv_fragmented: 0 >> >> dev.em.1.mac_stats.recv_undersize: 0 >> >> dev.em.1.mac_stats.recv_no_buff: 0 >> >> dev.em.1.mac_stats.missed_packets: 0 >> >> dev.em.1.mac_stats.defer_count: 0 >> >> dev.em.1.mac_stats.sequence_errors: 0 >> >> dev.em.1.mac_stats.symbol_errors: 0 >> >> dev.em.1.mac_stats.collision_count: 0 >> >> dev.em.1.mac_stats.late_coll: 0 >> >> dev.em.1.mac_stats.multiple_coll: 0 >> >> dev.em.1.mac_stats.single_coll: 0 >> >> dev.em.1.mac_stats.excess_coll: 0 >> >> dev.em.1.rxd_tail: 167 >> >> dev.em.1.rxd_head: 168 >> >> dev.em.1.txd_tail: 236 >> >> dev.em.1.txd_head: 236 >> >> dev.em.1.fifo_reset: 0 >> >> dev.em.1.fifo_workaround: 0 >> >> dev.em.1.fc_low_water: 45604 >> >> dev.em.1.fc_high_water: 47104 >> >> dev.em.1.rx_control: 32794 >> >> dev.em.1.device_control: 1086325321 >> >> dev.em.1.watchdog_timeouts: 0 >> >> dev.em.1.rx_overruns: 0 >> >> dev.em.1.tx_desc_fail2: 0 >> >> dev.em.1.tx_desc_fail1: 0 >> >> dev.em.1.tx_dma_fail: 0 >> >> dev.em.1.dropped: 0 >> >> dev.em.1.mbuf_defrag_fail: 0 >> >> dev.em.1.cluster_alloc_fail: 0 >> >> dev.em.1.flow_control: 3 >> >> dev.em.1.rx_processing_limit: 100 >> >> dev.em.1.itr: 488 >> >> dev.em.1.tx_abs_int_delay: 66 >> >> dev.em.1.rx_abs_int_delay: 66 >> >> dev.em.1.tx_int_delay: 66 >> >> dev.em.1.rx_int_delay: 0 >> >> dev.em.1.nvm: -1 >> >> dev.em.1.%parent: pci2 >> >> dev.em.1.%pnpinfo: vendor=3D0x8086 device=3D0x100f subvendor=3D0x15ad >> subdevice=3D0x0750 class=3D0x020000 >> >> dev.em.1.%location: slot=3D2 function=3D0 dbsf=3Dpci0:2:2:0 >> handle=3D\_SB_.PCI0.P2P0.S3F0 >> >> dev.em.1.%driver: em >> >> dev.em.1.%desc: Intel(R) PRO/1000 Legacy Network Connection 1.1.0 >> >> >>> >>> On Mon, Jul 8, 2019 at 9:15 AM Dan Lists <lists.dan@gmail.com> wrote: >>> >>> > I have a server running FreeBSD 11.2 that I am wanting to use as a >>> bridged >>> > firewall. I have it set up and it mostly works. The problem is tha= t >>> ARP >>> > replies are not being forwarded from the outside interface to the >>> inside >>> > interface. It appears to be working in the other direction. I see >>> the >>> > ARP request go out on the outside interface and the reply arrives bac= k >>> at >>> > the outside interface. The ARP reply is never getting to the bridge >>> or to >>> > the inside interface. >>> > >>> > The firewall server and the device behind it are in ESX. I think I'= ve >>> > worked all the ESX issues out. When I manually add an ARP entry >>> everything >>> > works. I've done this before with a physical server running FreeBSD >>> 8.4 >>> > and it works as expected. The differences are physical vs virtual, >>> and >>> > 8.4 vs 11.2. >>> > >>> > I'm at a loss as to why it is not working. I've searched the web an= d >>> > found noting. If anyone could offer suggestions on how to fix this o= r >>> > begin to debug it I would greatly appreciate it. >>> > >>> > Thanks, >>> > >>> > Dan >>> > _______________________________________________ >>> > freebsd-net@freebsd.org mailing list >>> > https://lists.freebsd.org/mailman/listinfo/freebsd-net >>> > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org= " >>> > >>> >>> >>> -- >>> >>> "Well," Brahm=C4=81 said, "even after ten thousand explanations, a fool= is no >>> wiser, but an intelligent person requires only two thousand five >>> hundred." >>> >>> - The Mah=C4=81bh=C4=81rata >>> _______________________________________________ >>> freebsd-net@freebsd.org mailing list >>> https://lists.freebsd.org/mailman/listinfo/freebsd-net >>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >>> >>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAPW8bZ3xcK9UZibk7o2vqNJcP67FUzS5_t7sUBifEm0DuoG8ag>