From owner-freebsd-questions@freebsd.org Fri Dec 6 22:17:30 2019 Return-Path: Delivered-To: freebsd-questions@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 039D51B88D3 for ; Fri, 6 Dec 2019 22:17:30 +0000 (UTC) (envelope-from kdunn@acm.org) Received: from azure.elm.relay.mailchannels.net (azure.elm.relay.mailchannels.net [23.83.212.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 47V6Qb4KN0z3yjW for ; Fri, 6 Dec 2019 22:17:27 +0000 (UTC) (envelope-from kdunn@acm.org) X-Sender-Id: hybczc27zq|env-sender|kdunn@acm.org Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id 53B8B1A1448 for ; Fri, 6 Dec 2019 22:17:25 +0000 (UTC) Received: from fly.hiwaay.net (100-96-4-107.trex.outbound.svc.cluster.local [100.96.4.107]) (Authenticated sender: hybczc27zq) by relay.mailchannels.net (Postfix) with ESMTPA id 2C4B61A111A for ; Fri, 6 Dec 2019 22:17:24 +0000 (UTC) X-Sender-Id: hybczc27zq|env-sender|kdunn@acm.org Received: from fly.hiwaay.net ([TEMPUNAVAIL]. [216.180.157.4]) (using TLSv1 with cipher DHE-RSA-AES256-SHA) by 0.0.0.0:2500 (trex/5.18.5); Fri, 06 Dec 2019 22:17:25 +0000 X-MC-Relay: Neutral X-MailChannels-SenderId: hybczc27zq|env-sender|kdunn@acm.org X-MailChannels-Auth-Id: hybczc27zq X-Lonely-Invention: 0dec270338c23116_1575670645130_659327469 X-MC-Loop-Signature: 1575670645130:1056181460 X-MC-Ingress-Time: 1575670645130 Received: from fly.hiwaay.net (localhost.localdomain [127.0.0.1]) by fly.hiwaay.net (8.13.8/8.13.8/fly) with ESMTP id xB6MHIFL014886 for ; Fri, 6 Dec 2019 16:17:18 -0600 Received: from localhost (kldunn@localhost) by fly.hiwaay.net (8.13.8/8.13.8/fly-submit) with ESMTP id xB6MHHCh014881 for ; Fri, 6 Dec 2019 16:17:17 -0600 X-Authentication-Warning: fly.hiwaay.net: kldunn owned process doing -bs Date: Fri, 6 Dec 2019 16:17:17 -0600 (CST) From: Karl Dunn X-X-Sender: kldunn@fly.hiwaay.net Reply-To: Karl Dunn To: freebsd-questions@freebsd.org Subject: Why doesn't ipfw log outgoing DHCP traffic? Message-ID: User-Agent: Alpine 2.03 (LRH 1266 2009-07-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII X-Rspamd-Queue-Id: 47V6Qb4KN0z3yjW X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=permerror (mx1.freebsd.org: domain of kdunn@acm.org uses mechanism not recognized by this client) smtp.mailfrom=kdunn@acm.org X-Spamd-Result: default: False [-1.45 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; HAS_REPLYTO(0.00)[kdunn@acm.org]; HAS_XAW(0.00)[]; TO_DN_NONE(0.00)[]; R_SPF_PERMFAIL(0.00)[~all]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:36483, ipnet:23.83.208.0/21, country:CA]; SUBJECT_ENDS_QUESTION(1.00)[]; MID_RHS_MATCH_FROM(0.00)[]; SH_EMAIL_ZRD(0.00)[0.0.0.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.993,0]; RCVD_COUNT_FIVE(0.00)[6]; REPLYTO_EQ_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; SH_EMAIL_DBL_DONT_QUERY_IPS(0.00)[0.0.0.0]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-0.998,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-questions@freebsd.org]; DMARC_NA(0.00)[acm.org]; RCPT_COUNT_ONE(0.00)[1]; RCVD_TLS_LAST(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[7.212.83.23.list.dnswl.org : 127.0.3.0]; IP_SCORE(-0.36)[ipnet: 23.83.208.0/21(-0.91), asn: 36483(-0.80), country: CA(-0.09)]; RWL_MAILSPIKE_POSSIBLE(0.00)[7.212.83.23.rep.mailspike.net : 127.0.0.17] X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Dec 2019 22:17:30 -0000 Why doesn't ipfw log outgoing DHCP IP traffic? A Dell Optiplex 745 serves as a gateway between my LAN at home and a cable modem. It runs (among other things) ntpd, local DNS, and ISC dhcpd for the local LAN. I wanted to log DHCP traffic through this gateway. However, I have not yet found out why ipfw logs none of the outgoing packets in layer1 (it DOES log them in layer2, though). I have reduced the 745's firewall script as far as I can so I can show the symptoms, and maybe find the problem. I configured a Dell Optiplex GX260 with this reduced script, so it serves as a gateway from another FreeBSD machine to my 192.168.71.0/24 LAN: 745 fw -- GX260 reduced-fw test gateway -- client cable net 192.168.71.28 -- (DHCP client) 192.168.72.28 -- 192.168.72.85 The Dell Optiplex GX260 (the test gateway) runs: uname -a FreeBSD dellgx260.xxxxxx.org 11.3-RELEASE-p2 FreeBSD 11.3-RELEASE-p2 #0: Tue Aug 6 05:03:27 UTC 2019 root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC i386 See below for the reduced ipfw script, output from ipfw show, the (snipped) all.log, the (snipped) dmesg, and the output of ps xjaww. Note that, in the all.log below, the DHCP sequence that aquires the address 192.168.71.24 from the 745 gateway shows a hit on the layer2 outgoing rules, but NO hit on ANY of the layer1 outgoing rules. DISCOVER and REQUEST: layer2 hits but no layer1 hits. OFFER and ACK: work as expected. Can anyone help me here? What have I missed / done wrong? I will provide un-snipped data and other stuff offline if you want. (This cry for help is already way too long.) Karl L. Dunn kdunn@acm.org The reduced firewall rule script used on the GX260: #!/bin/sh # @(#)rc.dellgx260_bareboned_layered_fw-benchtest_191205 KLD # Reduced firewall ruleset for gateway using dellgx260 # # Revisions: # 2019-Dec-05 Original # # Flush out the list before we begin. ipfw -q -f flush # Set rules command prefix fwcmd="ipfw -q add" # The interfaces pif="dc0" # NIC facing the public Internet (gets config via dhclient) lif="em0" # NIC facing our LAN (hard config) ##### Rules ##### # The first few rules control all traffic on the trusted internal interface # and on the loopback interface: # No restrictions on Loopback Interface ${fwcmd} 00010 allow all from any to any via lo0 # Skip to one of four rulesets, depending on which pass invoked us # packets from ether_demux or bdg_forward ${fwcmd} 00020 skipto 10000 all from any to any layer2 in # packets from ip_input ${fwcmd} 00021 skipto 20000 all from any to any not layer2 in # packets from ip_output ${fwcmd} 00022 skipto 30000 all from any to any not layer2 out # packets from ether_output_frame ${fwcmd} 00023 skipto 40000 all from any to any layer2 out # This is just paranoia - should never hit ${fwcmd} 00099 deny log all from any to any ### Incoming layer2 ### # Rules 10000-19999 inclusive # Allow arp to-from any MAC ${fwcmd} 10000 allow log ip from any to any layer2 mac-type arp # # Allow any MACs in from any interface ${fwcmd} 10100 allow log ip from any to any MAC any any in via ${lif} ${fwcmd} 10200 allow log ip from any to any MAC any any in via ${pif} # # Paranoia: Deny any other MACs to/from any interface ${fwcmd} 10300 deny log ip from any to any MAC any any ### End of Incoming layer2 ### ### Incoming layer1 ### # Rules 20000-29999 inclusive # Allow any MAC in from either interface ${fwcmd} 20000 allow log all from any to any in via ${lif} ${fwcmd} 20100 allow log all from any to any in via ${pif} # This is just paranoia - it should never hit ${fwcmd} 20300 deny log all from any to any ### End of Incoming layer1 ### ### Outgoing layer1 ### # Rules 30000-39999 inclusive # Allow everything else on local interface (LAN): log to find out what happens ${fwcmd} 30000 allow log all from any to any out via ${lif} # Allow everything else on public interface: log to find out what happens ${fwcmd} 30100 allow log all from any to any out via ${pif} # This is just paranoia - should never hit ${fwcmd} 30200 deny log all from any to any ### End of Outgoing layer1 ### ### Outgoing layer2 ### # Rules 40000-49999 inclusive # Allow arp to-from any MAC ${fwcmd} 40000 allow log ip from any to any layer2 mac-type arp # Allow any MAC out on either interface ${fwcmd} 40100 allow log ip from any to any MAC any any out via ${lif} ${fwcmd} 40200 allow log ip from any to any MAC any any out via ${pif} # Paranoia: Deny any other MACs to/from any interface ${fwcmd} 40300 deny log ip from any to any MAC any any ### End of Outgoing layer2 ### # The last rule logs all packets that do not match any of the rules in the ruleset: # Paranoia: everything else is denied and logged ${fwcmd} 65500 deny log all from any to any # Turn on a sysctl variable so MAC rules will work # Disable for test /sbin/sysctl net.link.ether.ipfw=1 The output of ipfw show after the end of the shown all.log content: 00010 48 3024 allow ip from any to any via lo0 00020 5 1662 skipto 10000 ip from any to any layer2 in 00021 5 1662 skipto 20000 ip from any to any not layer2 in 00022 0 0 skipto 30000 ip from any to any not layer2 out 00023 3 656 skipto 40000 ip from any to any layer2 out 00099 0 0 deny log ip from any to any 10000 0 0 allow log ip from any to any layer2 mac-type 0x0806 10100 3 984 allow log ip from any to any MAC any any in via em0 10200 2 678 allow log ip from any to any MAC any any in via dc0 10300 0 0 deny log ip from any to any MAC any any 20000 3 984 allow log ip from any to any in via em0 20100 2 678 allow log ip from any to any in via dc0 20300 0 0 deny log ip from any to any 30000 0 0 allow log ip from any to any out via em0 30100 0 0 allow log ip from any to any out via dc0 30200 0 0 deny log ip from any to any 40000 1 0 allow log ip from any to any layer2 mac-type 0x0806 40100 0 0 allow log ip from any to any MAC any any out via em0 40200 2 656 allow log ip from any to any MAC any any out via dc0 40300 0 0 deny log ip from any to any MAC any any 65500 0 0 deny log ip from any to any 65535 0 0 deny ip from any to any The snipped all.log: Dec 6 09:37:04 gw kernel: Copyright (c) 1992-2019 The FreeBSD Project. Dec 6 09:37:04 gw kernel: Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 Dec 6 09:37:04 gw kernel: The Regents of the University of California. All rights reserved. Dec 6 09:37:04 gw kernel: FreeBSD is a registered trademark of The FreeBSD Foundation. Dec 6 09:37:04 gw kernel: FreeBSD 11.3-RELEASE-p3 #0: Mon Aug 19 21:02:24 UTC 2019 Dec 6 09:37:04 gw kernel: root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC i386 Dec 6 09:37:04 gw kernel: FreeBSD clang version 8.0.0 (tags/RELEASE_800/final 356365) (based on LLVM 8.0.0) Dec 6 09:37:04 gw kernel: VT(vga): resolution 640x480 Dec 6 09:37:04 gw kernel: CPU: Intel(R) Pentium(R) 4 CPU 2.00GHz (2000.07-MHz 686-class CPU) Dec 6 09:37:04 gw kernel: Origin="GenuineIntel" Id=0xf27 Family=0xf Model=0x2 Stepping=7 Dec 6 09:37:04 gw kernel: Features=0xbfebfbff Dec 6 09:37:04 gw kernel: Features2=0x4400 Dec 6 09:37:04 gw kernel: real memory = 536870912 (512 MB) Dec 6 09:37:04 gw kernel: avail memory = 492662784 (469 MB) Dec 6 09:37:04 gw kernel: Event timer "LAPIC" quality 100 Dec 6 09:37:04 gw kernel: ACPI APIC Table: .... Dec 6 09:37:04 gw kernel: ipfw2 (+ipv6) initialized, divert loadable, nat loadable, default to deny, logging disabled .... Dec 6 09:37:04 gw kernel: Starting file system checks: Dec 6 09:37:04 gw kernel: /dev/ada0p2: FILE SYSTEM CLEAN; SKIPPING CHECKS Dec 6 09:37:04 gw kernel: /dev/ada0p2: clean, 1831547 free (31099 frags, 225056 blocks, 0.7% fragmentation) Dec 6 09:37:04 gw kernel: Mounting local filesystems:. Dec 6 09:37:04 gw kernel: Setting hostname: gw.kad-hg-benchtest.org. Dec 6 09:37:04 gw kernel: Setting up harvesting: [UMA],[FS_ATIME],SWI,INTERRUPT,NET_NG,NET_ETHER,NET_TUN,MOUSE,KEYBOARD,ATTACH,CACHED Dec 6 09:37:04 gw kernel: Feeding entropy: Dec 6 09:37:04 gw kernel: random: unblocking device. Dec 6 09:37:04 gw kernel: . Dec 6 09:37:04 gw kernel: lo0: link state changed to UP Dec 6 09:37:04 gw kernel: dc0: link state changed to UP Dec 6 09:37:04 gw kernel: Starting Network: lo0 dc0 em0. Dec 6 09:37:04 gw kernel: lo0: flags=8049 metric 0 mtu 16384 Dec 6 09:37:04 gw kernel: options=680003 Dec 6 09:37:04 gw kernel: inet6 ::1 prefixlen 128 Dec 6 09:37:04 gw kernel: inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 Dec 6 09:37:04 gw kernel: inet 127.0.0.1 netmask 0xff000000 Dec 6 09:37:04 gw kernel: nd6 options=21 Dec 6 09:37:04 gw kernel: groups: lo Dec 6 09:37:04 gw kernel: dc0: flags=8843 metric 0 mtu 1500 Dec 6 09:37:04 gw kernel: options=80008 Dec 6 09:37:04 gw kernel: ether 00:04:5a:8e:91:7e Dec 6 09:37:04 gw kernel: hwaddr 00:04:5a:8e:91:7e Dec 6 09:37:04 gw kernel: nd6 options=29 Dec 6 09:37:04 gw kernel: media: Ethernet autoselect (100baseTX ) Dec 6 09:37:04 gw kernel: status: active Dec 6 09:37:04 gw kernel: em0: flags=8843 metric 0 mtu 9014 Dec 6 09:37:04 gw kernel: options=209b Dec 6 09:37:04 gw kernel: ether 00:08:74:d2:ba:c7 Dec 6 09:37:04 gw kernel: hwaddr 00:08:74:d2:ba:c7 Dec 6 09:37:04 gw kernel: inet 192.168.72.28 netmask 0xffffff00 broadcast 192.168.72.255 Dec 6 09:37:04 gw kernel: nd6 options=29 Dec 6 09:37:04 gw kernel: media: Ethernet autoselect Dec 6 09:37:04 gw kernel: status: no carrier Dec 6 09:37:04 gw kernel: net.link.ether.ipfw: 0 -> 1 Dec 6 09:37:04 gw kernel: Firewall rules loaded. Dec 6 09:37:04 gw kernel: Firewall logging enabled. Dec 6 09:37:04 gw kernel: em0: link state changed to UP Dec 6 09:37:04 gw kernel: ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/compat/pkg /usr/local/lib/compat /usr/local/lib/compat/pkg /usr/local/lib/perl5/5.28/mach/CORE Dec 6 09:37:04 gw kernel: a.out ldconfig path: /usr/lib/aout /usr/lib/compat/aout Dec 6 09:37:04 gw kernel: Starting devd. Dec 6 09:37:04 gw kernel: Starting dhclient. Dec 6 09:37:04 gw kernel: DHCPDISCOVER on dc0 to 255.255.255.255 port 67 interval 5 Dec 6 09:37:04 gw kernel: ipfw: 40200 Accept UDP 0.0.0.0:68 255.255.255.255:67 out via dc0 Dec 6 09:37:04 gw kernel: ipfw: 10200 Accept UDP 192.168.71.28:67 192.168.71.24:68 in via dc0 Dec 6 09:37:04 gw kernel: ipfw: 20100 Accept UDP 192.168.71.28:67 192.168.71.24:68 in via dc0 Dec 6 09:37:04 gw kernel: DHCPOFFER from 192.168.71.28 Dec 6 09:37:04 gw kernel: DHCPREQUEST on dc0 to 255.255.255.255 port 67 Dec 6 09:37:04 gw kernel: ipfw: 40200 Accept UDP 0.0.0.0:68 255.255.255.255:67 out via dc0 Dec 6 09:37:04 gw kernel: ipfw: 10200 Accept UDP 192.168.71.28:67 192.168.71.24:68 in via dc0 Dec 6 09:37:04 gw kernel: ipfw: 20100 Accept UDP 192.168.71.28:67 192.168.71.24:68 in via dc0 Dec 6 09:37:04 gw kernel: DHCPACK from 192.168.71.28 Dec 6 09:37:04 gw kernel: ipfw: 40000 Accept MAC out via dc0 Dec 6 09:37:04 gw kernel: bound to 192.168.71.24 -- renewal in 21600 seconds. Dec 6 09:37:04 gw kernel: add host 127.0.0.1: gateway lo0 fib 0: route already in table Dec 6 09:37:04 gw kernel: add host ::1: gateway lo0 fib 0: route already in table Dec 6 09:37:04 gw kernel: add net fe80::: gateway ::1 Dec 6 09:37:04 gw kernel: add net ff02::: gateway ::1 Dec 6 09:37:04 gw kernel: add net ::ffff:0.0.0.0: gateway ::1 Dec 6 09:37:04 gw kernel: add net ::0.0.0.0: gateway ::1 Dec 6 09:37:04 gw kernel: Creating and/or trimming log files. Dec 6 09:37:04 gw kernel: Starting syslogd. Dec 6 09:37:05 gw kernel: No core dumps found. .... The (snipped) dmesg for the Dell Optiples GX260 test gateway: ... FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 11.3-RELEASE-p3 #0: Mon Aug 19 21:02:24 UTC 2019 root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC i386 FreeBSD clang version 8.0.0 (tags/RELEASE_800/final 356365) (based on LLVM 8.0.0) VT(vga): resolution 640x480 CPU: Intel(R) Pentium(R) 4 CPU 2.00GHz (2000.07-MHz 686-class CPU) Origin="GenuineIntel" Id=0xf27 Family=0xf Model=0x2 Stepping=7 Features=0xbfebfbff Features2=0x4400 real memory = 536870912 (512 MB) avail memory = 492662784 (469 MB) ... The ps xjaww output: USER PID PPID PGID SID JOBC STAT TT TIME COMMAND root 0 0 0 0 0 DLs - 0:00.00 [kernel] root 1 0 1 1 0 ILs - 0:00.01 /sbin/init -- root 2 0 0 0 0 DL - 0:00.00 [crypto] root 3 0 0 0 0 DL - 0:00.00 [crypto returns] root 4 0 0 0 0 DL - 0:00.04 [cam] root 5 0 0 0 0 DL - 0:00.00 [fdc0] root 6 0 0 0 0 DL - 0:00.00 [sctp_iterator] root 7 0 0 0 0 DL - 0:00.01 [rand_harvestq] root 8 0 0 0 0 DL - 0:00.00 [soaiod1] root 9 0 0 0 0 DL - 0:00.00 [soaiod2] root 10 0 0 0 0 DL - 0:00.00 [audit] root 11 0 0 0 0 RNL - 2:52.56 [idle] root 12 0 0 0 0 WL - 0:00.91 [intr] root 13 0 0 0 0 DL - 0:00.02 [geom] root 14 0 0 0 0 DL - 0:00.00 [sequencer 00] root 15 0 0 0 0 DL - 0:00.00 [usb] root 16 0 0 0 0 DL - 0:00.00 [soaiod3] root 17 0 0 0 0 DL - 0:00.00 [soaiod4] root 18 0 0 0 0 DL - 0:00.00 [pagedaemon] root 19 0 0 0 0 DL - 0:00.00 [vmdaemon] root 20 0 0 0 0 DNL - 0:00.00 [pagezero] root 21 0 0 0 0 DL - 0:00.01 [bufdaemon] root 22 0 0 0 0 DL - 0:00.00 [bufspacedaemon] root 23 0 0 0 0 DL - 0:00.01 [syncer] root 24 0 0 0 0 DL - 0:00.00 [vnlru] root 89 1 89 89 0 Is - 0:00.00 adjkerntz -i root 365 1 365 365 0 Is - 0:00.00 dhclient: dc0 [priv] (dhclient) _dhcp 417 1 417 417 0 ICs - 0:00.00 dhclient: dc0 (dhclient) root 422 1 422 422 0 Ss - 0:00.01 /sbin/devd -q root 494 1 494 494 0 Ss - 0:00.02 /usr/sbin/syslogd -s -4 root 687 1 687 687 0 Is - 0:00.00 /usr/sbin/sshd root 698 1 698 698 0 Ss - 0:00.01 sendmail: accepting connections (sendmail) smmsp 701 1 701 701 0 Is - 0:00.00 sendmail: Queue runner@00:30:00 for /var/spool/clientmqueue (sendmail) root 726 1 726 726 0 Is - 0:00.01 /usr/sbin/cron -s root 742 1 742 742 0 Is - 0:00.00 /usr/sbin/moused -p /dev/psm0 -t auto root 785 1 785 785 0 Is+ v0 0:00.00 /usr/libexec/getty Pc ttyv0 root 786 1 786 786 0 Is v1 0:00.05 login [pam] (login) root 793 786 793 786 1 R v1 0:00.04 -zsh (zsh) root 804 793 804 786 1 R+ v1 0:00.00 ps xjaww root 787 1 787 787 0 Is+ v2 0:00.00 /usr/libexec/getty Pc ttyv2 root 788 1 788 788 0 Is+ v3 0:00.00 /usr/libexec/getty Pc ttyv3 root 789 1 789 789 0 Is+ v4 0:00.00 /usr/libexec/getty Pc ttyv4 root 790 1 790 790 0 Is+ v5 0:00.00 /usr/libexec/getty Pc ttyv5 root 791 1 791 791 0 Is+ v6 0:00.00 /usr/libexec/getty Pc ttyv6 root 792 1 792 792 0 Is+ v7 0:00.00 /usr/libexec/getty Pc ttyv7