From owner-freebsd-bugs@freebsd.org Tue Jun 6 05:32:06 2017 Return-Path: Delivered-To: freebsd-bugs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D097EBF20EB for ; Tue, 6 Jun 2017 05:32:06 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BEEAC7E4E9 for ; Tue, 6 Jun 2017 05:32:06 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v565W6DE045890 for ; Tue, 6 Jun 2017 05:32:06 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-bugs@FreeBSD.org Subject: [Bug 219815] ipfw stops working when more than two tables are used Date: Tue, 06 Jun 2017 05:32:06 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.3-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: ecsd@transbay.net X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform op_sys bug_status bug_severity priority component assigned_to reporter Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Jun 2017 05:32:06 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D219815 Bug ID: 219815 Summary: ipfw stops working when more than two tables are used Product: Base System Version: 10.3-RELEASE Hardware: amd64 OS: Any Status: New Severity: Affects Many People Priority: --- Component: kern Assignee: freebsd-bugs@FreeBSD.org Reporter: ecsd@transbay.net This is not bug 165939. I do these things before doing anything with tables: After the original fw flush: sysctl kern.ipc.soacceptqueue=3D2048 ipfw table all flush { as recommended in 165939.} The behavior is bizarre. I started working with tables, loaded addresses in= to them, added rules via programs, and the rules seemed to work ... mostly. I = had a rule for table 86 to deny ip from any to any, and despite adding addresse= s to the table, some of those addresses still got through. When I added a lower numbered rule "deny ip from blah to any" that did not use tables, then the firewall stopped the packets. Ever After a reboot and a system crash (due to a failing disk), I can add O= NE TABLE that works, but despite adding addresses (I know connect to the machi= ne) to other tables, they NEVER TRIGGER. The crash did not injure system softwa= re. I originally manually (a) loaded the tables, (b) issued the ipfw command to effect a rule against the table. After a reboot before the crash, not only would the table rules refuse to trigger, higher-numbered firewall rules also refused to trigger. Causing havoc with my mail system where the tables form= erly firewalled off intense bad guy traffic. I then surmised that perhaps if I got rid of all the table-based firewall r= ules first, then loaded tables, then introduced rules to use them, that might fa= ke the error out, but no. Only the first table rule shows hits and the rest pretend they see nothing at all. It also appears that whatever this is, sto= ps the firing of any later-numbered rules. Some of the tables are very large, but I phased them in from small to large, testing between each load, and it seems that as soon as a 2nd table is introduced, things break. I am stymied because before the reboot and later = disk crash (not affecting the system main disk) the tables were working more than not, apart from the indication of a failure starting to occur when newly ad= ded "block all"-rule addresses were not being blocked. Some of the trouble tickets made reference to ipfw.conf and directories whe= re firewall control information is stored but man and apropos know nothing abo= ut it, nor does the online documentation. Since things were working for a whil= e, I thought I had smashed a limit (when I converted the tables back into "deny" rules there were 12,800 of them.) But nothing says there is any upper limit= to the content of a table (or any sysctl thingy to affect it.) Also I never us= ed a table numbered higher than 120 - the system said the limit was 127, I think, while the man page says the limit is 1024, and then I found I could set the limit with sysctl. I marked this as "affects many people" because if they see what I see ... t= hen it does. I see a couple trouble tickets that suggest similar funky behavior, but the contexts are different and I see nothing that says higher-numbered rules are ignored after tables enter the picture; I see "later rules don't /load/", n= ot "don't fire". Also, ipfw complains when the scripts try to re-enter a subnet already in the table, the message is ipfw: setsockopt(IP_FW_TABLE_XADD): File exists but I assume that is not an issue. However, I get that message even when the rules are sorted and run through uniq to eliminate duplicates. While the ta= bles were working I observed that X/N did not collide with X/M for N !=3D M.The = one hint of something going awry in the system guts is a curious message "No pr= ev search." that occurs sometimes after the "add" of many many addresses, after which the add script seems to die: I do ipfw table N add X1 ipfw table N add X2 ... ipfw add # deny ip from table\(N\) to any and when the "No prev search." message appears, the command to add the rule never executes. Has anyone else observed similar behavior? Does anyone know how to make it = stop (short of migrating to 11.0 from 10.3, if possible.) Are there any patches. Yada etcetera. =3D=3D As to suggestions: * It would be nice to be able to tell ipfw to list the rules in the order t= hey were added to the table. As is they are listed numerically and there is no choice about the order. * When ipfw complains 'XADD: file exists', it would be nice to see the addr= ess being complained of being duplicated. * program call ''system("ipfw -q table # add address/mask");'' still compla= ins about XADD dups to stderr, I thought '-q' was supposed to suppress that. * it would be /real/ cool to have an option for 'ipfw table list', to have = it output only the largest or smallest containing rules, and/or to have 'ipfw table add' delete a larger or smaller subnet upon insertion, e.g. the table contains X/24 X/16 and the modified list command would only report X/16 since X/24 is entirely subsumed by it; or vice-versa. The insertion to delete supersets or subsets, i.e. insert X/16 if X/24 is already present, deletes X/24 and adds X/16; or vice-versa. This is one of those things "left as an exercise for the reader= ", I think. ipfw show, shows me (excerpt): ... 01000 59589 5790205 allow ip from table(93) to any 01000 37042 6535481 allow ip from any to table(93) 01001 0 0 allow ip from table(94) to any dst-port 25 03000 0 0 deny ip from table(86) to any 03001 0 0 deny ip from table(101) to any dst-port 20-25,110,... 03001 0 0 deny ip from table(101) to any dst-port 20-25,110,... 03004 0 0 deny ip from table(103) to any dst-port 20-25,110,... 03005 0 0 deny ip from table(59) to any dst-port 25 03008 0 0 deny ip from table(102) to any dst-port 20-22,110,... 03009 0 0 deny ip from table(53) to any dst-port 25 03010 0 0 deny ip from table(80) to any dst-port 80,443 03011 0 0 deny ip from table(50) to any dst-port 25 03012 0 0 deny ip from table(58) to any dst-port 25 03013 0 0 deny ip from table(51) to any dst-port 25 03014 0 0 deny ip from table(21) to any dst-port 20,21 03015 0 0 deny ip from table(54) to any dst-port 25 03016 0 0 deny ip from table(61) to any dst-port 25 03017 0 0 deny ip from table(52) to any dst-port 25 03018 0 0 deny ip from table(20) to any dst-port 143,993 03019 0 0 deny ip from table(98) to any dst-port 20-22,110,... 03020 0 0 deny ip from table(11) to any dst-port 25 03021 0 0 deny ip from table(55) to any dst-port 25 03022 0 0 deny ip from table(10) to any dst-port 25 03023 0 0 deny ip from table(64) to any dst-port 25 05000 10 428 deny ip from 1.0.0.0/8 to any dst-port 20-25,110,... ... Table 93 (first added) is the only table that works. Table 94 should have registered hits and does not, and for sure addresses were connecting that should have added hits the the 3xxx series rules; and rule 5000 ceased to increment after the table rules were added, despite knowing for sure that m= ore hits should have accrued to it. For jollies to know, table 100 refers to all of CN, and contains some 7700 entries. 103 refers to all of VN, for 550 entries. --=20 You are receiving this mail because: You are the assignee for the bug.=