From owner-freebsd-ipfw@freebsd.org Sat Jul 30 14:17:24 2016 Return-Path: Delivered-To: freebsd-ipfw@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A8414BA83DB for ; Sat, 30 Jul 2016 14:17:24 +0000 (UTC) (envelope-from rj@obsigna.com) Received: from mo6-p00-ob.smtp.rzone.de (mo6-p00-ob.smtp.rzone.de [IPv6:2a01:238:20a:202:5300::10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.smtp.rzone.de", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4335F159A for ; Sat, 30 Jul 2016 14:17:23 +0000 (UTC) (envelope-from rj@obsigna.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1469888240; l=5068; s=domk; d=obsigna.com; h=To:References:Content-Transfer-Encoding:Date:In-Reply-To:From: Subject:Mime-Version:Content-Type; bh=9XGldtdeuDaUVC9//kKag858f/wox4J+OkYSZmVXIT0=; b=A9NpB7sYPmJONs4KRkwAnPrFrYRv9Ukmhu3HuXM82PTK9cch0QGeFWKgy4ZAcib+17k oRSc+P4k4OPlvy7Ns/KYJknNq6nw9tXsjDan9430etsObiJB2+Il6LDgDotMQfeSUv16E yFvGLwYRZQM+7lZUkvEbwhH+6JhhYsfyhUc= X-RZG-AUTH: :O2kGeEG7b/pS1EK7WHa0hxqKZr4lnx6UhToX1IWHkW4X7v2ImaU2B+3KSGnPFnK/TBWBAh4A X-RZG-CLASS-ID: mo00 Received: from mail.obsigna.com (bfb6bfbf.virtua.com.br [191.182.191.191]) by smtp.strato.de (RZmta 38.13 DYNA|AUTH) with ESMTPSA id D0b9bcs6UEHJ08N (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (curve secp521r1 with 521 ECDH bits, eq. 15360 bits RSA)) (Client did not present a certificate) for ; Sat, 30 Jul 2016 16:17:19 +0200 (CEST) Received: from rolf.projectworld.net (rolf.projectworld.net [192.168.222.25]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.obsigna.com (Postfix) with ESMTPSA id CAE18229861E for ; Sat, 30 Jul 2016 11:17:15 -0300 (BRT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: ipfw divert filter for IPv4 geo-blocking From: "Dr. Rolf Jansen" In-Reply-To: <9222BB10-C700-4DE7-83A3-BE7A38A11713@obsigna.com> Date: Sat, 30 Jul 2016 11:17:13 -0300 Content-Transfer-Encoding: quoted-printable Message-Id: <1B36CAD7-A139-436B-B7EC-0FFF232F9C6A@obsigna.com> References: <61DFB3E2-6E34-4EEA-8AC6-70094CEACA72@cyclaero.com> <4D047727-F7D0-4BEE-BD42-2501F44C9550@obsigna.com> <9641D08A-0501-4AA2-9DF6-D5AFE6CB2975@obsigna.com> <4d76a492-17ae-cbff-f92f-5bbbb1339aad@freebsd.org> <677900fb-c717-743f-fcfe-86b603466e33@freebsd.org> <0D3C9016-7A4A-46BA-B35F-3844D07562A8@obsigna.com> <1E1DB7E0-D354-4D7A-B657-0ECF94C12CE0@obsigna.com> <50d405a4-3f8f-a706-9cac-d1162925e56a@freebsd.org> <9222BB10-C700-4DE7-83A3-BE7A38A11713@obsigna.com> To: freebsd-ipfw@freebsd.org X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Jul 2016 14:17:24 -0000 > Am 29.07.2016 um 10:23 schrieb Dr. Rolf Jansen : >> Am 29.07.2016 um 06:50 schrieb Julian Elischer : >> On 29/07/2016 5:22 PM, Julian Elischer wrote: >>> On 29/07/2016 4:53 PM, Dr. Rolf Jansen wrote: >>>>> Am 28.07.2016 um 23:48 schrieb Lee Brown : >>>>>=20 >>>>> That makes sense to me. Your /20 range encompasses 201.222.16.0 - >>>>> 201.222.31.255. >>>>> If you want 201.222.20.0-201.222.31.255, you'll need 3 ranges: >>>>>=20 >>>>> 201.222.20.0/22 (201.222.20.0-201.222.23.255) >>>>> 201.222.24.0/22 (201.222.24.0-201.222.27.255) >>>>> 201.222.28.0/22 (201.222.28.0-201.222.31.255) >>>>=20 >>>> Ian, Julian and Lee, >>>>=20 >>>> Thank you vary much for your responses. In order not bloat the = thread, I answer only to one message. >>>>=20 >>>> I found the problem. As a matter of fact, the respective IP ranges = in the LACNIC delegation statistics file are 3 adjacent blocks with 1024 = addresses, i.e. those that you listed in your message above: >>>>=20 >>>> $grep 201.222.2 /usr/local/etc/ipdb/IPRanges/lacnic.dat >>>> lacnic|BR|ipv4|201.222.20.0|1024|20140710|allocated|164725 >>>> lacnic|BR|ipv4|201.222.24.0|1024|20140630|allocated|138376 >>>> lacnic|BR|ipv4|201.222.28.0|1024|20140701|allocated|129095 >>>>=20 >>>> However, my database compilation combines adjacent blocks with the = same country code, and the ranges above turn into one block of 3072 = addresses, which obviously doesn't have a valid netmask - log(3072) =3D = 11,5849625. >>>> ... >>>> ..., it is not sufficient to forget about optimization but I need = to check also whether, the delegation files contain already some = non-CIDR ranges, which need to be broken down. >>>=20 >>> there is code to take ranges and produce cidr sets. >>>=20 >>> We used to have exactly that code in the appletalk code before we = took it out. Appletalk uses ranges. >>> = https://svnweb.freebsd.org/base/release/3.2.0/sys/netatalk/at_control.c?vi= ew=3Dannotate#l703=20 >>=20 >> though htat uassumes input in the form af an appletak sockaddr.. >> there is also this python module >> = https://pythonhosted.org/netaddr/tutorial_01.html#support-for-non-standard= -address-ranges >>=20 >>> maybe you can find other versions on the net. >>> however if you fully populate the table, you will get the correct = result because more specific entries will >>> override less specific entries. To do that you would have to have a = way to describe to your program what >>> value each table entry should output. >>> If you did what you do now, then you would specify the value for the = required countries, and give a default falue for "all others". >>> aggregation of adjacent ranges with same value would be an = optimisation. >=20 > Don't worry, breaking down an arbitrary IP-range into a CIDR = conforming set of ranges, doesn't seem too difficult. ... > ... > Once I come to a conclusion, I will post it to this mailing list. I finished the work on CIDR conformity of the IP ranges tables generated = by the tool geoip. The main constraint is that the start and end address = of an IP block given by the delegation files MUST BE PRESERVED during = the transformation to a set of CIDR records. This target is achieved by: 1. Finding the largest common netmask boundary of the start address = utilizing int(log2(addr_count)); then iteration like Euclid's algorithm in = computing a GCD. 2. Output the CIDR with the given start address and the masklen = belonging to the found netmask. 3. If the CIDR does not match the whole original IP range then set the = start address of the next CIDR block to the next boundary of the common = netmask, and loop over starting at 1. until the original range has been = satisfied. I carefully tested the algorithm and a table that I pipe by the new = geoip tool into ipfw is 100 % identical to the output of the ipfw = command 'table N list'. It is worth to note, that already the original RIR delegation files = contain 457 non CIDR conforming IPv4 ranges in a total of 165815 = original records. I guess that this number will increase in the future = because the RIR's ran empty on new IPv4 ranges and are urged to = subdivide returned old ranges for new delegations. The above algorithm = is ready for this. Generally, CIDR conforming tables are more than twice as large as = optimized (joined adjacencies) IP range tables. All said changes have = been pushed to GitHup already. I am still a little bit amazed how ipfw come to accept incorrect CIDR = ranges and arbitrarily moves the start/end addresses in order to achieve = CIDR conformity, and that without any further notice, and that given = that ipfw can be considered as being quite relevant to system security. = Or, may I assume that ipfw knows always better than the user what should = be allowed or denied. Otherwise, perhaps I am the only one ever who = input incorrect CIDR ranges for processing by ipfw. Best regards Rolf