Date: Wed, 15 Oct 2014 07:04:55 +0400 From: Gleb Smirnoff <glebius@FreeBSD.org> To: Olivier =?iso-8859-1?Q?Cochard-Labb=E9?= <olivier@cochard.me> Cc: svn-src-head <svn-src-head@freebsd.org>, "George V. Neville-Neil" <gnn@freebsd.org>, svn-src-all <svn-src-all@freebsd.org>, src-committers <src-committers@freebsd.org> Subject: Re: svn commit: r272906 - in head/sys: conf libkern netpfil/pf sys Message-ID: <20141015030455.GC73266@FreeBSD.org> In-Reply-To: <CA%2Bq%2BTcpt0zEbgu_LndZb4dzh=xykHvppeykuiG6P9cTJgDGeWw@mail.gmail.com> References: <201410101926.s9AJQRY1049602@svn.freebsd.org> <CA%2Bq%2BTcpt0zEbgu_LndZb4dzh=xykHvppeykuiG6P9cTJgDGeWw@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--wAI/bQb0EMvlZCHl Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit On Mon, Oct 13, 2014 at 06:42:11PM +0200, Olivier Cochard-Labbé wrote: O> > Author: gnn O> > Date: Fri Oct 10 19:26:26 2014 O> > New Revision: 272906 O> > URL: https://svnweb.freebsd.org/changeset/base/272906 O> > O> > Log: O> > Change the PF hash from Jenkins to Murmur3. In forwarding tests O> > this showed a conservative 3% incrase in PPS. O> O> It's not easy to measure 3% of difference ;-) However, it is easy to estimate distribution of a hash function. Also doing the calculation speed test along the way. I've got very disappointing results with this murmur3 against jenkins. I am running 87k states just taken from a live system against a 32k state hash table (which is a default size). Look at results at the end of email. Guess what -j and -m mean. I want either proving me doing the test wrong, or backing the change out ASAP. The sources for the test attached. First you need to run dump2key program to store output of 'pfctl -s states' in a binary format. Then experiment with hashtest. Now dump2key would fail on a IPv6 state. You can either fix that or just filter out IPv6 states. glebius@think:~/pf/hashtest:|>./hashtest -j ../offcarp1.keys 0 collisions 2199 hits 1 collisions 6115 hits 2 collisions 8042 hits 3 collisions 7228 hits 4 collisions 4789 hits 5 collisions 2552 hits 6 collisions 1182 hits 7 collisions 466 hits 8 collisions 137 hits 9 collisions 44 hits 10 collisions 12 hits 11 collisions 2 hits Read 87787 keys, found 87787 keys in hash Hashes calculation time 8188957 nsec glebius@think:~/pf/hashtest:|>./hashtest -m ../offcarp1.keys 0 collisions 23979 hits 1 collisions 4118 hits 2 collisions 1707 hits 3 collisions 691 hits 4 collisions 411 hits 5 collisions 259 hits 6 collisions 231 hits 7 collisions 149 hits 8 collisions 131 hits 9 collisions 124 hits 10 collisions 88 hits 11 collisions 78 hits 12 collisions 58 hits 13 collisions 61 hits 14 collisions 45 hits 15 collisions 39 hits 16 collisions 31 hits 17 collisions 34 hits 18 collisions 28 hits 19 collisions 23 hits 20 collisions 20 hits 21 collisions 18 hits 22 collisions 20 hits 23 collisions 18 hits 24 collisions 10 hits 25 collisions 14 hits 26 collisions 13 hits 27 collisions 14 hits 28 collisions 12 hits 29 collisions 12 hits 30 collisions 13 hits 31 collisions 13 hits 32 collisions 6 hits 33 collisions 10 hits 34 collisions 8 hits 35 collisions 5 hits 36 collisions 9 hits 37 collisions 5 hits 38 collisions 9 hits 39 collisions 5 hits 40 collisions 7 hits 41 collisions 6 hits 42 collisions 11 hits 43 collisions 4 hits 44 collisions 6 hits 45 collisions 5 hits 46 collisions 10 hits 47 collisions 6 hits 48 collisions 3 hits 49 collisions 5 hits 50 collisions 4 hits 51 collisions 4 hits 52 collisions 6 hits 53 collisions 4 hits 54 collisions 7 hits 55 collisions 1 hits 56 collisions 9 hits 57 collisions 2 hits 58 collisions 3 hits 59 collisions 4 hits 60 collisions 2 hits 61 collisions 3 hits 62 collisions 3 hits 63 collisions 2 hits 64 collisions 3 hits 65 collisions 5 hits 67 collisions 2 hits 68 collisions 2 hits 69 collisions 3 hits 70 collisions 3 hits 71 collisions 1 hits 72 collisions 2 hits 73 collisions 1 hits 74 collisions 2 hits 75 collisions 2 hits 76 collisions 1 hits 77 collisions 1 hits 78 collisions 2 hits 79 collisions 2 hits 80 collisions 1 hits 81 collisions 2 hits 82 collisions 1 hits 83 collisions 1 hits 84 collisions 2 hits 85 collisions 3 hits 86 collisions 1 hits 87 collisions 2 hits 88 collisions 1 hits 89 collisions 1 hits 90 collisions 1 hits 91 collisions 2 hits 92 collisions 1 hits 93 collisions 1 hits 95 collisions 2 hits 96 collisions 1 hits 97 collisions 2 hits 98 collisions 1 hits >99 collisions 74 hits Read 87787 keys, found 47693 keys in hash Hashes calculation time 2835154 nsec -- Totus tuus, Glebius. --wAI/bQb0EMvlZCHl Content-Type: application/octet-stream Content-Disposition: attachment; filename="pf-hash-testing.tar.bz2" Content-Transfer-Encoding: base64 QlpoOTFBWSZTWVqaAs8ABid/2dywAoB7////L+/fvv//3/5AAAQAAAIAAIAIYAi/eAj1exp1 1W26642GQGlFGgHDQ0ZNGjRpoZGQwgDIAZBpoAAGQMgCSQmhPUwkejSmanqANMgAANA0AGgB o9qgA0E0Sah+qBppppoBtQYhoAAAAAAaAAJNSRKenpBGmmE00AABo0AAA0A0ANAAcNDRk0aN GmhkZDCAMgBkGmgAAZAyAJEhAEExMjKbCTIp5T8inqYmn6p6gAB6g0zUH6UB+qfWL+Iv1H7D y3HmDkSziZlTB3AAuPXxs5RYgX2ETcQiRgF4ExgBoCIxERBRioMGCMkQUFUQGIkOTr2zmXRG MZOaMkqMoqUKCIsRERVgqxUYxRVFFZ+uePHDnBxjHVGDB/N11rSrN/6KJcYUbiBRVrWsqKIq 1aoXjjS8WmhioiKmZVJeJQkJcJgiGmfLNMWO+sTRFBYsFP5RvtuoI35hk92gE4BOdi2CYvaH UTELYgtQcFziDL9R0Ys6ZeMj7K0LZ3ihQKasBsW9NSYE38bw2F7rKuDPGcEyrl8q+Zvji8nS GWfgzh0r6TzTR95j6+ozdxMoXYGXYsoBFqEJ4l72K6SgjU5GCH9OWn8BZZwnqqoqIdFiiTnN 3GdLb0Ros3FBPFx4dqrU2VUglL1Oua9KomqsWTXD3uQ8WQPZjPWEEWCGQjMEFFwDASORUAv7 kDOWDDEwyDI5wS0zEGGvA9RuIO6MYbMc2nWrzVvuqKMZWXxgECDvSXsaBteBICYM7zpICTLR pETIDS18tEHqEwO0WOlnD2UF2jqFs/22NpjbfOWOlpLUZui3CsvLiromRaoTmNkKVa8RKHFS arAzcUUR1RBOCgwlY6t30UlFCNowVjFqU0KwpGv2kl3cCAwuZ04Xkg9JYWbIPjunfPJuj1PP nHnOg1FrC3RF9cUdJls75Vx6Mc9CqkRdDx3ThbEjb80Da0RKVF0YLM7Deta0yF5iMCxl5ts3 HDpsNpQ3fvt+cnbkqk4iN7nKHVoxJwZsZrk41xKRmJHUPREdK8uRsMW7c6JRsc7jMa4zWcWv VPKF5PPhQXlNLNkKGQVY0YLJn1hZPPEN62HYSg1maV781gX8AE5j8Ccj0DwFWMAtoW8JgXv+ Hx5HBwRFQHf3nEhYikSFMGBJHiIgU/y0E9ffWvncpc8R3pJlI2NLzYiZWSTokbklxmBl49Ry ZPKDazVRVp7QolnFREWQnk9IpE25qZif0tErrSJFQ0yqXEWcNU6vcHMKYV5cCFXTH+ROYsCl ZEiRErGr5Xg6CCBKGezwRPECD4Aw8eUbAxrwIKgsBEyUQBfmLnMco7+eEeu5+3mmF9UGHNk2 /QU0vKTgruhL3PIzB5+XQYp1dTK1QA+e2vLPHdJ7ZVQofDYlrUMmgwk/VKCTbQTGJbjSIUFX ee/xRmPad5/ifiQRD1YbzFw0w8d4et0yRS68wWGusCpHYcVBarQeUzeIkQ3yGBYsoxUEsRHG XGQtvIkE+FDDqo4YlYb+ThKu4melfiKDGigvrXc2CSJMZsqGRnhlCFpmrR0A7Ia5QmzVhoV2 Brok5W1xfL1vR0KaNIopL9ixJUHHt7Y+Y8NfDZ+c5EKhpDUHeaV9AbCh+RrwD6RUaPgQcBe8 rJB3I/gQew7DuXroYL3mQXAMx3dxYQ1B8D6jtKwVgFDEL8DgcM4oxJDLzSQaAq+BBExyXrih hi86kUQVkGgEso2ZayuhHpkSe4Y20pUezDWxOB44rNI+ycoJlAKQSx4RmHrOVDPJGUJu23+7 Pa6hLIv5oHWsTEJAlx31tOOaMZ2jCRE6BRfg1G9loqq2dmBULD0D2C2qwLTZ9aDflALWc6XI YGFpp0CyoDilaIS33tJHVSoJgETHYGZZ8eol2MmN5VFRXaa0dVlTfI9wsSysC7sZtgadnYIU uyYTuTKut+XLyn5CuZXag5iYpIIGF4HBS44CRxuKGo3j2ou3FRWaBgKhVMg2G09Yz07Deg/E TMw16EigM46C/nr5r7U5SiSdKgNPmOjA5/iLBJHhiYLENH2zoO08xBdXzNiqY9mWXCJajTs2 L7zhhr7uF4lQYUwOQLUuDYbGLsD0P0ivt2nDQXMLWXSNJgpqiOo10BL1dPA25MDPexs85ai8 VW9A9rNrCVeXN8QcCZ9xYtQ9IoJMWFFzpCcQREjGFxkyDlwMarQZ1F9YMaNxKRIoxIysoUI5 kbV15SSJjRnB1WgFCQfcvLxjSLJg2QcQ3ZHdQZ2rHehGXESn1owEmL5TNNvX2Zw6SYci7ZZY tVS6xLWmKgtFlrVV2pnl+Ab8jENXCKmjlS7UEpqhMzTU3rCVyC23HYXIVDWGwuQbZsmAUIQF qETSMQKhauhhn0VY0pqIUXdZI0jrUgtsrowrN8CrTTbKy6JOpZzqDMMkoBZrr1bYB62RtGyZ LnOkctxEncnVI0KgT8eXFUFZyFUgZTKMYVSN2UsTTY2mdL4o0MobpNxMBmeTgTFuEWJHNOQm iYmgMrIYoKg3CrDNsMkYFpowrkzKkMaZcK4mGCQWhiHPA2RkoWajVm0be3M4uSkCTDWYazsJ EhnNy4i1bCQg0CyeQtvE0gxaDzmTyFnOmnLZo1AaT8O1CDrLCElamLQkiZiBsP/EEGQDzoD/ 4u5IpwoSC1NAWeA= --wAI/bQb0EMvlZCHl--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20141015030455.GC73266>