From owner-freebsd-arch@freebsd.org Wed Jan 9 20:47:54 2019 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 028CE148FE94 for ; Wed, 9 Jan 2019 20:47:54 +0000 (UTC) (envelope-from gavin.d.howard@gmail.com) Received: from mail-ua1-x943.google.com (mail-ua1-x943.google.com [IPv6:2607:f8b0:4864:20::943]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 70FED75C76 for ; Wed, 9 Jan 2019 20:47:52 +0000 (UTC) (envelope-from gavin.d.howard@gmail.com) Received: by mail-ua1-x943.google.com with SMTP id c24so2892385uak.1 for ; Wed, 09 Jan 2019 12:47:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=W/K0QXZ/T31Q1KXu9scBMSWv9yAtB97KIxMWQhNVCXA=; b=PehLjtUSnVvqzzU6Ei9N0IIStsOGfSIev6SATgvDbLL7RQ54/qUkYyHymHwuASnzFB VZfh/xG2M3aZcCJkqRFYAclovnIJV84aVhYVhemrs8GG8KRpDGVmsC9fGpHL+Nm/Mkpz 9EcnRH22GY0K/tsagXwCXOZgIeNaiWY/wSkzQ+c8F5zZxLCd83FTWRmozIPo/9HbKssj 5SKia7HG69UFC6sECOY+dBdeiFvyUdmDWYyE3RsbkDwc+1WBYKLD5vj8Qutd3v8yjUNK YkafAUFU/EeNVgfxnMv8wQk3WCFZw6sT0ILo7dkzZ79ljTqUWP1GOjP9LCjtNjb+Wtkq kztg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=W/K0QXZ/T31Q1KXu9scBMSWv9yAtB97KIxMWQhNVCXA=; b=TTFWSyw+p4dNeW11wY03kUK/kWC3wM/8BmdNS/7eLkcQedcBkecfaMkd+05nFO8yk7 kUZ3STqGEGvQt22ticnn5KK8K/LqMUxRMwRCuO/6V9IAvr50MZjiIqLDyBIEq+QPt0Rp vCCpT7ohawpyM5Vzil0L/CqKoaJtvOKZX0TxIcwZwojk02/l9iLx/2LplSV2M4lTZcCZ WC316M3lNeT64t/VIqRIpn5M50UCKFpf26DTNApB73hj+ndR9GFMWW1MLS+UPX2anAIN K7uXN5r0Y0d9GmPObH6CRm3CyquduWa7fSgxiqFjsZjX8Li/efv/Sto8cL5Og+el6i43 YRbQ== X-Gm-Message-State: AJcUukcn7GUp2tv6kC04oc31ukHMv48XN+JYyBX2rfM1dLzbW3/qVrFr oh8z6zu2Fwla8r8SbvXIYqEesmxgURXmhm8JeA5kjw== X-Google-Smtp-Source: ALg8bN6Mtw3aYZ5LycTsLmHy9m4s+N31EgBHyoJNAL3BWBKN+Wkr++HcE/i1lYSxnkgVTyzN+MCsytNCLJ8UNdowd9I= X-Received: by 2002:a9f:376c:: with SMTP id a41mr2792458uae.131.1547066871370; Wed, 09 Jan 2019 12:47:51 -0800 (PST) MIME-Version: 1.0 References: <8FFA4578-0BAE-4F9F-8A06-AE83283BDEA4@FreeBSD.org> <61F802DC-2E59-4E0A-955D-899EBD7874A1@FreeBSD.org> In-Reply-To: From: Gavin Howard Date: Wed, 9 Jan 2019 13:47:14 -0700 Message-ID: Subject: Re: GNU-compatible, BSD-licensed bc To: "freebsd-arch@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 70FED75C76 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=PehLjtUS; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of gavindhoward@gmail.com designates 2607:f8b0:4864:20::943 as permitted sender) smtp.mailfrom=gavindhoward@gmail.com X-Spamd-Result: default: False [-3.55 / 15.00]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; NEURAL_HAM_SHORT(-0.30)[-0.296,0]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; TAGGED_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.990,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-0.999,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-arch@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; IP_SCORE(-0.25)[ip: (2.81), ipnet: 2607:f8b0::/32(-2.27), asn: 15169(-1.72), country: US(-0.08)]; RCVD_IN_DNSWL_NONE(0.00)[3.4.9.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; TO_DN_EQ_ADDR_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jan 2019 20:47:54 -0000 On Wed, Jan 9, 2019 at 12:58 PM Devin Teske wrote: > > > > On Jan 7, 2019, at 11:45 PM, Gavin Howard wrote: > > On Mon, Jan 7, 2019 at 10:57 PM Devin Teske wrote: > > > > > On Jan 7, 2019, at 4:42 PM, Gavin Howard wrote: > > On Mon, Jan 7, 2019 at 5:38 PM Conrad Meyer wrote: > > > On Mon, Jan 7, 2019 at 4:00 PM Devin Teske wrote: > > How do you handle arbitrary arithmetic precision? > > > Looks like https://github.com/gavinhoward/bc/blob/master/src/num.c . > > > That is correct. Because this bc is meant to help bootstrap the Linux > kernel and have no dependencies other than POSIX 2008, I wrote my own. > > > Impressive. It might be worth turning this into a library itself. > > > Eh...it is completely tuned for bc. And it won't be fast, > unfortunately. See below. > > Also, the POSIX bc standard mandates doing math in decimal. OpenSSL > would not be smart if they did that. > > > Not sure I understand what you mean here. > > > Well, for starters, OpenSSL's BIGNUM is integer only. Yes, those > integers can be any size, but they are only integers. That is not good > enough for bc; it has to allow arbitrary precision, including > non-integers, and its fractional part can be any size, up to a certain > limit, which you can get from my bc by typing "limits" at the prompt > and looking for the value of BC_NUM_MAX (which is actually the maximum > number of decimal digits, period). > > > Thanks for explaining that further. > > [snip] > > There are also a few > peculiarities with the POSIX bc standard that (more or less) require a > standalone implementation. > > > How hard would it be to convert a bn(3)-based library to use your code? > Would there be any performance impact -- I've benchmarked bn(3) to > be pretty fast. > > > It would not be terribly hard, but as I said above, it would not be > fast at all, at least compared to a well-written hardware-based, > binary bignum implementation. But if you want to, go ahead; I would > appreciate the credit (though the license does not even require that). > > > Well, unfortunately, my needs are purely whole-integer arithmetic but > speed is paramount. > > My application of OpenSSL bn(3) is here: > https://reviews.freebsd.org/D16132 > > Though worth noting that I haven't updated the review since November. > Since then, I have made many changes which can be seen on GitHub: > https://FrauBSD.org/libcmb > > > > Also, right now I am working on getting a release candidate out that > will enable me to make a quick port that Stefan could use as a jumping > off point. My build system changed between 1.0 and now, and I would > like to be able to test it. > > > Cool. Looking forward to it. > > > It's out. It works great. The Makefile that I sent to the mailing list > a few messages back does the job well enough, though I was told in a > private message not to use the GNU bc port's pkg-descr file, since it > might be under the GPL. > > However, note that this is not the final 1.1 release; it is just for > testing, even though it is high quality. > > > I'm wondering why you chose to write your own configure.sh instead of > leveraging autotools. I had a goal of absolutely zero dependencies (that is one reason why I imported and adapted an existing history implementation instead of just making readline or editline an optional dependency). That was also why I wrote a custom parser, even though it is complicated, rather than using Lex/Flex and YACC/Bison. Autotools would be a dependency, so I wrote a custom one. > Also, it looks like you have a high number of build-time options. I also > notice that you're into writing tests for your software. It might be interesting > to apply my tool for combining all possible combinations of build options. > > Seen here: > https://github.com/FrauBSD/pkgcenter/blob/master/depend/libcmb/release/Makefile > > It's a great way to make sure all the various build options work together. > -- > Cheers, > Devin Thank you. It looks interesting, though unfortunately, some of my build options are exclusive (for example, the `-b` and `-d` flags to the `configure.sh` script cannot both be used at the same time), and from a cursory glance, I can't tell if libcmb can handle them. I do have a way of testing all of the (valid) build option combinations in `release.sh`, which is a script I run before every release. It tests that it compiles without error and/or warnings on gcc and clang, in debug, release, reldebug, and min size modes, running the test suite on every build. It also runs the test suite under ASan, UBSan, MSan, and Valgrind. (I think it runs 532 builds and does the test suite for all of them.) It takes 5 hours 20 minutes to run on my fast, custom-built machine. And then on top of that, I have a script to generate random math problems (`tests/randmath.py`), which I run for about 10 million iterations. And then I run the afl fuzzer for at least 100 cycles on both calculators, fixing every single crash it finds. I am doing the latter two right now, and if it passes, the release will be out right after that. Gavin Howard