From owner-svn-src-head@freebsd.org Fri Oct 18 07:55:02 2019 Return-Path: Delivered-To: svn-src-head@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 523DE149237; Fri, 18 Oct 2019 07:55:02 +0000 (UTC) (envelope-from phk@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46vdc61GTcz4Gp4; Fri, 18 Oct 2019 07:55:02 +0000 (UTC) (envelope-from phk@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id E546F5DB5; Fri, 18 Oct 2019 07:55:01 +0000 (UTC) (envelope-from phk@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x9I7t152036486; Fri, 18 Oct 2019 07:55:01 GMT (envelope-from phk@FreeBSD.org) Received: (from phk@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x9I7t1da036485; Fri, 18 Oct 2019 07:55:01 GMT (envelope-from phk@FreeBSD.org) Message-Id: <201910180755.x9I7t1da036485@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: phk set sender to phk@FreeBSD.org using -f From: Poul-Henning Kamp Date: Fri, 18 Oct 2019 07:55:01 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r353718 - head/usr.bin/ministat X-SVN-Group: head X-SVN-Commit-Author: phk X-SVN-Commit-Paths: head/usr.bin/ministat X-SVN-Commit-Revision: 353718 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Oct 2019 07:55:02 -0000 Author: phk Date: Fri Oct 18 07:55:01 2019 New Revision: 353718 URL: https://svnweb.freebsd.org/changeset/base/353718 Log: Improve the way we calculate variance to reduce the rounding errors when variance is small relative to data points. Now [0, 1, 2] shows same standard deviation as [10000000000000, ...1, ...2] Also: Various nitpickery from my own tree. Modified: head/usr.bin/ministat/ministat.c Modified: head/usr.bin/ministat/ministat.c ============================================================================== --- head/usr.bin/ministat/ministat.c Fri Oct 18 03:38:02 2019 (r353717) +++ head/usr.bin/ministat/ministat.c Fri Oct 18 07:55:01 2019 (r353718) @@ -18,6 +18,7 @@ __FBSDID("$FreeBSD$"); #include #include +#include #include #include #include @@ -31,7 +32,7 @@ __FBSDID("$FreeBSD$"); #define NSTUDENT 100 #define NCONF 6 static double const studentpct[] = { 80, 90, 95, 98, 99, 99.5 }; -static double student[NSTUDENT + 1][NCONF] = { +static double const student[NSTUDENT + 1][NCONF] = { /* inf */ { 1.282, 1.645, 1.960, 2.326, 2.576, 3.090 }, /* 1. */ { 3.078, 6.314, 12.706, 31.821, 63.657, 318.313 }, /* 2. */ { 1.886, 2.920, 4.303, 6.965, 9.925, 22.327 }, @@ -152,8 +153,11 @@ NewSet(void) struct dataset *ds; ds = calloc(1, sizeof *ds); + assert(ds != NULL); ds->lpoints = 100000; ds->points = calloc(sizeof *ds->points, ds->lpoints); + assert(ds->points != NULL); + ds->syy = NAN; return(ds); } @@ -166,55 +170,58 @@ AddPoint(struct dataset *ds, double a) dp = ds->points; ds->lpoints *= 4; ds->points = calloc(sizeof *ds->points, ds->lpoints); + assert(ds->points != NULL); memcpy(ds->points, dp, sizeof *dp * ds->n); free(dp); } ds->points[ds->n++] = a; ds->sy += a; - ds->syy += a * a; } static double -Min(struct dataset *ds) +Min(const struct dataset *ds) { return (ds->points[0]); } static double -Max(struct dataset *ds) +Max(const struct dataset *ds) { return (ds->points[ds->n -1]); } static double -Avg(struct dataset *ds) +Avg(const struct dataset *ds) { return(ds->sy / ds->n); } static double -Median(struct dataset *ds) +Median(const struct dataset *ds) { + const unsigned m = ds->n / 2; + if ((ds->n % 2) == 0) - return ((ds->points[ds->n / 2] + (ds->points[(ds->n / 2) - 1])) / 2); - else - return (ds->points[ds->n / 2]); + return ((ds->points[m] + (ds->points[m - 1])) / 2); + return (ds->points[m]); } static double Var(struct dataset *ds) { + unsigned n; + const double a = Avg(ds); - /* - * Due to limited precision it is possible that sy^2/n > syy, - * but variance cannot actually be negative. - */ - if (ds->syy <= ds->sy * ds->sy / ds->n) - return (0); - return (ds->syy - ds->sy * ds->sy / ds->n) / (ds->n - 1.0); + if (isnan(ds->syy)) { + ds->syy = 0.0; + for (n = 0; n < ds->n; n++) + ds->syy += (ds->points[n] - a) * (ds->points[n] - a); + } + + return (ds->syy / (ds->n - 1.0)); } static double @@ -265,7 +272,7 @@ Relative(struct dataset *ds, struct dataset *rs, int c re = t * sqrt(re); if (fabs(d) > e) { - + printf("Difference at %.1f%% confidence\n", studentpct[confidx]); printf(" %g +/- %g\n", d, e); printf(" %g%% +/- %g%%\n", d * 100 / Avg(rs), re * 100 / Avg(rs)); @@ -349,13 +356,17 @@ PlotSet(struct dataset *ds, int val) else bar = 0; - if (pl->bar == NULL) + if (pl->bar == NULL) { pl->bar = calloc(sizeof(char *), pl->num_datasets); + assert(pl->bar != NULL); + } + if (pl->bar[bar] == NULL) { pl->bar[bar] = malloc(pl->width); + assert(pl->bar[bar] != NULL); memset(pl->bar[bar], 0, pl->width); } - + m = 1; i = -1; j = 0; @@ -373,6 +384,7 @@ PlotSet(struct dataset *ds, int val) m += 1; if (m > pl->height) { pl->data = realloc(pl->data, pl->width * m); + assert(pl->data != NULL); memset(pl->data + pl->height * pl->width, 0, (m - pl->height) * pl->width); } @@ -477,6 +489,7 @@ ReadSet(FILE *f, const char *n, int column, const char s = NewSet(); s->name = strdup(n); + assert(s->name != NULL); line = 0; while (fgets(buf, sizeof buf, f) != NULL) { line++; @@ -619,7 +632,10 @@ main(int argc, char **argv) nds = argc; for (i = 0; i < nds; i++) { setfilenames[i] = argv[i]; - setfiles[i] = fopen(argv[i], "r"); + if (!strcmp(argv[i], "-")) + setfiles[0] = stdin; + else + setfiles[i] = fopen(argv[i], "r"); if (setfiles[i] == NULL) err(2, "Cannot open %s", argv[i]); } @@ -639,10 +655,11 @@ main(int argc, char **argv) for (i = 0; i < nds; i++) { ds[i] = ReadSet(setfiles[i], setfilenames[i], column, delim); - fclose(setfiles[i]); + if (setfiles[i] != stdin) + fclose(setfiles[i]); } - for (i = 0; i < nds; i++) + for (i = 0; i < nds; i++) printf("%c %s\n", symbol[i+1], ds[i]->name); if (!flag_n && !suppress_plot) {