From owner-svn-src-stable-11@freebsd.org Sat Nov 26 03:39:03 2016 Return-Path: Delivered-To: svn-src-stable-11@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DC59AC54E48; Sat, 26 Nov 2016 03:39:03 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9EAAAF96; Sat, 26 Nov 2016 03:39:03 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id uAQ3d2T5077496; Sat, 26 Nov 2016 03:39:02 GMT (envelope-from emaste@FreeBSD.org) Received: (from emaste@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id uAQ3d22m077495; Sat, 26 Nov 2016 03:39:02 GMT (envelope-from emaste@FreeBSD.org) Message-Id: <201611260339.uAQ3d22m077495@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: emaste set sender to emaste@FreeBSD.org using -f From: Ed Maste Date: Sat, 26 Nov 2016 03:39:02 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r309183 - stable/11/contrib/mdocml X-SVN-Group: stable-11 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-stable-11@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for only the 11-stable src tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Nov 2016 03:39:04 -0000 Author: emaste Date: Sat Nov 26 03:39:02 2016 New Revision: 309183 URL: https://svnweb.freebsd.org/changeset/base/309183 Log: MFC r307003, r307564: makewhatis: make output reproducible r307003: Instead: 1) provide fts_open() with a comparison function to process directories and files in a deterministic order 2) in addition to the existing hash, insert pages into a linked list which will be sorted (by virtue of 1) 3) iterate over pages by the list in 2, instead of hash order Idea from: des r307564: makewhatis: avoid skipping another page after one with no mlinks Submitted by: Ingo Schwarze Sponsored by: The FreeBSD Foundation Modified: stable/11/contrib/mdocml/mandocdb.c Directory Properties: stable/11/ (props changed) Modified: stable/11/contrib/mdocml/mandocdb.c ============================================================================== --- stable/11/contrib/mdocml/mandocdb.c Sat Nov 26 02:53:35 2016 (r309182) +++ stable/11/contrib/mdocml/mandocdb.c Sat Nov 26 03:39:02 2016 (r309183) @@ -103,6 +103,7 @@ struct mpage { char *arch; /* architecture from file content */ char *title; /* title from file content */ char *desc; /* description from file content */ + struct mpage *next; /* singly linked list */ struct mlink *mlinks; /* singly linked list */ int form; /* format from file content */ int name_head_done; @@ -146,6 +147,7 @@ static void dbadd_mlink_name(const stru static int dbopen(int); static void dbprune(void); static void filescan(const char *); +static int fts_compare(const FTSENT *const *, const FTSENT *const *); static void mlink_add(struct mlink *, const struct stat *); static void mlink_check(struct mpage *, struct mlink *); static void mlink_free(struct mlink *); @@ -204,6 +206,7 @@ static struct ohash strings; /* table o static sqlite3 *db = NULL; /* current database */ static sqlite3_stmt *stmts[STMT__MAX]; /* current statements */ static uint64_t name_mask; +static struct mpage *mpage_head; static const struct mdoc_handler mdocs[MDOC_MAX] = { { NULL, 0 }, /* Ap */ @@ -571,6 +574,20 @@ usage: return (int)MANDOCLEVEL_BADARG; } +static int +fts_compare(const FTSENT *const *a, const FTSENT *const *b) +{ + + /* + * The mpage list is processed in the opposite order to which pages are + * added, so traverse the hierarchy in reverse alpha order, resulting + * in database inserts in alpha order. This is not required for correct + * operation, but is helpful when inspecting the database during + * development. + */ + return -strcmp((*a)->fts_name, (*b)->fts_name); +} + /* * Scan a directory tree rooted at "basedir" for manpages. * We use fts(), scanning directory parts along the way for clues to our @@ -600,8 +617,8 @@ treescan(void) argv[0] = "."; argv[1] = (char *)NULL; - f = fts_open((char * const *)argv, - FTS_PHYSICAL | FTS_NOCHDIR, NULL); + f = fts_open((char * const *)argv, FTS_PHYSICAL | FTS_NOCHDIR, + fts_compare); if (f == NULL) { exitcode = (int)MANDOCLEVEL_SYSERR; say("", "&fts_open"); @@ -966,6 +983,8 @@ mlink_add(struct mlink *mlink, const str mpage = mandoc_calloc(1, sizeof(struct mpage)); mpage->inodev.st_ino = inodev.st_ino; mpage->inodev.st_dev = inodev.st_dev; + mpage->next = mpage_head; + mpage_head = mpage; ohash_insert(&mpages, slot, mpage); } else mlink->next = mpage->mlinks; @@ -989,20 +1008,18 @@ mpages_free(void) { struct mpage *mpage; struct mlink *mlink; - unsigned int slot; - mpage = ohash_first(&mpages, &slot); - while (NULL != mpage) { + while (NULL != (mpage = mpage_head)) { while (NULL != (mlink = mpage->mlinks)) { mpage->mlinks = mlink->next; mlink_free(mlink); } + mpage_head = mpage->next; free(mpage->sec); free(mpage->arch); free(mpage->title); free(mpage->desc); free(mpage); - mpage = ohash_next(&mpages, &slot); } } @@ -1123,18 +1140,14 @@ mpages_merge(struct mparse *mp) char *sodest; char *cp; int fd; - unsigned int pslot; if ( ! nodb) SQL_EXEC("BEGIN TRANSACTION"); - mpage = ohash_first(&mpages, &pslot); - while (mpage != NULL) { + for (mpage = mpage_head; mpage != NULL; mpage = mpage->next) { mlinks_undupe(mpage); - if ((mlink = mpage->mlinks) == NULL) { - mpage = ohash_next(&mpages, &pslot); + if ((mlink = mpage->mlinks) == NULL) continue; - } name_mask = NAME_MASK; mandoc_ohash_init(&names, 4, offsetof(struct str, key)); @@ -1256,7 +1269,6 @@ mpages_merge(struct mparse *mp) nextpage: ohash_delete(&strings); ohash_delete(&names); - mpage = ohash_next(&mpages, &pslot); } if (0 == nodb)