Date: Fri, 7 Aug 2015 23:53:32 +0000 (UTC) From: Baptiste Daroussin <bapt@FreeBSD.org> To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r286432 - in projects/collation/usr.bin: . localedef Message-ID: <201508072353.t77NrW0L014715@repo.freebsd.org>
next in thread | raw e-mail | index | archive | help
Author: bapt Date: Fri Aug 7 23:53:31 2015 New Revision: 286432 URL: https://svnweb.freebsd.org/changeset/base/286432 Log: Add localedef(1), a locale definition generator tool The localedef tool can read entire (and unmodified) CLDR posix definition files, and generate all 6 LC categories: LC_COLLATE, LC_CTYPE, LC_TIME, LC_NUMERIC, LC_MONETARY and LC_MESSAGES. This tool has a long history with Solaris. The Nexenta developers modified it to read CLDR files and created the much richer collation formats. The libc collation functions have to be modified to read the new format (called "BSD-1.0") and to handle the new data structures. The result will be that locale-sensitive tools and functions will now properly sort multibyte and unicode strings. Obtained from: Dragonfly Added: projects/collation/usr.bin/localedef/ projects/collation/usr.bin/localedef/Makefile (contents, props changed) projects/collation/usr.bin/localedef/README projects/collation/usr.bin/localedef/charmap.c (contents, props changed) projects/collation/usr.bin/localedef/collate.c (contents, props changed) projects/collation/usr.bin/localedef/ctype.c (contents, props changed) projects/collation/usr.bin/localedef/localedef.1 (contents, props changed) projects/collation/usr.bin/localedef/localedef.c (contents, props changed) projects/collation/usr.bin/localedef/localedef.h (contents, props changed) projects/collation/usr.bin/localedef/messages.c (contents, props changed) projects/collation/usr.bin/localedef/monetary.c (contents, props changed) projects/collation/usr.bin/localedef/numeric.c (contents, props changed) projects/collation/usr.bin/localedef/parser.y projects/collation/usr.bin/localedef/scanner.c (contents, props changed) projects/collation/usr.bin/localedef/time.c (contents, props changed) projects/collation/usr.bin/localedef/wide.c (contents, props changed) Modified: projects/collation/usr.bin/Makefile Modified: projects/collation/usr.bin/Makefile ============================================================================== --- projects/collation/usr.bin/Makefile Fri Aug 7 23:45:32 2015 (r286431) +++ projects/collation/usr.bin/Makefile Fri Aug 7 23:53:31 2015 (r286432) @@ -84,6 +84,7 @@ SUBDIR= ${_addr2line} \ lesskey \ limits \ locale \ + localedef \ lock \ lockf \ logger \ Added: projects/collation/usr.bin/localedef/Makefile ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ projects/collation/usr.bin/localedef/Makefile Fri Aug 7 23:53:31 2015 (r286432) @@ -0,0 +1,30 @@ +# $FreeBSD$ + +PROG= localedef +SRCS= charmap.c \ + collate.c \ + ctype.c \ + localedef.c \ + messages.c \ + monetary.c \ + numeric.c \ + parser.y \ + scanner.c \ + time.c \ + wide.c + +WARNS= 3 +${SRCS:M*.c}: parser.h +parser.h: parser.y +LIBADD= avl + +IGNORE_PRAGMA= yes + +CFLAGS+= -DNEED_SOLARIS_BOOLEAN +CFLAGS+= -I. -I${.CURDIR} +CFLAGS+= -I${.CURDIR}/../../lib/libc/locale +CFLAGS+= -I${.CURDIR}/../../lib/libc/stdtime +CFLAGS+= -I${.CURDIR}/../../sys/cddl/compat/opensolaris +CFLAGS+= -I${.CURDIR}/../../sys/cddl/contrib/opensolaris/uts/common + +.include <bsd.prog.mk> Added: projects/collation/usr.bin/localedef/README ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ projects/collation/usr.bin/localedef/README Fri Aug 7 23:53:31 2015 (r286432) @@ -0,0 +1,11 @@ +While there are tools called "localedef" in Solaris and Linux, this +tool does not share heritage with any other implementation. It was +written independently by Garrett D'Amore while employed at Nexenta +Systems, and thus carries the Nexenta Copyright. + +It was initially released under the CDDL license, but on 4 July 2014, +Nexenta reissued the source under the BSD 2-clause license. This +code is part of the Illumos project. + +see: +https://github.com/Nexenta/illumos-nexenta/commit/cf17542a37fc83d0ae093777e30d480423858c29 Added: projects/collation/usr.bin/localedef/charmap.c ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ projects/collation/usr.bin/localedef/charmap.c Fri Aug 7 23:53:31 2015 (r286432) @@ -0,0 +1,364 @@ +/* + * Copyright 2010 Nexenta Systems, Inc. All rights reserved. + * Copyright 2015 John Marino <draco@marino.st> + * + * This source code is derived from the illumos localedef command, and + * provided under BSD-style license terms by Nexenta Systems, Inc. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" + * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE + * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR + * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF + * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS + * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN + * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) + * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE + * POSSIBILITY OF SUCH DAMAGE. + */ + +/* + * CHARMAP file handling for localedef. + */ +#include <sys/cdefs.h> +__FBSDID("$FreeBSD$"); + +#include <sys/types.h> +#include <sys/avl.h> + +#include <stdio.h> +#include <stdlib.h> +#include <string.h> +#include <limits.h> +#include <unistd.h> +#include <stddef.h> +#include <unistd.h> +#include "localedef.h" +#include "parser.h" + +static avl_tree_t cmap_sym; +static avl_tree_t cmap_wc; + +typedef struct charmap { + const char *name; + wchar_t wc; + avl_node_t avl_sym; + avl_node_t avl_wc; +} charmap_t; + + +/* + * Array of POSIX specific portable characters. + */ + +#pragma GCC diagnostic push +#pragma GCC diagnostic ignored "-Wdiscarded-qualifiers" + +static const struct { + char *name; + int ch; +} portable_chars[] = { + { "NUL", '\0' }, + { "alert", '\a' }, + { "backspace", '\b' }, + { "tab", '\t' }, + { "carriage-return", '\r' }, + { "newline", '\n' }, + { "vertical-tab", '\v' }, + { "form-feed", '\f' }, + { "space", ' ' }, + { "exclamation-mark", '!' }, + { "quotation-mark", '"' }, + { "number-sign", '#' }, + { "dollar-sign", '$' }, + { "percent-sign", '%' }, + { "ampersand", '&' }, + { "apostrophe", '\'' }, + { "left-parenthesis", '(' }, + { "right-parenthesis", '(' }, + { "asterisk", '*' }, + { "plus-sign", '+' }, + { "comma", ','}, + { "hyphen-minus", '-' }, + { "hyphen", '-' }, + { "full-stop", '.' }, + { "period", '.' }, + { "slash", '/' }, + { "solidus", '/' }, + { "zero", '0' }, + { "one", '1' }, + { "two", '2' }, + { "three", '3' }, + { "four", '4' }, + { "five", '5' }, + { "six", '6' }, + { "seven", '7' }, + { "eight", '8' }, + { "nine", '9' }, + { "colon", ':' }, + { "semicolon", ';' }, + { "less-than-sign", '<' }, + { "equals-sign", '=' }, + { "greater-than-sign", '>' }, + { "question-mark", '?' }, + { "commercial-at", '@' }, + { "left-square-bracket", '[' }, + { "backslash", '\\' }, + { "reverse-solidus", '\\' }, + { "right-square-bracket", ']' }, + { "circumflex", '^' }, + { "circumflex-accent", '^' }, + { "low-line", '_' }, + { "underscore", '_' }, + { "grave-accent", '`' }, + { "left-brace", '{' }, + { "left-curly-bracket", '{' }, + { "vertical-line", '|' }, + { "right-brace", '}' }, + { "right-curly-bracket", '}' }, + { "tilde", '~' }, + { "A", 'A' }, + { "B", 'B' }, + { "C", 'C' }, + { "D", 'D' }, + { "E", 'E' }, + { "F", 'F' }, + { "G", 'G' }, + { "H", 'H' }, + { "I", 'I' }, + { "J", 'J' }, + { "K", 'K' }, + { "L", 'L' }, + { "M", 'M' }, + { "N", 'N' }, + { "O", 'O' }, + { "P", 'P' }, + { "Q", 'Q' }, + { "R", 'R' }, + { "S", 'S' }, + { "T", 'T' }, + { "U", 'U' }, + { "V", 'V' }, + { "W", 'W' }, + { "X", 'X' }, + { "Y", 'Y' }, + { "Z", 'Z' }, + { "a", 'a' }, + { "b", 'b' }, + { "c", 'c' }, + { "d", 'd' }, + { "e", 'e' }, + { "f", 'f' }, + { "g", 'g' }, + { "h", 'h' }, + { "i", 'i' }, + { "j", 'j' }, + { "k", 'k' }, + { "l", 'l' }, + { "m", 'm' }, + { "n", 'n' }, + { "o", 'o' }, + { "p", 'p' }, + { "q", 'q' }, + { "r", 'r' }, + { "s", 's' }, + { "t", 't' }, + { "u", 'u' }, + { "v", 'v' }, + { "w", 'w' }, + { "x", 'x' }, + { "y", 'y' }, + { "z", 'z' }, + { NULL, 0 } +}; + +#pragma GCC diagnostic pop + +static int +cmap_compare_sym(const void *n1, const void *n2) +{ + const charmap_t *c1 = n1; + const charmap_t *c2 = n2; + int rv; + + rv = strcmp(c1->name, c2->name); + return ((rv < 0) ? -1 : (rv > 0) ? 1 : 0); +} + +static int +cmap_compare_wc(const void *n1, const void *n2) +{ + const charmap_t *c1 = n1; + const charmap_t *c2 = n2; + + return ((c1->wc < c2->wc) ? -1 : (c1->wc > c2->wc) ? 1 : 0); +} + +void +init_charmap(void) +{ + avl_create(&cmap_sym, cmap_compare_sym, sizeof (charmap_t), + offsetof(charmap_t, avl_sym)); + + avl_create(&cmap_wc, cmap_compare_wc, sizeof (charmap_t), + offsetof(charmap_t, avl_wc)); +} + +static void +add_charmap_impl(char *sym, wchar_t wc, int nodups) +{ + charmap_t srch; + charmap_t *n = NULL; + avl_index_t where; + + srch.wc = wc; + srch.name = sym; + + /* + * also possibly insert the wide mapping, although note that there + * can only be one of these per wide character code. + */ + if ((wc != -1) && ((avl_find(&cmap_wc, &srch, &where)) == NULL)) { + if ((n = calloc(1, sizeof (*n))) == NULL) { + errf("out of memory"); + return; + } + n->wc = wc; + avl_insert(&cmap_wc, n, where); + } + + if (sym) { + if (avl_find(&cmap_sym, &srch, &where) != NULL) { + if (nodups) { + errf("duplicate character definition"); + } + return; + } + if ((n == NULL) && ((n = calloc(1, sizeof (*n))) == NULL)) { + errf("out of memory"); + return; + } + n->wc = wc; + n->name = sym; + + avl_insert(&cmap_sym, n, where); + } +} + +void +add_charmap(char *sym, int c) +{ + add_charmap_impl(sym, c, 1); +} + +void +add_charmap_undefined(char *sym) +{ + charmap_t srch; + charmap_t *cm = NULL; + + srch.name = sym; + cm = avl_find(&cmap_sym, &srch, NULL); + + if ((undefok == 0) && ((cm == NULL) || (cm->wc == -1))) { + warn("undefined symbol <%s>", sym); + add_charmap_impl(sym, -1, 0); + } else { + free(sym); + } +} + +void +add_charmap_range(char *s, char *e, int wc) +{ + int ls, le; + int si; + int sn, en; + int i; + + static const char *digits = "0123456789"; + + ls = strlen(s); + le = strlen(e); + + if (((si = strcspn(s, digits)) == 0) || (si == ls) || + (strncmp(s, e, si) != 0) || + ((int)strspn(s + si, digits) != (ls - si)) || + ((int)strspn(e + si, digits) != (le - si)) || + ((sn = atoi(s + si)) > ((en = atoi(e + si))))) { + errf("malformed charmap range"); + return; + } + + s[si] = 0; + + for (i = sn; i <= en; i++) { + char *nn; + (void) asprintf(&nn, "%s%0*u", s, ls - si, i); + if (nn == NULL) { + errf("out of memory"); + return; + } + + add_charmap_impl(nn, wc, 1); + wc++; + } + free(s); + free(e); +} + +void +add_charmap_char(char *name, int val) +{ + add_charmap_impl(name, val, 0); +} + +/* + * POSIX insists that certain entries be present, even when not in the + * orginal charmap file. + */ +void +add_charmap_posix(void) +{ + int i; + + for (i = 0; portable_chars[i].name; i++) { + add_charmap_char(portable_chars[i].name, portable_chars[i].ch); + } +} + +int +lookup_charmap(const char *sym, wchar_t *wc) +{ + charmap_t srch; + charmap_t *n; + + srch.name = sym; + n = avl_find(&cmap_sym, &srch, NULL); + if (n && n->wc != -1) { + if (wc) + *wc = n->wc; + return (0); + } + return (-1); +} + +int +check_charmap(wchar_t wc) +{ + charmap_t srch; + + srch.wc = wc; + return (avl_find(&cmap_wc, &srch, NULL) ? 0 : -1); +} Added: projects/collation/usr.bin/localedef/collate.c ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ projects/collation/usr.bin/localedef/collate.c Fri Aug 7 23:53:31 2015 (r286432) @@ -0,0 +1,1299 @@ +/* + * Copyright 2010 Nexenta Systems, Inc. All rights reserved. + * Copyright 2015 John Marino <draco@marino.st> + * + * This source code is derived from the illumos localedef command, and + * provided under BSD-style license terms by Nexenta Systems, Inc. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" + * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE + * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR + * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF + * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS + * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN + * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) + * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE + * POSSIBILITY OF SUCH DAMAGE. + */ + +/* + * LC_COLLATE database generation routines for localedef. + */ +#include <sys/cdefs.h> +__FBSDID("$FreeBSD$"); + +#include <sys/types.h> +#include <sys/avl.h> + +#include <stdio.h> +#include <stddef.h> +#include <stdlib.h> +#include <errno.h> +#include <string.h> +#include <unistd.h> +#include <wchar.h> +#include <limits.h> +#include "localedef.h" +#include "parser.h" +#include "collate.h" + +/* + * Design notes. + * + * It will be extremely helpful to the reader if they have access to + * the localedef and locale file format specifications available. + * Latest versions of these are available from www.opengroup.org. + * + * The design for the collation code is a bit complex. The goal is a + * single collation database as described in collate.h (in + * libc/port/locale). However, there are some other tidbits: + * + * a) The substitution entries are now a directly indexable array. A + * priority elsewhere in the table is taken as an index into the + * substitution table if it has a high bit (COLLATE_SUBST_PRIORITY) + * set. (The bit is cleared and the result is the index into the + * table. + * + * b) We eliminate duplicate entries into the substitution table. + * This saves a lot of space. + * + * c) The priorities for each level are "compressed", so that each + * sorting level has consecutively numbered priorities starting at 1. + * (O is reserved for the ignore priority.) This means sort levels + * which only have a few distinct priorities can represent the + * priority level in fewer bits, which makes the strxfrm output + * smaller. + * + * d) We record the total number of priorities so that strxfrm can + * figure out how many bytes to expand a numeric priority into. + * + * e) For the UNDEFINED pass (the last pass), we record the maximum + * number of bits needed to uniquely prioritize these entries, so that + * the last pass can also use smaller strxfrm output when possible. + * + * f) Priorities with the sign bit set are verboten. This works out + * because no active character set needs that bit to carry significant + * information once the character is in wide form. + * + * To process the entire data to make the database, we actually run + * multiple passes over the data. + * + * The first pass, which is done at parse time, identifies elements, + * substitutions, and such, and records them in priority order. As + * some priorities can refer to other priorities, using forward + * references, we use a table of references indicating whether the + * priority's value has been resolved, or whether it is still a + * reference. + * + * The second pass walks over all the items in priority order, noting + * that they are used directly, and not just an indirect reference. + * This is done by creating a "weight" structure for the item. The + * weights are stashed in an AVL tree sorted by relative "priority". + * + * The third pass walks over all the weight structures, in priority + * order, and assigns a new monotonically increasing (per sort level) + * weight value to them. These are the values that will actually be + * written to the file. + * + * The fourth pass just writes the data out. + */ + +/* + * In order to resolve the priorities, we create a table of priorities. + * Entries in the table can be in one of three states. + * + * UNKNOWN is for newly allocated entries, and indicates that nothing + * is known about the priority. (For example, when new entries are created + * for collating-symbols, this is the value assigned for them until the + * collating symbol's order has been determined. + * + * RESOLVED is used for an entry where the priority indicates the final + * numeric weight. + * + * REFER is used for entries that reference other entries. Typically + * this is used for forward references. A collating-symbol can never + * have this value. + * + * The "pass" field is used during final resolution to aid in detection + * of referencing loops. (For example <A> depends on <B>, but <B> has its + * priority dependent on <A>.) + */ +typedef enum { + UNKNOWN, /* priority is totally unknown */ + RESOLVED, /* priority value fully resolved */ + REFER /* priority is a reference (index) */ +} res_t; + +typedef struct weight { + int32_t pri; + int opt; + avl_node_t avl; +} weight_t; + +typedef struct priority { + res_t res; + int32_t pri; + int pass; + int lineno; +} collpri_t; + +#define NUM_WT collinfo.directive_count + +/* + * These are the abstract collating symbols, which are just a symbolic + * way to reference a priority. + */ +struct collsym { + char *name; + int32_t ref; + avl_node_t avl; +}; + +/* + * These are also abstract collating symbols, but we allow them to have + * different priorities at different levels. + */ +typedef struct collundef { + char *name; + int32_t ref[COLL_WEIGHTS_MAX]; + avl_node_t avl; +} collundef_t; + +/* + * These are called "chains" in libc. This records the fact that two + * more characters should be treated as a single collating entity when + * they appear together. For example, in Spanish <C><h> gets collated + * as a character between <C> and <D>. + */ +struct collelem { + char *symbol; + wchar_t *expand; + int32_t ref[COLL_WEIGHTS_MAX]; + avl_node_t avl_bysymbol; + avl_node_t avl_byexpand; +}; + +/* + * Individual characters have a sequence of weights as well. + */ +typedef struct collchar { + wchar_t wc; + int32_t ref[COLL_WEIGHTS_MAX]; + avl_node_t avl; +} collchar_t; + +/* + * Substitution entries. The key is itself a priority. Note that + * when we create one of these, we *automatically* wind up with a + * fully resolved priority for the key, because creation of + * substitutions creates a resolved priority at the same time. + */ +typedef struct { + int32_t key; + int32_t ref[COLLATE_STR_LEN]; + avl_node_t avl; + avl_node_t avl_ref; +} subst_t; + +static avl_tree_t collsyms; +static avl_tree_t collundefs; +static avl_tree_t elem_by_symbol; +static avl_tree_t elem_by_expand; +static avl_tree_t collchars; +static avl_tree_t substs[COLL_WEIGHTS_MAX]; +static avl_tree_t substs_ref[COLL_WEIGHTS_MAX]; +static avl_tree_t weights[COLL_WEIGHTS_MAX]; +static int32_t nweight[COLL_WEIGHTS_MAX]; + +/* + * This is state tracking for the ellipsis token. Note that we start + * the initial values so that the ellipsis logic will think we got a + * magic starting value of NUL. It starts at minus one because the + * starting point is exclusive -- i.e. the starting point is not + * itself handled by the ellipsis code. + */ +static int currorder = EOF; +static int lastorder = EOF; +static collelem_t *currelem; +static collchar_t *currchar; +static collundef_t *currundef; +static wchar_t ellipsis_start = 0; +static int32_t ellipsis_weights[COLL_WEIGHTS_MAX]; + +/* + * We keep a running tally of weights. + */ +static int nextpri = 1; +static int nextsubst[COLL_WEIGHTS_MAX] = { 0 }; + +/* + * This array collects up the weights for each level. + */ +static int32_t order_weights[COLL_WEIGHTS_MAX]; +static int curr_weight = 0; +static int32_t subst_weights[COLLATE_STR_LEN]; +static int curr_subst = 0; + +/* + * Some initial priority values. + */ +static int32_t pri_undefined[COLL_WEIGHTS_MAX]; +static int32_t pri_ignore; + +static collate_info_t collinfo; + +static collpri_t *prilist = NULL; +static int numpri = 0; +static int maxpri = 0; + +static void start_order(int); + +static int32_t +new_pri(void) +{ + int i; + + if (numpri >= maxpri) { + maxpri = maxpri ? maxpri * 2 : 1024; + prilist = realloc(prilist, sizeof (collpri_t) * maxpri); + if (prilist == NULL) { + fprintf(stderr,"out of memory"); + return (-1); + } + for (i = numpri; i < maxpri; i++) { + prilist[i].res = UNKNOWN; + prilist[i].pri = 0; + prilist[i].pass = 0; + } + } + return (numpri++); +} + +static collpri_t * +get_pri(int32_t ref) +{ + if ((ref < 0) || (ref > numpri)) { + INTERR; + return (NULL); + } + return (&prilist[ref]); +} + +static void +set_pri(int32_t ref, int32_t v, res_t res) +{ + collpri_t *pri; + + pri = get_pri(ref); + + if ((res == REFER) && ((v < 0) || (v >= numpri))) { + INTERR; + } + + /* Resolve self references */ + if ((res == REFER) && (ref == v)) { + v = nextpri; + res = RESOLVED; + } + + if (pri->res != UNKNOWN) { + warn("repeated item in order list (first on %d)", + pri->lineno); + return; + } + pri->lineno = lineno; + pri->pri = v; + pri->res = res; +} + +static int32_t +resolve_pri(int32_t ref) +{ + collpri_t *pri; + static int32_t pass = 0; + + pri = get_pri(ref); + pass++; + while (pri->res == REFER) { + if (pri->pass == pass) { + /* report a line with the circular symbol */ + lineno = pri->lineno; + fprintf(stderr,"circular reference in order list"); + return (-1); + } + if ((pri->pri < 0) || (pri->pri >= numpri)) { + INTERR; + return (-1); + } + pri->pass = pass; + pri = &prilist[pri->pri]; + } + + if (pri->res == UNKNOWN) { + return (-1); + } + if (pri->res != RESOLVED) + INTERR; + + return (pri->pri); +} + +static int +weight_compare(const void *n1, const void *n2) +{ + int32_t k1 = ((const weight_t *)n1)->pri; + int32_t k2 = ((const weight_t *)n2)->pri; + + return (k1 < k2 ? -1 : k1 > k2 ? 1 : 0); +} + +static int +collsym_compare(const void *n1, const void *n2) +{ + const collsym_t *c1 = n1; + const collsym_t *c2 = n2; + int rv; + + rv = strcmp(c1->name, c2->name); + return ((rv < 0) ? -1 : (rv > 0) ? 1 : 0); +} + +static int +collundef_compare(const void *n1, const void *n2) +{ + const collundef_t *c1 = n1; + const collundef_t *c2 = n2; + int rv; + + rv = strcmp(c1->name, c2->name); + return ((rv < 0) ? -1 : (rv > 0) ? 1 : 0); +} + +static int +element_compare_symbol(const void *n1, const void *n2) +{ + const collelem_t *c1 = n1; + const collelem_t *c2 = n2; + int rv; + + rv = strcmp(c1->symbol, c2->symbol); + return ((rv < 0) ? -1 : (rv > 0) ? 1 : 0); +} + +static int +element_compare_expand(const void *n1, const void *n2) +{ + const collelem_t *c1 = n1; + const collelem_t *c2 = n2; + int rv; + + rv = wcscmp(c1->expand, c2->expand); + return ((rv < 0) ? -1 : (rv > 0) ? 1 : 0); +} + +static int +collchar_compare(const void *n1, const void *n2) +{ + wchar_t k1 = ((const collchar_t *)n1)->wc; + wchar_t k2 = ((const collchar_t *)n2)->wc; + + return (k1 < k2 ? -1 : k1 > k2 ? 1 : 0); +} + +static int +subst_compare(const void *n1, const void *n2) +{ + int32_t k1 = ((const subst_t *)n1)->key; + int32_t k2 = ((const subst_t *)n2)->key; + + return (k1 < k2 ? -1 : k1 > k2 ? 1 : 0); +} + +#pragma GCC diagnostic push +#pragma GCC diagnostic ignored "-Wcast-qual" + +static int +subst_compare_ref(const void *n1, const void *n2) +{ + int32_t *c1 = ((subst_t *)n1)->ref; + int32_t *c2 = ((subst_t *)n2)->ref; + int rv; + + rv = wcscmp((wchar_t *)c1, (wchar_t *)c2); + return ((rv < 0) ? -1 : (rv > 0) ? 1 : 0); +} + +#pragma GCC diagnostic pop + +void +init_collate(void) +{ + int i; + + avl_create(&collsyms, collsym_compare, sizeof (collsym_t), + offsetof(collsym_t, avl)); + + avl_create(&collundefs, collundef_compare, sizeof (collsym_t), + offsetof(collundef_t, avl)); + + avl_create(&elem_by_symbol, element_compare_symbol, sizeof (collelem_t), + offsetof(collelem_t, avl_bysymbol)); + avl_create(&elem_by_expand, element_compare_expand, sizeof (collelem_t), + offsetof(collelem_t, avl_byexpand)); + + avl_create(&collchars, collchar_compare, sizeof (collchar_t), + offsetof(collchar_t, avl)); + + for (i = 0; i < COLL_WEIGHTS_MAX; i++) { + avl_create(&substs[i], subst_compare, sizeof (subst_t), + offsetof(subst_t, avl)); + avl_create(&substs_ref[i], subst_compare_ref, + sizeof (subst_t), offsetof(subst_t, avl_ref)); + avl_create(&weights[i], weight_compare, sizeof (weight_t), + offsetof(weight_t, avl)); + nweight[i] = 1; + } + + (void) memset(&collinfo, 0, sizeof (collinfo)); + + /* allocate some initial priorities */ + pri_ignore = new_pri(); + + set_pri(pri_ignore, 0, RESOLVED); + + for (i = 0; i < COLL_WEIGHTS_MAX; i++) { + pri_undefined[i] = new_pri(); + + /* we will override this later */ + set_pri(pri_undefined[i], COLLATE_MAX_PRIORITY, UNKNOWN); + } +} + +void +define_collsym(char *name) +{ + collsym_t *sym; + avl_index_t where; + + if ((sym = calloc(sizeof (*sym), 1)) == NULL) { + fprintf(stderr,"out of memory"); + return; + } + sym->name = name; + sym->ref = new_pri(); + + if (avl_find(&collsyms, sym, &where) != NULL) { + /* + * This should never happen because we are only called + * for undefined symbols. + */ + INTERR; + return; + } + avl_insert(&collsyms, sym, where); +} + +collsym_t * +lookup_collsym(char *name) +{ + collsym_t srch; + + srch.name = name; + return (avl_find(&collsyms, &srch, NULL)); +} + +collelem_t * +lookup_collelem(char *symbol) +{ + collelem_t srch; + + srch.symbol = symbol; + return (avl_find(&elem_by_symbol, &srch, NULL)); +} + +static collundef_t * +get_collundef(char *name) +{ + collundef_t srch; + collundef_t *ud; + avl_index_t where; + int i; + + srch.name = name; + if ((ud = avl_find(&collundefs, &srch, &where)) == NULL) { + if (((ud = calloc(sizeof (*ud), 1)) == NULL) || + ((ud->name = strdup(name)) == NULL)) { + fprintf(stderr,"out of memory"); + return (NULL); + } + for (i = 0; i < NUM_WT; i++) { + ud->ref[i] = new_pri(); + } + avl_insert(&collundefs, ud, where); + } + add_charmap_undefined(name); + return (ud); +} + +static collchar_t * +get_collchar(wchar_t wc, int create) +{ + collchar_t srch; + collchar_t *cc; + avl_index_t where; + int i; + + srch.wc = wc; + cc = avl_find(&collchars, &srch, &where); + if ((cc == NULL) && create) { + if ((cc = calloc(sizeof (*cc), 1)) == NULL) { + fprintf(stderr, "out of memory"); + return (NULL); + } + for (i = 0; i < NUM_WT; i++) { + cc->ref[i] = new_pri(); + } + cc->wc = wc; + avl_insert(&collchars, cc, where); + } + return (cc); +} + *** DIFF OUTPUT TRUNCATED AT 1000 LINES ***
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201508072353.t77NrW0L014715>