From owner-svn-src-stable-12@freebsd.org Wed Mar 27 14:47:12 2019 Return-Path: Delivered-To: svn-src-stable-12@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 894CF15520C5; Wed, 27 Mar 2019 14:47:12 +0000 (UTC) (envelope-from bapt@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2C8C177B9E; Wed, 27 Mar 2019 14:47:12 +0000 (UTC) (envelope-from bapt@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 055228ACE; Wed, 27 Mar 2019 14:47:12 +0000 (UTC) (envelope-from bapt@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x2RElBq3024127; Wed, 27 Mar 2019 14:47:11 GMT (envelope-from bapt@FreeBSD.org) Received: (from bapt@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x2RElBxk024122; Wed, 27 Mar 2019 14:47:11 GMT (envelope-from bapt@FreeBSD.org) Message-Id: <201903271447.x2RElBxk024122@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: bapt set sender to bapt@FreeBSD.org using -f From: Baptiste Daroussin Date: Wed, 27 Mar 2019 14:47:11 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-12@freebsd.org Subject: svn commit: r345577 - stable/12/usr.bin/calendar X-SVN-Group: stable-12 X-SVN-Commit-Author: bapt X-SVN-Commit-Paths: stable/12/usr.bin/calendar X-SVN-Commit-Revision: 345577 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 2C8C177B9E X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.96 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_SHORT(-0.96)[-0.965,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_MEDIUM(-1.00)[-0.996,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: svn-src-stable-12@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: SVN commit messages for only the 12-stable src tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Mar 2019 14:47:12 -0000 Author: bapt Date: Wed Mar 27 14:47:10 2019 New Revision: 345577 URL: https://svnweb.freebsd.org/changeset/base/345577 Log: MFC r344340: calendar: use iconv to respect the output encoding calendar(1) can have input in various encoding, specifying LANG= to enable calendar(1) to determine which one to use. The problem is the content of the calendar itself is exposed as is making it unreadable in many cases. For example french calendar which is encoded ISO8859-1 is rendered badly in a fr_FR.UTF-8 environment. Using iconv allows to solve this issue. This will also allow to keep only 1 encoding in base for those files without breaking user existing setup Reported by: many Differential Revision: https://reviews.freebsd.org/D19221 Modified: stable/12/usr.bin/calendar/Makefile stable/12/usr.bin/calendar/calendar.c stable/12/usr.bin/calendar/calendar.h stable/12/usr.bin/calendar/events.c stable/12/usr.bin/calendar/io.c Directory Properties: stable/12/ (props changed) Modified: stable/12/usr.bin/calendar/Makefile ============================================================================== --- stable/12/usr.bin/calendar/Makefile Wed Mar 27 14:35:38 2019 (r345576) +++ stable/12/usr.bin/calendar/Makefile Wed Mar 27 14:47:10 2019 (r345577) @@ -13,6 +13,10 @@ INTER= de_AT.ISO_8859-15 de_DE.ISO8859-1 fr_F DE_LINKS= de_DE.ISO8859-15 FR_LINKS= fr_FR.ISO8859-15 +.if ${MK_ICONV} == "yes" +CFLAGS+= -DWITH_ICONV +.endif + FILESGROUPS+= CALS CALS= calendars/calendar.all \ calendars/calendar.australia \ Modified: stable/12/usr.bin/calendar/calendar.c ============================================================================== --- stable/12/usr.bin/calendar/calendar.c Wed Mar 27 14:35:38 2019 (r345576) +++ stable/12/usr.bin/calendar/calendar.c Wed Mar 27 14:47:10 2019 (r345577) @@ -47,6 +47,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #include #include #include @@ -66,6 +67,9 @@ static char *DEBUG = NULL; static time_t f_time = 0; double UTCOffset = UTCOFFSET_NOTSET; int EastLongitude = LONGITUDE_NOTSET; +#ifdef WITH_ICONV +const char *outputEncoding; +#endif static void usage(void) __dead2; @@ -80,6 +84,12 @@ main(int argc, char *argv[]) struct tm tp1, tp2; (void)setlocale(LC_ALL, ""); +#ifdef WITH_ICONV + /* save the information about the encoding used in the terminal */ + outputEncoding = strdup(nl_langinfo(CODESET)); + if (outputEncoding == NULL) + errx(1, "cannot allocate memory"); +#endif while ((ch = getopt(argc, argv, "-A:aB:D:dF:f:l:t:U:W:?")) != -1) switch (ch) { Modified: stable/12/usr.bin/calendar/calendar.h ============================================================================== --- stable/12/usr.bin/calendar/calendar.h Wed Mar 27 14:35:38 2019 (r345576) +++ stable/12/usr.bin/calendar/calendar.h Wed Mar 27 14:47:10 2019 (r345577) @@ -59,6 +59,9 @@ extern struct fixs neaster, npaskha, ncny, nfullmoon, extern struct fixs nmarequinox, nsepequinox, njunsolstice, ndecsolstice; extern double UTCOffset; extern int EastLongitude; +#ifdef WITH_ICONV +extern const char *outputEncoding; +#endif #define isleap(y) ((((y) % 4) == 0 && ((y) % 100) != 0) || ((y) % 400) == 0) @@ -197,3 +200,7 @@ void fpom(int year, double utcoffset, double *ffms, do void equinoxsolstice(int year, double UTCoffset, int *equinoxdays, int *solsticedays); void fequinoxsolstice(int year, double UTCoffset, double *equinoxdays, double *solsticedays); int calculatesunlongitude30(int year, int degreeGMToffset, int *ichinesemonths); + +#ifdef WITH_ICONV +void set_new_encoding(void); +#endif Modified: stable/12/usr.bin/calendar/events.c ============================================================================== --- stable/12/usr.bin/calendar/events.c Wed Mar 27 14:35:38 2019 (r345576) +++ stable/12/usr.bin/calendar/events.c Wed Mar 27 14:47:10 2019 (r345577) @@ -35,10 +35,120 @@ __FBSDID("$FreeBSD$"); #include #include #include +#ifdef WITH_ICONV +#include +#include +#include +static iconv_t conv = (iconv_t)-1; +static char *currentEncoding = NULL; + +#endif + #include "pathnames.h" #include "calendar.h" +#ifdef WITH_ICONV +void +set_new_encoding(void) +{ + const char *newenc; + + newenc = nl_langinfo(CODESET); + if (currentEncoding == NULL) { + currentEncoding = strdup(newenc); + if (currentEncoding == NULL) + errx(1, "set_new_encoding: cannot allocate memory"); + return; + } + if (strcmp(currentEncoding, newenc) == 0) + return; + free(currentEncoding); + currentEncoding = strdup(newenc); + if (currentEncoding == NULL) + errx(1, "set_new_encoding: cannot allocate memory"); + if (conv != (iconv_t) -1) { + iconv_close(conv); + conv = (iconv_t) -1; + } +} +#endif + +static char * +convert(char *input) +{ + char *output; +#ifdef WITH_ICONV + size_t inleft, outleft, converted = 0; + char *outbuf, *tmp; + char *inbuf; + size_t outlen; + + if (currentEncoding == NULL) { + output = strdup(input); + if (output == NULL) + errx(1, "convert: cannot allocate memory"); + return (output); + } + if (conv == (iconv_t)-1) { + conv = iconv_open(outputEncoding, currentEncoding); + if (conv == (iconv_t)-1) { + if (errno == EINVAL) + errx(1, "Conversion is not supported"); + else + err(1, "Initialization failure"); + } + } + + inleft = strlen(input); + inbuf = input; + + outlen = inleft; + if ((output = malloc(outlen + 1)) == NULL) + errx(1, "convert: cannot allocate memory"); + + for (;;) { + errno = 0; + outbuf = output + converted; + outleft = outlen - converted; + + converted = iconv(conv, (char **) &inbuf, &inleft, &outbuf, &outleft); + if (converted != (size_t) -1 || errno == EINVAL) { + /* finished or invalid multibyte, so truncate and ignore */ + break; + } + + if (errno != E2BIG) { + free(output); + err(1, "convert"); + } + + converted = outbuf - output; + outlen += inleft * 2; + + if ((tmp = realloc(output, outlen + 1)) == NULL) { + free(output); + errx(1, "convert: cannot allocate memory"); + } + + output = tmp; + outbuf = output + converted; + } + + /* flush the iconv conversion */ + iconv(conv, NULL, NULL, &outbuf, &outleft); + + /* null terminate the string */ + *outbuf = '\0'; +#else + output = strdup(input); + if (output == NULL) + errx(1, "convert: cannot allocate memory"); +#endif + + return (output); +} + struct event * event_add(int year, int month, int day, char *date, int var, char *txt, char *extra) @@ -58,15 +168,15 @@ event_add(int year, int month, int day, char *date, in e->month = month; e->day = day; e->var = var; - e->date = strdup(date); + e->date = convert(date); if (e->date == NULL) errx(1, "event_add: cannot allocate memory"); - e->text = strdup(txt); + e->text = convert(txt); if (e->text == NULL) errx(1, "event_add: cannot allocate memory"); e->extra = NULL; if (extra != NULL && extra[0] != '\0') - e->extra = strdup(extra); + e->extra = convert(extra); addtodate(e, year, month, day); return (e); } @@ -74,23 +184,17 @@ event_add(int year, int month, int day, char *date, in void event_continue(struct event *e, char *txt) { - char *text; + char *oldtext, *text; - /* - * Adding text to the event: - * - Save a copy of the old text (unknown length, so strdup()) - * - Allocate enough space for old text + \n + new text + 0 - * - Store the old text + \n + new text - * - Destroy the saved copy. - */ - text = strdup(e->text); - if (text == NULL) + text = convert(txt); + oldtext = e->text; + if (oldtext == NULL) errx(1, "event_continue: cannot allocate memory"); - free(e->text); - asprintf(&e->text, "%s\n%s", text, txt); + asprintf(&e->text, "%s\n%s", oldtext, text); if (e->text == NULL) errx(1, "event_continue: cannot allocate memory"); + free(oldtext); free(text); return; Modified: stable/12/usr.bin/calendar/io.c ============================================================================== --- stable/12/usr.bin/calendar/io.c Wed Mar 27 14:35:38 2019 (r345576) +++ stable/12/usr.bin/calendar/io.c Wed Mar 27 14:47:10 2019 (r345577) @@ -294,6 +294,9 @@ cal_parse(FILE *in, FILE *out) if (strncmp(buf, "LANG=", 5) == 0) { (void)setlocale(LC_ALL, buf + 5); d_first = (*nl_langinfo(D_MD_ORDER) == 'd'); +#ifdef WITH_ICONV + set_new_encoding(); +#endif setnnames(); continue; }