Date: Sat, 13 Jun 2020 14:11:02 +0000 (UTC) From: Yuri Pankov <yuripv@FreeBSD.org> To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r362148 - head/contrib/nvi/common Message-ID: <202006131411.05DEB2mP097868@repo.freebsd.org>
next in thread | raw e-mail | index | archive | help
Author: yuripv Date: Sat Jun 13 14:11:02 2020 New Revision: 362148 URL: https://svnweb.freebsd.org/changeset/base/362148 Log: nvi: fallback to ISO8859-1 as last resort Current logic of using user's locale encoding that is UTF-8 doesn't make much sense if we already failed the looks_utf8() check and skipped encoding set using "fileencoding" as being UTF-8 as well; fallback to ISO8859-1 in that case. Reviewed by: Zhihao Yuan <lichray@gmail.com> Differential Revision: https://reviews.freebsd.org/D24919 Modified: head/contrib/nvi/common/exf.c Modified: head/contrib/nvi/common/exf.c ============================================================================== --- head/contrib/nvi/common/exf.c Sat Jun 13 09:16:07 2020 (r362147) +++ head/contrib/nvi/common/exf.c Sat Jun 13 14:11:02 2020 (r362148) @@ -1237,7 +1237,10 @@ file_encinit(SCR *sp) } /* - * Detect UTF-8 and fallback to the locale/preset encoding. + * 1. Check for valid UTF-8. + * 2. Check if fallback fileencoding is set and is NOT UTF-8. + * 3. Check if user locale's encoding is NOT UTF-8. + * 4. Use ISO8859-1 as last resort. * * XXX * A manually set O_FILEENCODING indicates the "fallback @@ -1246,9 +1249,13 @@ file_encinit(SCR *sp) */ if (looks_utf8(buf, blen) > 1) o_set(sp, O_FILEENCODING, OS_STRDUP, "utf-8", 0); - else if (!O_ISSET(sp, O_FILEENCODING) || - !strcasecmp(O_STR(sp, O_FILEENCODING), "utf-8")) + else if (O_ISSET(sp, O_FILEENCODING) && + strcasecmp(O_STR(sp, O_FILEENCODING), "utf-8") != 0) + /* Use fileencoding as is */ ; + else if (strcasecmp(codeset(), "utf-8") != 0) o_set(sp, O_FILEENCODING, OS_STRDUP, codeset(), 0); + else + o_set(sp, O_FILEENCODING, OS_STRDUP, "iso8859-1", 0); conv_enc(sp, O_FILEENCODING, 0); #endif
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?202006131411.05DEB2mP097868>