Date: Sun, 16 Sep 2007 12:54:33 +0400 From: Andrey Chernov <ache@nagual.pp.ru> To: Petr Hroudny <petr.hroudny@gmail.com> Cc: jkoshy@FreeBSD.ORG, freebsd-gnats-submit@FreeBSD.ORG, perky@FreeBSD.ORG, i18n@FreeBSD.ORG Subject: Re: gnu/116363: isspace broken for UTF-8 locales Message-ID: <20070916085432.GA8884@nagual.pp.ru> In-Reply-To: <200709150908.l8F981jj075109@www.freebsd.org> References: <200709150908.l8F981jj075109@www.freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sat, Sep 15, 2007 at 09:08:01AM +0000, Petr Hroudny wrote: > > >Number: 116363 > >Category: gnu > >Synopsis: isspace broken for UTF-8 locales > >Confidential: no > >Severity: non-critical > >Priority: medium > >Responsible: freebsd-bugs > >State: open > >Quarter: > >Keywords: > >Date-Required: > >Class: sw-bug > >Submitter-Id: current-users > >Arrival-Date: Sat Sep 15 09:10:02 GMT 2007 > >Closed-Date: > >Last-Modified: > >Originator: Petr Hroudny > >Release: 6-stable, 7-current > >Organization: > >Environment: > >Description: > In UTF-8 locales, isspace(0xA0) returns 1 which is wrong. > > In UTF-8, 0xA0 could only be the second or third byte of multibyte character, but never a space. > > As a consequence, operations like str.upper() and/or str.split() are broken, when > UTF-8 character with 0xA0 byte is encountered. It seems that our UTF-8.src is completely wrong, it is just plain Unicode and not UTF-8 which multibyte values should start from C2-DF E0-EF F0-F4 only (as stated in http://en.wikipedia.org/wiki/UTF-8 f.e.) Can anybody write replacement for it? -- http://ache.pp.ru/
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20070916085432.GA8884>