Date: Sun, 16 Sep 2007 21:01:43 +0400 From: Andrey Chernov <ache@nagual.pp.ru> To: freebsd-bugs@FreeBSD.ORG, jkoshy@FreeBSD.ORG, perky@FreeBSD.ORG, i18n@FreeBSD.ORG, petr.hroudny@gmail.com Subject: Re: gnu/116363: isspace broken for UTF-8 locales Message-ID: <20070916170142.GA11047@nagual.pp.ru> In-Reply-To: <200709161640.l8GGe7iQ077745@freefall.freebsd.org> References: <200709161640.l8GGe7iQ077745@freefall.freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sun, Sep 16, 2007 at 04:40:07PM +0000, Andrey Chernov wrote:
> The following reply was made to PR gnu/116363; it has been noted by GNATS.
>
> From: Andrey Chernov <ache@nagual.pp.ru>
> To: Hye-Shik Chang <perky@FreeBSD.org>
> Cc: Petr Hroudny <petr.hroudny@gmail.com>, freebsd-gnats-submit@FreeBSD.org,
> jkoshy@FreeBSD.org, i18n@FreeBSD.org
> Subject: Re: gnu/116363: isspace broken for UTF-8 locales
> Date: Sun, 16 Sep 2007 20:34:07 +0400
>
> On Mon, Sep 17, 2007 at 01:22:14AM +0900, Hye-Shik Chang wrote:
> > In fact, UTF-8.src defines values for not UTF-8 but Unicode codepoints.
> > Using the Unicode codepoint as wchar_t's internal representation gives
> > much benefit. I think we would be better to make isspace() and
> > other ctypes functions aware of "encoding". IIRC, tjr@ provided the
> > workaround as in the URL mentioned above and said that it would get
> > a chance to be fixed in 6 or 7 on 2004.
>
> Currently wchar_t represents given encoding in all places including
> wc<->mbr conversions. To make it UCS-4-only instead we need to rewrite the
Oops, sorry for my overlook, we really have UCS-4 as wchar_t, no
UTF-8.src replacement is needed.
In that case iswspace(0xA0) should be 1 but not isspace(0xA0) so it seems
it is isspace() (and others plain ctype) bug. It seems even isspace(' ')
is illegal in UTF-8 locale because all chars are wide, but I am not sure.
--
http://ache.pp.ru/
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20070916170142.GA11047>