From owner-freebsd-bugs@FreeBSD.ORG  Sun Sep 16 09:10:06 2007
Return-Path: <owner-freebsd-bugs@FreeBSD.ORG>
Delivered-To: freebsd-bugs@hub.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 483C216A46B
	for <freebsd-bugs@hub.freebsd.org>;
	Sun, 16 Sep 2007 09:10:06 +0000 (UTC)
	(envelope-from gnats@FreeBSD.org)
Received: from freefall.freebsd.org (freefall.freebsd.org
	[IPv6:2001:4f8:fff6::28])
	by mx1.freebsd.org (Postfix) with ESMTP id 2842013C46A
	for <freebsd-bugs@hub.freebsd.org>;
	Sun, 16 Sep 2007 09:10:06 +0000 (UTC)
	(envelope-from gnats@FreeBSD.org)
Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1])
	by freefall.freebsd.org (8.14.1/8.14.1) with ESMTP id l8G9A6MY050907
	for <freebsd-bugs@freefall.freebsd.org>; Sun, 16 Sep 2007 09:10:06 GMT
	(envelope-from gnats@freefall.freebsd.org)
Received: (from gnats@localhost)
	by freefall.freebsd.org (8.14.1/8.14.1/Submit) id l8G9A6ts050905;
	Sun, 16 Sep 2007 09:10:06 GMT (envelope-from gnats)
Date: Sun, 16 Sep 2007 09:10:06 GMT
Message-Id: <200709160910.l8G9A6ts050905@freefall.freebsd.org>
To: freebsd-bugs@FreeBSD.org
From: Andrey Chernov <ache@nagual.pp.ru>
Cc: 
Subject: Re: gnu/116363: isspace broken for UTF-8 locales
X-BeenThere: freebsd-bugs@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
Reply-To: Andrey Chernov <ache@nagual.pp.ru>
List-Id: Bug reports <freebsd-bugs.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-bugs>,
	<mailto:freebsd-bugs-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-bugs>
List-Post: <mailto:freebsd-bugs@freebsd.org>
List-Help: <mailto:freebsd-bugs-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-bugs>,
	<mailto:freebsd-bugs-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sun, 16 Sep 2007 09:10:06 -0000

The following reply was made to PR gnu/116363; it has been noted by GNATS.

From: Andrey Chernov <ache@nagual.pp.ru>
To: Petr Hroudny <petr.hroudny@gmail.com>
Cc: freebsd-gnats-submit@FreeBSD.ORG, jkoshy@FreeBSD.ORG, perky@FreeBSD.ORG,
        i18n@FreeBSD.ORG
Subject: Re: gnu/116363: isspace broken for UTF-8 locales
Date: Sun, 16 Sep 2007 12:54:33 +0400

 On Sat, Sep 15, 2007 at 09:08:01AM +0000, Petr Hroudny wrote:
 > 
 > >Number:         116363
 > >Category:       gnu
 > >Synopsis:       isspace broken for UTF-8 locales
 > >Confidential:   no
 > >Severity:       non-critical
 > >Priority:       medium
 > >Responsible:    freebsd-bugs
 > >State:          open
 > >Quarter:        
 > >Keywords:       
 > >Date-Required:
 > >Class:          sw-bug
 > >Submitter-Id:   current-users
 > >Arrival-Date:   Sat Sep 15 09:10:02 GMT 2007
 > >Closed-Date:
 > >Last-Modified:
 > >Originator:     Petr Hroudny
 > >Release:        6-stable, 7-current
 > >Organization:
 > >Environment:
 > >Description:
 > In UTF-8 locales, isspace(0xA0) returns 1 which is wrong.
 > 
 > In UTF-8, 0xA0 could only be the second or third byte of multibyte character, but never a space.
 > 
 > As a consequence, operations like str.upper() and/or str.split() are broken, when
 > UTF-8 character with 0xA0 byte is encountered.
 
 It seems that our UTF-8.src is completely wrong, it is just plain Unicode 
 and not UTF-8 which multibyte values should start from
 C2-DF
 E0-EF
 F0-F4
 only (as stated in http://en.wikipedia.org/wiki/UTF-8 f.e.)
 Can anybody write replacement for it?
 
 -- 
 http://ache.pp.ru/