From owner-freebsd-bugs@FreeBSD.ORG Thu Jul 13 15:00:35 2006 Return-Path: X-Original-To: freebsd-bugs@hub.freebsd.org Delivered-To: freebsd-bugs@hub.freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2839516A4F1 for ; Thu, 13 Jul 2006 15:00:35 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id BE16F43D53 for ; Thu, 13 Jul 2006 15:00:30 +0000 (GMT) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.13.4/8.13.4) with ESMTP id k6DF0UUb084839 for ; Thu, 13 Jul 2006 15:00:30 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.13.4/8.13.4/Submit) id k6DF0UAb084838; Thu, 13 Jul 2006 15:00:30 GMT (envelope-from gnats) Resent-Date: Thu, 13 Jul 2006 15:00:30 GMT Resent-Message-Id: <200607131500.k6DF0UAb084838@freefall.freebsd.org> Resent-From: FreeBSD-gnats-submit@FreeBSD.org (GNATS Filer) Resent-To: freebsd-bugs@FreeBSD.org Resent-Reply-To: FreeBSD-gnats-submit@FreeBSD.org, "J.R. Oldroyd" Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5E26316A4E6 for ; Thu, 13 Jul 2006 14:51:13 +0000 (UTC) (envelope-from jr@opal.com) Received: from smtp.vzavenue.net (smtp.vzavenue.net [66.171.59.140]) by mx1.FreeBSD.org (Postfix) with ESMTP id C9C0543D70 for ; Thu, 13 Jul 2006 14:51:06 +0000 (GMT) (envelope-from jr@opal.com) Received: from 118.79.171.66.subscriber.vzavenue.net (HELO linwhf.opal.com) ([66.171.79.118]) by smtp.vzavenue.net with ESMTP; 13 Jul 2006 10:51:05 -0400 Received: from linwhf.opal.com (localhost [127.0.0.1]) by linwhf.opal.com (8.13.6/8.13.6) with ESMTP id k6DEp4OH093702 for ; Thu, 13 Jul 2006 10:51:04 -0400 (EDT) (envelope-from jr@opal.com) Received: from 127.0.0.1 ([127.0.0.1] helo=linwhf.opal.com) by ASSP-nospam; 13 Jul 2006 10:51:04 -0400 Received: (from jr@localhost) by linwhf.opal.com (8.13.6/8.13.6/Submit) id k6DEp4Gq093701; Thu, 13 Jul 2006 10:51:04 -0400 (EDT) (envelope-from jr) Message-Id: <200607131451.k6DEp4Gq093701@linwhf.opal.com> Date: Thu, 13 Jul 2006 10:51:04 -0400 (EDT) From: "J.R. Oldroyd" To: FreeBSD-gnats-submit@FreeBSD.org X-Send-Pr-Version: 3.113 Cc: Subject: misc/100212: UTF-8 zero-width character patch X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: "J.R. Oldroyd" List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Jul 2006 15:00:35 -0000 >Number: 100212 >Category: misc >Synopsis: UTF-8 zero-width character patch >Confidential: no >Severity: serious >Priority: medium >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: change-request >Submitter-Id: current-users >Arrival-Date: Thu Jul 13 15:00:30 GMT 2006 >Closed-Date: >Last-Modified: >Originator: J.R. Oldroyd >Release: FreeBSD 6.1-STABLE i386 >Organization: >Environment: System: FreeBSD linwhf.opal.com 6.1-STABLE FreeBSD 6.1-STABLE #1: Thu May 18 16:03:24 EDT 2006 xxx@linwhf.opal.com:/usr/obj/usr/src/sys/LINWHF i386 >Description: This patch makes the so-called zero-width, non-spacing, or overstriking characters of the UTF-8 encoding exactly that. At the present time, these characters are coded with a width of 1 which is wrong. They should have a width of 0. >How-To-Repeat: Save this file: http://opal.com/freebsd/unicode/utf8demo.txt On an xterm, cat the file and examine the "Combining characters" and the "Thai (UCS Level 2)" sections. Without the patch, the non-spacing characters do not overstrike the previous character. With the patch, they do. This patch has been posted to -current and downloaded and reviewed many times following that posting: http://lists.freebsd.org/pipermail/freebsd-current/2006-June/064218.html >Fix: --- /usr/src/share/mklocale/UTF-8.src.orig Sat Mar 27 03:14:14 2004 +++ /usr/src/share/mklocale/UTF-8.src Mon Jun 26 23:15:34 2006 @@ -487,9 +487,9 @@ * U+0300 - U+036F : Combining Diacritical Marks */ -GRAPH 0x0300 - 0x034f 0x0360 - 0x036f -PRINT 0x0300 - 0x034f 0x0360 - 0x036f -SWIDTH1 0x0300 - 0x034f 0x0360 - 0x036f +GRAPH 0x0300 - 0x036f +PRINT 0x0300 - 0x036f +SWIDTH0 0x0300 - 0x036f MAPUPPER < 0x0345 0x0399 > @@ -593,7 +593,8 @@ UPPER 0x04e2 0x04e4 0x04e6 0x04e8 0x04ea 0x04ec 0x04ee UPPER 0x04f0 0x04f2 0x04f4 0x04f8 PRINT 0x0400 - 0x0486 0x0488 - 0x04ce 0x04d0 - 0x04f5 0x04f8 0x04f9 -SWIDTH1 0x0400 - 0x0486 0x0488 - 0x04ce 0x04d0 - 0x04f5 0x04f8 0x04f9 +SWIDTH1 0x0400 - 0x0482 0x048a - 0x04ce 0x04d0 - 0x04f5 0x04f8 0x04f9 +SWIDTH0 0x0483 - 0x0486 0x0488 - 0x0489 MAPUPPER < 0x0430 - 0x044f : 0x0410 > MAPUPPER < 0x0450 - 0x045f : 0x0400 > @@ -1016,7 +1017,8 @@ GRAPH 0x0e01 - 0x0e3a 0x0e3f - 0x0e5b PUNCT 0x0e3f 0x0e4f 0x0e5a 0x0e5b PRINT 0x0e01 - 0x0e3a 0x0e3f - 0x0e5b -SWIDTH1 0x0e01 - 0x0e3a 0x0e3f - 0x0e5b +SWIDTH0 0x0e31 0x0e34 - 0x0e3a 0x0e47 - 0x0e4e +SWIDTH1 0x0e01 - 0x0e30 0x0e32 - 0x0e33 0x0e3f - 0x0e46 0x0e4f - 0x0e5b /* @@ -1647,9 +1649,9 @@ * U+20D0 - U+20FF : Combining Diacritical Marks for Symbols */ -GRAPH 0x20d0 - 0x20ea -PRINT 0x20d0 - 0x20ea -SWIDTH1 0x20d0 - 0x20ea +GRAPH 0x20d0 - 0x20ff +PRINT 0x20d0 - 0x20ff +SWIDTH0 0x20d0 - 0x20ff /* @@ -1927,7 +1929,8 @@ PUNCT 0x309b 0x309c PRINT 0x3041 - 0x3096 0x3099 - 0x309f PHONOGRAM 0x3041 - 0x3096 0x309f -SWIDTH2 0x3041 - 0x3096 0x3099 - 0x309f +SWIDTH2 0x3041 - 0x3096 0x309b - 0x309f +SWIDTH0 0x3099 - 0x309a /* @@ -2149,9 +2152,9 @@ * U+FE20 - U+FE2F : Combining Half Marks */ -GRAPH 0xfe20 - 0xfe23 -PRINT 0xfe20 - 0xfe23 -SWIDTH1 0xfe20 - 0xfe23 +GRAPH 0xfe20 - 0xfe2f +PRINT 0xfe20 - 0xfe2f +SWIDTH0 0xfe20 - 0xfe2f /* @@ -2272,7 +2275,8 @@ PUNCT 0x1d100 - 0x1d126 0x1d12a - 0x1d164 0x1d16a - 0x1d16c PUNCT 0x1d183 0x1d184 0x1d18c - 0x1d1a9 0x1d1ae - 0x1d1dd PRINT 0x1d100 - 0x1d126 0x1d12a - 0x1d172 0x1d17b - 0x1d1dd -SWIDTH1 0x1d100 - 0x1d126 0x1d12a - 0x1d172 0x1d17b - 0x1d1dd +SWIDTH1 0x1d100 - 0x1d126 0x1d12a - 0x1d164 0x1d16a - 0x1d172 0x1d183 0x1d184 0x1d18c - 0x1d1a9 0x1d1ae - 0x1d1dd +SWIDTH0 0x1d165 - 0x1d169 0x1d17b - 0x1d182 0x1d185 - 0x1d18b 0x1d1aa - 0x1d1ad /* >Release-Note: >Audit-Trail: >Unformatted: