From owner-freebsd-hackers@FreeBSD.ORG Wed Jun 18 11:44:59 2008 Return-Path: Delivered-To: hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 421FF1065670; Wed, 18 Jun 2008 11:44:59 +0000 (UTC) (envelope-from ache@nagual.pp.ru) Received: from nagual.pp.ru (nagual.pp.ru [194.87.13.69]) by mx1.freebsd.org (Postfix) with ESMTP id 861918FC0C; Wed, 18 Jun 2008 11:44:58 +0000 (UTC) (envelope-from ache@nagual.pp.ru) Received: from nagual.pp.ru (ache@localhost [127.0.0.1]) by nagual.pp.ru (8.14.2/8.14.2) with ESMTP id m5IBiUti089601; Wed, 18 Jun 2008 15:44:30 +0400 (MSD) (envelope-from ache@nagual.pp.ru) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=nagual.pp.ru; s=default; t=1213789470; bh=g7eodq+WsWlW6ADC0l/55w5ojXG/8e06wJbXYp2 u0Ns=; l=1008; h=Date:From:To:Cc:Subject:Message-ID:References: MIME-Version:Content-Type:In-Reply-To; b=YAs5brQrTzea5TVfKV84+llBn IHzZgKyuhnlLa05KkosL5pFa0XyYkQUBXv+RjTP+V79ChL0Zw48Cmo6imxPkmsq7Flq lDiMMzJATAdonNMHfpsCQ8xP4TruTQ4gnckDBY4qGN60gcTUwJHgsiMFrzUTSBzfJTf 9zqTi7xWh7jo= Received: (from ache@localhost) by nagual.pp.ru (8.14.2/8.14.2/Submit) id m5IBiT4w089600; Wed, 18 Jun 2008 15:44:29 +0400 (MSD) (envelope-from ache) Date: Wed, 18 Jun 2008 15:44:28 +0400 From: Andrey Chernov To: Dag-Erling Sm??rgrav Message-ID: <20080618114428.GA89383@nagual.pp.ru> Mail-Followup-To: Andrey Chernov , Dag-Erling Sm??rgrav , Gabor Kovesdan , Konrad Jankowski , Diomidis Spinellis , Doug Barton , K?vesd?n G?bor , hackers@FreeBSD.org, current@FreeBSD.org, "Sean C. Farley" , Max Khon References: <20080617004647.GA16546@nagual.pp.ru> <48576610.9080808@FreeBSD.org> <48577510.4020007@aueb.gr> <48577BD2.4070205@bluemedia.pl> <20080617102900.GA46479@nagual.pp.ru> <485798C4.2050605@FreeBSD.org> <20080618055851.GA85018@nagual.pp.ru> <86zlpjduew.fsf@ds4.des.no> <20080618083739.GA87100@nagual.pp.ru> <867icndqv5.fsf@ds4.des.no> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <867icndqv5.fsf@ds4.des.no> User-Agent: Mutt/1.5.18 (2008-05-17) X-Mailman-Approved-At: Wed, 18 Jun 2008 12:15:07 +0000 Cc: Doug Barton , current@FreeBSD.org, Konrad Jankowski , Diomidis Spinellis , hackers@FreeBSD.org, Gabor Kovesdan , Max Khon , "Sean C. Farley" , K?vesd?n G?bor Subject: Re: CFT: BSD-licensed grep [Fwd: cvs commit: ports/textproc/bsdgrep Makefile distinfo] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jun 2008 11:44:59 -0000 On Wed, Jun 18, 2008 at 11:39:10AM +0200, Dag-Erling Sm??rgrav wrote: > Does that mean our wcsxfrm() doesn't work? IIUC, it should convert > wide strings to strings that can be compared directly with strcmp()? (directly with wcscmp()) For single byte locales wcsxfrm() and wcscoll() works, but for multibyte they do just raw binary. > In any case, this is a libc issue, right? As long as sort / grep uses > the API correctly, they will work fine once libc is fixed? GNU grep and sort will work just fine. BSD grep not calls setlocale() but even it will be added, BSD grep have other places where multibyte is not handled proberly. I already notice two of them: ignore case comparison and word boundary sensing, perhaps other places exists, I not study the code enough to cach them all. BSD sort uses upper half of 256 char table on its own purposes so badly damage both single byte and multibyte locales and of couse not use wcscoll() at all etc. -- http://ache.pp.ru/