From owner-freebsd-hackers@FreeBSD.ORG Sun Jun 22 13:53:55 2008 Return-Path: Delivered-To: hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3F9851065670; Sun, 22 Jun 2008 13:53:55 +0000 (UTC) (envelope-from ache@nagual.pp.ru) Received: from nagual.pp.ru (nagual.pp.ru [194.87.13.69]) by mx1.freebsd.org (Postfix) with ESMTP id 8EFA18FC1B; Sun, 22 Jun 2008 13:53:54 +0000 (UTC) (envelope-from ache@nagual.pp.ru) Received: from nagual.pp.ru (ache@localhost [127.0.0.1]) by nagual.pp.ru (8.14.2/8.14.2) with ESMTP id m5MDrhUv073074; Sun, 22 Jun 2008 17:53:43 +0400 (MSD) (envelope-from ache@nagual.pp.ru) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=nagual.pp.ru; s=default; t=1214142823; bh=5S72Ee7gNbUc0lWxQUdiE8E9bTcUASlDNLB/d6h fcx0=; l=982; h=Date:From:To:Cc:Subject:Message-ID:References: MIME-Version:Content-Type:In-Reply-To; b=RN5gAGnDGZHHtxgph+dsh8qpV fOdgXn35wqHmEfl1Ge85ygkF+nkQl0ScO+/ly+MPInOS4qK/GHqfAjAomw9j+dkRHwz PGq3hLn7wh65L0uDm05bbuMJJTbgMd/P4/FwWquYuUsM2RMJ4yhPunmYhgh742OuZsf DZaV9gsbIQUk= Received: (from ache@localhost) by nagual.pp.ru (8.14.2/8.14.2/Submit) id m5MDrhfU073073; Sun, 22 Jun 2008 17:53:43 +0400 (MSD) (envelope-from ache) Date: Sun, 22 Jun 2008 17:53:43 +0400 From: Andrey Chernov To: Gabor Kovesdan Message-ID: <20080622135343.GA72068@nagual.pp.ru> Mail-Followup-To: Andrey Chernov , Gabor Kovesdan , Dag-Erling Sm??rgrav , Konrad Jankowski , Diomidis Spinellis , Doug Barton , hackers@FreeBSD.org, current@FreeBSD.org, "Sean C. Farley" , Max Khon References: <20080617102900.GA46479@nagual.pp.ru> <485798C4.2050605@FreeBSD.org> <20080618055851.GA85018@nagual.pp.ru> <86zlpjduew.fsf@ds4.des.no> <20080618083739.GA87100@nagual.pp.ru> <867icndqv5.fsf@ds4.des.no> <4858DBF6.5070001@bluemedia.pl> <86skvbc9gn.fsf@ds4.des.no> <20080618114917.GB89383@nagual.pp.ru> <485E4C69.1080805@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <485E4C69.1080805@FreeBSD.org> User-Agent: Mutt/1.5.18 (2008-05-17) X-Mailman-Approved-At: Sun, 22 Jun 2008 14:04:55 +0000 Cc: Doug Barton , current@FreeBSD.org, Konrad Jankowski , Diomidis Spinellis , hackers@FreeBSD.org, Dag-Erling Sm??rgrav , "Sean C. Farley" , Max Khon Subject: Re: CFT: BSD-licensed grep [Fwd: cvs commit: ports/textproc/bsdgrep Makefile distinfo] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Jun 2008 13:53:55 -0000 On Sun, Jun 22, 2008 at 02:58:17PM +0200, Gabor Kovesdan wrote: > Andrey Chernov escribi?: > > On Wed, Jun 18, 2008 at 12:40:24PM +0200, Dag-Erling Sm??rgrav wrote: > > > >> For grep, I believe it should simply be a matter of calling setlocale(), > >> using wide strings, and using a multibyte regex engine (for appropriate > >> values of "simply"). > >> > > > > See my prev reply telling more details. Using wide strings is not so easy, > > f.e. all ctype BSD grep now uses should be converted to wctype, input > > conversion added, etc. > > > I've started to work on doing this big change, the first step: > http://kovesdan.org/patches/grep-i18n.diff 1) You can't convert just whole buffer after fread() since it can be ended in the middle of multibyte sequence on BUFSIZ edge. Look how GNU utils do it. 2) Better use iswspace and iswcntrl instead of iswctype. 3) util.c needs to be fixed in several places too. -- http://ache.pp.ru/