From owner-freebsd-hackers@freebsd.org Sat Dec 5 21:13:26 2020 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 62E874A82C9 for ; Sat, 5 Dec 2020 21:13:26 +0000 (UTC) (envelope-from yuripv@yuripv.dev) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4CpMkG1Gb3z4hqn for ; Sat, 5 Dec 2020 21:13:26 +0000 (UTC) (envelope-from yuripv@yuripv.dev) Received: by mailman.nyi.freebsd.org (Postfix) id 2B8494A842D; Sat, 5 Dec 2020 21:13:26 +0000 (UTC) Delivered-To: hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 2B44C4A842C for ; Sat, 5 Dec 2020 21:13:26 +0000 (UTC) (envelope-from yuripv@yuripv.dev) Received: from wnew1-smtp.messagingengine.com (wnew1-smtp.messagingengine.com [64.147.123.26]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4CpMkF70v0z4hbX; Sat, 5 Dec 2020 21:13:25 +0000 (UTC) (envelope-from yuripv@yuripv.dev) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailnew.west.internal (Postfix) with ESMTP id 7982A66D; Sat, 5 Dec 2020 16:13:23 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Sat, 05 Dec 2020 16:13:23 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yuripv.dev; h= subject:to:cc:references:from:message-id:date:mime-version :in-reply-to:content-type:content-transfer-encoding; s=fm1; bh=S 5IDdZ8ouUhQ1E+c0FRsXa7ixuwWJjOW3CASqsv6FTw=; b=amU2VWAvo2sUorcyD 7fFsdy3hiT3XBjc5MfkWI9rjNQgZA47pKc6X0ZrS5C9nLzCa8GJ+KKgNWQxENIzq /HZTM8ANVvuRt5HqplGO038Sotd8/vPdkcY/Fc2O8reys5ONvkQDLaowFbyRXHGC cIRQ5U057Qg8aBuLKz/Qtn4x0sxaSgXuPhVcOwurc4SM+AoZWnyIjzdlhJfvW3WU 8seX3XsV5wEq1XECx9W2BuGbxomdsvLahLvOlagyoa0x1LgiEoZEtYw+DvENO1sF v23n0lafrkp/JkBKStOyeyyJN46ZJ4tambk+nE7HUhwvsOatvrBP/hz9wIA8SOck Q4xIw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; bh=S5IDdZ8ouUhQ1E+c0FRsXa7ixuwWJjOW3CASqsv6F Tw=; b=LHl5qPlVKkp+R3+I5YiBLgKSNO6riAn87oQzdTPF/lEC2bIjgVQKNum42 +PyE6MOmmGWFeMR+0N4jImCX1bf50keUMWA62NRTwaOvd4KUpl1UA+f22gyTCplH x5vL+vwcGrcwyh7+2E361YSIIedbxahF7lZUjvQ8nkZJabBJPKQh2BbBNCTFVJnq WE8IrOZEfZh+/nzez2eBcrw+Mk+yDV7qnCGZg5H781sDO03quJwxLOEWGN2Mqni7 RxHnPpFQ1jPBQQD9liUqCnPreLOHjPlMJLbwrHJNG2kHZJpaErnfcoiiVeWsYI6m 7GsqbG191WH2mrYhNatBmkCjZUDdw== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedujedrudejtddgudehtdcutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpefuvfhfhffkffgfgggjtgfgsehtkeertddtfeejnecuhfhrohhmpegjuhhr ihcurfgrnhhkohhvuceohihurhhiphhvseihuhhrihhpvhdruggvvheqnecuggftrfgrth htvghrnhepudeuffegtdehffdtffefkefhgfelieeitefghfeugeelfeduffegtdeufeek gfdvnecukfhppeeluddrvdegtddruddvgedrudehjeenucevlhhushhtvghrufhiiigvpe dtnecurfgrrhgrmhepmhgrihhlfhhrohhmpeihuhhrihhpvheshihurhhiphhvrdguvghv X-ME-Proxy: Received: from [192.168.1.6] (unknown [91.240.124.157]) by mail.messagingengine.com (Postfix) with ESMTPA id 26E27240057; Sat, 5 Dec 2020 16:13:21 -0500 (EST) Subject: Re: locale-related review, wcwidth() data To: Baptiste Daroussin , Thomas Munro Cc: hackers@freebsd.org References: <559840f6-ee81-1303-2986-1eafb2104e1b@yuripv.dev> <20201204133111.uyu55cl7zgll4vk2@ivaldir.net> <5b8f1016-bcb3-42ff-9e70-c4c0240ea685@FreeBSD.org> From: Yuri Pankov Message-ID: <6b967ee3-5fc1-51cb-2cbf-8f5f4b3e0120@yuripv.dev> Date: Sun, 6 Dec 2020 00:13:20 +0300 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.5.1 MIME-Version: 1.0 In-Reply-To: <5b8f1016-bcb3-42ff-9e70-c4c0240ea685@FreeBSD.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4CpMkF70v0z4hbX X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; TAGGED_RCPT(0.00)[]; REPLY(-4.00)[] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 05 Dec 2020 21:13:26 -0000 Baptiste Daroussin wrote: > > 5 déc. 2020 02:25:29 Thomas Munro : > >> On Sat, Dec 5, 2020 at 2:31 AM Baptiste Daroussin wrote: >>> I do like what I see here, the only reason I haven't review is that I can't >>> test, since the last modification from hrs@ in the locale generation tools each >>> time I try to regenerate the locales it fails. >> >> During install?  I noticed that too but wasn't sure of the correct fix, perhaps: >> >> diff --git a/tools/tools/locale/Makefile b/tools/tools/locale/Makefile >> index 76fff6acb17..b6ae2feadac 100644 >> --- a/tools/tools/locale/Makefile >> +++ b/tools/tools/locale/Makefile >> @@ -95,7 +95,7 @@ install: install-${t} >> install-${t}: >>         cd ${LOCALESRCDIR}/${t} && \ >>             rm -f Makefile *.src && \ >> -           install -c ${t}/* ${LOCALESRCDIR}/${t} >> +           install -c ${.OBJDIR}/${t}/* ${LOCALESRCDIR}/${t} >> .  endif >> .endfor > > > Nope that one was easy to figure out, but once the locales are regenerated, localdef dies on plenty of them (non unicode mostly). Now that you mentioned it, there's something I was thinking about for a long time now -- it's probably a sign that we need to mark all single-byte locales as set in stone, and stop regenerating the source for them as well as adding more and more workarounds against utf-8 charmap. This is something I'm going to look into eventually as I believe it will make updating to new CLDR releases much easier, that is, if I'm not missing something here. > It does with the current setup as well as with an update of both cldr and un unicode