From owner-freebsd-ports@FreeBSD.ORG Fri Dec 6 10:17:43 2013 Return-Path: Delivered-To: freebsd-ports@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1EE56AA5 for ; Fri, 6 Dec 2013 10:17:43 +0000 (UTC) Received: from mail.grem.de (outcast.grem.de [213.239.217.27]) by mx1.freebsd.org (Postfix) with SMTP id 7BBEB1014 for ; Fri, 6 Dec 2013 10:17:41 +0000 (UTC) Received: (qmail 31434 invoked by uid 89); 6 Dec 2013 10:17:39 -0000 Received: from unknown (HELO bsd64.grem.de) (mg@grem.de@194.97.158.66) by mail.grem.de with ESMTPA; 6 Dec 2013 10:17:39 -0000 Date: Fri, 6 Dec 2013 11:17:39 +0100 From: Michael Gmelin To: Hiroki Sato Subject: Re: Differences between iconv from ports and iconv in base (transliteration) Message-ID: <20131206111739.3d08e33a@bsd64.grem.de> In-Reply-To: <20131206.113657.559308112148365646.hrs@allbsd.org> References: <20131206001554.0d9d3e23@bsd64.grem.de> <20131206.113657.559308112148365646.hrs@allbsd.org> X-Mailer: Claws Mail 3.9.2 (GTK+ 2.24.18; amd64-portbld-freebsd9.1) Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-ports@FreeBSD.org X-BeenThere: freebsd-ports@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Porting software to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Dec 2013 10:17:43 -0000 On Fri, 06 Dec 2013 11:36:57 +0900 (JST) Hiroki Sato wrote: > Michael Gmelin wrote > in <20131206001554.0d9d3e23@bsd64.grem.de>: >=20 > fr> I'm in the process of changing ports from ports iconv to iconv in > fr> base. I noticed that transliteration doesn't work in base as it > fr> does with iconv from ports. Examples: > fr> > fr> "T\xc5\xbdst" > fr> ports: "TZst" > fr> base: "Tst" > fr> > fr> "T\xe2\x82\xacst" > fr> ports: "TEURst" > fr> base: "Tst" > fr> > fr> Conversion done using: > fr> iconv_open("ISO8859-1//IGNORE//TRANSLIT", "UTF-8"); > fr> > fr> Any ideas? >=20 > //TRANSLIT is a GNU iconv specific extension and iconv in base (and > other POSIX systems) does not support it. Does your software depend > on it? >=20 > -- Hiroki The porters handbook implies that USES=3D iconv and the macros it provides (ICONV_LIB etc.) are a plugin replacement for GNU iconv. This is obviously not the case, so I think two things are required: 1. The documentation should point out this fact (something like "A port might depend on GNU iconv specific features, one way to figure out if your port might be affected is egrep -Rl "//(TRANSLIT|IGNORE)" work ) 2. Provide some way to specify that GNU iconv is required by the port. (assuming that it is possible to use both at the same time on one system) Otherwise people will get bitten by this change, especially if such features are used on top of the stack (e.g. PHP5's iconv function is officially documented to support "//TRANSLIT", but completely depends on the underlying iconv library to provide it). Imagine you're updating your application server to FreeBSD 10 and all the sudden your third party web application written in PHP will break in ways that are really hard to track down (I just verified this, I chose PHP since it's quite popular and has a user base that's probably not aware of the underlying libraries). An example I found in production code is converting a person's name to ASCII to transmit it to an old credit card system, transliterating central European characters (e.g. =C3=A4 =3D> ae, =C3=A1 = =3D> a etc.). All of this is independent of the question if GNU iconv is POSIX compliant or not and if it should behave like this or not. For practical reasons the ports system must provide a way to select GNU iconv for software that had been written with GNU iconv in mind (or mark it as BROKEN instead). Well, IMHO at least. Cheers, Michael --=20 Michael Gmelin