From owner-freebsd-i18n@FreeBSD.ORG Tue Jun 15 00:31:47 2010 Return-Path: Delivered-To: i18n@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3DD21106566B for ; Tue, 15 Jun 2010 00:31:47 +0000 (UTC) (envelope-from gabor@FreeBSD.org) Received: from server.mypc.hu (server.mypc.hu [87.229.73.95]) by mx1.freebsd.org (Postfix) with ESMTP id A44438FC08 for ; Tue, 15 Jun 2010 00:31:46 +0000 (UTC) Received: from server.mypc.hu (localhost [127.0.0.1]) by server.mypc.hu (Postfix) with ESMTP id 6469F14DBB31; Tue, 15 Jun 2010 02:13:46 +0200 (CEST) X-Virus-Scanned: amavisd-new at server.mypc.hu Received: from server.mypc.hu ([127.0.0.1]) by server.mypc.hu (server.mypc.hu [127.0.0.1]) (amavisd-new, port 10024) with LMTP id L2OMla-WlAF8; Tue, 15 Jun 2010 02:13:44 +0200 (CEST) Received: from [192.168.1.105] (catv-80-99-92-167.catv.broadband.hu [80.99.92.167]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by server.mypc.hu (Postfix) with ESMTPSA id 289A814DBB0E; Tue, 15 Jun 2010 02:13:44 +0200 (CEST) Message-ID: <4C16C5B5.1070308@FreeBSD.org> Date: Tue, 15 Jun 2010 02:13:41 +0200 From: Gabor Kovesdan User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.2; es-ES; rv:1.9.1.9) Gecko/20100317 Thunderbird/3.0.4 MIME-Version: 1.0 To: FreeBSD Current Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Xin LI , i18n@FreeBSD.org Subject: [CFT] BSDL iconv in base system X-BeenThere: freebsd-i18n@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: FreeBSD Internationalization Effort List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Jun 2010 00:31:47 -0000 Hello Folks, during the last summer, Google generously founded my Summer of Code project, which was providing a BSD-licensed iconv implementation for FreeBSD. I'm proud to announce that the work has been completed and a patch is available to add it to the base system. The results of this work are: - The Citrus implementation has been ported from NetBSD. - Some utilities have been added. There is a conversion table generator, which can compare conversion tables to reference data generated by GNU libiconv. This helps ensuring conversion compatibility. - UTF-16 surrogate support and some endianness issues have been fixed. - The rather chaotic Makefiles to build metadata have been refactored and cleaned up, now it is easy to read and it is also easier to add support for new encodings. - A bunch of new encodings and encoding aliases have been added. - Support for 1->2, 1->3 and 1->4 mappings, which is needed for transliterating with flying accents as GNU does, like "u. - Lots of warnings have been fixed, the major part of the code is now WARNS=6 clean. - New section 1 and section 5 manual pages have been added. - Some GNU-specific calls have been implemented: iconvlist(), iconvctl(), iconv_canonicalize(), iconv_open_into() - Support for GNU's //IGNORE suffix has been added. - The "-" argument for stdin is now recognized in iconv(1) as per POSIX. - The Big5 conversion module has been fixed. - The iconv.h header files is supposed to be compatible with the GNU version, i.e. sources should build with base iconv.h and GNU libiconv. I've just did a very quick test and it seems ports can safely link to GNU libiconv, there's no conflict. - Various cleanups and style(9) fixes. - A bachelor thesis written in Hungarian language: http://www.kovesdan.org/files/bsc_iconv.pdf The rather big patch (42,5M) is available here: http://www.kovesdan.org/patches/iconv_base_integrate.diff Any comments, suggestions or bugreports are very welcome. -- Gabor Kovesdan FreeBSD Volunteer EMAIL:gabor@FreeBSD.org .:|:.gabor@kovesdan.org WEB:http://people.FreeBSD.org/~gabor .:|:.http://kovesdan.org From owner-freebsd-i18n@FreeBSD.ORG Tue Jun 15 00:48:51 2010 Return-Path: Delivered-To: i18n@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A37B1106567A; Tue, 15 Jun 2010 00:48:51 +0000 (UTC) (envelope-from jamesbrandongooch@gmail.com) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 5CFC28FC17; Tue, 15 Jun 2010 00:48:51 +0000 (UTC) Received: by iwn7 with SMTP id 7so5756489iwn.13 for ; Mon, 14 Jun 2010 17:48:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type; bh=L+naH46Ea+hQdBw9taDGmU/Sxcv/VbtMj4u/rkLjSi8=; b=YDx397BPe6lwZDpqJWcybhIwwTJO/rVfj5p0p1w52tPkEuIwcu7F9JfBSRDH08i6Nu +2+h90pFotAhBjbMupQzyRT2XmAVyIW38hPgRrGM6X6XHl9fkodhD5eyo+oQEgzW8xwK I5CKMmY0gVVHvjTdKZh/ydR4otmdkyBSlkEJI= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=LLOy2Dld04oUVs1F0RaVWtSR9JUJSJdqiZu8Vt7uLD6mhv2CZHSjOtqWk8Q9MHm3H7 OhWKBDtOV04norRyJFxCta+6BArIxudZeeI8xjwAgRdeG7aOFT62Qf4eIMRBGlcsPH+w LcK6h9Qu4Qnlyfj7PXilL3mErKPfw04XR5ZQU= MIME-Version: 1.0 Received: by 10.231.69.71 with SMTP id y7mr6278984ibi.136.1276561508541; Mon, 14 Jun 2010 17:25:08 -0700 (PDT) Received: by 10.231.182.196 with HTTP; Mon, 14 Jun 2010 17:25:08 -0700 (PDT) In-Reply-To: <4C16C5B5.1070308@FreeBSD.org> References: <4C16C5B5.1070308@FreeBSD.org> Date: Mon, 14 Jun 2010 19:25:08 -0500 Message-ID: From: Brandon Gooch To: Gabor Kovesdan Content-Type: text/plain; charset=ISO-8859-1 Cc: FreeBSD Current , Xin LI , i18n@freebsd.org Subject: Re: [CFT] BSDL iconv in base system X-BeenThere: freebsd-i18n@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: FreeBSD Internationalization Effort List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Jun 2010 00:48:51 -0000 On Mon, Jun 14, 2010 at 7:13 PM, Gabor Kovesdan wrote: > Hello Folks, > > during the last summer, Google generously founded my Summer of Code project, > which was providing a BSD-licensed iconv implementation for FreeBSD. I'm > proud to announce that the work has been completed and a patch is available > to add it to the base system. > > The results of this work are: > - The Citrus implementation has been ported from NetBSD. > - Some utilities have been added. There is a conversion table generator, > which can compare conversion tables to reference data generated by GNU > libiconv. This helps ensuring conversion compatibility. > - UTF-16 surrogate support and some endianness issues have been fixed. > - The rather chaotic Makefiles to build metadata have been refactored and > cleaned up, now it is easy to read and it is also easier to add support for > new encodings. > - A bunch of new encodings and encoding aliases have been added. > - Support for 1->2, 1->3 and 1->4 mappings, which is needed for > transliterating with flying accents as GNU does, like "u. > - Lots of warnings have been fixed, the major part of the code is now > WARNS=6 clean. > - New section 1 and section 5 manual pages have been added. > - Some GNU-specific calls have been implemented: iconvlist(), iconvctl(), > iconv_canonicalize(), iconv_open_into() > - Support for GNU's //IGNORE suffix has been added. > - The "-" argument for stdin is now recognized in iconv(1) as per POSIX. > - The Big5 conversion module has been fixed. > - The iconv.h header files is supposed to be compatible with the GNU > version, i.e. sources should build with base iconv.h and GNU libiconv. I've > just did a very quick test and it seems ports can safely link to GNU > libiconv, there's no conflict. > - Various cleanups and style(9) fixes. > - A bachelor thesis written in Hungarian language: > http://www.kovesdan.org/files/bsc_iconv.pdf > > The rather big patch (42,5M) is available here: > http://www.kovesdan.org/patches/iconv_base_integrate.diff Over 40 Megabytes?! WOW. Thank you for this incredible amount of work, I know the FreeBSD community will benefit greatly from it. I think this effort deserves some hardcore testing, so now to the FreeBSD community -- I know it will get the attention it deserves :) -Brandon From owner-freebsd-i18n@FreeBSD.ORG Tue Jun 15 17:11:04 2010 Return-Path: Delivered-To: i18n@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 01D47106566B; Tue, 15 Jun 2010 17:11:04 +0000 (UTC) (envelope-from gabor@FreeBSD.org) Received: from server.mypc.hu (server.mypc.hu [87.229.73.95]) by mx1.freebsd.org (Postfix) with ESMTP id AD7658FC08; Tue, 15 Jun 2010 17:10:56 +0000 (UTC) Received: from server.mypc.hu (localhost [127.0.0.1]) by server.mypc.hu (Postfix) with ESMTP id D929214DBB5C; Tue, 15 Jun 2010 19:10:54 +0200 (CEST) X-Virus-Scanned: amavisd-new at server.mypc.hu Received: from server.mypc.hu ([127.0.0.1]) by server.mypc.hu (server.mypc.hu [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 66O6HdmzmIHP; Tue, 15 Jun 2010 19:10:52 +0200 (CEST) Received: from [192.168.1.105] (catv-80-99-92-167.catv.broadband.hu [80.99.92.167]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by server.mypc.hu (Postfix) with ESMTPSA id A17B814DB9B6; Tue, 15 Jun 2010 19:10:52 +0200 (CEST) Message-ID: <4C17B419.6020507@FreeBSD.org> Date: Tue, 15 Jun 2010 19:10:49 +0200 From: Gabor Kovesdan User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.2; es-ES; rv:1.9.1.9) Gecko/20100317 Thunderbird/3.0.4 MIME-Version: 1.0 To: Gleb Kurtsou References: <4C16C5B5.1070308@FreeBSD.org> <20100615170158.GA2131@tops> In-Reply-To: <20100615170158.GA2131@tops> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD Current , i18n@FreeBSD.org Subject: Re: [CFT] BSDL iconv in base system X-BeenThere: freebsd-i18n@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: FreeBSD Internationalization Effort List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Jun 2010 17:11:04 -0000 > Are there any plans to resurrect/finish multibyte collation support > GSoC'2008 project: > http://wiki.freebsd.org/KonradJankowski/Collation > Yes, my queue is just so long that I haven't got there yet. I'm in SoC 2010 again with a different project and there's still BSD grep from SoC 2008. I'm also fixing the last nits of that. And there are also personal things, like a one-year internship in Portugal, which is going to start in September. But I hope once I'll find time or this. > And are you aware of any plans on adding utf8-aware regex? I think > NetBSD has already imported one: > http://blog.netbsd.org/tnf/entry/efficient_wide_character_regular_expressions > Yes, again but same issues. :) Besides, we need/should add a more relaxed regex support to TRE before we can adopt it. GNU regex allows things like [a|], which is not standard, so we should support them to maintain compatibility. This will be important for ports. This is also the reason why BSD grep is linked to GNU regex instead of libc-regex. -- Gabor Kovesdan FreeBSD Volunteer EMAIL: gabor@FreeBSD.org .:|:. gabor@kovesdan.org WEB: http://people.FreeBSD.org/~gabor .:|:. http://kovesdan.org From owner-freebsd-i18n@FreeBSD.ORG Tue Jun 15 17:31:27 2010 Return-Path: Delivered-To: i18n@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 634E11065675 for ; Tue, 15 Jun 2010 17:31:27 +0000 (UTC) (envelope-from gleb.kurtsou@gmail.com) Received: from mail-bw0-f54.google.com (mail-bw0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id DD1FF8FC14 for ; Tue, 15 Jun 2010 17:31:26 +0000 (UTC) Received: by bwz8 with SMTP id 8so276713bwz.13 for ; Tue, 15 Jun 2010 10:31:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:date:from:to:cc:subject :message-id:references:mime-version:content-type:content-disposition :in-reply-to:user-agent; bh=CDUii5vuCia8DK1TewIs2tp/HTzoattqdc/br6m2OJs=; b=pxhh2aWJ9+0y+JQycuydDdnwBufv1Um2agCfJFBuGG2VWA04Etv1dV1glmh7Ai03aS C98SSVY3DVrYVqLpwHzMdPOz0SxxPctgGihvX4756+/vecf5h8f3tjh2ngBBnM5DtkeT tkPZq1B51fqSPbGq0bFQa1Jwf8YEBwIDt/d4E= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; b=Bv+pyAOrfaXfizucfPnYTM/orKhFn4o7Vo77YMo7fORWjw/jZyQgsHpgMx0cRtCsZh 80f0BCaO3zpKLATMfw82bj7OlxnEWoha9FUOWgzQRPeXvVxdLUGV+7nRB62lUSZINEKL Cqmmpwb92YiWdaZTfhL8Pw61zQTnPmimS9G8I= Received: by 10.204.47.25 with SMTP id l25mr5637974bkf.46.1276621312316; Tue, 15 Jun 2010 10:01:52 -0700 (PDT) Received: from localhost ([212.98.186.134]) by mx.google.com with ESMTPS id f13sm25202151bka.23.2010.06.15.10.01.50 (version=TLSv1/SSLv3 cipher=RC4-MD5); Tue, 15 Jun 2010 10:01:51 -0700 (PDT) Date: Tue, 15 Jun 2010 20:01:58 +0300 From: Gleb Kurtsou To: Gabor Kovesdan Message-ID: <20100615170158.GA2131@tops> References: <4C16C5B5.1070308@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <4C16C5B5.1070308@FreeBSD.org> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: FreeBSD Current , Xin LI , i18n@FreeBSD.org Subject: Re: [CFT] BSDL iconv in base system X-BeenThere: freebsd-i18n@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: FreeBSD Internationalization Effort List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Jun 2010 17:31:27 -0000 On (15/06/2010 02:13), Gabor Kovesdan wrote: > Hello Folks, > > during the last summer, Google generously founded my Summer of Code > project, which was providing a BSD-licensed iconv implementation for > FreeBSD. I'm proud to announce that the work has been completed and a > patch is available to add it to the base system. > > The results of this work are: > - The Citrus implementation has been ported from NetBSD. > - Some utilities have been added. There is a conversion table generator, > which can compare conversion tables to reference data generated by GNU > libiconv. This helps ensuring conversion compatibility. > - UTF-16 surrogate support and some endianness issues have been fixed. > - The rather chaotic Makefiles to build metadata have been refactored > and cleaned up, now it is easy to read and it is also easier to add > support for new encodings. > - A bunch of new encodings and encoding aliases have been added. > - Support for 1->2, 1->3 and 1->4 mappings, which is needed for > transliterating with flying accents as GNU does, like "u. > - Lots of warnings have been fixed, the major part of the code is now > WARNS=6 clean. > - New section 1 and section 5 manual pages have been added. > - Some GNU-specific calls have been implemented: iconvlist(), > iconvctl(), iconv_canonicalize(), iconv_open_into() > - Support for GNU's //IGNORE suffix has been added. > - The "-" argument for stdin is now recognized in iconv(1) as per POSIX. > - The Big5 conversion module has been fixed. > - The iconv.h header files is supposed to be compatible with the GNU > version, i.e. sources should build with base iconv.h and GNU libiconv. > I've just did a very quick test and it seems ports can safely link to > GNU libiconv, there's no conflict. > - Various cleanups and style(9) fixes. > - A bachelor thesis written in Hungarian language: > http://www.kovesdan.org/files/bsc_iconv.pdf > > The rather big patch (42,5M) is available here: > http://www.kovesdan.org/patches/iconv_base_integrate.diff > > Any comments, suggestions or bugreports are very welcome. Awesome! Thanks for working on it. Are there any plans to resurrect/finish multibyte collation support GSoC'2008 project: http://wiki.freebsd.org/KonradJankowski/Collation And are you aware of any plans on adding utf8-aware regex? I think NetBSD has already imported one: http://blog.netbsd.org/tnf/entry/efficient_wide_character_regular_expressions Thanks, Gleb. > -- > Gabor Kovesdan > FreeBSD Volunteer > > EMAIL:gabor@FreeBSD.org .:|:.gabor@kovesdan.org > WEB:http://people.FreeBSD.org/~gabor .:|:.http://kovesdan.org > > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" From owner-freebsd-i18n@FreeBSD.ORG Wed Jun 16 14:50:48 2010 Return-Path: Delivered-To: i18n@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AFFF41065672; Wed, 16 Jun 2010 14:50:48 +0000 (UTC) (envelope-from lichave@gmail.com) Received: from fg-out-1718.google.com (fg-out-1718.google.com [72.14.220.153]) by mx1.freebsd.org (Postfix) with ESMTP id CCE938FC15; Wed, 16 Jun 2010 14:50:47 +0000 (UTC) Received: by fg-out-1718.google.com with SMTP id d23so1181965fga.13 for ; Wed, 16 Jun 2010 07:50:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:reply-to :in-reply-to:references:date:message-id:subject:from:to:cc :content-type; bh=j/HqinKM1k8Z3NWeX+Rm2HnbC0UdR9PhljzBj0H8tdU=; b=LB5lBmgJGnyZ09ykpa0JtdD8V9+hdaxfI+mnyjpL0ng3gRAcqtYj6lGeN2i1cfAhug rEg3eCMwa0clf6Fe/tS7SDa2JbFiy7K2ALCVvePPzxFHne9IJGhcGF8/w1mpxRBDy+j5 h/AQM8jpGCXloCfCMy8+d5ht1y8WvlHNlwTGY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:reply-to:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; b=QUSxqH9SPuP9SOgttn07yQVbJRUVTy/cc97CkhOW2cF1SpyYBO6w3reKf+7VMfWDdX U8dlRoLT43WlmwiEZIo2R52c/annhIdyjWzqCDiDonrdhxc+cqXYdpFdshar+9zxjj5k uPARN5fzN4aHsJE/CWmz0sc1cLTT6NY7sdoP4= MIME-Version: 1.0 Received: by 10.204.3.65 with SMTP id 1mr6630272bkm.210.1276697954576; Wed, 16 Jun 2010 07:19:14 -0700 (PDT) Received: by 10.204.139.28 with HTTP; Wed, 16 Jun 2010 07:19:14 -0700 (PDT) In-Reply-To: <20100615170158.GA2131@tops> References: <4C16C5B5.1070308@FreeBSD.org> <20100615170158.GA2131@tops> Date: Wed, 16 Jun 2010 16:19:14 +0200 Message-ID: From: Konrad Jankowski To: Gleb Kurtsou Content-Type: text/plain; charset=UTF-8 Cc: FreeBSD Current , Xin LI , Gabor Kovesdan , i18n@freebsd.org Subject: Re: [CFT] BSDL iconv in base system X-BeenThere: freebsd-i18n@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: konrad@freebsd.org List-Id: FreeBSD Internationalization Effort List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Jun 2010 14:50:48 -0000 On Tue, Jun 15, 2010 at 7:01 PM, Gleb Kurtsou wrote: > On (15/06/2010 02:13), Gabor Kovesdan wrote: >> Hello Folks, >> >> during the last summer, Google generously founded my Summer of Code >> project, which was providing a BSD-licensed iconv implementation for >> FreeBSD. I'm proud to announce that the work has been completed and a >> patch is available to add it to the base system. >> >> The results of this work are: >> - The Citrus implementation has been ported from NetBSD. >> - Some utilities have been added. There is a conversion table generator, >> which can compare conversion tables to reference data generated by GNU >> libiconv. This helps ensuring conversion compatibility. >> - UTF-16 surrogate support and some endianness issues have been fixed. >> - The rather chaotic Makefiles to build metadata have been refactored >> and cleaned up, now it is easy to read and it is also easier to add >> support for new encodings. >> - A bunch of new encodings and encoding aliases have been added. >> - Support for 1->2, 1->3 and 1->4 mappings, which is needed for >> transliterating with flying accents as GNU does, like "u. >> - Lots of warnings have been fixed, the major part of the code is now >> WARNS=6 clean. >> - New section 1 and section 5 manual pages have been added. >> - Some GNU-specific calls have been implemented: iconvlist(), >> iconvctl(), iconv_canonicalize(), iconv_open_into() >> - Support for GNU's //IGNORE suffix has been added. >> - The "-" argument for stdin is now recognized in iconv(1) as per POSIX. >> - The Big5 conversion module has been fixed. >> - The iconv.h header files is supposed to be compatible with the GNU >> version, i.e. sources should build with base iconv.h and GNU libiconv. >> I've just did a very quick test and it seems ports can safely link to >> GNU libiconv, there's no conflict. >> - Various cleanups and style(9) fixes. >> - A bachelor thesis written in Hungarian language: >> http://www.kovesdan.org/files/bsc_iconv.pdf >> >> The rather big patch (42,5M) is available here: >> http://www.kovesdan.org/patches/iconv_base_integrate.diff >> >> Any comments, suggestions or bugreports are very welcome. > > Awesome! Thanks for working on it. > > Are there any plans to resurrect/finish multibyte collation support > GSoC'2008 project: > http://wiki.freebsd.org/KonradJankowski/Collation Hi. The project is not dead. I've resumed actively working on it. Expect some patches/commits soon. -- Konrad From owner-freebsd-i18n@FreeBSD.ORG Wed Jun 16 19:23:22 2010 Return-Path: Delivered-To: i18n@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C2F1B1065678; Wed, 16 Jun 2010 19:23:22 +0000 (UTC) (envelope-from jh@FreeBSD.org) Received: from gw03.mail.saunalahti.fi (gw03.mail.saunalahti.fi [195.197.172.111]) by mx1.freebsd.org (Postfix) with ESMTP id 781618FC12; Wed, 16 Jun 2010 19:23:21 +0000 (UTC) Received: from a91-153-117-195.elisa-laajakaista.fi (a91-153-117-195.elisa-laajakaista.fi [91.153.117.195]) by gw03.mail.saunalahti.fi (Postfix) with SMTP id 57C6B2167FE; Wed, 16 Jun 2010 22:04:16 +0300 (EEST) Date: Wed, 16 Jun 2010 22:04:16 +0300 From: Jaakko Heinonen To: Gabor Kovesdan Message-ID: <20100616190416.GA3896@a91-153-117-195.elisa-laajakaista.fi> References: <4C16C5B5.1070308@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4C16C5B5.1070308@FreeBSD.org> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: FreeBSD Current , Xin LI , i18n@FreeBSD.org Subject: Re: [CFT] BSDL iconv in base system X-BeenThere: freebsd-i18n@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: FreeBSD Internationalization Effort List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Jun 2010 19:23:22 -0000 Hi, On 2010-06-15, Gabor Kovesdan wrote: > - The iconv.h header files is supposed to be compatible with the GNU > version, i.e. sources should build with base iconv.h and GNU libiconv. > I've just did a very quick test and it seems ports can safely link to > GNU libiconv, there's no conflict. > The rather big patch (42,5M) is available here: > http://www.kovesdan.org/patches/iconv_base_integrate.diff iconv(3) prototype doesn't conform to POSIX.1-2008. Is it a well-considered decision? -- Jaakko From owner-freebsd-i18n@FreeBSD.ORG Wed Jun 16 20:02:31 2010 Return-Path: Delivered-To: i18n@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 442F71065675; Wed, 16 Jun 2010 20:02:31 +0000 (UTC) (envelope-from gabor@FreeBSD.org) Received: from server.mypc.hu (server.mypc.hu [87.229.73.95]) by mx1.freebsd.org (Postfix) with ESMTP id ECE118FC12; Wed, 16 Jun 2010 20:02:30 +0000 (UTC) Received: from server.mypc.hu (localhost [127.0.0.1]) by server.mypc.hu (Postfix) with ESMTP id E4F3914DBBBE; Wed, 16 Jun 2010 22:02:29 +0200 (CEST) X-Virus-Scanned: amavisd-new at server.mypc.hu Received: from server.mypc.hu ([127.0.0.1]) by server.mypc.hu (server.mypc.hu [127.0.0.1]) (amavisd-new, port 10024) with LMTP id FBsdIGH-b48B; Wed, 16 Jun 2010 22:02:27 +0200 (CEST) Received: from [192.168.1.105] (catv-80-99-92-167.catv.broadband.hu [80.99.92.167]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by server.mypc.hu (Postfix) with ESMTPSA id A60E114DBBB4; Wed, 16 Jun 2010 22:02:27 +0200 (CEST) Message-ID: <4C192DD1.5070401@FreeBSD.org> Date: Wed, 16 Jun 2010 22:02:25 +0200 From: Gabor Kovesdan User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.2; es-ES; rv:1.9.1.9) Gecko/20100317 Thunderbird/3.0.4 MIME-Version: 1.0 To: Jaakko Heinonen References: <4C16C5B5.1070308@FreeBSD.org> <20100616190416.GA3896@a91-153-117-195.elisa-laajakaista.fi> In-Reply-To: <20100616190416.GA3896@a91-153-117-195.elisa-laajakaista.fi> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD Current , Xin LI , i18n@FreeBSD.org Subject: Re: [CFT] BSDL iconv in base system X-BeenThere: freebsd-i18n@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: FreeBSD Internationalization Effort List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Jun 2010 20:02:31 -0000 > iconv(3) prototype doesn't conform to POSIX.1-2008. Is it a > well-considered decision? > No, it was just like that in the Citrus version and I didn't notice the const qualifier. Fixed in my working copy, will be available soon with some minor modifications. Thanks for reporting this. -- Gabor Kovesdan FreeBSD Volunteer EMAIL: gabor@FreeBSD.org .:|:. gabor@kovesdan.org WEB: http://people.FreeBSD.org/~gabor .:|:. http://kovesdan.org From owner-freebsd-i18n@FreeBSD.ORG Wed Jun 16 20:09:37 2010 Return-Path: Delivered-To: i18n@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C792C106566B; Wed, 16 Jun 2010 20:09:37 +0000 (UTC) (envelope-from jilles@stack.nl) Received: from mx1.stack.nl (relay04.stack.nl [IPv6:2001:610:1108:5010::107]) by mx1.freebsd.org (Postfix) with ESMTP id 8A4F48FC1C; Wed, 16 Jun 2010 20:09:37 +0000 (UTC) Received: from turtle.stack.nl (turtle.stack.nl [IPv6:2001:610:1108:5010::132]) by mx1.stack.nl (Postfix) with ESMTP id 255901DD68B; Wed, 16 Jun 2010 22:09:36 +0200 (CEST) Received: by turtle.stack.nl (Postfix, from userid 1677) id 10D9F1729C; Wed, 16 Jun 2010 22:09:36 +0200 (CEST) Date: Wed, 16 Jun 2010 22:09:36 +0200 From: Jilles Tjoelker To: Jaakko Heinonen Message-ID: <20100616200935.GA26321@stack.nl> References: <4C16C5B5.1070308@FreeBSD.org> <20100616190416.GA3896@a91-153-117-195.elisa-laajakaista.fi> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100616190416.GA3896@a91-153-117-195.elisa-laajakaista.fi> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: FreeBSD Current , Gabor Kovesdan , i18n@FreeBSD.org Subject: Re: [CFT] BSDL iconv in base system X-BeenThere: freebsd-i18n@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: FreeBSD Internationalization Effort List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Jun 2010 20:09:37 -0000 On Wed, Jun 16, 2010 at 10:04:16PM +0300, Jaakko Heinonen wrote: > On 2010-06-15, Gabor Kovesdan wrote: > > - The iconv.h header files is supposed to be compatible with the GNU > > version, i.e. sources should build with base iconv.h and GNU libiconv. > > I've just did a very quick test and it seems ports can safely link to > > GNU libiconv, there's no conflict. > > The rather big patch (42,5M) is available here: > > http://www.kovesdan.org/patches/iconv_base_integrate.diff > iconv(3) prototype doesn't conform to POSIX.1-2008. Is it a > well-considered decision? I think the difference from POSIX.1-2008 is pretty common and may therefore cause less compilation problems. NetBSD's Citrus iconv and GNU iconv have the extra 'const', and so does the default Solaris iconv (Solaris has a separate iconv for standards-conforming applications with the POSIX prototype.) -- Jilles Tjoelker From owner-freebsd-i18n@FreeBSD.ORG Wed Jun 16 20:27:14 2010 Return-Path: Delivered-To: i18n@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3C9251065672 for ; Wed, 16 Jun 2010 20:27:14 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id EBBD68FC12 for ; Wed, 16 Jun 2010 20:27:13 +0000 (UTC) Received: from ds4.des.no (des.no [84.49.246.2]) by smtp.des.no (Postfix) with ESMTP id 42B5B1FFC34; Wed, 16 Jun 2010 20:10:09 +0000 (UTC) Received: by ds4.des.no (Postfix, from userid 1001) id F2EF08444F; Wed, 16 Jun 2010 22:07:58 +0200 (CEST) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Jaakko Heinonen References: <4C16C5B5.1070308@FreeBSD.org> <20100616190416.GA3896@a91-153-117-195.elisa-laajakaista.fi> Date: Wed, 16 Jun 2010 22:07:58 +0200 In-Reply-To: <20100616190416.GA3896@a91-153-117-195.elisa-laajakaista.fi> (Jaakko Heinonen's message of "Wed, 16 Jun 2010 22:04:16 +0300") Message-ID: <86bpba7nc1.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.0.95 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: FreeBSD Current , Xin LI , Gabor Kovesdan , i18n@FreeBSD.org Subject: Re: [CFT] BSDL iconv in base system X-BeenThere: freebsd-i18n@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: FreeBSD Internationalization Effort List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Jun 2010 20:27:14 -0000 Jaakko Heinonen writes: > iconv(3) prototype doesn't conform to POSIX.1-2008. Is it a > well-considered decision? Probably not, because it breaks the interface. Imagine that inbuf were just a char *, not a char **. It would be perfectly safe to change it to const char *, because you can always assign a char * to a const char *. However, inbuf is a char **, which is a pointer to a pointer to char. Gabor changed it to const char **, which is a pointer to a pointer to const char. Unfortunately, the two types are incompatible. If foo is a char *, you can't pass &foo as inbuf. % cat >/tmp/const.c < void fs(char *s) { puts(++s); } void gs(const char *s) { puts(++s); } void fsp(char **sp) { puts(++*sp); } void gsp(const char **sp) { puts(++*sp); } int main() { char *s =3D "xyzzy", **sp =3D &s; fs(s); gs(s); fsp(sp); gsp(s= p); } EOF % cc -Wall -Wextra -Werror -std=3Dc99 -o/dev/null /tmp/const.c cc1: warnings being treated as errors /tmp/const.c: In function =E2=80=98main=E2=80=99: /tmp/const.c:6: error: passing argument 1 of =E2=80=98gsp=E2=80=99 from inc= ompatible pointer type /tmp/const.c:5: note: expected =E2=80=98const char **=E2=80=99 but argument= is of type =E2=80=98char **=E2=80=99 This means you can't, say, read data from a file into a buffer and then pass that buffer to iconv, because the buffer is not const (otherwise you couldn't have read data into it). That seems like a pretty fundamental flaw. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no