From owner-freebsd-hackers@FreeBSD.ORG Tue May 25 11:29:20 2010 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F407F1065672 for ; Tue, 25 May 2010 11:29:19 +0000 (UTC) (envelope-from jhs@berklix.com) Received: from tower.berklix.org (tower.berklix.org [83.236.223.114]) by mx1.freebsd.org (Postfix) with ESMTP id 669C08FC15 for ; Tue, 25 May 2010 11:29:18 +0000 (UTC) Received: from park.js.berklix.net (p549A37B2.dip.t-dialin.net [84.154.55.178]) (authenticated bits=0) by tower.berklix.org (8.14.2/8.14.2) with ESMTP id o4PBTEUg094743 for ; Tue, 25 May 2010 11:29:17 GMT (envelope-from jhs@berklix.com) Received: from fire.js.berklix.net (fire.js.berklix.net [192.168.91.41]) by park.js.berklix.net (8.13.8/8.13.8) with ESMTP id o4PBTDlB050777 for ; Tue, 25 May 2010 13:29:15 +0200 (CEST) (envelope-from jhs@berklix.com) Received: from fire.js.berklix.net (localhost [127.0.0.1]) by fire.js.berklix.net (8.14.3/8.14.3) with ESMTP id o4PBSsLS049798 for ; Tue, 25 May 2010 13:29:11 +0200 (CEST) (envelope-from jhs@fire.js.berklix.net) Message-Id: <201005251129.o4PBSsLS049798@fire.js.berklix.net> To: hackers@freebsd.org From: "Julian H. Stacey" Organization: http://www.berklix.com BSD Linux Unix Consultancy, Munich Germany User-agent: EXMH on FreeBSD http://www.berklix.com/free/ X-URL: http://www.berklix.com/~jhs/cv/ Date: Tue, 25 May 2010 13:28:54 +0200 Sender: jhs@berklix.com Cc: Subject: patch for /usr/src/usr.bin/fmt/ (not 8 bit clean) for German & French X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 May 2010 11:29:20 -0000 Hi Hackers@ Problem: /usr/src/usr.bin/fmt/fmt.c is not 8 bit clean. (ie French or German etc text with accents in gets broken) Alternate Solutions: Solution 1: Discard "& 0x7f checks. Allow pretty much all 8 bit as a printable. This would need some patching. It might freak out some from the English speaking world ? ( We should also consider whether other tools like cat sed awk grep etc are 8 bit clean ? ) Solution 2: Apply my patches. They work for English German & French German for 9 years, French added yesterday (maybe I missed an accent ?). If we extended for more languages, would we regret not choosing Solution 1 ? http://www.berklix.com/~jhs/src/bsd/fixes/FreeBSD/src/gen/usr.bin/fmt/ Links for all releases from 4.10 to 8.0, reduce to just 4 files: Jul 26 2001 ./fmt.1.REL=ALL.diff Nov 9 2004 ./fmt.c.tail.diff.REL=4.10-RELEASE.diff May 25 00:20 ./fmt.c.tail.diff.REL=5.3-RELEASE.diff May 25 02:35 ./fmt.c.top.diff.REL=ALL.diff Solution 3: Import a single Locale ? Inadequate, Would fail to support multiple font sets simunltaneously. Example: Pasting notes into an xterm, clauses from http://seafrance.com in English then French original & German, to get the feel of what an unclear English translation really meant on booking amendment fees) Solution 4: Support Multiple Fonts simultaneously How ? Solution 1 or 2 ? or use wchar ? Isocode ? ... Or ? A non native English speaker, best to do this I guess. Not me: I am English; My patches work for me. I don't want to work more to support fonts I dislike (Before FreeBSD started I already listed 3 different incompatible German umlaut representations, & had developed Russian fonts for SCO, & now with BSD use groff & HTML font escapes, & for decades I've experienced regular change of German & English & American keyboards eg at customer sites, when all I (English) want is Ascii & BIOS conformant American), national accent fonts etc I don't enjoy) PS I'll be off line soon, won't see replies for a while, please discuss/ improve/ send-pr/ commit/ whatever in my absence :-). Cheers, Julian -- Julian Stacey, BSD Unix Linux C Sys Eng Consultants Munich http://berklix.com Mail plain text, Not HTML quoted-printable Base64 http://www.asciiribbon.org