From owner-freebsd-questions@FreeBSD.ORG Wed Nov 9 03:00:01 2011 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BC501106564A for ; Wed, 9 Nov 2011 03:00:01 +0000 (UTC) (envelope-from conrads@cox.net) Received: from eastrmfepo101.cox.net (eastrmfepo101.cox.net [68.230.241.213]) by mx1.freebsd.org (Postfix) with ESMTP id 570448FC18 for ; Wed, 9 Nov 2011 03:00:00 +0000 (UTC) Received: from eastrmimpo109.cox.net ([68.230.241.222]) by eastrmfepo101.cox.net (InterMail vM.8.01.04.00 201-2260-137-20101110) with ESMTP id <20111109025955.CCQN3808.eastrmfepo101.cox.net@eastrmimpo109.cox.net>; Tue, 8 Nov 2011 21:59:55 -0500 Received: from serene.no-ip.org ([98.164.86.236]) by eastrmimpo109.cox.net with bizsmtp id uqzu1h00C55wwzE02qzuNr; Tue, 08 Nov 2011 21:59:55 -0500 X-CT-Class: Clean X-CT-Score: 0.00 X-CT-RefID: str=0001.0A020207.4EB9ECAB.002D,ss=2,re=0.000,fgs=0 X-CT-Spam: 0 X-Authority-Analysis: v=1.1 cv=n8j2qrTChidZYK+driZvPGNPpB1vlUiTfZeEYAw05jg= c=1 sm=1 a=G8Uczd0VNMoA:10 a=8nJEP1OIZ-IA:10 a=uAbGmPAyUfLL1M3oYAsfuA==:17 a=kviXuzpPAAAA:8 a=uw2IMaIg2Va7rkmCooEA:9 a=-0GJvpSeB4-JCZbJfPEA:7 a=wPNLvfGTeEIA:10 a=jKRid43F7UoA:10 a=4vB-4DCPJfMA:10 a=uAbGmPAyUfLL1M3oYAsfuA==:117 X-CM-Score: 0.00 Authentication-Results: cox.net; none Received: from cox.net (localhost [127.0.0.1]) by serene.no-ip.org (8.14.5/8.14.5) with ESMTP id pA92xrlL016450; Tue, 8 Nov 2011 20:59:53 -0600 (CST) (envelope-from conrads@cox.net) Date: Tue, 8 Nov 2011 20:59:48 -0600 From: "Conrad J. Sabatier" To: Polytropon Message-ID: <20111108205948.54daef43@cox.net> In-Reply-To: <20111109031024.fb4c617e.freebsd@edvax.de> References: <20111108184236.3a78ebf6@cox.net> <20111109031024.fb4c617e.freebsd@edvax.de> X-Mailer: Claws Mail 3.7.10 (GTK+ 2.24.6; amd64-portbld-freebsd9.0) Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Michael Ross , freebsd-questions@freebsd.org Subject: Re: "Unprintable" 8-bit characters X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Nov 2011 03:00:01 -0000 On Wed, 9 Nov 2011 03:10:24 +0100 Polytropon wrote: > On Wed, 09 Nov 2011 02:51:31 +0100, Michael Ross wrote: > > Am 09.11.2011, 01:42 Uhr, schrieb Conrad J. Sabatier > > : [snip] > > > I've been trying to understand what the deal is with regards to > > > the displaying of the "extended" 8-bit character set, i.e., 8-bit > > > characters with the MSB set. [snip]=20 > > Unsure if I understand you correctly. > > ("extended" 8-bit character set with MSB? utf-16?) > > I'm confused by this charset stuff in general. > >=20 > > Assuming you want \0xfc displayed as "=FC", [snip] > > here is what works for me: > >=20 > > in my login class in /etc/login.conf: > >=20 > > :charset=3DISO-8859-1:\ > > :lang=3Dde_DE.ISO8859-1:\ > >=20 > > ``cap_mkdb /etc/login.conf'' after changes >=20 > Ah, thanks - that seems to be the proper way to have > the environmental variables set - instead of my (ab)use > of setenv's in the csh config file. :-) Same here. I've been "guilty" as well of neglecting to properly adjust my console configuration. > Note the "precedence" of $LANG vs. $LC_* (as they can > be used to configure things more precisely, e. g. > regarding system messages or date formats; see example > following). >=20 >=20 >=20 > > in /etc/rc.conf: > >=20 > > scrnmap=3D"iso-8859-1_to_cp437" >=20 > Hm? CP437? Codepage? Isn't that some MS-DOS thing? > I've never needed a screenmap to make "extended > characters" (everything beyong US-ASCII) work. >=20 >=20 >=20 > > font8x8=3D"cp850-8x8" > > font8x14=3D"cp850-8x14" > > font8x16=3D"cp850-8x16" > >=20 > >=20 > > and in /etc/ttys, console type is set to ``cons25l1'' >=20 > I have a similar setting here, but that does _not_ work > wuth UTF-8 codec characters. If I want to use them, I > have to change some environmental variables, from >=20 > #-------GERMAN/ENGLISH------------------------ <=3D=3D=3D DEFAULT > setenv LC_ALL en_US.ISO8859-1 > setenv LC_MESSAGES en_US.ISO8859-1 > setenv LC_COLLATE de_DE.ISO8859-1 > setenv LC_CTYPE de_DE.ISO8859-1 > setenv LC_MONETARY de_DE.ISO8859-1 > setenv LC_NUMERIC de_DE.ISO8859-1 > setenv LC_TIME de_DE.ISO8859-1 > unsetenv LANG >=20 > to >=20 > #-------INTERNATIONAL------------------------- > setenv LC_ALL en_US.UTF-8 > setenv LC_MESSAGES en_US.UTF-8 > setenv LC_COLLATE de_DE.UTF-8 > setenv LC_CTYPE de_DE.UTF-8 > setenv LC_MONETARY de_DE.UTF-8 > setenv LC_NUMERIC de_DE.UTF-8 > setenv LC_TIME de_DE.UTF-8 > setenv LANG de_DE.UTF-8 Doesn't using "LC_ALL" obviate the need to set any of the other LC_* variables? At least, that's always been my understanding of it. But, getting back to something you said earlier, what did you mean exactly about the precedence of LANG vs. LC_*? > Then I can use UTF-8 characters inside rxvt-unicode. Of > course, text mode console is limited to the first set > of configuration, using the ISO 8859-1 character set. >=20 > This worked long before UTF-8 arrived with the glorious > idea that I should have 2 bytes where one is sufficient, > to describe our (german) 6 umlauts and the Eszett ligature. :-) Yes, and this is one area where the labels are more than a little misleading as well. My natural inclination is think of UTF-8 as being a single-byte representation for each character in the set, whereas UTF-16, as the name implies, would be the "wide", 2-byte version. Nonetheless, as I posted earlier in this thread, according to the info in gucharmap, the representations of the umlauted "u" are just the opposite of this: UTF-8: 0xC3 0xBC UTF-16: 0x00FC Go figure, huh? :-) =20 > Improper settings will result in [][] or A-tilde three > quarters upside-down question mark, depending on editor > or terminal used. Yes, I will definitely have to try using the recommendations that have come up in this thread re: the console. > But returning to the original question, I think Robert > did explain it very well: There is no real consensus > about what the different codings should mean. They > were meant to unify the representation of a very large > set of characters, but basically there are many inter- > pretations now, and how they show up to the user depends > on the font in use, _if_ it has this mapping or that, > or none. This seems rather unfortunate to me. You would think that, by now, some "standard" character set might have emerged that would allow one to use, at the very least, the "Western" characters (as opposed to the "Eastern" or "Oriental" or "Asian", if you will) with a reasonable expectation that others will see what was intended. > For running ls, -w is the right option to use - but IN > COMBINATION with correct settings for the terminal > emulation AND the presence of a font that will do. Yes. I'm still a little embarrassed for having completely overlooked that option earlier. Hasty (impatient) reading of man pages. :-) > Again a fine demonstration why file names should be > limited to printable ASCII and no spaces if you want > them to work everywhere. :-) Well, for myself, personally, I'm a bit of a stickler for "language authenticity", you might call it. Having studied both German and French rather extensively in my younger days, I'm quite fond of both languages, and rather keen on seeing them represented accurately (I especially wince at the use of the plain, unaccented vowel followed by an "e" in place of the umlaut, and to a lesser degree, the use of "ss" in place of Esszett), which has caused me no small amount of confusion, aggravation and frustration over the years, to be sure! :-) --=20 Conrad J. Sabatier conrads@cox.net