From owner-freebsd-questions@freebsd.org Thu Apr 4 22:04:47 2019 Return-Path: Delivered-To: freebsd-questions@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 36594155DC84 for ; Thu, 4 Apr 2019 22:04:47 +0000 (UTC) (envelope-from rockyhotas@post.com) Received: from mout.gmx.com (mout.gmx.com [74.208.4.200]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "mout.gmx.com", Issuer "GeoTrust RSA CA 2018" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D9DC68EEB2 for ; Thu, 4 Apr 2019 22:04:45 +0000 (UTC) (envelope-from rockyhotas@post.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=mail.com; s=dbd5af2cbaf7; t=1554415484; bh=kYxmorsGWhZQLjFD7P+tftzxNgR6QwpK3modGmgJ0ko=; h=X-UI-Sender-Class:From:To:Cc:Subject:Date:In-Reply-To:References; b=M4jQqoRmxsjOjeUbVABy/qwi5aw15teN3+/mlS6o/cLGDmEUAv7kWMfwoqOQABpoy bEq66/LqfhzV8Ft7nX1m0dkBgpx6KcNm8rZbDYnkqo8agATJCgb44AYu7FUMDj6oPz Icthu0rjWnBFQFjtQClfJ7EyVFBe+YcK3J2UrzGI= X-UI-Sender-Class: 214d933f-fd2f-45c7-a636-f5d79ae31a79 Received: from [95.246.186.146] ([95.246.186.146]) by web-mail.mail.com (3c-app-mailcom-lxa08.server.lan [10.76.45.9]) (via HTTP); Thu, 4 Apr 2019 23:59:24 +0200 MIME-Version: 1.0 Message-ID: From: "Rocky Hotas" To: "FreeBSD Questions" Cc: guru@unixarea.de Subject: Re: vi(1) and ISO 8859-1 Content-Type: text/plain; charset=UTF-8 Date: Thu, 4 Apr 2019 23:59:24 +0200 Importance: normal Sensitivity: Normal In-Reply-To: <20190404190642.GA4609@sh4-5.1blu.de> References: <20190404190642.GA4609@sh4-5.1blu.de> Content-Transfer-Encoding: quoted-printable X-UI-Message-Type: mail X-Priority: 3 X-Provags-ID: V03:K1:tntiZZVITFhJSeBdmsB58j5KpMSUS40/mct91zKws7goWgHJXVsSNIjaYz8GVgkppJBd1 n0+/mlrjilwJOCUD4FPC2ROk0GsUBxk3WxtB8si3nW5DoxJIMa865C7ha94JMbcvI8gK0fRQUske /CJqpVMsoJVFbsTdikvjGofpyKIEOE2JZEDAfVvqkrBDoW02BoMzUhYyXeRIzSnW0ifIoGsiqWrt CUeT4B12DwvsAxFrApduVNNORnP4JUTbYa1IPkUTB3gRswdWCyFQiKU1LuY7oXmr5BdF2K33FRAo Z4= X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V03:K0:cv5xiExQqyo=:4fV+qoiYmuy7F7pMPzx9C9 10JWPQBjD1fPsenNI8d7PcAHBmoy05tv752dqLd9WqfVLPSWhzBSWNtdNi4pplcOPell2dqL2 dUL/Q+vtmayq1HSIRXSeo6NB8eW53HRKt67MqeaIyYmF7PwF9+Cltbw52nkCsx9tiMCSOuNaC T57dO1JMnCayQcNXTRU/KuizeCriofsGt7OO0AcncK11izCVtwKGJPf1jHqNWmIIjOHbz/not orEs8nv4nE+43Fsk1vGzdixOq5ArKKvimj8DccTONXlmdVXRY+WSaNRXQvUYYRIraxRVwika2 zCPl+hcG1hzOYcEsb2ilZwJokWnTDiV206yKAmvjOgnAzdCdHEWhMCe9fuGI6WCE+vwoO2bSH 49MrUsaElZqkI83m6gEmfjWC4c50Kb94QrwqOncsw040jsc+Pkq37Qv1LUVlG6JZhF46VDq18 Dsyn4ATO1T4eIMmat4b8DoYVKnB0EhZM0QXKhrJQMXIfnm6DSkYrxlOiFV84Cw4xno1rbdbiw SCTLu++Y04kxudp5ODU05/gugM/DkJEZ7QtUH3nHo+m4//CTK2jQAIvX2CTq7GvXMKF8DKQfV GIgjdChgVmFaI= X-Rspamd-Queue-Id: D9DC68EEB2 X-Spamd-Bar: +++ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=mail.com header.s=dbd5af2cbaf7 header.b=M4jQqoRm; spf=pass (mx1.freebsd.org: domain of rockyhotas@post.com designates 74.208.4.200 as permitted sender) smtp.mailfrom=rockyhotas@post.com X-Spamd-Result: default: False [3.13 / 15.00]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:74.208.4.192/26]; FREEMAIL_FROM(0.00)[post.com]; DKIM_TRACE(0.00)[mail.com:+]; MX_GOOD(-0.01)[cached: mx01.mail.com]; HAS_X_PRIO_THREE(0.00)[3]; RCPT_COUNT_TWO(0.00)[2]; RECEIVED_SPAMHAUS_PBL(0.00)[146.186.246.95.zen.spamhaus.org : 127.0.0.11]; RCVD_IN_DNSWL_LOW(-0.10)[200.4.208.74.list.dnswl.org : 127.0.3.1]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:8560, ipnet:74.208.0.0/16, country:DE]; FREEMAIL_ENVFROM(0.00)[post.com]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[mail.com:s=dbd5af2cbaf7]; FROM_HAS_DN(0.00)[]; NEURAL_SPAM_SHORT(0.93)[0.927,0]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[post.com]; NEURAL_SPAM_MEDIUM(0.68)[0.682,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_SPAM_LONG(0.86)[0.857,0]; IP_SCORE(0.77)[ip: (-2.42), ipnet: 74.208.0.0/16(4.75), asn: 8560(1.54), country: DE(-0.01)]; MID_RHS_NOT_FQDN(0.50)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Apr 2019 22:04:47 -0000 > Sent: Thursday, April 04, 2019 at 9:06 PM > From: "Matthias Apitz" > To: freebsd-questions@freebsd=2Eorg > Subject: Re: vi(1) and ISO 8859-1 [=2E=2E=2E] > Are you sure that the file is completely ISO-8859-1 encoded?=20 Yes: > What does > file(1) says about $ file myfile myfile: ISO-8859 text > and what happens if you run it through iconv(1) > translating from ISO-8859-1 to UTF-8 into /dev/null, will iconv(1) do it > or complain? $ iconv -f ISO-8859-1 -t UTF-8 < myfile > /dev/null It completes successfully, exit status is 0=2E You can easily create such = a file and reproduce this: just open vi(1), set the encoding with `:se fe=3Diso-8859-1', and just type a letter with an accent, like: hello =C3=AC If you save it and reopen it with vi(1), it will use UTF-8 by default and = it won't be able to read the line with `=C3=AC', unless you again use `:se fe=3Diso-8859-1'=2E The correctness (and integrity) of the ISO 8859-1 encoding is also shown b= y hexdump: $ hexdump -C myfile 00000000 68 65 6c 6c 6f 0a ec 0a |hello=2E=2E= =2E| 00000008 Where `ec' is `=C3=AC' in ISO 8859-1=2E Rocky >=20 > matthias > --=20 > Matthias Apitz, =E2=9C=89 guru@unixarea=2Ede, http://www=2Eunixarea=2Ede= / +49-176-38902045 > Public GnuPG key: http://www=2Eunixarea=2Ede/key=2Epub > 70 years of NATO - 70 years of wars (Jugoslavia, Afghanistan, Syria, =2E= =2E=2E) and 70 years > of war preparation against Russia=2E -- PEACE instead of NATO !