From owner-freebsd-current@FreeBSD.ORG Wed Jun 17 03:33:39 2015 Return-Path: Delivered-To: freebsd-current@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BAABE78D for ; Wed, 17 Jun 2015 03:33:39 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from mx1.scaleengine.net (mx1.scaleengine.net [209.51.186.6]) by mx1.freebsd.org (Postfix) with ESMTP id 8D3713F8 for ; Wed, 17 Jun 2015 03:33:39 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from [192.168.1.2] (unknown [192.168.1.2]) (Authenticated sender: allanjude.freebsd@scaleengine.com) by mx1.scaleengine.net (Postfix) with ESMTPSA id 757A896F7 for ; Wed, 17 Jun 2015 03:33:33 +0000 (UTC) Message-ID: <5580EA90.2060503@freebsd.org> Date: Tue, 16 Jun 2015 23:33:36 -0400 From: Allan Jude User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.7.0 MIME-Version: 1.0 To: freebsd-current@freebsd.org Subject: Re: ls is broken for non-C locales due to libxo References: <48B53C84-E4F7-4E88-B07D-7489FF585430@freebsd.org> <5580DEB0.7080908@freebsd.org> In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="o9v9h7TvnimLFw6KjStrVBIOHVn7XsjPJ" X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jun 2015 03:33:39 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --o9v9h7TvnimLFw6KjStrVBIOHVn7XsjPJ Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 2015-06-16 23:23, Marcel Moolenaar wrote: >=20 >> On Jun 16, 2015, at 7:42 PM, Andrey Chernov wrote: >> >> On 17.06.2015 5:18, Andrey Chernov wrote: >>> Recent -current, LANG=3Dru_RU.KOI8-R locale. ls -t eats whole date co= lumn, just ls eats whole filenames with printable national characters ins= ide. Long live libxo. >>> >> >> I mean ls -l. To be precise, for 8bit non-C locales at least: >> ls -lt: skip date column and eat filename with even single 8bit >> character inside. >> ls -l: just eat filename as above. >=20 > Date/time is fixed. I don=E2=80=99t know how to reproduce the filename > problem, so make sure sources are up-to-date and if still a > problem, provide me with a way to reproduce. >=20 > Thanks, >=20 > -- > Marcel Moolenaar > marcel@xcllnt.net >=20 Simple reproduction: env LANG=3Dru_RU.KOI8-R ./ls -l --libxo warn total 200 ls: invalid UTF-8 character: c9/2 -rw-r--r-- 1 allan allan 315 Makefile ls: invalid UTF-8 character: c9/2 -rw-r--r-- 1 allan allan 405 Makefile.depend ls: invalid UTF-8 character: cd/2 -rw-r--r-- 1 allan allan 4922 cmp.c ls: invalid UTF-8 character: c9/2 -rw-r--r-- 1 allan allan 4096 cmp.o ls: invalid UTF-8 character: c9/2 -rw-r--r-- 1 allan allan 2987 extern.h ls: invalid UTF-8 character: c9/2 -rwxr-xr-x 1 allan allan 38934 ls ls: invalid UTF-8 character: c9/2 It seems like the problem is that xo_utf8_to_wc_len() doesn't know how to deal with 8 bit characters that are not UTF-8. If the highest bit is set, but it is not UTF-8, libxo pukes. --=20 Allan Jude --o9v9h7TvnimLFw6KjStrVBIOHVn7XsjPJ Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32) iQIcBAEBAgAGBQJVgOqbAAoJEBmVNT4SmAt+yaoQALkESUUfR3f+D5lEKbRyhxr+ DcOk4N04xWVzAekmmnjU/9m9tTsxlU6jeZoA9fd4W9Y0zmc+8pGZgOUoymkZHV09 vcvq/vnz+CvHjFjJS27EKHcFQFj6yGsjRCtAcQZv2ejZETRIoN1ZDTffet0KXO70 QIGyFyffrp5fG/0gYhKv8f9sfPBHljGABMmuqciVeDsfsNjMbj40JcPrc57oQb3j ng4Sfv9vt57+7jfW2O+pfE//B2b9mI1KDHB9rl/Gi7yxlOdQfwWt+gaUvtxVyYk/ OsW/wJi2ROr+PMv9SmNm3DQx3xVDQbzhTPsq0w0aeLHlmtEE8gh90jcxlTFhFZDA NmMc+dQ7J1Gnz/xr4cDBfAIVbwK7+fqBncvAvR1keqUCBDPHZ5fqOIEnN3qU9Noi O2YUAFpz3QF4uPm5xeCJ9MJbhwtiF/fQT4V2SbcEj2abHiGzA1DLL/j95zLY8594 qNKJpRZVYrL/4h59EAje3IuSArWc7cnGRJJ2Rcz8a9a83vx4F+pyYen7U5mwxttQ 1jb9t9z40K7L0xkTjc0jJBHaqwALeCmIg7kbjzVMas0f8MBVXfqLcfuVUnHwM+Rt iKMTwEkXdu9eHKh2GveMU/sRGHd+EA6MBPmQih4lL7QKWpOF+lIu/gDdcrADfmWT 9qS1kB23cxzxuZAdGL+t =yCLL -----END PGP SIGNATURE----- --o9v9h7TvnimLFw6KjStrVBIOHVn7XsjPJ--