From owner-freebsd-python@FreeBSD.ORG Thu Jan 29 07:30:21 2015 Return-Path: Delivered-To: freebsd-python@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F1455E47 for ; Thu, 29 Jan 2015 07:30:21 +0000 (UTC) Received: from lb1-smtp-cloud6.xs4all.net (lb1-smtp-cloud6.xs4all.net [194.109.24.24]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (Client CN "Bizanga Labs SMTP Client Certificate", Issuer "Bizanga Labs CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 873CEBED for ; Thu, 29 Jan 2015 07:30:20 +0000 (UTC) Received: from slackbox.erewhon.home ([83.162.243.5]) by smtp-cloud6.xs4all.net with ESMTP id ljV81p00G07iGuj01jV969; Thu, 29 Jan 2015 08:29:09 +0100 Received: by slackbox.erewhon.home (Postfix, from userid 1001) id 46DCB123EB; Thu, 29 Jan 2015 08:29:08 +0100 (CET) Date: Thu, 29 Jan 2015 08:29:08 +0100 From: Roland Smith To: Robert Simmons Subject: Re: Unicode Problem Message-ID: <20150129072908.GA37127@slackbox.erewhon.home> Mail-Followup-To: Robert Simmons , freebsd-python@freebsd.org References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="6c2NcOVqGQ03X4Wi" Content-Disposition: inline In-Reply-To: X-GPG-Fingerprint: 1A2B 477F 9970 BA3C 2914 B7CE 1277 EFB0 C321 A725 X-GPG-Key: http://www.xs4all.nl/~rsmith/pubkey.txt X-GPG-Notice: If this message is not signed, don't assume I sent it! User-Agent: Mutt/1.5.23 (2014-03-12) Cc: freebsd-python@freebsd.org X-BeenThere: freebsd-python@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: FreeBSD-specific Python issues List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Jan 2015 07:30:22 -0000 --6c2NcOVqGQ03X4Wi Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Jan 29, 2015 at 01:38:21AM -0500, Robert Simmons wrote: > I'm having a unicode problem on FreeBSD lang/python34 that does not > appear on MacOS X. I've condensed the problem to one single line to > enter in the interpreter: >=20 > FreeBSD: > Python 3.4.2 (default, Jan 28 2015, 22:23:57) > [GCC 4.2.1 Compatible FreeBSD Clang 3.4.1 (tags/RELEASE_34/dot1-final > 208032)] on freebsd10 > Type "help", "copyright", "credits" or "license" for more information. > >>> b'\xc3\xa2'.decode('utf-8') > '\xe2' >=20 > MacOS X: > Python 3.4.2 (default, Oct 19 2014, 17:55:38) > [GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.54)] on darwin > Type "help", "copyright", "credits" or "license" for more information. > >>> b'\xc3\xa2'.decode('utf-8') > '=C3=A2' >=20 > Why is Python on FreeBSD incorrectly decoding this? Works fine here (FreeBSD 10.1-STABLE #0 r276653 amd64): Python 3.4.2 (default, Nov 4 2014, 19:34:48) [GCC 4.2.1 Compatible FreeBSD Clang 3.4.1 (tags/RELEASE_34/dot1-final 2= 08032)] on freebsd10 Type "help", "copyright", "credits" or "license" for more information. >>> b'\xc3\xa2'.decode('utf-8') '=C3=A2' Roland --=20 R.F.Smith http://rsmith.home.xs4all.nl/ [plain text _non-HTML_ PGP/GnuPG encrypted/signed email much appreciated] pgp: 5753 3324 1661 B0FE 8D93 FCED 40F6 D5DC A38A 33E0 (keyID: A38A33E0) --6c2NcOVqGQ03X4Wi Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAABAgAGBQJUyeFEAAoJEED21dyjijPgye8P/1Nvv+9aOnvZl+Y6JcKCjBC4 5L8Tx3gK+kdunK5frr+8NjVbHrTablsLez0fcAFP5z0APq63mt/HBz/KSrtD5+91 sTXPkHlnVYJiwXw1wHqIfIpg5B6sN14LG39iZWl3MKZCApS8yrnytOUIeoR9FRWC 1fwdpD9ty+MztwnGrbr8Vi0Kv89nrIg0sLOGFS8B41z+YQoEd0bJQ87bjhmHFYDn JHLa+aPEHF0878lPzMwL5O06uMl3NppyuN189N0UTCFnoOr1AEsNn3SPHUl1Tmrb L1cMvqTZrBPCR9Hx0hIKj5QyyrDntz9W3VdCyiLKjCKgMnSp6F5uixswA7p7NpAM pNY7EaD39Wz7hmCMI7iC303jcs22/9eJBVyytEze32kUECpYoOIQ5kvp99YjbHHm HEgkODspPhpClAHdaF/yBkmSCcWWsXhd1yBSoYluFKaJy3ourXxbSptmG35vgg4Y /hxXPaGmflNAnp878bWe7XuZyFwUp8IRr0en0GXEUFHda6o+MOq5ykRCpN/PVj4J TddMFqk7lXMbKpaYWzU/+b4t/M6YrAsVy8PpCl+RmesKFc4j2BRaULB+RwzrD8be fTo5zeyba35/fx4ulHyZ2zjqbjeZJve4RxojdZNhATO2pr/aC9YJjnLyx2J+119o mRAjUpeUw2sRMidU2V3q =Tq3c -----END PGP SIGNATURE----- --6c2NcOVqGQ03X4Wi--