From owner-freebsd-python@FreeBSD.ORG  Thu Jan 29 07:13:44 2015
Return-Path: <owner-freebsd-python@FreeBSD.ORG>
Delivered-To: freebsd-python@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115])
 (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by hub.freebsd.org (Postfix) with ESMTPS id 42023B42
 for <freebsd-python@freebsd.org>; Thu, 29 Jan 2015 07:13:44 +0000 (UTC)
Received: from mail-ie0-x233.google.com (mail-ie0-x233.google.com
 [IPv6:2607:f8b0:4001:c03::233])
 (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits))
 (Client CN "smtp.gmail.com",
 Issuer "Google Internet Authority G2" (verified OK))
 by mx1.freebsd.org (Postfix) with ESMTPS id 0B283A9D
 for <freebsd-python@freebsd.org>; Thu, 29 Jan 2015 07:13:44 +0000 (UTC)
Received: by mail-ie0-f179.google.com with SMTP id x19so30154437ier.10
 for <freebsd-python@freebsd.org>; Wed, 28 Jan 2015 23:13:43 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
 h=mime-version:in-reply-to:references:date:message-id:subject:from:to
 :content-type:content-transfer-encoding;
 bh=zzRpGXwbVm0WeyW+nR0TquV8KpHnAlVFAS6R4B+RFUw=;
 b=HwYTJDHhuFtOOi7y4T/aSmvhVPyFRGatkWd22OpnrBev02hA8OBvRcGQOXZh7pZs+X
 rWuTMgTwhgtP4aukVqdFa7zl6iRAqaXzbW0Zy2YzPbk/FeHv3iXqZ2U8DlBqG6mjkGTx
 S1iP+ZXS81jHolEoueVtfKMH/x6LhbNQYiCQ+nDqWQrOrNmZVlyF6aZO4w7jmrvs2huh
 jksL36y3FSi5y090TrucaQmZudlmHaWRKKy88XOC2Zfv8MH/O6Gdt/ds0pfI4dODLQpg
 FGSEiIZZKAPjHjfmkI6Oyg7/lZCwgyhAFXr7w6505IKSMayvVrxANolcYbTgxZAzVdUU
 l2rQ==
MIME-Version: 1.0
X-Received: by 10.50.142.106 with SMTP id rv10mr743902igb.18.1422515623443;
 Wed, 28 Jan 2015 23:13:43 -0800 (PST)
Received: by 10.36.8.215 with HTTP; Wed, 28 Jan 2015 23:13:43 -0800 (PST)
In-Reply-To: <CA+QLa9DnD4mYSBc3B_SUZ6shH64v8Lej_CMMVaK=s+PrQp5Gvw@mail.gmail.com>
References: <CA+QLa9DnD4mYSBc3B_SUZ6shH64v8Lej_CMMVaK=s+PrQp5Gvw@mail.gmail.com>
Date: Thu, 29 Jan 2015 02:13:43 -0500
Message-ID: <CA+QLa9Amvu+1hg4Oo5J_CjoJNF2Ahs6LAkPrOAEncNU_iWXEtQ@mail.gmail.com>
Subject: Re: Unicode Problem
From: Robert Simmons <rsimmons0@gmail.com>
To: freebsd-python@freebsd.org
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
X-BeenThere: freebsd-python@freebsd.org
X-Mailman-Version: 2.1.18-1
Precedence: list
List-Id: FreeBSD-specific Python issues <freebsd-python.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-python>,
 <mailto:freebsd-python-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-python/>
List-Post: <mailto:freebsd-python@freebsd.org>
List-Help: <mailto:freebsd-python-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-python>,
 <mailto:freebsd-python-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 29 Jan 2015 07:13:44 -0000

On further inspection I've found the following:

FreeBSD
>>> import sys
>>> print(sys.getdefaultencoding())
utf-8
>>> print(sys.stdout.encoding)
US-ASCII

MacOS X:
>>> import sys
>>> print(sys.getdefaultencoding())
utf-8
>>> print(sys.stdout.encoding)
UTF-8

How do I modify stdout encoding to set it to UTF-8 in FreeBSD?

On Thu, Jan 29, 2015 at 1:38 AM, Robert Simmons <rsimmons0@gmail.com> wrote=
:
> I'm having a unicode problem on FreeBSD lang/python34 that does not
> appear on MacOS X. I've condensed the problem to one single line to
> enter in the interpreter:
>
> FreeBSD:
> Python 3.4.2 (default, Jan 28 2015, 22:23:57)
> [GCC 4.2.1 Compatible FreeBSD Clang 3.4.1 (tags/RELEASE_34/dot1-final
> 208032)] on freebsd10
> Type "help", "copyright", "credits" or "license" for more information.
>>>> b'\xc3\xa2'.decode('utf-8')
> '\xe2'
>
> MacOS X:
> Python 3.4.2 (default, Oct 19 2014, 17:55:38)
> [GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.54)] on darwin
> Type "help", "copyright", "credits" or "license" for more information.
>>>> b'\xc3\xa2'.decode('utf-8')
> '=C3=A2'
>
> Why is Python on FreeBSD incorrectly decoding this?