From owner-freebsd-python@FreeBSD.ORG Thu Jan 29 09:53:34 2015
Return-Path: <owner-freebsd-python@FreeBSD.ORG>
Delivered-To: freebsd-python@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
[IPv6:2001:1900:2254:206a::19:1])
(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
(No client certificate requested)
by hub.freebsd.org (Postfix) with ESMTPS id 02DCAF98;
Thu, 29 Jan 2015 09:53:34 +0000 (UTC)
Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1])
(using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
(Client did not present a certificate)
by mx1.freebsd.org (Postfix) with ESMTPS id 680E0EB9;
Thu, 29 Jan 2015 09:53:33 +0000 (UTC)
Received: from tom.home (kostik@localhost [127.0.0.1])
by kib.kiev.ua (8.14.9/8.14.9) with ESMTP id t0T9rSO6089542
(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO);
Thu, 29 Jan 2015 11:53:28 +0200 (EET)
(envelope-from kostikbel@gmail.com)
DKIM-Filter: OpenDKIM Filter v2.9.2 kib.kiev.ua t0T9rSO6089542
Received: (from kostik@localhost)
by tom.home (8.14.9/8.14.9/Submit) id t0T9rSxO089541;
Thu, 29 Jan 2015 11:53:28 +0200 (EET)
(envelope-from kostikbel@gmail.com)
X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com
using -f
Date: Thu, 29 Jan 2015 11:53:28 +0200
From: Konstantin Belousov <kostikbel@gmail.com>
To: Kubilay Kocak <koobs@FreeBSD.org>
Subject: Re: Unicode Problem
Message-ID: <20150129095328.GQ42409@kib.kiev.ua>
References: <CA+QLa9DnD4mYSBc3B_SUZ6shH64v8Lej_CMMVaK=s+PrQp5Gvw@mail.gmail.com>
<CA+QLa9Amvu+1hg4Oo5J_CjoJNF2Ahs6LAkPrOAEncNU_iWXEtQ@mail.gmail.com>
<54C9FE33.2070307@FreeBSD.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <54C9FE33.2070307@FreeBSD.org>
User-Agent: Mutt/1.5.23 (2014-03-12)
X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00,
DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no
autolearn_force=no version=3.4.0
X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on tom.home
Cc: Robert Simmons <rsimmons0@gmail.com>, freebsd-python@freebsd.org
X-BeenThere: freebsd-python@freebsd.org
X-Mailman-Version: 2.1.18-1
Precedence: list
List-Id: FreeBSD-specific Python issues <freebsd-python.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-python>,
<mailto:freebsd-python-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-python/>
List-Post: <mailto:freebsd-python@freebsd.org>
List-Help: <mailto:freebsd-python-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-python>,
<mailto:freebsd-python-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 29 Jan 2015 09:53:34 -0000
On Thu, Jan 29, 2015 at 08:32:35PM +1100, Kubilay Kocak wrote:
> On 29/01/2015 6:13 PM, Robert Simmons wrote:
> > On further inspection I've found the following:
> >
> > FreeBSD
> >>>> import sys
> >>>> print(sys.getdefaultencoding())
> > utf-8
> >>>> print(sys.stdout.encoding)
> > US-ASCII
> >
> > MacOS X:
> >>>> import sys
> >>>> print(sys.getdefaultencoding())
> > utf-8
> >>>> print(sys.stdout.encoding)
> > UTF-8
> >
> > How do I modify stdout encoding to set it to UTF-8 in FreeBSD?
>
> Another data point from my 9-STABLE:
>
> Python 3.4.2 (default, Nov 3 2014, 13:38:18)
> [GCC 4.2.1 Compatible FreeBSD Clang 3.4.1 (tags/RELEASE_34/dot1-final
> 208032)] on freebsd9
> Type "help", "copyright", "credits" or "license" for more information.
> >>> b'\xc3\xa2'.decode('utf-8')
> '??'
> >>> import sys
> >>> print(sys.getdefaultencoding())
> utf-8
> >>> print(sys.stdout.encoding)
> UTF-8
> >>>
>
> Python 2.7.9 (default, Jan 24 2015, 20:39:40)
> [GCC 4.2.1 Compatible FreeBSD Clang 3.4.1 (tags/RELEASE_34/dot1-final
> 208032)] on freebsd9
> Type "help", "copyright", "credits" or "license" for more information.
> >>> b'\xc3\xa2'.decode('utf-8')
> u'\xe2'
> >>> import sys
> >>> print(sys.getdefaultencoding())
> ascii
> >>> print(sys.stdout.encoding)
> UTF-8
> >>>
Encoding of the sys.stdout is determined by the language environment.
Set LANG environment variable to something like en_US.UTF-8 to get
UTF-8 output from python.
The 'data points' about python version, FreeBSD version, compiler
version, colour of the enclosure for hard disk used, or even phase of
the Moon have lesser impact on the sys.stdout.encoding.