From owner-freebsd-questions@FreeBSD.ORG Wed Jul 21 23:03:53 2010 Return-Path: Delivered-To: FreeBSD-Questions@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2B9A31065672 for ; Wed, 21 Jul 2010 23:03:53 +0000 (UTC) (envelope-from cswiger@mac.com) Received: from asmtpout028.mac.com (asmtpout028.mac.com [17.148.16.103]) by mx1.freebsd.org (Postfix) with ESMTP id 139C58FC12 for ; Wed, 21 Jul 2010 23:03:52 +0000 (UTC) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; charset=us-ascii Received: from cswiger1.apple.com ([17.209.4.71]) by asmtp028.mac.com (Sun Java(tm) System Messaging Server 6.3-8.01 (built Dec 16 2008; 32bit)) with ESMTPSA id <0L5X00KXIK2BNX60@asmtp028.mac.com> for FreeBSD-Questions@FreeBSD.org; Wed, 21 Jul 2010 16:03:47 -0700 (PDT) X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 ipscore=0 phishscore=0 bulkscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx engine=6.0.2-1004200000 definitions=main-1007210103 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.0.10011,1.0.148,0.0.0000 definitions=2010-07-21_09:2010-07-21, 2010-07-21, 1970-01-01 signatures=0 From: Chuck Swiger In-reply-to: <20100721223445.GA44260@Grumpy.DynDNS.org> Date: Wed, 21 Jul 2010 16:03:46 -0700 Message-id: References: <20100721223445.GA44260@Grumpy.DynDNS.org> To: David Kelly X-Mailer: Apple Mail (2.1081) Cc: FreeBSD-Questions@FreeBSD.org Subject: Re: Strip high bit from text? X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Jul 2010 23:03:53 -0000 Hi, all-- On Jul 21, 2010, at 3:34 PM, David Kelly wrote: > I regularly get email from a Blackberry user which my ISP then adds > this header, "X-MIME-Autoconverted: from quoted-printable to 8bit". So > far so good but the result always contains a number of 0xa0's in places > a plain old space belongs. Mutt/vim renders these as "?" making a > complete mess of things. Those are Unicode/UTF-8 non-breaking-space characters, generally coming from HTML-formatted email. > Already use procmail so adding an automatic filter should not be > difficult if only I can come up with on. > > Tried "tr \240 ' ' < testfile | hd" and was not able to change the 0xa0 > into anything. Have already spent much more time trying to make tr or > sed do the job than it would have taken to knock something out in C, but > I think there should be something laying around already in the base > system to perform this task. > > Suggestions? Repair the email while procmail has it? Reconfigure mutt and/or vim? If you've got procmail in the loop already, then calling iconv as a filter like so: iconv -f utf-8 -t ascii ...is likely to help. Another choice would be to switch to using a MIME+Unicode/UTF-8 aware mail reader. Regards, -- -Chuck