From owner-freebsd-questions Mon Feb 15 16:29:46 1999 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id QAA01234 for freebsd-questions-outgoing; Mon, 15 Feb 1999 16:29:46 -0800 (PST) (envelope-from owner-freebsd-questions@FreeBSD.ORG) Received: from post.mail.demon.net (finch-post-10.mail.demon.net [194.217.242.38]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id QAA01228 for ; Mon, 15 Feb 1999 16:29:39 -0800 (PST) (envelope-from marko@uk.radan.com) Received: from [158.152.75.22] (helo=uk.radan.com) by post.mail.demon.net with smtp (Exim 2.12 #1) id 10CYOH-0004MS-00; Tue, 16 Feb 1999 00:29:29 +0000 Organisation: Radan Computational Ltd., Bath, UK. Phone: +44-1225-320320 Fax: +44-1225-320311 Received: from marder-1. (rasnt-1 [193.114.228.211]) by uk.radan.com (8.6.10/8.6.10) with ESMTP id AAA00363; Tue, 16 Feb 1999 00:28:54 GMT Received: (from marko@localhost) by marder-1. (8.8.8/8.8.8) id AAA00407; Tue, 16 Feb 1999 00:27:03 GMT (envelope-from marko) Message-ID: <19990216002703.A337@localhost> Date: Tue, 16 Feb 1999 00:27:03 +0000 From: Mark Ovens To: Sue Blake Cc: questions@FreeBSD.ORG Subject: Re: cleaning a text file References: <19990215201056.19929@welearn.com.au> <19990216095232.J2207@lemis.com> <19990216103740.60271@welearn.com.au> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 0.93.2i In-Reply-To: <19990216103740.60271@welearn.com.au>; from Sue Blake on Tue, Feb 16, 1999 at 10:37:40AM +1100 Sender: owner-freebsd-questions@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Tue, Feb 16, 1999 at 10:37:40AM +1100, Sue Blake wrote: > On Tue, Feb 16, 1999 at 09:52:32AM +1030, Greg Lehey wrote: > > On Monday, 15 February 1999 at 1:10:36 -0800, rick hamell wrote: > > > > > >> Also, this file has some very long lines which would get truncated > > >> or unexpectedly wrapped when sent as email. And if there is something > > >> strange, I have to read it and guess what it should have been. > > >> > > >> Maybe someone will come up with something for this particular case. > > >> I can't believe there's not some little untility for this that's been > > >> hanging around unloved for years. > > > > > > Oy! Ok... how does Greg reformat all those emails? > > > > With Emacs. I have a collection of macros which I'm constantly > > changing to catch up with new tricks that mailers discover. > > > > To Sue's original question: it depends on what your text looks like. > > tr(1) will remove characters if you ask it to. > > If I knew which characters were there (so I could ask tr to remove > them) I would have already removed them with my text editor. > > > fmt(1) might be useful for wrapping lines. > > I don't see the long line lengths as a big problem at this stage, but > fmt might be useful later. > > The problem is that I don't know which funny characters exist in the > file, if any. I want to find out what they are, so I can search for > them and eyeball them before killing them. > > > Just knowing which characters they are would give me many solutions > immediately. There still doesn't seem to be a way to find this out :-( > First you need to identify the offending characters. Use od(1) or hexdump(1) to identify them and then work out a filter. Are they all extended ASCII (>127) chars? or are some of them control (<32) chars?. You could possibly use awk(1) as a filter, or write a simple C prog using issprint() and isspace(). HTH > Maybe there's a long way... somehow put a linefeed after each character > in the file (with sed?) and then sort it and look at the top and bottom > of the sorted file. > > -- > > Regards, > -*Sue*- > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-questions" in the body of the message > -- FreeBSD - The Power To Serve http://www.freebsd.org My Webpage http://www.users.globalnet.co.uk/~markov _______________________________________________________________ Mark Ovens, CNC Apps Engineer, Radan Computational Ltd. Bath UK CAD/CAM solutions for Sheetmetal Working Industry mailto:marko@uk.radan.com http://www.radan.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-questions" in the body of the message