From owner-freebsd-questions@freebsd.org Tue Nov 7 18:54:45 2017 Return-Path: Delivered-To: freebsd-questions@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 62D38E61F1E for ; Tue, 7 Nov 2017 18:54:45 +0000 (UTC) (envelope-from byrnejb@harte-lyne.ca) Received: from inet08.hamilton.harte-lyne.ca (inet08.hamilton.harte-lyne.ca [216.185.71.28]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "inet08.hamilton.harte-lyne.ca", Issuer "CA_HLL_ISSUER_2016" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 2F7DD8078B for ; Tue, 7 Nov 2017 18:54:44 +0000 (UTC) (envelope-from byrnejb@harte-lyne.ca) Received: from localhost (localhost [127.0.0.1]) by inet08.hamilton.harte-lyne.ca (Postfix) with ESMTP id 36E4E622A3; Tue, 7 Nov 2017 13:54:43 -0500 (EST) X-Virus-Scanned: amavisd-new at harte-lyne.ca Received: from inet08.hamilton.harte-lyne.ca ([127.0.0.1]) by localhost (inet08.hamilton.harte-lyne.ca [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tWHrozEr3K77; Tue, 7 Nov 2017 13:54:41 -0500 (EST) Received: from webmail.harte-lyne.ca (inet04.hamilton.harte-lyne.ca [216.185.71.24]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by inet08.hamilton.harte-lyne.ca (Postfix) with ESMTPSA id 4FA2562223; Tue, 7 Nov 2017 13:54:40 -0500 (EST) Received: from 216.185.71.44 (SquirrelMail authenticated user byrnejb_hll) by webmail.harte-lyne.ca with HTTP; Tue, 7 Nov 2017 13:54:41 -0500 Message-ID: <76aef2fd3792a0d9291b90cb74b6924f.squirrel@webmail.harte-lyne.ca> In-Reply-To: <20171107193652.7b0aa08f.freebsd@edvax.de> References: <20171107193652.7b0aa08f.freebsd@edvax.de> Date: Tue, 7 Nov 2017 13:54:41 -0500 Subject: Re: sed - remove nul lines from file From: "James B. Byrne" To: "Polytropon" Cc: freebsd-questions@freebsd.org Reply-To: byrnejb@harte-lyne.ca User-Agent: SquirrelMail/1.4.22-5.el6 MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Nov 2017 18:54:45 -0000 On Tue, November 7, 2017 13:36, Polytropon wrote: > On Tue, 7 Nov 2017 12:12:55 -0500, James B. Byrne via > freebsd-questions wrote: >> I have a data file created by an ancient proprietary scripting >> language called QTP. There is a bug in this program which, on >> occasion, manifests itself by inserting output records consisting >> entirely of nul (^@) (\x00) bytes at regular intervals. In the >> present case every 47th. record consists entirely of nuls. > ... > In this case, awk can also help: > > $ awk '(length > 0)' < infile.txt > outfile.txt > > This will print all lines which are longer than 0 characters. > Thank you very much. This worked exactly as I required. I infer from this that awk does not consider nul a character and its presence does not count towards the length of a record. Which is counter intuitive to me. A nul takes up the same space as any other character so why is it not counted? I would not have tried this construction for that reason. -- *** e-Mail is NOT a SECURE channel *** Do NOT transmit sensitive data via e-Mail Do NOT open attachments nor follow links sent by e-Mail James B. Byrne mailto:ByrneJB@Harte-Lyne.ca Harte & Lyne Limited http://www.harte-lyne.ca 9 Brockley Drive vox: +1 905 561 1241 Hamilton, Ontario fax: +1 905 561 0757 Canada L8E 3C3