From owner-freebsd-questions@freebsd.org Tue Nov 7 18:53:49 2017 Return-Path: Delivered-To: freebsd-questions@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 28A30E61DA2 for ; Tue, 7 Nov 2017 18:53:49 +0000 (UTC) (envelope-from yuripv@gmx.com) Received: from mout.gmx.net (mout.gmx.net [212.227.17.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9D85B804C1 for ; Tue, 7 Nov 2017 18:53:48 +0000 (UTC) (envelope-from yuripv@gmx.com) Received: from thor.xvoid.org ([62.183.126.160]) by mail.gmx.com (mrgmx101 [212.227.17.174]) with ESMTPSA (Nemesis) id 0M4ScS-1fAsSe0rOu-00ylO8; Tue, 07 Nov 2017 19:53:40 +0100 Subject: Re: sed - remove nul lines from file To: byrnejb@harte-lyne.ca Cc: freebsd-questions@freebsd.org References: <88a59a82-2902-9f63-0a94-bd23b910e7ad@gmx.com> <88b1870184a8810072fe503917cd86be.squirrel@webmail.harte-lyne.ca> <656b08b1-b79d-8e7e-27ec-e39d883f2355@gmx.com> From: Yuri Pankov Message-ID: <7fa1cb30-9a6f-a799-682b-1b352f1a1a58@gmx.com> Date: Tue, 7 Nov 2017 21:53:38 +0300 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:52.0) Gecko/20100101 Thunderbird/52.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:4i6HrDdHoFswdIPkDRotfROMFD9IwEnKlrJBjz3YpnpuqLxTIxF Tdo+NVPckWobKztjsA6Euft+zkYh8gDLXD2LtuZNcBZetGw+hyB0w/jzFW567JTaYovXZ4s MMBgrLRFepPpeZPFcHmrQ6aGE8qpXNnQFCa6COLetbN76sSkFoLdWNbRDi9fsGWAANjZ1cl rPLmGZgN1svoC1FLwTTzQ== X-UI-Out-Filterresults: notjunk:1;V01:K0:+Vj8t6qlD24=:mqyPKUKABKowbj9KUgGC2M ZxZdixo7TilFQS4FmcR5Uz6WQ3zaftprWajVyTcNkQPCXfGGYIlGAFQx+y5+PWZkrUkcsIno2 f/z7z7rkN6319Gk+ecgMCDGvmV5g1wXGIcdalfqKvm4MN/GF51Wy4tZ3uwfmtu3fjP3X+zwK3 /8+1PbaW+kEkbAibETYJdrfbg3J8NXDjtvJb07yTwKZgytaVOqyTlmMPABtSqF4PH3sXYmA4n jD7ciW7HnLcyD5Lmbjb3Qd4MlSysSnD1oaTbpVPlZd5vEFjdgfRyLcTP/41XKndw2UT3tSWEH J/ldeut2kw+hakOm62QDqHMterMTpgwDWwDs9oiaLEu5Rst/V0RaujdlExasOU7a9LzseTdJR ArADg7kh74M+FR8tiA84cjtf/eRHHVcIA/BqWesw/ITcrNExpN+NcMKNY784Pd1KknlmQHS1k eFh+v88C3gK3Q22YNscTUnSR/L5U6z9xM/Mys++NTSWnb89+qTvM1jSqJJaIVINqrRmR1oItf SX1LHu5boDfcmlydkMyZcV+j5TxPqwHQwlvLFPCBOP3UJUVidS9IG7QbxNh5RXUv6shExkULr 5lDBJb/7TXV1rK7UckDgREQOtXvmIR3yqyrpt0cY1LQaQadocqML1a5bPyhE2RFdNZQeKkViM ZpSIjKmIr3IZoPdhzLofVG3oU+bH2NPzrY/xdbk4OndASiVesJxKPCVce3caxYef3+vQJw2CY l6OVSvH+HBjc6VQYRa6HBdaqF3/VkYQ7i3aTye0Oc78pu/3820JXpeOsBwyOC8hXA+7aWBHT5 NwJu+iVkoOTxPIsDgClFCOWurObA8wg1EjvoIW17r2xenZBZbEmyL4K7roxlEh1Q831Crf1 X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Nov 2017 18:53:49 -0000 On Tue, 7 Nov 2017 13:46:44 -0500, James B Byrne Via Freebsd-questions wrote: > > On Tue, November 7, 2017 13:28, Yuri Pankov wrote: >> On Tue, 7 Nov 2017 21:20:40 +0300, Yuri Pankov wrote: >>> On Tue, 7 Nov 2017 13:14:08 -0500, James B Byrne Via >>> Freebsd-questions >>> wrote: >>>> >>>> On Tue, November 7, 2017 13:03, Yuri Pankov wrote: >>>> >>>>> >>>>> You want /d, not /g, to delete the *lines* which contain NUL >>>>> symbols >>>>> (that's what your subject line said). >>>>> >>>> >>>> Sigh. Thank you. That works. However, it also deletes any line >>>> that >>>> has even one NUL in it regardless of the presence of other non-nul >>>> characters on the line. >>>> >>>> What I wish to accomplish is to delete only the lines that are >>>> completely nul. I thought that this could be accomplished by >>>> prefacing the match sting with the start of line anchor ^ and >>>> ending >>>> it with the end of line anchor $ but this does not work as I >>>> expect. >>> >>> "[[.NUL.]]" is just a character specified by its collation name, so >>> treat as any other ordinary character: >>> >>> sed -E '/^[[.NUL.]]+$/d' INFILE > OUTFILE >>> >>> Need extended regexp here for '+' to work. >> >> >> Or, after looking at re_format(7), it could be written using BREs, >> your >> choice :-) >> >> sed '/^[[.NUL.]]\{1,\}$/d' >> > > sed '/^[[.NUL.]]\{0,\}$/d' INFILE > OUTFILE > > Which has no effect. OUTFILE and INFILE remain identical. I get the > exact same result from the first invocation as well. Likewise: > > sed '/^[[.NUL.]]\{1,\}$/d' INFILE > OUTFILE > > and > > sed -E '/^[[.NUL.]]+$/d' INFILE > OUTFILE > > # diff INFILE OUTFILE > # > # ll INFILE OUTFILE > -rw-r--r-- 1 root wheel 61480 Nov 7 13:09 INFILE > -rw-r--r-- 1 root wheel 61480 Nov 7 13:38 OUTFILE > > > I had actually tried these combinations, or at least I believe that I > tried these, before I wrote. Given the complexity and arcane nature > of whatever flavour of RE one is working with I may have transgressed > and written them slightly differently. But the examples you provided > me with give the results I obtained exactly. > Then there's probably something else on those lines with \0's as it works for me with a simple example created using the following command: printf "abc\0\n\0\0\nd\0ef\n" > 1