Date: Tue, 7 Nov 2017 15:22:27 -0600 From: Noel <noeldude@gmail.com> To: freebsd-questions@freebsd.org Subject: Re: sed - remove nul lines from file Message-ID: <5ca5d4fd-122b-814d-5632-553c497b25ec@gmail.com> In-Reply-To: <20171107200908.f9358f33.freebsd@edvax.de> References: <b21bf201363c34a90ab55c4a05ff8fd7.squirrel@webmail.harte-lyne.ca> <20171107193652.7b0aa08f.freebsd@edvax.de> <76aef2fd3792a0d9291b90cb74b6924f.squirrel@webmail.harte-lyne.ca> <20171107200908.f9358f33.freebsd@edvax.de>
next in thread | previous in thread | raw e-mail | index | archive | help
On 11/7/2017 1:09 PM, Polytropon wrote: > On Tue, 7 Nov 2017 13:54:41 -0500, James B. Byrne wrote: >> On Tue, November 7, 2017 13:36, Polytropon wrote: >>> On Tue, 7 Nov 2017 12:12:55 -0500, James B. Byrne via >>> freebsd-questions wrote: >>>> I have a data file created by an ancient proprietary scripting >>>> language called QTP. There is a bug in this program which, on >>>> occasion, manifests itself by inserting output records consisting >>>> entirely of nul (^@) (\x00) bytes at regular intervals. In the >>>> present case every 47th. record consists entirely of nuls. >> ... >>> In this case, awk can also help: >>> >>> $ awk '(length > 0)' < infile.txt > outfile.txt >>> >>> This will print all lines which are longer than 0 characters. >>> >> Thank you very much. This worked exactly as I required. >> >> I infer from this that awk does not consider nul a character and its >> presence does not count towards the length of a record. Which is >> counter intuitive to me. A nul takes up the same space as any other >> character so why is it not counted? I would not have tried this >> construction for that reason. > Even though this example was actually meant for empty lines, > i. e., those where the NULs have already been removed (for > example with the tr -d command), but it seems that awk does > actually ignores the NULs. > > Let's say this is the test input: > > foo > bar > ^@^@^@^@^@^@^@^@ > baz > meow > > When fed into the awk command mentioned above, the NULs are > magically removed: > > $ awk '(length > 0)' < nul.txt > foo > bar > baz > meow > > This is an interesting behaviour, but fits the current problem > quite well: It removes NULs and emoty lines. :-) > > > > > I'd probably just use grep. grep '^[[:print:]]' INFILE > OUTFILE ie. any line that starts with a printable character is copied to OUTFILE. This will skip all-null and empty lines.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?5ca5d4fd-122b-814d-5632-553c497b25ec>