Date: Tue, 7 Nov 2017 20:09:08 +0100 From: Polytropon <freebsd@edvax.de> To: byrnejb@harte-lyne.ca Cc: freebsd-questions@freebsd.org Subject: Re: sed - remove nul lines from file Message-ID: <20171107200908.f9358f33.freebsd@edvax.de> In-Reply-To: <76aef2fd3792a0d9291b90cb74b6924f.squirrel@webmail.harte-lyne.ca> References: <b21bf201363c34a90ab55c4a05ff8fd7.squirrel@webmail.harte-lyne.ca> <20171107193652.7b0aa08f.freebsd@edvax.de> <76aef2fd3792a0d9291b90cb74b6924f.squirrel@webmail.harte-lyne.ca>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, 7 Nov 2017 13:54:41 -0500, James B. Byrne wrote: > > On Tue, November 7, 2017 13:36, Polytropon wrote: > > On Tue, 7 Nov 2017 12:12:55 -0500, James B. Byrne via > > freebsd-questions wrote: > >> I have a data file created by an ancient proprietary scripting > >> language called QTP. There is a bug in this program which, on > >> occasion, manifests itself by inserting output records consisting > >> entirely of nul (^@) (\x00) bytes at regular intervals. In the > >> present case every 47th. record consists entirely of nuls. > > > ... > > In this case, awk can also help: > > > > $ awk '(length > 0)' < infile.txt > outfile.txt > > > > This will print all lines which are longer than 0 characters. > > > > Thank you very much. This worked exactly as I required. > > I infer from this that awk does not consider nul a character and its > presence does not count towards the length of a record. Which is > counter intuitive to me. A nul takes up the same space as any other > character so why is it not counted? I would not have tried this > construction for that reason. Even though this example was actually meant for empty lines, i. e., those where the NULs have already been removed (for example with the tr -d command), but it seems that awk does actually ignores the NULs. Let's say this is the test input: foo bar ^@^@^@^@^@^@^@^@ baz meow When fed into the awk command mentioned above, the NULs are magically removed: $ awk '(length > 0)' < nul.txt foo bar baz meow This is an interesting behaviour, but fits the current problem quite well: It removes NULs and emoty lines. :-) -- Polytropon Magdeburg, Germany Happy FreeBSD user since 4.0 Andra moi ennepe, Mousa, ...
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20171107200908.f9358f33.freebsd>