Date: Sat, 11 Feb 2006 16:45:49 -0500 From: Parv <parv@pair.com> To: Kristian Vaaf <vaaf@broadpark.no> Cc: questions@freebsd.org Subject: Re: Script to clean text files Message-ID: <20060211214549.GA1674@holestein.holy.cow> In-Reply-To: <7.0.1.0.2.20060211172807.0214a4b8@broadpark.no> References: <7.0.1.0.2.20060211172807.0214a4b8@broadpark.no>
next in thread | previous in thread | raw e-mail | index | archive | help
in message <7.0.1.0.2.20060211172807.0214a4b8@broadpark.no>, wrote Kristian Vaaf thusly... > > > Among other things, this script is suppose to add an empty line at > the bottom of a file. > > But somehow it always removes the first line in a text file, > how do I stop this? Can you provide a small sample file complete w/ things that you want to remove? > #!/usr/local/bin/bash > # > # Remove CRLF, trailing whitespace and double lines. What are "double lines"? > # $ARBA: clean.sh,v 1.0 2007/11/11 15:09:05 vaaf Exp $ > # > for file in `find -s . -type f -not -name ".*"`; do > if file -b "$file" | grep -q 'text'; then > echo >> "$file" > perl -i -pe 's/\015$//' "$file" > perl -i -pe 's/[^\S\n]+$//g' "$file" Why do you have two perl runs? More importantly, you will remove anything which is not whitespace or not newline. That means, in the end, you should have a file filled w/ whitespace only. > > perl -pi -00 -e 1 "$file" > echo "$file: Done" > fi > done To remove CRLF, trailing whitespace, and 2 consecutive blank lines ... { tr -d '\r' < "$file" \ | sed -E -e 's/[[:space:]]+$//' \ | cat -s - > "${file}.tmp" } && mv -f "${file}.tmp" "$file" - Parv --
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20060211214549.GA1674>