Date: Mon, 18 Aug 2008 06:29:03 +0300 From: Giorgos Keramidas <keramida@ceid.upatras.gr> To: David Wolfskill <david@catwhisker.org> Cc: freebsd-questions@freebsd.org Subject: Re: Shell scripts: variable assignment within read loops Message-ID: <87ljyvypa8.fsf@kobe.laptop> In-Reply-To: <20080818013328.GY44815@bunrab.catwhisker.org> (David Wolfskill's message of "Sun, 17 Aug 2008 18:33:28 -0700") References: <20080818013328.GY44815@bunrab.catwhisker.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sun, 17 Aug 2008 18:33:28 -0700, David Wolfskill <david@catwhisker.org> wrote:
> I am writing a (Bourne) shell script that is intended (among other
> things) to obtain information from a command, such as:
>
> netstat -nibd -f inet
>
> by reading and parsing the output.
>
> However, the "obvious" (to me) approach of piping the output of the
> command to the standard input of a "while read ..." statement turns out
> to be not very useful; it appears that while
>
> foo=""
> while read bar ... ; do
> ...
> foo=$bar
> ...
> done <$filename
> echo $foo
>
> will assign to foo the value of the bar variable form the last record
> read (in FreeBSD 6.3-STABLE, at least), the following fails to do so:
>
> foo=""
> cat $filename | while read bar ... ; do
> ...
> foo=$bar
> ...
> done
> echo $foo
>
> Well, that's not *quite* accurate:the assignment is done all right, but
> in the latter case, it appears to be done in a subshell, so by the time
> we get to the "echo" statement, any variable assignments from within the
> read loop have vanished.
Hi David,
You are right that feeding data to a looping construct through a pipe
may run in a subshell. The ``Single UNIX Specification'' says
"... each command of a multi-command pipeline is in a subshell
environment; as an extension, however, any or all commands in a
pipeline may be executed in the current environment."
You can read the online text of the Single UNIX Specification at:
http://www.unix.org/single_unix_specification/
A simple 'registration' is required, and you need a browser that
supports cookies. But after that, you can follow the links to the shell
command language page at
http://www.opengroup.org/onlinepubs/000095399/utilities/xcu_chap02.html
Near section `` 2.12 Shell Execution Environment'' you can find the text
I quoted.
What I usually do in similar shell scripts is something like:
cat "${filename}" | sed -n -e '/foo/ s/bar/baz/' | \
xargs -n1 blah
This isn't exactly the same as assigning $foo to the results of the
loop, but you can also use:
foo=`cat $filename | while read bar ; do \
stuff ...
echo "$bar"
more stuff...
done`
> Now here's a copy of the in-development script:
>
> #! /bin/sh
>
> cmd="netstat -nibd -f inet"
> ctr=0
> clist=""
> hlist=`$cmd | head -1`
> for f in $hlist; do
> ctr=$(( $ctr + 1 ))
> eval c$ctr=\"$f\"
> eval h_$f=c$ctr
> done
> cmax=$ctr
>
> t_file=`mktemp /tmp/XXXXXXXXX`
> $cmd | tail +2 >$t_file
> while read $hlist dummy; do
> if [ "$Name" = "lo0" ]; then
> continue
> fi
> for f in $hlist; do
> eval val=\"\$$f\"
> case $val in
> -) eval ${f}_$Name=0;;
> *) eval ${f}_$Name="$val";;
> esac;
> done
> nics="$Name $nics";
> done</$t_file
> rm $t_file
> echo "(end) NICs: $nics"
> for n in $nics; do
> for f in $hlist; do
> eval "echo ${f}_$n: \$${f}_$n"
> done
> done
>
> exit 0
>
> And its output on my laptop:
>
> (end) NICs: ath0
> Name_ath0: ath0
> Mtu_ath0: 1500
> Network_ath0: 172.17
> Address_ath0: 172.17.1.30
> Ipkts_ath0: 725191
> Ierrs_ath0: 0
> Ibytes_ath0: 185144197
> Opkts_ath0: 821917
> Oerrs_ath0: 0
> Obytes_ath0: 74260936
> Coll_ath0: 0
> Drop_ath0: 0
>
> and (somewhat more interestingly) on my "firewall" machine:
>
> (end) NICs: dc0 de0 fxp0
> Name_dc0: dc0
> Mtu_dc0: 1500
> Network_dc0: 172.16.8/24
> Address_dc0: 172.16.8.1
> Ipkts_dc0: 2501577
> Ierrs_dc0: 0
> Ibytes_dc0: 215386153
> Opkts_dc0: 20269087
> Oerrs_dc0: 0
> Obytes_dc0: 2553930555
> Coll_dc0: 0
> Drop_dc0: 0
> Name_de0: de0
> Mtu_de0: 1500
> Network_de0: 63.193.123/24
> Address_de0: 63.193.123.122
> Ipkts_de0: 5936847
> Ierrs_de0: 0
> Ibytes_de0: 734092787
> Opkts_de0: 18557543
> Oerrs_de0: 0
> Obytes_de0: 2551089632
> Coll_de0: 0
> Drop_de0: 0
> Name_fxp0: fxp0
> Mtu_fxp0: 1500
> Network_fxp0: 172.17
> Address_fxp0: 172.17.0.1
> Ipkts_fxp0: 10013
> Ierrs_fxp0: 0
> Ibytes_fxp0: 1366082
> Opkts_fxp0: 1253115
> Oerrs_fxp0: 0
> Obytes_fxp0: 70429903
> Coll_fxp0: 0
> Drop_fxp0: 0
>
> As you see, I am circumventing the issue by writing to a transient
> file. In the intended application, the script is to be used to gather
> resource-utilization information; thus, I want its "footprint" to be
> smaller, rather than larger. Granted, in my case, I would be writing
> a tiny text file to a swap-backed tmpfs, but in production, I won't
> have the luxury of knowing that in advance: the intent is that the
> script must run on a minimal FreeBSD system, with no "ports" or other
> 3rd-party software installed.
>
> Is there some other -- possibly better -- way to do this (using Bourne
> shell scripting)?
Ah, that's much better. Now I see what you are trying to do. Would you
be ok with an awk(1) script instead of /bin/sh? It tends to be nicer
for this sort of thing, i.e.:
$ expand david.awk | cat -n
1 #
2 # Gather the field names if this is a header-line.
3 #
4 $0 ~ /^Name/ {
5 for (k = 1; k <= NF; k++)
6 tag[k] = $k;
7 }
8
9 #
10 # For all other lines, just print the tagged field values.
11 #
12 $0 !~ /^Name/ {
13 name = $1;
14 for (k = 1; k <= NF; k++) {
15 if ($k == "-")
16 $k = "0";
17 printf "%s_%s: %s\n", tag[k], name, $k;
18 }
19 }
$ netstat -nibd -f inet | awk -f david.awk
Name_re0: re0
Mtu_re0: 1500
Network_re0: 192.168.1.0/2
Address_re0: 192.168.1.3
Ipkts_re0: 1672873
Ierrs_re0: 0
Ibytes_re0: 1411912899
Opkts_re0: 1418782
Oerrs_re0: 0
Obytes_re0: 948973554
Coll_re0: 0
Drop_re0: 0
Name_lo0: lo0
Mtu_lo0: 16384
Network_lo0: 127.0.0.0/8
Address_lo0: 127.0.0.1
Ipkts_lo0: 9019
Ierrs_lo0: 0
Ibytes_lo0: 2835806
Opkts_lo0: 9019
Oerrs_lo0: 0
Obytes_lo0: 2835806
Coll_lo0: 0
Drop_lo0: 0
With a bit of preprocessing, it may be possible to extract the network
names and print the "(end) NICs: XXX XXX" part too.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?87ljyvypa8.fsf>