From owner-freebsd-questions@FreeBSD.ORG Mon Aug 18 03:30:51 2008 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 88E57106566B for ; Mon, 18 Aug 2008 03:30:51 +0000 (UTC) (envelope-from keramida@ceid.upatras.gr) Received: from igloo.linux.gr (igloo.linux.gr [62.1.205.36]) by mx1.freebsd.org (Postfix) with ESMTP id 210F18FC19 for ; Mon, 18 Aug 2008 03:30:50 +0000 (UTC) (envelope-from keramida@ceid.upatras.gr) Received: from kobe.laptop (adsl144-198.kln.forthnet.gr [195.74.243.198]) (authenticated bits=128) by igloo.linux.gr (8.14.3/8.14.3/Debian-5) with ESMTP id m7I3T4fM016329 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Mon, 18 Aug 2008 06:29:10 +0300 Received: from kobe.laptop (kobe.laptop [127.0.0.1]) by kobe.laptop (8.14.2/8.14.2) with ESMTP id m7I3T43w048101; Mon, 18 Aug 2008 06:29:04 +0300 (EEST) (envelope-from keramida@ceid.upatras.gr) Received: (from keramida@localhost) by kobe.laptop (8.14.2/8.14.2/Submit) id m7I3T3Dw048100; Mon, 18 Aug 2008 06:29:03 +0300 (EEST) (envelope-from keramida@ceid.upatras.gr) From: Giorgos Keramidas To: David Wolfskill References: <20080818013328.GY44815@bunrab.catwhisker.org> Date: Mon, 18 Aug 2008 06:29:03 +0300 In-Reply-To: <20080818013328.GY44815@bunrab.catwhisker.org> (David Wolfskill's message of "Sun, 17 Aug 2008 18:33:28 -0700") Message-ID: <87ljyvypa8.fsf@kobe.laptop> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.0.60 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-MailScanner-ID: m7I3T4fM016329 X-Hellug-MailScanner: Found to be clean X-Hellug-MailScanner-SpamCheck: not spam, SpamAssassin (not cached, score=-3.707, required 5, autolearn=not spam, ALL_TRUSTED -1.80, AWL 0.69, BAYES_00 -2.60) X-Hellug-MailScanner-From: keramida@ceid.upatras.gr X-Spam-Status: No Cc: freebsd-questions@freebsd.org Subject: Re: Shell scripts: variable assignment within read loops X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Aug 2008 03:30:51 -0000 On Sun, 17 Aug 2008 18:33:28 -0700, David Wolfskill wrote: > I am writing a (Bourne) shell script that is intended (among other > things) to obtain information from a command, such as: > > netstat -nibd -f inet > > by reading and parsing the output. > > However, the "obvious" (to me) approach of piping the output of the > command to the standard input of a "while read ..." statement turns out > to be not very useful; it appears that while > > foo="" > while read bar ... ; do > ... > foo=$bar > ... > done <$filename > echo $foo > > will assign to foo the value of the bar variable form the last record > read (in FreeBSD 6.3-STABLE, at least), the following fails to do so: > > foo="" > cat $filename | while read bar ... ; do > ... > foo=$bar > ... > done > echo $foo > > Well, that's not *quite* accurate:the assignment is done all right, but > in the latter case, it appears to be done in a subshell, so by the time > we get to the "echo" statement, any variable assignments from within the > read loop have vanished. Hi David, You are right that feeding data to a looping construct through a pipe may run in a subshell. The ``Single UNIX Specification'' says "... each command of a multi-command pipeline is in a subshell environment; as an extension, however, any or all commands in a pipeline may be executed in the current environment." You can read the online text of the Single UNIX Specification at: http://www.unix.org/single_unix_specification/ A simple 'registration' is required, and you need a browser that supports cookies. But after that, you can follow the links to the shell command language page at http://www.opengroup.org/onlinepubs/000095399/utilities/xcu_chap02.html Near section `` 2.12 Shell Execution Environment'' you can find the text I quoted. What I usually do in similar shell scripts is something like: cat "${filename}" | sed -n -e '/foo/ s/bar/baz/' | \ xargs -n1 blah This isn't exactly the same as assigning $foo to the results of the loop, but you can also use: foo=`cat $filename | while read bar ; do \ stuff ... echo "$bar" more stuff... done` > Now here's a copy of the in-development script: > > #! /bin/sh > > cmd="netstat -nibd -f inet" > ctr=0 > clist="" > hlist=`$cmd | head -1` > for f in $hlist; do > ctr=$(( $ctr + 1 )) > eval c$ctr=\"$f\" > eval h_$f=c$ctr > done > cmax=$ctr > > t_file=`mktemp /tmp/XXXXXXXXX` > $cmd | tail +2 >$t_file > while read $hlist dummy; do > if [ "$Name" = "lo0" ]; then > continue > fi > for f in $hlist; do > eval val=\"\$$f\" > case $val in > -) eval ${f}_$Name=0;; > *) eval ${f}_$Name="$val";; > esac; > done > nics="$Name $nics"; > done rm $t_file > echo "(end) NICs: $nics" > for n in $nics; do > for f in $hlist; do > eval "echo ${f}_$n: \$${f}_$n" > done > done > > exit 0 > > And its output on my laptop: > > (end) NICs: ath0 > Name_ath0: ath0 > Mtu_ath0: 1500 > Network_ath0: 172.17 > Address_ath0: 172.17.1.30 > Ipkts_ath0: 725191 > Ierrs_ath0: 0 > Ibytes_ath0: 185144197 > Opkts_ath0: 821917 > Oerrs_ath0: 0 > Obytes_ath0: 74260936 > Coll_ath0: 0 > Drop_ath0: 0 > > and (somewhat more interestingly) on my "firewall" machine: > > (end) NICs: dc0 de0 fxp0 > Name_dc0: dc0 > Mtu_dc0: 1500 > Network_dc0: 172.16.8/24 > Address_dc0: 172.16.8.1 > Ipkts_dc0: 2501577 > Ierrs_dc0: 0 > Ibytes_dc0: 215386153 > Opkts_dc0: 20269087 > Oerrs_dc0: 0 > Obytes_dc0: 2553930555 > Coll_dc0: 0 > Drop_dc0: 0 > Name_de0: de0 > Mtu_de0: 1500 > Network_de0: 63.193.123/24 > Address_de0: 63.193.123.122 > Ipkts_de0: 5936847 > Ierrs_de0: 0 > Ibytes_de0: 734092787 > Opkts_de0: 18557543 > Oerrs_de0: 0 > Obytes_de0: 2551089632 > Coll_de0: 0 > Drop_de0: 0 > Name_fxp0: fxp0 > Mtu_fxp0: 1500 > Network_fxp0: 172.17 > Address_fxp0: 172.17.0.1 > Ipkts_fxp0: 10013 > Ierrs_fxp0: 0 > Ibytes_fxp0: 1366082 > Opkts_fxp0: 1253115 > Oerrs_fxp0: 0 > Obytes_fxp0: 70429903 > Coll_fxp0: 0 > Drop_fxp0: 0 > > As you see, I am circumventing the issue by writing to a transient > file. In the intended application, the script is to be used to gather > resource-utilization information; thus, I want its "footprint" to be > smaller, rather than larger. Granted, in my case, I would be writing > a tiny text file to a swap-backed tmpfs, but in production, I won't > have the luxury of knowing that in advance: the intent is that the > script must run on a minimal FreeBSD system, with no "ports" or other > 3rd-party software installed. > > Is there some other -- possibly better -- way to do this (using Bourne > shell scripting)? Ah, that's much better. Now I see what you are trying to do. Would you be ok with an awk(1) script instead of /bin/sh? It tends to be nicer for this sort of thing, i.e.: $ expand david.awk | cat -n 1 # 2 # Gather the field names if this is a header-line. 3 # 4 $0 ~ /^Name/ { 5 for (k = 1; k <= NF; k++) 6 tag[k] = $k; 7 } 8 9 # 10 # For all other lines, just print the tagged field values. 11 # 12 $0 !~ /^Name/ { 13 name = $1; 14 for (k = 1; k <= NF; k++) { 15 if ($k == "-") 16 $k = "0"; 17 printf "%s_%s: %s\n", tag[k], name, $k; 18 } 19 } $ netstat -nibd -f inet | awk -f david.awk Name_re0: re0 Mtu_re0: 1500 Network_re0: 192.168.1.0/2 Address_re0: 192.168.1.3 Ipkts_re0: 1672873 Ierrs_re0: 0 Ibytes_re0: 1411912899 Opkts_re0: 1418782 Oerrs_re0: 0 Obytes_re0: 948973554 Coll_re0: 0 Drop_re0: 0 Name_lo0: lo0 Mtu_lo0: 16384 Network_lo0: 127.0.0.0/8 Address_lo0: 127.0.0.1 Ipkts_lo0: 9019 Ierrs_lo0: 0 Ibytes_lo0: 2835806 Opkts_lo0: 9019 Oerrs_lo0: 0 Obytes_lo0: 2835806 Coll_lo0: 0 Drop_lo0: 0 With a bit of preprocessing, it may be possible to extract the network names and print the "(end) NICs: XXX XXX" part too.