Date: Fri, 27 May 2005 14:49:08 -0700 From: Ted Faber <faber@isi.edu> To: freebsd-current@freebsd.org Cc: Peter Jeremy <PeterJeremy@optushome.com.au> Subject: amd/nfs deadlock on -current w/example code (was Re: hard deadlock(?) on -current; some debugging info, need help) Message-ID: <20050527214908.GA1031@pun.isi.edu> In-Reply-To: <20050527204302.GC18914@cirb503493.alcatel.com.au> References: <20050526001806.GA1008@pun.isi.edu> <20050526080928.GE12640@cirb503493.alcatel.com.au> <20050526160846.GA6851@pun.isi.edu> <20050526203243.GB1055@pun.isi.edu> <20050527083734.GA18696@cirb503493.alcatel.com.au> <20050527152752.GA10069@pun.isi.edu> <20050527204302.GC18914@cirb503493.alcatel.com.au>
next in thread | previous in thread | raw e-mail | index | archive | help
--4jXrM3lyYWu4nBt5 Content-Type: multipart/mixed; boundary="gj572EiMnwbLXET9" Content-Disposition: inline --gj572EiMnwbLXET9 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable With a bunch of help from Peter Jeremy, I've been able to narrow my deadlock problem down to a repeatable case. If you're just tuning in, -current running amd and nfs deadlocks to the point where panic or call doadump from the debugger fails to generate a crash dump. I think that the included code and configs will let you cause this on your own machine. The scenario is this: amd mounted NFS directory reasonable read/write load from different processes eventually the machine deadlocks Amd configuration files and test code are attached. On my machines, compiling loadit and running loadit 35 /nfs/jade/faber (which just happens to be an nfs directory I use) locks up the system repeatably. loadit just forks n processes (arg1) to create files called lock${n} in the given directory (arg2). There's no locking done, but I named the files when I thought locking might be at issue. Each process accesses only its own lock file and makes some small writes/reads to it 10000 times. It seems to matter that the amd is managing multiple maps, though I only touch the /nfs one. I couldn't get it to lock only loading the nfs map. The amd configurations and code are attached. in etc/rc.conf I do: amd_enable=3D"YES" amd_flags=3D"-F /usr/local/etc/amd.conf" I'm happy to supply more data, file a pr, test patches or anything. I'm also working around by statically mounting the most common filesystems I use. Thanks again to Peter for holding my hand until I could get a repeatable case. --=20 Ted Faber http://www.isi.edu/~faber PGP: http://www.isi.edu/~faber/pubkeys.= asc Unexpected attachment on this mail? See http://www.isi.edu/~faber/FAQ.html#= SIG --gj572EiMnwbLXET9 Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="amd.conf" [global] nfs_proto = udp auto_dir = /auto log_file = syslog restart_mounts = yes [/nfs] map_name = /usr/local/etc/amd.test [/isi] map_name = /usr/local/etc/amd.isi [/home] map_name = /usr/local/etc/amd.home [/removable] map_name = /usr/local/etc/amd.removable --gj572EiMnwbLXET9 Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="amd.test" Content-Transfer-Encoding: quoted-printable # -- IMPORTANT NOTICE -- # # The master version of this file is kept on VENERA.ISI.EDU # You *must* make all modifications to this file on that host. # It will automatically update the other servers overnight. # You may then have to run "amq -f" to reset AMD. # =20 # Your cooperation in this matter is greatly appreciated!!! # /defaults type:=3Dnfs;fs:=3D${autodir}/${rhost}/${key};rfs:=3D${autodir}/${= rhost}/${key};opts:=3Dtimeo=3D8,retrans=3D9,intr jade rhost:=3Dboreas --gj572EiMnwbLXET9 Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="amd.home" ravichan type:=link;fs:="/nfs/isd3/ravichan" mlievens type:=link;fs:="/nfs/ivory/mlievens" fukumoto type:=link;fs:="/nfs/isd2/fukumoto" carolina type:=link;fs:="/nfs/div2/carolina" scoyazo type:=link;fs:="/nfs/ipc/scoyazo" jwhalen type:=link;fs:="/nfs/neon/jwhalen" insley type:=link;fs:="/nfs/guest/insley" bsmith type:=link;fs:="/nfs/m_usr/bsmith" blythe type:=link;fs:="/nfs/isd/blythe" apache type:=link;fs:="/var/www" melz type:=link;fs:="/nfs/topaz/melz" workshop type:=link;fs:="/nfs/gren/workshop" spundun type:=link;fs:="/nfs/div1/spundun" smartin type:=link;fs:="/nfs/guest/smartin" oracle7 type:=link;fs:="/nfs/div2/oracle7" koranda type:=link;fs:="/nfs/guest/koranda" mwilde type:=link;fs:="/nfs/guest/mwilde" umdom type:=link;fs:="/nfs/ruby/umdom" puid2 type:=link;fs:="/none" faber type:=link;fs:="/usr/home/faber" thussain type:=link;fs:="/nfs/ivory/thussain" cmerchan type:=link;fs:="/nfs/isd2/cmerchan" christos type:=link;fs:="/nfs/gren/christos" melissa type:=link;fs:="/nfs/ivory/melissa" bhavsar type:=link;fs:="/nfs/guest/bhavsar" postel type:=link;fs:="/nfs/div7/postel" knight type:=link;fs:="/nfs/isd3/knight" kathyt type:=link;fs:="/nfs/asd2/kathyt" pipet type:=link;fs:="/nfs/topaz/pipet" ryankash type:=link;fs:="/nfs/isd/ryankash" kjohnson type:=link;fs:="/nfs/guest/kjohnson" alqahtan type:=link;fs:="/nfs/isd2/alqahtan" ziegler type:=link;fs:="/nfs/div1/ziegler" joechiu type:=link;fs:="/nfs/isd3/joechiu" jinbo type:=link;fs:="/nfs/div2/jinbo" juan type:=link;fs:="/nfs/guest/juan" bud_bkup type:=link;fs:="/" terrell type:=link;fs:="/nfs/v5/terrell" tania type:=link;fs:="/nfs/isd2/tania" yuri type:=link;fs:="/nfs/jade/yuri" wong type:=link;fs:="/nfs/guest/wong" shen type:=link;fs:="/nfs/isd3/shen" youngkan type:=link;fs:="/nfs/guest/youngkan" personne type:=link;fs:="/nfs/ivory/personne" jonesej type:=link;fs:="/nfs/ipc/jonesej" trang type:=link;fs:="/nfs/m_usr/trang" alanr type:=link;fs:="/nfs/ivory/alanr" web-ican type:=link;fs:="/" tshaulis type:=link;fs:="/nfs/guest/tshaulis" sridhar type:=link;fs:="/nfs/asd/sridhar" fleisch type:=link;fs:="/nfs/guest/fleisch" vsrini type:=link;fs:="/nfs/asd/vsrini" kakani type:=link;fs:="/nfs/isd2/kakani" mubin type:=link;fs:="/nfs/ivory/mubin" marcs type:=link;fs:="/nfs/topaz/marcs" arens type:=link;fs:="/nfs/topaz/arens" vay type:=link;fs:="/nfs/div1/vay" ulf type:=link;fs:="/nfs/isd/ulf" amarshal type:=link;fs:="/nfs/topaz/amarshal" web-rfc type:=link;fs:="/" pkamath type:=link;fs:="/nfs/jade/pkamath" gurmeet type:=link;fs:="/nfs/v6/gurmeet" eandre type:=link;fs:="/nfs/guest/eandre" arcada type:=link;fs:="/nfs/m_app1/arcada" amosis type:=link;fs:="/nfs/m_usr/amosis" weiye type:=link;fs:="/nfs/ruby/weiye" lane type:=link;fs:="/nfs/guest/lane" efab type:=link;fs:="/nfs/eisd/efab" bcn type:=link;fs:="/nfs/ruby/bcn" ohlander type:=link;fs:="/nfs/neon/ohlander" mwebster type:=link;fs:="/nfs/m_usr/mwebster" bmanning type:=link;fs:="/nfs/jade/bmanning" sfranke type:=link;fs:="/nfs/isd2/sfranke" pcadmin type:=link;fs:="/nfs/ipc/pcadmin" tjkwon type:=link;fs:="/nfs/div1/tjkwon" ndss00 type:=link;fs:="/nfs/jade/ndss00" iscapc type:=link;fs:="/nfs/v5/iscapc" mhall type:=link;fs:="/nfs/div1/mhall" jihie type:=link;fs:="/nfs/isd3/jihie" apachmos type:=link;fs:="/nfs/m_app1/apachmos" ars_adm type:=link;fs:="/usr/ar" maggon type:=link;fs:="/nfs/div2/maggon" koenig type:=link;fs:="/nfs/guest/koenig" nkwon type:=link;fs:="/nfs/topaz/nkwon" glenn type:=link;fs:="/nfs/m_usr/glenn" dhawe type:=link;fs:="/nfs/ipc/dhawe" cey type:=link;fs:="/nfs/jade/cey" informix type:=link;fs:="/nfs/isd/informix" ims_test type:=link;fs:="/nfs/m_prb1/probing/ims" gonzalez type:=link;fs:="/nfs/ipc/gonzalez" versant type:=link;fs:="/nfs/sevak-data/versant" hylafax type:=link;fs:="/var/spool/hylafax" rajeev type:=link;fs:="/nfs/guest/rajeev" soar-doc type:=link;fs:="/nfs/soar2/soar-doc" rstoker type:=link;fs:="/nfs/ivory/rstoker" diaz type:=link;fs:="/nfs/div7/diaz" touch-mm type:=link;fs:="/nfs/ruby/touch-mm" suifspec type:=link;fs:="/nfs/v5/suifspec" granacki type:=link;fs:="/nfs/v6/granacki" devmosis type:=link;fs:="/nfs/m_app1/devmosis" cacuser type:=link;fs:="/nfs/share/cacuser" xmosis type:=link;fs:="/nfs/m_app1/xmosis" ginoza type:=link;fs:="/nfs/gren/ginoza" timc type:=link;fs:="/nfs/isd/timc" xiw type:=link;fs:="/nfs/jade/xiw" iko type:=link;fs:="/nfs/guest/iko" cox type:=link;fs:="/nfs/m_usr/cox" web-usrx type:=link;fs:="/" prmsrv type:=link;fs:="/nfs/gost/products/prm" tyree type:=link;fs:="/nfs/m_usr/tyree" saltz type:=link;fs:="/nfs/neon/saltz" abocc type:=link;fs:="/nfs/jade/abocc" blau type:=link;fs:="/nfs/guest/blau" bgs type:=link;fs:="/nfs/guest/bgs" wdelorie type:=link;fs:="/nfs/m_usr/wdelorie" div10rpt type:=link;fs:="/nfs/neon/div10rpt" laura type:=link;fs:="/nfs/asd/laura" soar type:=link;fs:="/nfs/soar2/soar" delacruz type:=link;fs:="/nfs/gren/delacruz" web-bor type:=link;fs:="/" techlib type:=link;fs:="/nfs/ivory/techlib" cporter type:=link;fs:="/nfs/ivory/cporter" rwhois type:=link;fs:="/nfs/coms/rwhois" gmehta type:=link;fs:="/nfs/asd2/gmehta" dasher type:=link;fs:="/nfs/div2/dasher" kagy type:=link;fs:="/nfs/ipc/kagy" lwinston type:=link;fs:="/nfs/ivory/lwinston" chunchen type:=link;fs:="/nfs/div1/chunchen" germann type:=link;fs:="/nfs/guest/germann" web-bb type:=link;fs:="/" shetty type:=link;fs:="/nfs/v6/shetty" fstann type:=link;fs:="/nfs/jade/fstann" amyers type:=link;fs:="/nfs/isd2/amyers" noori type:=link;fs:="/nfs/guest/noori" alain type:=link;fs:="/nfs/v5/alain" mei type:=link;fs:="/nfs/asd/mei" penguest type:=link;fs:="/nfs/isd/penguest" sondeen type:=link;fs:="/nfs/div1/sondeen" hzhang3 type:=link;fs:="/nfs/guest/hzhang3" popadm type:=link;fs:="/tmp" draper type:=link;fs:="/nfs/div1/draper" d7east type:=link;fs:="/nfs/ruby/d7east" keith type:=link;fs:="/nfs/ivory/keith" och type:=link;fs:="/nfs/guest/och" web-clr2 type:=link;fs:="/" ruiwang type:=link;fs:="/nfs/ipc/ruiwang" muslea type:=link;fs:="/nfs/guest/muslea" action type:=link;fs:="/nfs/u2/action" marcu type:=link;fs:="/nfs/isd/marcu" brian type:=link;fs:="/nfs/gren/brian" webadmin type:=link;fs:="/tmp" rnelson type:=link;fs:="/nfs/ipc/rnelson" default type:=link;fs:="/nfs/u2/action" d7lists type:=link;fs:="/nfs/div7/d7lists" robots type:=link;fs:="/nfs/isd2/robots" craig type:=link;fs:="/nfs/ipc/craig" hlee type:=link;fs:="/nfs/topaz/hlee" sdr type:=link;fs:="/nfs/m_usr/sdr" asd type:=link;fs:="/nfs/asd/asd" mmguest type:=link;fs:="/nfs/conf/mmguest" div3rpt type:=link;fs:="/nfs/neon/div3rpt" rbates type:=link;fs:="/nfs/ipc/rbates" melody type:=link;fs:="/nfs/guest/melody" mieke type:=link;fs:="/nfs/div7/mieke" terrawor type:=link;fs:="/nfs/isd3/terrawor" smchenry type:=link;fs:="/nfs/div7/smchenry" in-notes type:=link;fs:="/nfs/ftp/in-notes" rapteam type:=link;fs:="/nfs/isd2/rapteam" erikabn type:=link;fs:="/nfs/topaz/erikabn" daemon type:=link;fs:="/" guillory type:=link;fs:="/nfs/isd2/guillory" facility type:=link;fs:="/nfs/ivory/facility" resumes type:=link;fs:="/nfs/ivory/resumes" dhollen type:=link;fs:="/nfs/m_usr/dhollen" web-ln type:=link;fs:="/" dlhawe type:=link;fs:="/nfs/ipc/dlhawe" boapps type:=link;fs:="/nfs/ivory/boapps" mason type:=link;fs:="/nfs/ipc/mason" carte type:=link;fs:="/nfs/amber/carte" preetham type:=link;fs:="/nfs/isd3/preetham" jamesrey type:=link;fs:="/nfs/ipc/jamesrey" ambastha type:=link;fs:="/nfs/guest/ambastha" oramos8 type:=link;fs:="/nfs/m_app1/oramos8" jonmay type:=link;fs:="/nfs/isd/jonmay" eiffel type:=link;fs:="/nfs/m_app1/eiffel" abebeh type:=link;fs:="/nfs/m_usr/abebeh" rfccp type:=link;fs:="/nfs/ruby/rfccp" difac type:=link;fs:="/nfs/ruby/difac" yuki type:=link;fs:="/nfs/isd2/yuki" hutchins type:=link;fs:="/nfs/guest/hutchins" schuler type:=link;fs:="/nfs/v5/schuler" probing type:=link;fs:="/nfs/m_prb1/probing" web-ntro type:=link;fs:="/" web-mman type:=link;fs:="/" vitanshu type:=link;fs:="/nfs/v6/vitanshu" nlsummer type:=link;fs:="/nfs/topaz/nlsummer" mmanpcen type:=link;fs:="/nfs/ruby/mmanpcen" omega type:=link;fs:="/nfs/isd2/omega" govindan type:=link;fs:="/nfs/div7/govindan" web-mon type:=link;fs:="/" valente type:=link;fs:="/nfs/topaz/valente" tmurray type:=link;fs:="/nfs/isd/tmurray" shahabi type:=link;fs:="/nfs/guest/shahabi" linnett type:=link;fs:="/nfs/neon/linnett" datatag type:=link;fs:="/nfs/asd/datatag" cpina type:=link;fs:="/nfs/m_usr/cpina" sims type:=link;fs:="/nfs/isd3/sims" ipc type:=link;fs:="/nfs/ipc/ipc" oramosis type:=link;fs:="/nfs/m_app1/oramosis" mhopkins type:=link;fs:="/nfs/isd/mhopkins" jdamoula type:=link;fs:="/nfs/guest/jdamoula" hfaxmail type:=link;fs:="/nfs/m_app1/hfaxmail" fhouston type:=link;fs:="/nfs/guest/fhouston" sharmaa type:=link;fs:="/nfs/div1/sharmaa" bschott type:=link;fs:="/nfs/guest/bschott" rzhou type:=link;fs:="/nfs/div7/rzhou" isl type:=link;fs:="/nfs/asd/asd" cad type:=link;fs:="/nfs/m_cad/cad" mboseman type:=link;fs:="/nfs/ivory/mboseman" jasapara type:=link;fs:="/nfs/guest/jasapara" hongsuda type:=link;fs:="/nfs/guest/hongsuda" imudom type:=link;fs:="/nfs/guest/imudom" hannes type:=link;fs:="/nfs/isd/hannes" barish type:=link;fs:="/nfs/guest/barish" jkrey type:=link;fs:="/nfs/jade/jkrey" teamhost type:=link;fs:="/nfs/amber/teamhost" isdguest type:=link;fs:="/nfs/isd/isdguest" mailman type:=link;fs:="/nfs/neon/mailman" mmconf type:=link;fs:="/nfs/conf/diamond/mmconf" ramya type:=link;fs:="/nfs/v6/ramya" mosis type:=link;fs:="/nfs/m_app1/mosis" hobbs type:=link;fs:="/nfs/topaz/hobbs" veda type:=link;fs:="/nfs/ivory/veda" mta type:=link;fs:="/" staruser type:=link;fs:="/nfs/ivory/staruser" maechlin type:=link;fs:="/nfs/asd2/maechlin" contract type:=link;fs:="/nfs/neon/contract" benbrown type:=link;fs:="/nfs/m_usr/benbrown" rneches type:=link;fs:="/nfs/div2/rneches" lindell type:=link;fs:="/nfs/guest/lindell" root type:=link;fs:="/" ragy type:=link;fs:="/nfs/guest/ragy" jair-ed type:=link;fs:="/nfs/isd2/jair-ed" lapin type:=link;fs:="/nfs/ivory/lapin" koehn type:=link;fs:="/nfs/guest/koehn" ih-mm type:=link;fs:="/nfs/ruby/ih-mm" zach type:=link;fs:="/nfs/m_usr/zach" gost type:=link;fs:="/nfs/gost" dkauchak type:=link;fs:="/nfs/guest/dkauchak" thakkar type:=link;fs:="/nfs/amber/thakkar" stefan type:=link;fs:="/nfs/topaz/stefan" frank type:=link;fs:="/nfs/guest/frank" dmkrx type:=link;fs:="/nfs/div2/dmkrx" acmos type:=link;fs:="/nfs/div2/acmos" gose type:=link;fs:="/nfs/guest/gose" wdb type:=link;fs:="/nfs/neon/wdb" ito type:=link;fs:="/nfs/isd2/ito" pynadath type:=link;fs:="/nfs/isd3/pynadath" whitney type:=link;fs:="/nfs/isd3/whitney" faxmail type:=link;fs:="/var/spool/faxmail" dartisi type:=link;fs:="/nfs/dart/dartisi" jbaker type:=link;fs:="/nfs/ipc/jbaker" fraser type:=link;fs:="/nfs/isd/fraser" div7ew type:=link;fs:="/nfs/ruby/div7ew" berger type:=link;fs:="/nfs/guest/berger" ning type:=link;fs:="/nfs/topaz/ning" carl type:=link;fs:="/nfs/v6/carl" lac type:=link;fs:="/nfs/isd/lac" comments type:=link;fs:="/nfs/jade/comments" ahonka type:=link;fs:="/nfs/m_usr/ahonka" falk type:=link;fs:="/nfs/jade/falk" gil type:=link;fs:="/nfs/isd/gil" shutdown type:=link;fs:="/tmp" web-iab type:=link;fs:="/" nobody type:=link;fs:="/" mariam type:=link;fs:="/nfs/isd3/mariam" leana type:=link;fs:="/nfs/gren/leana" astt type:=link;fs:="/nfs/soar2/astt" www type:=link;fs:="/nfs/web/htdocs" loomlist type:=link;fs:="/nfs/topaz/loomlist" rutland type:=link;fs:="/nfs/isd/rutland" lerman type:=link;fs:="/nfs/isd3/lerman" cjansa type:=link;fs:="/nfs/neon/cjansa" bbuser type:=link;fs:="/nfs/ipc/bbuser" d2adm type:=link;fs:="/nfs/eisd/d2adm" elai type:=link;fs:="/nfs/isd2/elai" mman-rfc type:=link;fs:="/nfs/ruby/mman-rfc" salemi type:=link;fs:="/nfs/div2/salemi" pantel type:=link;fs:="/nfs/isd3/pantel" puid4 type:=link;fs:="/none" cohen type:=link;fs:="/nfs/topaz/cohen" geoworld type:=link;fs:="/nfs/eisd/geoworld" pegasus type:=link;fs:="/nfs/asd2/pegasus" gganger type:=link;fs:="/nfs/m_usr/gganger" dolbear type:=link;fs:="/nfs/guest/dolbear" div2rpt type:=link;fs:="/nfs/neon/div2rpt" d7admin type:=link;fs:="/nfs/ruby/d7admin" yoonju type:=link;fs:="/nfs/asd2/yoonju" thiago type:=link;fs:="/nfs/topaz/thiago" thayer type:=link;fs:="/nfs/topaz/thayer" npckge type:=link;fs:="/nfs/v6/npckge" oasmosis type:=link;fs:="/nfs/m_app1/oasmosis" ns-users type:=link;fs:="/nfs/neon/ns-users" dceg_cvs type:=link;fs:="/nfs/div2/dceg_cvs" bbarrett type:=link;fs:="/nfs/guest/bbarrett" soluser type:=link;fs:="/nfs/ivory/soluser" matthew type:=link;fs:="/nfs/ivory/matthew" whiten type:=link;fs:="/nfs/isd2/whiten" girish type:=link;fs:="/nfs/isd2/girish" pedro type:=link;fs:="/nfs/div1/pedro" oard type:=link;fs:="/nfs/guest/oard" itf type:=link;fs:="/nfs/guest/itf" tstmosis type:=link;fs:="/nfs/m_app1/tstmosis" rosenblo type:=link;fs:="/nfs/neon/rosenblo" majordom type:=link;fs:="/nfs/infr/majordom" aserrano type:=link;fs:="/nfs/jade/aserrano" jlacoss type:=link;fs:="/nfs/v6/jlacoss" ariadne type:=link;fs:="/nfs/isd2/ariadne" qcao type:=link;fs:="/nfs/guest/qcao" esam type:=link;fs:="/nfs/isd2/esam" rfc-onli type:=link;fs:="/nfs/infr/rfc-onli" rramani type:=link;fs:="/nfs/div7/rramani" kintali type:=link;fs:="/nfs/div1/kintali" mdarcy type:=link;fs:="/nfs/v6/mdarcy" alau type:=link;fs:="/nfs/ivory/alau" usatlas1 type:=link;fs:="/nfs/asd/usatlas1" list-mgr type:=link;fs:="/nfs/infr/list-mgr" apopescu type:=link;fs:="/nfs/guest/apopescu" stellac type:=link;fs:="/nfs/topaz/stellac" steele type:=link;fs:="/nfs/asd/steele" nadim type:=link;fs:="/nfs/isd3/nadim" brad type:=link;fs:="/nfs/m_usr/brad" annc type:=link;fs:="/nfs/asd/annc" tex type:=link;fs:="/local/tex/lib/tex/macros" weathers type:=link;fs:="/nfs/isd2/weathers" nabbasi type:=link;fs:="/nfs/isd2/nabbasi" lsam type:=link;fs:="/nfs/conf/lsam" hans type:=link;fs:="/nfs/topaz/hans" knoblock type:=link;fs:="/nfs/isd3/knoblock" khoshnev type:=link;fs:="/nfs/neon/khoshnev" rbakshi type:=link;fs:="/nfs/guest/rbakshi" oraboff type:=link;fs:="/oracle/oraboff" johnson type:=link;fs:="/nfs/amber/johnson" isisrb type:=link;fs:="/nfs/asd2/isisrb" wygr type:=link;fs:="/nfs/ivory/wygr" will type:=link;fs:="/nfs/eisd/will" bso type:=link;fs:="/nfs/guest/bso" marsella type:=link;fs:="/nfs/amber/marsella" cescobar type:=link;fs:="/nfs/ivory/cescobar" isiconf type:=link;fs:="/nfs/conf/isiconf" ecannon type:=link;fs:="/nfs/isd2/ecannon" jerryc type:=link;fs:="/nfs/v5/jerryc" jcole type:=link;fs:="/nfs/guest/jcole" ivdgl type:=link;fs:="/nfs/asd/ivdgl" ldap-usrn type:=link;fs:="/" web-usrn type:=link;fs:="/" web-smos type:=link;fs:="/" nsnamsim type:=link;fs:="/nfs/ruby/nsnamsim" bosybase type:=link;fs:="/nfs/ivory/bosybase" icarmi type:=link;fs:="/nfs/div1/icarmi" cmah type:=link;fs:="/nfs/ivory/cmah" web-usrs type:=link;fs:="/" par_serv type:=link;fs:="/nfs/cad10/par_serv" mbrasser type:=link;fs:="/nfs/guest/mbrasser" lorraine type:=link;fs:="/nfs/ivory/lorraine" kkmercha type:=link;fs:="/nfs/guest/kkmercha" mcurry type:=link;fs:="/nfs/m_usr/mcurry" fujita type:=link;fs:="/nfs/jade/fujita" raghu type:=link;fs:="/nfs/ruby/raghu" kolahdoz type:=link;fs:="/nfs/isd2/kolahdoz" rparker type:=link;fs:="/nfs/guest/rparker" skrimm type:=link;fs:="/nfs/ivory/skrimm" agents type:=link;fs:="/nfs/isd2/agents" traum type:=link;fs:="/nfs/guest/traum" leiqu type:=link;fs:="/nfs/amber/leiqu" jackw type:=link;fs:="/nfs/asd/jackw" anava type:=link;fs:="/nfs/topaz/anava" http type:=link;fs:="/nfs/web/htdocs" stergiou type:=link;fs:="/nfs/ivory/stergiou" stamant type:=link;fs:="/nfs/topaz/stamant" nrathod type:=link;fs:="/nfs/isd/nrathod" madduri type:=link;fs:="/nfs/guest/madduri" d7share type:=link;fs:="/nfs/gren/d7share" klink type:=link;fs:="/nfs/isd2/klink" vlsi type:=link;fs:="/nfs/m_app1/vlsi" shaw type:=link;fs:="/nfs/amber/shaw" news type:=link;fs:="/usr/spool/news" gdt type:=link;fs:="/nfs/v6/gdt" fry type:=link;fs:="/nfs/m_usr/fry" pdox-adm type:=link;fs:="/nfs/ivory/pdox-adm" shishir type:=link;fs:="/nfs/v5/shishir" touch2 type:=link;fs:="/nfs/ruby/touch2" bhuang type:=link;fs:="/nfs/isd3/bhuang" opr type:=link;fs:="/nfs/u2/opr" galstyan type:=link;fs:="/nfs/isd3/galstyan" alfredau type:=link;fs:="/nfs/isd2/alfredau" shalini type:=link;fs:="/nfs/neon/shalini" onruser type:=link;fs:="/nfs/share/onruser" div9rpt type:=link;fs:="/nfs/neon/div9rpt" kokje type:=link;fs:="/nfs/jade/kokje" cbeal type:=link;fs:="/nfs/isd/cbeal" bbean type:=link;fs:="/nfs/ivory/bbean" bagga type:=link;fs:="/nfs/ruby/bagga" zhou type:=link;fs:="/nfs/jade/zhou" alba type:=link;fs:="/nfs/jade/alba" web-omos type:=link;fs:="/web-omos" div5rpt type:=link;fs:="/nfs/neon/div5rpt" liangz type:=link;fs:="/nfs/isd2/liangz" cmcdir type:=link;fs:="/nfs/m_cad/cmcdir" aharth type:=link;fs:="/nfs/guest/aharth" yasin type:=link;fs:="/nfs/guest/yasin" bacon type:=link;fs:="/nfs/guest/bacon" iana type:=link;fs:="/nfs/jade/iana" pan type:=link;fs:="/nfs/topaz/pan" mib type:=link;fs:="/nfs/ftp/mib" pcbackup type:=link;fs:="/nfs/ipc/pcbackup" defactom type:=link;fs:="/nfs/asd/defactom" web-cur type:=link;fs:="/" theseus type:=link;fs:="/nfs/isd3/theseus" rflucas type:=link;fs:="/nfs/div1/rflucas" counter type:=link;fs:="/nfs/infr/counter" adibi type:=link;fs:="/nfs/isd/adibi" web-pstl type:=link;fs:="/" web-iana type:=link;fs:="/" ldapuser type:=link;fs:="/" package type:=link;fs:="/nfs/m_mod1/package" buskirk type:=link;fs:="/nfs/guest/buskirk" rogers type:=link;fs:="/nfs/div2/rogers" jmoore type:=link;fs:="/nfs/isd3/jmoore" yaser type:=link;fs:="/nfs/guest/yaser" zita type:=link;fs:="/nfs/jade/zita" yushunwa type:=link;fs:="/nfs/ruby/yushunwa" rajgarhi type:=link;fs:="/nfs/guest/rajgarhi" anuraagm type:=link;fs:="/nfs/guest/anuraagm" sonjain type:=link;fs:="/nfs/isd2/sonjain" nlserv type:=link;fs:="/nfs/isd3/nlserv" lara type:=link;fs:="/nfs/isd2/lara" drew type:=link;fs:="/nfs/ipc/drew" pvm type:=link;fs:="/nfs/guest/pvm" pinfosrv type:=link;fs:="/nfs/gost/pfs" m_shared type:=link;fs:="/nfs/m_app1/m_shared" abramson type:=link;fs:="/nfs/div1/abramson" srinath type:=link;fs:="/nfs/guest/srinath" samtani type:=link;fs:="/nfs/isd2/samtani" jchame type:=link;fs:="/nfs/div1/jchame" model type:=link;fs:="/nfs/m_mod1/model" dford type:=link;fs:="/nfs/isd2/dford" eric type:=link;fs:="/nfs/neon/eric" goldberg type:=link;fs:="/nfs/m_usr/goldberg" sidjain type:=link;fs:="/nfs/v6/sidjain" carlos type:=link;fs:="/nfs/m_usr/carlos" cnpa type:=link;fs:="/nfs/div7/cnpa" p4 type:=link;fs:="/nfs/div1/cad/p4" bartlett type:=link;fs:="/nfs/jade/bartlett" hussain type:=link;fs:="/nfs/ruby/hussain" bhatti type:=link;fs:="/nfs/div1/bhatti" neno type:=link;fs:="/nfs/guest/neno" paradox type:=link;fs:="/nfs/neon/paradox" neesdaq type:=link;fs:="/nfs/asd2/neesdaq" loomora type:=link;fs:="/nfs/isd/loomora" cousins type:=link;fs:="/nfs/m_usr/cousins" clayton type:=link;fs:="/nfs/isd3/clayton" minton type:=link;fs:="/nfs/guest/minton" madmin type:=link;fs:="/nfs/m_usr/madmin" smmsp type:=link;fs:="/var/spool/mqueue" inc-work type:=link;fs:="/nfs/jade/inc-work" crnmosis type:=link;fs:="/nfs/m_app1/crnmosis" pkhoury type:=link;fs:="/nfs/isd2/pkhoury" mgalley type:=link;fs:="/nfs/guest/mgalley" lnadmin type:=link;fs:="/nfs/ruby/lnadmin" div8rpt type:=link;fs:="/nfs/neon/div8rpt" div4rpt type:=link;fs:="/nfs/neon/div4rpt" ddutta type:=link;fs:="/nfs/ruby/ddutta" d7adm type:=link;fs:="/nfs/jade/d7adm" kenj type:=link;fs:="/nfs/ipc/kenj" isx type:=link;fs:="/nfs/isd/isx" ora_alto type:=link;fs:="/nfs/alto/ora_alto" kyamada type:=link;fs:="/nfs/guest/kyamada" schorr type:=link;fs:="/nfs/neon/schorr" nouhi type:=link;fs:="/nfs/isd/nouhi" elves type:=link;fs:="/nfs/isd3/elves" dono type:=link;fs:="/nfs/isd/dono" wcadmin type:=link;fs:="/nfs/ipc/wcadmin" tbenzel type:=link;fs:="/nfs/neon/tbenzel" margiej type:=link;fs:="/nfs/ivory/margiej" sharad type:=link;fs:="/nfs/v5/sharad" remedy type:=link;fs:="/nfs/neon/remedy" tilp type:=link;fs:="/nfs/div1/tilp" hjma type:=link;fs:="/nfs/m_usr/hjma" publicpc type:=link;fs:="/nfs/isd/publicpc" maj-domo type:=link;fs:="/nfs/ipc/maj-domo" actiondb type:=link;fs:="/nfs/u2/actiondb" mallard type:=link;fs:="/nfs/amber/mallard" shumin type:=link;fs:="/nfs/isd3/shumin" xbone type:=link;fs:="/nfs/jade/xbone" puid3 type:=link;fs:="/none" vmjoyner type:=link;fs:="/nfs/m_usr/vmjoyner" echihabi type:=link;fs:="/nfs/guest/echihabi" hdaume type:=link;fs:="/nfs/isd3/hdaume" foukia type:=link;fs:="/nfs/ruby/foukia" expect type:=link;fs:="/nfs/topaz/expect" coreyc type:=link;fs:="/nfs/ivory/coreyc" sweep type:=link;fs:="/local/sophos" puid5 type:=link;fs:="/none" ch-mm type:=link;fs:="/nfs/div7/ch-mm" loom type:=link;fs:="/nfs/isd2/loom" pat type:=link;fs:="/nfs/m_usr/pat" chaudhry type:=link;fs:="/nfs/isd2/chaudhry" aeronaut type:=link;fs:="/nfs/ipc/aeronaut" masseyd type:=link;fs:="/nfs/guest/masseyd" martinm type:=link;fs:="/nfs/topaz/martinm" donghui type:=link;fs:="/nfs/isd2/donghui" graehl type:=link;fs:="/nfs/topaz/graehl" baoshi type:=link;fs:="/nfs/eisd/baoshi" jtran type:=link;fs:="/nfs/div1/jtran" brent type:=link;fs:="/nfs/isd2/brent" sdeneefe type:=link;fs:="/nfs/isd/sdeneefe" quamrul type:=link;fs:="/nfs/topaz/quamrul" mib-adm type:=link;fs:="/nfs/coms/mib-adm" safari type:=link;fs:="/nfs/share/safari" berson type:=link;fs:="/nfs/guest/berson" touch type:=link;fs:="/nfs/ruby/touch" kurinsky type:=link;fs:="/nfs/topaz/kurinsky" uscms01 type:=link;fs:="/nfs/asd/uscms01" div7rpt type:=link;fs:="/nfs/neon/div7rpt" labore type:=link;fs:="/nfs/amber/labore" unify type:=link;fs:="/usr/unify" mcgee type:=link;fs:="/nfs/asd2/mcgee" skim type:=link;fs:="/nfs/isd2/skim" mack type:=link;fs:="/nfs/topaz/mack" finn type:=link;fs:="/nfs/ruby/finn" lcr type:=link;fs:="/nfs/m_usr/lcr" sullivan type:=link;fs:="/nfs/ivory/sullivan" johnmgce type:=link;fs:="/nfs/asd2/johnmgce" istouser type:=link;fs:="/nfs/share/istouser" d8recrut type:=link;fs:="/nfs/asd/d8recrut" div6rpt type:=link;fs:="/nfs/neon/div6rpt" toonen type:=link;fs:="/nfs/guest/toonen" shanky type:=link;fs:="/nfs/guest/shanky" yvaid type:=link;fs:="/nfs/ivory/yvaid" rahul type:=link;fs:="/nfs/topaz/rahul" nsnam type:=link;fs:="/nfs/ruby/nsnam" rayshon type:=link;fs:="/nfs/m_usr/rayshon" scadds type:=link;fs:="/nfs/gren/scadds" dragos type:=link;fs:="/nfs/isd/dragos" bester type:=link;fs:="/nfs/guest/bester" ojeil type:=link;fs:="/nfs/isd/ojeil" argos type:=link;fs:="/nfs/isd2/argos" tar type:=link;fs:="/nfs/topaz/tar" shengnan type:=link;fs:="/nfs/isd/shengnan" 10recept type:=link;fs:="/nfs/ivory/10recept" sudafed type:=link;fs:="/nfs/ruby/sudafed" mkramer type:=link;fs:="/nfs/v6/mkramer" macgreg type:=link;fs:="/nfs/div2/macgreg" acttest type:=link;fs:="/nfs/ipc/acttest" olomu type:=link;fs:="/nfs/v5/olomu" uucp type:=link;fs:="/var/spool/uucppublic" kemp type:=link;fs:="/nfs/div7/kemp" ravinder type:=link;fs:="/nfs/div1/ravinder" ershaghi type:=link;fs:="/nfs/div7/ershaghi" wormuth type:=link;fs:="/nfs/m_usr/wormuth" jaewook type:=link;fs:="/nfs/div1/jaewook" mmop type:=link;fs:="/nfs/conf/mmop" bob type:=link;fs:="/nfs/neon/bob" bin type:=link;fs:="/bin" linkscan type:=link;fs:="/nfs/m_app1/linkscan" vision type:=link;fs:="/nfs/div2/vision" karlcz type:=link;fs:="/nfs/guest/karlcz" ddavis type:=link;fs:="/nfs/div1/ddavis" tdey type:=link;fs:="/nfs/isd/tdey" purestat type:=link;fs:="/nfs/ipc/purestat" ora_dmkr type:=link;fs:="/nfs/div2/ora_dmkr" info-sys type:=link;fs:="/nfs/div7/info-sys" web-ict type:=link;fs:="/" tsacore type:=link;fs:="/nfs/asd/tsacore" isiuser type:=link;fs:="/nfs/share/isiuser" div1rpt type:=link;fs:="/nfs/neon/div1rpt" prober type:=link;fs:="/nfs/m_prb1/prober" braden type:=link;fs:="/nfs/gren/braden" pltfrm03 type:=link;fs:="/nfs/v6/pltfrm03" bkrishna type:=link;fs:="/nfs/gren/bkrishna" rs1test type:=link;fs:="/nfs/m_app1/rs1test" loomftp type:=link;fs:="/nfs/isd2/loomftp/loomftp" aridemo type:=link;fs:="/nfs/isd3/aridemo" sumitm type:=link;fs:="/nfs/div1/sumitm" dianee type:=link;fs:="/nfs/ipc/dianee" neven type:=link;fs:="/nfs/div2/neven" asyed type:=link;fs:="/nfs/guest/asyed" hmt type:=link;fs:="/nfs/ipc/hmt" gjthomas type:=link;fs:="/nfs/div7/gjthomas" philpot type:=link;fs:="/nfs/isd3/philpot" haldar type:=link;fs:="/nfs/ruby/haldar" e2e-mm type:=link;fs:="/nfs/ruby/e2e-mm" bbnrs1 type:=link;fs:="/nfs/m_app1/bbnrs1" sdlin type:=link;fs:="/nfs/isd2/sdlin" d7cnd type:=link;fs:="/nfs/gren/d7cnd" vahi type:=link;fs:="/nfs/asd2/vahi" tomw type:=link;fs:="/nfs/ipc/tomw" radu type:=link;fs:="/nfs/isd/radu" payroll type:=link;fs:="/nfs/ivory/payroll" lnxuser type:=link;fs:="/nfs/div1/lnxuser" gridcvs type:=link;fs:="/nfs/asd/gridcvs" imr-ed type:=link;fs:="/nfs/coms/imr-ed" bmoore type:=link;fs:="/nfs/isd/bmoore" mjtorres type:=link;fs:="/nfs/isd2/mjtorres" gritchey type:=link;fs:="/nfs/guest/gritchey" cinquini type:=link;fs:="/nfs/guest/cinquini" liyuan type:=link;fs:="/nfs/jade/liyuan" jsweet type:=link;fs:="/nfs/guest/jsweet" eromo type:=link;fs:="/nfs/ivory/eromo" dserv type:=link;fs:="/nfs/div2/dserv" hovy type:=link;fs:="/nfs/isd2/hovy" web-usrl type:=link;fs:="/" tryutov type:=link;fs:="/nfs/gren/tryutov" deelman type:=link;fs:="/nfs/asd/deelman" d7guest type:=link;fs:="/nfs/div7/d7guest" johnh type:=link;fs:="/nfs/gren/johnh" koda type:=link;fs:="/nfs/u2/koda" benc type:=link;fs:="/nfs/v6/benc" rim type:=link;fs:="/nfs/guest/rim" web-mos type:=link;fs:="/" leiding type:=link;fs:="/nfs/topaz/leiding" lindam type:=link;fs:="/nfs/ivory/lindam" joseph type:=link;fs:="/nfs/gren/joseph" dongho type:=link;fs:="/nfs/div7/dongho" fabio type:=link;fs:="/nfs/jade/fabio" crago type:=link;fs:="/nfs/guest/crago" june type:=link;fs:="/nfs/v6/june" d7unsupp type:=link;fs:="/nfs/gren/d7unsupp" jshetty type:=link;fs:="/nfs/guest/jshetty" allcock type:=link;fs:="/nfs/guest/allcock" web-bo type:=link;fs:="/" kapoor type:=link;fs:="/nfs/div7/kapoor" jordan type:=link;fs:="/nfs/asd/jordan" carley type:=link;fs:="/nfs/ivory/carley" rynge type:=link;fs:="/nfs/asd/rynge" puid1 type:=link;fs:="/none" svn type:=link;fs:="/nfs/jade/svn" jtb type:=link;fs:="/nfs/div1/jtb" beverlyh type:=link;fs:="/nfs/ivory/beverlyh" yuhe type:=link;fs:="/nfs/v6/yuhe" mote type:=link;fs:="/nfs/isd3/mote" dson type:=link;fs:="/nfs/guest/dson" ilt type:=link;fs:="/nfs/ivory/ilt" yuseonki type:=link;fs:="/nfs/guest/yuseonki" usdomreg type:=link;fs:="/nfs/coms/usdomreg" mulligan type:=link;fs:="/nfs/guest/mulligan" michelle type:=link;fs:="/nfs/guest/michelle" federico type:=link;fs:="/nfs/guest/federico" vernier type:=link;fs:="/nfs/m_usr/vernier" gregor type:=link;fs:="/nfs/guest/gregor" genew type:=link;fs:="/nfs/div1/genew" ecoe type:=link;fs:="/nfs/div7/ecoe" purchase type:=link;fs:="/nfs/ivory/purchase" pszekely type:=link;fs:="/nfs/div2/pszekely" imsuser type:=link;fs:="/" varunr type:=link;fs:="/nfs/isd/varunr" rfc-ed type:=link;fs:="/nfs/jade/rfc-ed" padmin type:=link;fs:="/nfs/ivory/padmin" globus type:=link;fs:="/nfs/asd2/globus" moody type:=link;fs:="/nfs/isd3/moody" tdw type:=link;fs:="/nfs/ipc/tdw" thiebaux type:=link;fs:="/nfs/asd/thiebaux" letstrav type:=link;fs:="/nfs/guest/letstrav" cristina type:=link;fs:="/nfs/asd2/cristina" umosis type:=link;fs:="/nfs/m_app1/umosis" moses type:=link;fs:="/nfs/ivory/moses" chris type:=link;fs:="/nfs/ipc/chris" zyda type:=link;fs:="/nfs/neon/zyda" ipgr type:=link;fs:="/nfs/infr/ipgr" jairmail type:=link;fs:="/nfs/isd3/jairmail" isiguest type:=link;fs:="/nfs/guest/isiguest" deschon type:=link;fs:="/nfs/jade/deschon" d3admin type:=link;fs:="/nfs/amber/d3admin" vgupta type:=link;fs:="/nfs/guest/vgupta" sheena type:=link;fs:="/nfs/isd2/sheena" tambe type:=link;fs:="/nfs/guest/tambe" mcai type:=link;fs:="/nfs/div2/mcai" rfl type:=link;fs:="/nfs/div1/rfl" cyl type:=link;fs:="/nfs/isd/cyl" dimitrap type:=link;fs:="/nfs/isd/dimitrap" web-usd type:=link;fs:="/" library type:=link;fs:="/nfs/ivory/library" icsuser type:=link;fs:="/" griphyn type:=link;fs:="/nfs/v5/griphyn" cmassey type:=link;fs:="/nfs/ipc/cmassey" victor type:=link;fs:="/nfs/isd2/victor" fredh type:=link;fs:="/nfs/ivory/fredh" tbds type:=link;fs:="/nfs/tufa/tbds" sys type:=link;fs:="/" etg type:=link;fs:="/nfs/amber/etg" hansford type:=link;fs:="/nfs/m_usr/hansford" pingali type:=link;fs:="/nfs/ruby/pingali" modtool type:=link;fs:="/nfs/cad10/modtool" rrios type:=link;fs:="/nfs/ivory/rrios" mikez type:=link;fs:="/nfs/ipc/mikez" meisi type:=link;fs:="/nfs/topaz/meisi" rich type:=link;fs:="/nfs/guest/rich" kyao type:=link;fs:="/nfs/eisd/kyao" sri type:=link;fs:="/nfs/isd3/sri" umdomreg type:=link;fs:="/nfs/ruby/umdomreg" vendors type:=link;fs:="/nfs/m_mod1/vendors" mdorosz type:=link;fs:="/nfs/div2/mdorosz" dktest type:=link;fs:="/nfs/gren/dktest" flhou type:=link;fs:="/nfs/isd/flhou" yamazaki type:=link;fs:="/nfs/div7/yamazaki" wordperf type:=link;fs:="/nfs/m_app1/wordperf" bedernik type:=link;fs:="/nfs/isd2/bedernik" jeanine type:=link;fs:="/nfs/div7/jeanine" cakers type:=link;fs:="/nfs/m_usr/cakers" divam type:=link;fs:="/nfs/asd/divam" jjg type:=link;fs:="/nfs/asd/jjg" michelso type:=link;fs:="/nfs/isd2/michelso" akanauka type:=link;fs:="/nfs/neon/akanauka" dominik type:=link;fs:="/nfs/ipc/dominik" bugacov type:=link;fs:="/nfs/guest/bugacov" condor type:=link;fs:="/nfs/v5/condor" helen type:=link;fs:="/nfs/m_usr/helen" chase type:=link;fs:="/nfs/asd/chase" rutu type:=link;fs:="/nfs/isd2/rutu" kary type:=link;fs:="/nfs/amber/kary" pvenugop type:=link;fs:="/nfs/div2/pvenugop" profiles type:=link;fs:="/local/profiles" nastaran type:=link;fs:="/nfs/div1/nastaran" ettelaie type:=link;fs:="/nfs/topaz/ettelaie" ambite type:=link;fs:="/nfs/amber/ambite" jiao type:=link;fs:="/nfs/ruby/jiao" qa type:=link;fs:="/nfs/m_app1/qa" szekely type:=link;fs:="/nfs/div2/szekely" ryarber type:=link;fs:="/nfs/ivory/ryarber" jessica type:=link;fs:="/nfs/ipc/jessica" crogers type:=link;fs:="/nfs/div2/crogers" xiangf type:=link;fs:="/nfs/v6/xiangf" allard type:=link;fs:="/nfs/div2/allard" meder type:=link;fs:="/nfs/guest/meder" --gj572EiMnwbLXET9 Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="amd.removable" cdrom type:=cdfs;addopts:=ro,nocache;dev:=/dev/acd0 --gj572EiMnwbLXET9 Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="amd.isi" etc wire==boreas-net.isi.edu;type:=nfs;rhost:=boreas;rfs:=/usr/share/etc; faber wire==boreas-net.isi.edu;type:=nfs;rhost:=boreas;rfs:=/home/faber; --gj572EiMnwbLXET9-- --4jXrM3lyYWu4nBt5 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (FreeBSD) iD8DBQFCl5XUaUz3f+Zf+XsRAkmHAKC52Xg3+DvdtVJzMDaykW2hVn9Q0ACdFiNS OIYBTk/qPlGzc6gZlTbyXfw= =moHK -----END PGP SIGNATURE----- --4jXrM3lyYWu4nBt5--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20050527214908.GA1031>