From owner-freebsd-stable@FreeBSD.ORG Thu Nov 20 07:11:41 2008 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 109B7106564A for ; Thu, 20 Nov 2008 07:11:41 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from QMTA07.westchester.pa.mail.comcast.net (qmta07.westchester.pa.mail.comcast.net [76.96.62.64]) by mx1.freebsd.org (Postfix) with ESMTP id C9F488FC18 for ; Thu, 20 Nov 2008 07:11:39 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from OMTA07.westchester.pa.mail.comcast.net ([76.96.62.59]) by QMTA07.westchester.pa.mail.comcast.net with comcast id hKAA1a0011GhbT857KAoUk; Thu, 20 Nov 2008 07:10:48 +0000 Received: from koitsu.dyndns.org ([69.181.141.110]) by OMTA07.westchester.pa.mail.comcast.net with comcast id hKBc1a0032P6wsM3TKBdjv; Thu, 20 Nov 2008 07:11:38 +0000 X-Authority-Analysis: v=1.0 c=1 a=6I5d2MoRAAAA:8 a=QycZ5dHgAAAA:8 a=AFzJLNVnVYuBpOHGbGgA:9 a=3telo9Gqa0uFKkogSGIA:7 a=r9g8t6t6mHK2g1TyIS-V3RKyoAAA:4 a=EoioJ0NPDVgA:10 a=SV7veod9ZcQA:10 a=LY0hPdMaydYA:10 Received: by icarus.home.lan (Postfix, from userid 1000) id A903E33C1C; Wed, 19 Nov 2008 23:11:36 -0800 (PST) Date: Wed, 19 Nov 2008 23:11:36 -0800 From: Jeremy Chadwick To: Ganbold Message-ID: <20081120071136.GA19577@icarus.home.lan> References: <4922906D.4060805@micom.mng.net> <492508C6.3090005@micom.mng.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <492508C6.3090005@micom.mng.net> User-Agent: Mutt/1.5.18 (2008-05-17) Cc: Kostik Belousov , kib@FreeBSD.org, freebsd-stable@FreeBSD.ORG Subject: Re: shutdown -p now crashes X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Nov 2008 07:11:41 -0000 On Thu, Nov 20, 2008 at 02:50:46PM +0800, Ganbold wrote: > Ganbold wrote: >> #3 0xc06ab8b3 in vput (vp=0xc3d11ac8) at >> /usr/src/sys/kern/vfs_subr.c:2202 >> #4 0xc06a69e6 in dounmount (mp=0xc3e59b30, flags=524288, >> td=0xc3b31d20) at /usr/src/sys/kern/vfs_mount.c:1288 >> #5 0xc06aa493 in vfs_unmountall () at /usr/src/sys/kern/vfs_subr.c:2939 >> #6 0xc062b7f4 in boot (howto=16392) at >> /usr/src/sys/kern/kern_shutdown.c:400 >> #7 0xc062bc17 in reboot (td=0xc3b31d20, uap=0xeef55cfc) at >> /usr/src/sys/kern/kern_shutdown.c:172 >> #8 0xc08171f5 in syscall (frame=0xeef55d38) at >> /usr/src/sys/i386/i386/trap.c:1090 >> #9 0xc07fd710 in Xint0x80_syscall () at >> /usr/src/sys/i386/i386/exception.s:255 >> #10 0x00000033 in ?? () >> (kgdb) >> .. > I was able to reproduce the panic. I plugged in external ZFS/GELI HDD > via USB > and used 'zpool import' to import disk and did some 'ls' and then used > 'zpool export' to umount disk. > Then I detached the disk from desktop and tried to "shutdown -p now". > This way panic is > reproduced 2 times. Are we sure that the problem isn't the well-known "do not yank a device out from underneathe the rest of the OS" problem, e.g. removing removable devices while filesystems are still mounted? If so, this problem is very well-known, documented on my Wiki, and work is being done in CURRENT to fix it (official ETA is 2009/02). I see geli(8) has a "detach" option, which you might need to do after the "zpool export", being as the GEOM provider is still attached to the USB HDD. I would recommend trying that first. > More info: > > daemon% uname -an > FreeBSD daemon.micom.mng.net 7.1-PRERELEASE FreeBSD 7.1-PRERELEASE #8 > r185085M: Thu Nov 20 12:50:33 ULAT 2008 > tsgan@daemon.micom.mng.net:/usr/obj/usr/src/sys/GDAEMON i386 > daemon% > > > <118>. > <118>Writing entropy file: > <118>. > <118>Terminated > <118>. > <118>Nov 20 14:30:12 daemon syslogd: exiting on signal 15 > rootvp 0xc3d11ac8 v_usecount 2 > KDB: stack backtrace: > db_trace_self_wrapper(c086b1ee,f2c6bbac,c06abb23,c086faa8,c3d11ac8,...) > at db_trace_self_wrapper+0x26 > kdb_backtrace(c086faa8,c3d11ac8,2,b,0,...) at kdb_backtrace+0x29 > vrele(c3d11ac8,c3e6f690,0,0,0,...) at vrele+0x83 > fdfree(c3e6f690,c3e8f000,48077bf0,f2c6bc80,c062ccfe,...) at fdfree+0x8b6 > exit1(c3e6f690,0,f2c6bd2c,c0817245,c3e6f690,...) at exit1+0x5e9 > sys_exit(c3e6f690,f2c6bcfc,4,c3e6f690,f2c6bd2c,...) at sys_exit+0x1d > syscall(f2c6bd38) at syscall+0x335 > Xint0x80_syscall() at Xint0x80_syscall+0x20 > --- syscall (1, FreeBSD ELF32, sys_exit), eip = 0x4810ed2f, esp = > 0xbfbfe6bc, ebp = 0xbfbfe6c8 --- > rootvp 0xc3d11ac8 v_usecount 1 > KDB: stack backtrace: > db_trace_self_wrapper(c086b1ee,f2c6bbac,c06abb23,c086faa8,c3d11ac8,...) > at db_trace_self_wrapper+0x26 > kdb_backtrace(c086faa8,c3d11ac8,1,b,0,...) at kdb_backtrace+0x29 > vrele(c3d11ac8,c3e6f690,0,0,0,...) at vrele+0x83 > fdfree(c3e6f690,c3e8f000,48077bf0,f2c6bc80,c062ccfe,...) at fdfree+0x8c3 > exit1(c3e6f690,0,f2c6bd2c,c0817245,c3e6f690,...) at exit1+0x5e9 > sys_exit(c3e6f690,f2c6bcfc,4,c3e6f690,f2c6bd2c,...) at sys_exit+0x1d > syscall(f2c6bd38) at syscall+0x335 > Xint0x80_syscall() at Xint0x80_syscall+0x20 > --- syscall (1, FreeBSD ELF32, sys_exit), eip = 0x4810ed2f, esp = > 0xbfbfe6bc, ebp = 0xbfbfe6c8 --- > Waiting (max 60 seconds) for system process `vnlru' to stop...done > Waiting (max 60 seconds) for system process `bufdaemon' to stop...done > Waiting (max 60 seconds) for system process `syncer' to stop... > Syncing disks, vnodes remaining...2 1 0 0 done > All buffers synced. > rootvp 0xc3d11ac8 v_usecount 1 > KDB: stack backtrace: > db_trace_self_wrapper(c086b1ee,eef55ba8,c06abb23,c086faa8,c3d11ac8,...) > at db_trace_self_wrapper+0x26 > kdb_backtrace(c086faa8,c3d11ac8,1,c3b31d20,eef55ba8,...) at > kdb_backtrace+0x29 > vrele(c3d11ac8,2,eef55bf0,c3b31d20,0,...) at vrele+0x83 > dounmount(c3e56b30,80000,c3b31d20,e3b61aec,0,...) at dounmount+0x491 > vfs_unmountall(c0869007,0,c086906b,117,eef55c50,...) at vfs_unmountall+0x33 > boot(c3b31d20,8,0,fffffffe,c3b31d20,...) at boot+0x444 > reboot(c3b31d20,eef55cfc,4,8048290,9e7c9054,...) at reboot+0x67 > syscall(eef55d38) at syscall+0x335 > Xint0x80_syscall() at Xint0x80_syscall+0x20 > --- syscall (55, FreeBSD ELF32, reboot), eip = 0x8050bb3, esp = > 0xbfbfe8ec, ebp = 0xbfbfe9b8 --- > panic: vput: negative ref cnt > cpuid = 0 > KDB: stack backtrace: > db_trace_self_wrapper(c086b1ee,eef55b68,c062bd39,c08832ea,0,...) at > db_trace_self_wrapper+0x26 > kdb_backtrace(c08832ea,0,c086fac1,eef55b74,0,...) at kdb_backtrace+0x29 > panic(c086fac1,eef55ba8,c06abcd0,c3d11ac8,eef55b90,...) at panic+0x119 > vput(c3d11ac8,2,eef55bf0,c3b31d20,0,...) at vput+0xf3 > dounmount(c3e56b30,80000,c3b31d20,e3b61aec,0,...) at dounmount+0x4a6 > vfs_unmountall(c0869007,0,c086906b,117,eef55c50,...) at vfs_unmountall+0x33 > boot(c3b31d20,8,0,fffffffe,c3b31d20,...) at boot+0x444 > reboot(c3b31d20,eef55cfc,4,8048290,9e7c9054,...) at reboot+0x67 > syscall(eef55d38) at syscall+0x335 > Xint0x80_syscall() at Xint0x80_syscall+0x20 > --- syscall (55, FreeBSD ELF32, reboot), eip = 0x8050bb3, esp = > 0xbfbfe8ec, ebp = 0xbfbfe9b8 --- > Uptime: 7m3s > Physical memory: 1013 MB > Dumping 54 MB: 39 23 7 > (kgdb) where > #0 doadump () at pcpu.h:196 > #1 0xc062ba67 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:418 > #2 0xc062bd72 in panic (fmt=Variable "fmt" is not available. > ) at /usr/src/sys/kern/kern_shutdown.c:574 > #3 0xc06ab8e3 in vput (vp=0xc3d11ac8) at /usr/src/sys/kern/vfs_subr.c:2211 > #4 0xc06a69e6 in dounmount (mp=0xc3e56b30, flags=524288, td=0xc3b31d20) > at /usr/src/sys/kern/vfs_mount.c:1288 > #5 0xc06aa493 in vfs_unmountall () at /usr/src/sys/kern/vfs_subr.c:2948 > #6 0xc062b7f4 in boot (howto=16392) at > /usr/src/sys/kern/kern_shutdown.c:400 > #7 0xc062bc17 in reboot (td=0xc3b31d20, uap=0xeef55cfc) at > /usr/src/sys/kern/kern_shutdown.c:172 > #8 0xc0817245 in syscall (frame=0xeef55d38) at > /usr/src/sys/i386/i386/trap.c:1090 > #9 0xc07fd760 in Xint0x80_syscall () at > /usr/src/sys/i386/i386/exception.s:255 > #10 0x00000033 in ?? () > (kgdb) frame 4 > #4 0xc06a69e6 in dounmount (mp=0xc3e56b30, flags=524288, td=0xc3b31d20) > at /usr/src/sys/kern/vfs_mount.c:1288 > 1288 vput(fsrootvp); > (kgdb) p *mp > $1 = {mnt_lock = {lk_object = {lo_name = 0xc086f234 "vfslock", lo_type = > 0xc086f234 "vfslock", lo_flags = 70844416, lo_witness_data = {lod_list = > { > stqe_next = 0x0}, lod_witness = 0x0}}, lk_interlock = > 0xc0956fa8, lk_flags = 1310720, lk_sharecount = 0, lk_waitcount = 0, > lk_exclusivecount = 1, > lk_prio = 80, lk_timo = 0, lk_lockholder = 0xc3b31d20, lk_newlock = > 0x0}, mnt_mtx = {lock_object = {lo_name = 0xc086f223 "struct mount mtx", > lo_type = 0xc086f223 "struct mount mtx", lo_flags = 16973824, > lo_witness_data = {lod_list = {stqe_next = 0x0}, lod_witness = 0x0}}, > mtx_lock = 4, > mtx_recurse = 0}, mnt_gen = 1, mnt_list = {tqe_next = 0x0, tqe_prev = > 0xc091c224}, mnt_op = 0xc0926080, mnt_vfc = 0xc0926040, mnt_vnodecovered > = 0x0, > mnt_syncer = 0x0, mnt_ref = 736, mnt_nvnodelist = {tqh_first = > 0xc3d11ac8, tqh_last = 0xc67fd350}, mnt_nvnodelistsize = 736, > mnt_writeopcount = 1, > mnt_kern_flag = 553648137, mnt_flag = 20480, mnt_noasync = 0, mnt_opt = > 0xc3c4cc90, mnt_optnew = 0x0, mnt_maxsymlinklen = 120, mnt_stat = { > f_version = 537068824, f_type = 1, f_flags = 20480, f_bsize = 2048, > f_iosize = 16384, f_blocks = 2538519, f_bfree = 2339932, f_bavail = > 2136851, > f_files = 659454, f_ffree = 651413, f_syncwrites = 0, f_asyncwrites = > 0, f_syncreads = 0, f_asyncreads = 0, f_spare = {0, 0, 0, 0, 0, 0, 0, 0, > 0, 0}, > f_namemax = 255, f_owner = 0, f_fsid = {val = {1149777033, > 1944990175}}, f_charspare = '\0' , > f_fstypename = "ufs", '\0' , f_mntfromname = > "/dev/ad4s1a", '\0' , f_mntonname = "/", '\0' 86 times>}, > mnt_cred = 0xc3e52700, mnt_data = 0xc3e52600, mnt_time = 1227162073, > mnt_iosize_max = 131072, mnt_export = 0x0, mnt_label = 0x0, > mnt_hashseed = 2358371174, mnt_markercnt = 0, mnt_holdcnt = 0, > mnt_holdcntwaiters = 0, mnt_secondary_writes = 0, > mnt_secondary_accwrites = 2874, > mnt_gjprovider = 0x0, mnt_explock = {lk_object = {lo_name = 0xc086f23c > "explock", lo_type = 0xc086f23c "explock", lo_flags = 70844416, > lo_witness_data = { > lod_list = {stqe_next = 0x0}, lod_witness = 0x0}}, lk_interlock = > 0xc0956fc0, lk_flags = 0, lk_sharecount = 0, lk_waitcount = 0, > lk_exclusivecount = 0, lk_prio = 80, lk_timo = 0, lk_lockholder = > 0xffffffff, lk_newlock = 0x0}} > (kgdb) p fsrootvp $2 = (struct vnode *) 0xc3d11ac8 > (kgdb) p *fsrootvp > $3 = {v_type = VDIR, v_tag = 0xc0864e51 "ufs", v_op = 0xc0926280, v_data > = 0xc3e5d000, v_mount = 0xc3e56b30, v_nmntvnodes = {tqe_next = > 0xc3d119b4, > tqe_prev = 0xc3e56b98}, v_un = {vu_mount = 0x0, vu_socket = 0x0, > vu_cdev = 0x0, vu_fifoinfo = 0x0, vu_yield = 0}, v_hashlist = {le_next = > 0x0, > le_prev = 0xc3d09da0}, v_hash = 2, v_cache_src = {lh_first = 0x0}, > v_cache_dst = {tqh_first = 0x0, tqh_last = 0xc3d11af8}, v_dd = 0x0, > v_cstart = 0, > v_lasta = 0, v_lastw = 0, v_clen = 0, v_lock = {lk_object = {lo_name = > 0xc0864e51 "ufs", lo_type = 0xc0864e51 "ufs", lo_flags = 70844416, > lo_witness_data = {lod_list = {stqe_next = 0x0}, lod_witness = > 0x0}}, lk_interlock = 0xc0956510, lk_flags = 262208, lk_sharecount = 0, > lk_waitcount = 0, lk_exclusivecount = 1, lk_prio = 80, lk_timo = 51, > lk_lockholder = 0xc3b31d20, lk_newlock = 0x0}, v_interlock = > {lock_object = { > lo_name = 0xc086fb51 "vnode interlock", lo_type = 0xc086fb51 "vnode > interlock", lo_flags = 16973824, lo_witness_data = {lod_list = > {stqe_next = 0x0}, > lod_witness = 0x0}}, mtx_lock = 3283295520, mtx_recurse = 0}, > v_vnlock = 0xc3d11b20, v_holdcnt = 2, v_usecount = 0, v_iflag = 0, > v_vflag = 1, > v_writecount = 0, v_freelist = {tqe_next = 0x0, tqe_prev = 0x0}, > v_bufobj = {bo_mtx = 0xc3d11b50, bo_clean = {bv_hd = {tqh_first = > 0xe3d02594, > tqh_last = 0xe3d025cc}, bv_root = 0xe3d02594, bv_cnt = 1}, > bo_dirty = {bv_hd = {tqh_first = 0x0, tqh_last = 0xc3d11b9c}, bv_root = > 0x0, > bv_cnt = 0}, bo_numoutput = 0, bo_flag = 0, bo_ops = 0xc091ae00, > bo_bsize = 16384, bo_object = 0xc106183c, bo_synclist = {le_next = 0x0, > le_prev = 0x0}, bo_private = 0xc3d11ac8, __bo_vnode = 0xc3d11ac8}, > v_pollinfo = 0x0, v_label = 0x0, v_lockf = 0x0} > (kgdb) p rootvnode > $4 = (struct vnode *) 0x0 > (kgdb) p *rootvnode > Cannot access memory at address 0x0 > (kgdb) > > > -- > They are called computers simply because computation is the only > significant job that has so far been given to them. > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB |