From owner-freebsd-fs@FreeBSD.ORG Sun Oct 18 02:45:42 2009 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EECB8106566C; Sun, 18 Oct 2009 02:45:42 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id C5B2D8FC15; Sun, 18 Oct 2009 02:45:42 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n9I2jgMt020361; Sun, 18 Oct 2009 02:45:42 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id n9I2jgGT020357; Sun, 18 Oct 2009 02:45:42 GMT (envelope-from linimon) Date: Sun, 18 Oct 2009 02:45:42 GMT Message-Id: <200910180245.n9I2jgGT020357@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-fs@FreeBSD.org From: linimon@FreeBSD.org Cc: Subject: Re: kern/139715: [zfs] vfs.numvnodes leak on busy zfs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Oct 2009 02:45:43 -0000 Old Synopsis: vfs.numvnodes leak on bussy zfs New Synopsis: [zfs] vfs.numvnodes leak on busy zfs Responsible-Changed-From-To: freebsd-bugs->freebsd-fs Responsible-Changed-By: linimon Responsible-Changed-When: Sun Oct 18 02:45:27 UTC 2009 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=139715 From owner-freebsd-fs@FreeBSD.ORG Sun Oct 18 08:39:41 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BFFB31065670 for ; Sun, 18 Oct 2009 08:39:41 +0000 (UTC) (envelope-from ndenev@gmail.com) Received: from fg-out-1718.google.com (fg-out-1718.google.com [72.14.220.158]) by mx1.freebsd.org (Postfix) with ESMTP id 4EFDA8FC25 for ; Sun, 18 Oct 2009 08:39:40 +0000 (UTC) Received: by fg-out-1718.google.com with SMTP id d23so1462224fga.13 for ; Sun, 18 Oct 2009 01:39:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:subject:mime-version :content-type:from:in-reply-to:date:cc:content-transfer-encoding :message-id:references:to:x-mailer; bh=wnLV2d5Evrk9DVKZBzodqdd0IKUGJe/X4zyEeyzfA4U=; b=C4yYaj+iD+/ONF5DQWyo7TGywmDh2jEKzvHj6dqNWhnc6KBWdgi6rCgOvjXlIIzAne S4QKPjZyrxYqPCRXyQt3xfOzj7ecErE2hKArnZXc8FQ6+GGkMvwa9N0PpBIn6SRtCmUa AC9YOMEunsog8ORqlwly8z3lE+wMMSq2oSzeU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer; b=gtgIjibUOp/DSWHbq6YRDYwZyCplGgkK+vKIw9iZldSt0gwEzdtRh1GZxO25oTz1A4 V22EsgkCvlHOKTdVdDD4cXAfDMUSLtXKoolwK9z2Jl0j62r2Zb2ji4nqC7MgTazVera+ h7fcCQkcpxVVZPCUYHSF/bXcg5FIs+q6IV4l8= Received: by 10.103.81.12 with SMTP id i12mr1510401mul.37.1255853416682; Sun, 18 Oct 2009 01:10:16 -0700 (PDT) Received: from ?10.0.0.10? (93-152-151-19.ddns.onlinedirect.bg [93.152.151.19]) by mx.google.com with ESMTPS id y2sm5151018mug.19.2009.10.18.01.10.15 (version=TLSv1/SSLv3 cipher=RC4-MD5); Sun, 18 Oct 2009 01:10:15 -0700 (PDT) Mime-Version: 1.0 (Apple Message framework v1076) Content-Type: text/plain; charset=us-ascii; format=flowed; delsp=yes From: Nikolay Denev In-Reply-To: <4AD77556.7050607@tzim.net> Date: Sun, 18 Oct 2009 11:10:12 +0300 Content-Transfer-Encoding: 7bit Message-Id: References: <4AD77556.7050607@tzim.net> To: Arnaud Houdelette X-Mailer: Apple Mail (2.1076) Cc: freebsd-fs@freebsd.org Subject: Re: ZFS send from FreeBSD 7.1 (v6) to 8.0 (v13). X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Oct 2009 08:39:41 -0000 On 15 Oct, 2009, at 22:17 , Arnaud Houdelette wrote: > Hi. > > I use ZFS send/receive to do daily (or nightly) remote replication > from a dedicated server (at ISP's ) to my Home server. > > Dedicated server use 7.1-RELEASE. > Home server is 7.2-RELEASE. > Each night, a snapshot is taken on the dedicated server and an > incremental send used to replicate the content to my Home server. No > problem so far. > > With incoming 8.0, I'd like to upgrade my home server (and maybe > upgrade pool to v13) before even thinking to do anything to the > dedicated server. Will the send/receive scheme continue to work ? > > Regards, > > Arnaud Check in the zfs manual page. It says this : ... The format of the stream is evolving. No backwards compatibility is guaranteed. You may not be able to receive your streams on future versions of ZFS. ... Regards, Niki From owner-freebsd-fs@FreeBSD.ORG Sun Oct 18 15:45:48 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BECD6106566B for ; Sun, 18 Oct 2009 15:45:48 +0000 (UTC) (envelope-from kickbsd@ya.ru) Received: from forward7.yandex.ru (forward7.yandex.ru [77.88.61.37]) by mx1.freebsd.org (Postfix) with ESMTP id 3B7E78FC16 for ; Sun, 18 Oct 2009 15:45:48 +0000 (UTC) Received: from webmail6.yandex.ru (webmail6.yandex.ru [77.88.61.50]) by forward7.yandex.ru (Yandex) with ESMTP id 87E5931088B; Sun, 18 Oct 2009 19:45:46 +0400 (MSD) Received: from localhost (localhost.localdomain [127.0.0.1]) by webmail6.yandex.ru (Yandex) with ESMTP id 7044E1028011; Sun, 18 Oct 2009 19:45:46 +0400 (MSD) X-Yandex-Spam: 1 X-Yandex-Front: webmail6 X-Yandex-TimeMark: 1255880746 Received: from modemcable177.130-37-24.mc.videotron.ca (modemcable177.130-37-24.mc.videotron.ca [24.37.130.177]) by mail.yandex.ru with HTTP; Sun, 18 Oct 2009 19:45:46 +0400 From: kickbsd kickbsd To: linimon@freebsd.org In-Reply-To: <200910180245.n9I2jgGT020357@freefall.freebsd.org> References: <200910180245.n9I2jgGT020357@freefall.freebsd.org> MIME-Version: 1.0 Message-Id: <10921255880746@webmail6.yandex.ru> Date: Sun, 18 Oct 2009 19:45:46 +0400 X-Mailer: Yamail [ http://yandex.ru ] 5.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=koi8-r Cc: freebsd-fs@freebsd.org, freebsd-bugs@freebsd.org Subject: Re: kern/139715: [zfs] vfs.numvnodes leak on busy zfs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Oct 2009 15:45:48 -0000 There is the link to leaking vnodes grpaph http://xs.to/xs.php?h=xs1144&d=09420&f=vnodes-leak215.jpg 18.10.09, 02:45, linimon@FreeBSD.org: > Old Synopsis: vfs.numvnodes leak on bussy zfs > New Synopsis: [zfs] vfs.numvnodes leak on busy zfs > Responsible-Changed-From-To: freebsd-bugs->freebsd-fs > Responsible-Changed-By: linimon > Responsible-Changed-When: Sun Oct 18 02:45:27 UTC 2009 > Responsible-Changed-Why: > Over to maintainer(s). > http://www.freebsd.org/cgi/query-pr.cgi?pr=139715 > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" -- óÉÌØÎÁÑ ÐÏÞÔÁ ÎÁÈÏÄÉÔÓÑ ÚÄÅÓØ: http://mail.yandex.ru/promo/new/attachments From owner-freebsd-fs@FreeBSD.ORG Sun Oct 18 16:36:30 2009 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DB82A106566C; Sun, 18 Oct 2009 16:36:30 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id B3E0E8FC1D; Sun, 18 Oct 2009 16:36:30 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n9IGaUJg073454; Sun, 18 Oct 2009 16:36:30 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id n9IGaUaF073450; Sun, 18 Oct 2009 16:36:30 GMT (envelope-from linimon) Date: Sun, 18 Oct 2009 16:36:30 GMT Message-Id: <200910181636.n9IGaUaF073450@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-fs@FreeBSD.org From: linimon@FreeBSD.org Cc: Subject: Re: kern/139718: [reboot] all mounted fs don't get synced during reboot/shutdown with >= 1 mounted inaccessible device X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Oct 2009 16:36:30 -0000 Old Synopsis: all mounted fs don't get synced during reboot/shutdown with >= 1 mounted inaccessible device New Synopsis: [reboot] all mounted fs don't get synced during reboot/shutdown with >= 1 mounted inaccessible device Responsible-Changed-From-To: freebsd-bugs->freebsd-fs Responsible-Changed-By: linimon Responsible-Changed-When: Sun Oct 18 16:34:44 UTC 2009 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=139718 From owner-freebsd-fs@FreeBSD.ORG Sun Oct 18 16:40:04 2009 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AD9AF106566B for ; Sun, 18 Oct 2009 16:40:04 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 8598C8FC12 for ; Sun, 18 Oct 2009 16:40:04 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n9IGe416073545 for ; Sun, 18 Oct 2009 16:40:04 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id n9IGe4iH073544; Sun, 18 Oct 2009 16:40:04 GMT (envelope-from gnats) Date: Sun, 18 Oct 2009 16:40:04 GMT Message-Id: <200910181640.n9IGe4iH073544@freefall.freebsd.org> To: freebsd-fs@FreeBSD.org From: Mark Linimon Cc: Subject: Re: kern/139715: [zfs] vfs.numvnodes leak on busy zfs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Mark Linimon List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Oct 2009 16:40:04 -0000 The following reply was made to PR kern/139715; it has been noted by GNATS. From: Mark Linimon To: bug-followup@FreeBSD.org Cc: Subject: Re: kern/139715: [zfs] vfs.numvnodes leak on busy zfs Date: Sun, 18 Oct 2009 11:33:33 -0500 ----- Forwarded message from kickbsd kickbsd ----- From: kickbsd kickbsd To: linimon@freebsd.org Cc: freebsd-bugs@freebsd.org, freebsd-fs@freebsd.org Subject: Re: kern/139715: [zfs] vfs.numvnodes leak on busy zfs There is the link to leaking vnodes grpaph http://xs.to/xs.php?h=xs1144&d=09420&f=vnodes-leak215.jpg ----- End forwarded message ----- From owner-freebsd-fs@FreeBSD.ORG Sun Oct 18 18:42:21 2009 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A78A5106566B; Sun, 18 Oct 2009 18:42:21 +0000 (UTC) (envelope-from trasz@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 7EAD68FC14; Sun, 18 Oct 2009 18:42:21 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n9IIgLxH084244; Sun, 18 Oct 2009 18:42:21 GMT (envelope-from trasz@freefall.freebsd.org) Received: (from trasz@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id n9IIgLl4084240; Sun, 18 Oct 2009 18:42:21 GMT (envelope-from trasz) Date: Sun, 18 Oct 2009 18:42:21 GMT Message-Id: <200910181842.n9IIgLl4084240@freefall.freebsd.org> To: trasz@FreeBSD.org, freebsd-fs@FreeBSD.org, trasz@FreeBSD.org From: trasz@FreeBSD.org Cc: Subject: Re: kern/139718: [reboot] all mounted fs don't get synced during reboot/shutdown with >= 1 mounted inaccessible device X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Oct 2009 18:42:21 -0000 Synopsis: [reboot] all mounted fs don't get synced during reboot/shutdown with >= 1 mounted inaccessible device Responsible-Changed-From-To: freebsd-fs->trasz Responsible-Changed-By: trasz Responsible-Changed-When: Sun Oct 18 18:42:21 UTC 2009 Responsible-Changed-Why: I'll take it. http://www.freebsd.org/cgi/query-pr.cgi?pr=139718 From owner-freebsd-fs@FreeBSD.ORG Sun Oct 18 21:09:34 2009 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 248B81065670; Sun, 18 Oct 2009 21:09:34 +0000 (UTC) (envelope-from gavin@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id EF8B68FC0A; Sun, 18 Oct 2009 21:09:33 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n9IL9XqF004259; Sun, 18 Oct 2009 21:09:33 GMT (envelope-from gavin@freefall.freebsd.org) Received: (from gavin@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id n9IL9XV6004255; Sun, 18 Oct 2009 21:09:33 GMT (envelope-from gavin) Date: Sun, 18 Oct 2009 21:09:33 GMT Message-Id: <200910182109.n9IL9XV6004255@freefall.freebsd.org> To: gavin@FreeBSD.org, freebsd-i386@FreeBSD.org, freebsd-fs@FreeBSD.org From: gavin@FreeBSD.org Cc: Subject: Re: i386/139725: zdb(1) dumps core on i386 when examining zpool contents: Assertion failed: (rwlp->rw_count == 0) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Oct 2009 21:09:34 -0000 Old Synopsis: zdb dumps core on FreeBSD/i386 when examining zpool contents New Synopsis: zdb(1) dumps core on i386 when examining zpool contents: Assertion failed: (rwlp->rw_count == 0) Responsible-Changed-From-To: freebsd-i386->freebsd-fs Responsible-Changed-By: gavin Responsible-Changed-When: Sun Oct 18 21:06:32 UTC 2009 Responsible-Changed-Why: Over to maintainer(s) http://www.freebsd.org/cgi/query-pr.cgi?pr=139725 From owner-freebsd-fs@FreeBSD.ORG Mon Oct 19 11:06:52 2009 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A458B1065725 for ; Mon, 19 Oct 2009 11:06:52 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 6F47D8FC1A for ; Mon, 19 Oct 2009 11:06:52 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n9JB6q7T063428 for ; Mon, 19 Oct 2009 11:06:52 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id n9JB6pK2063426 for freebsd-fs@FreeBSD.org; Mon, 19 Oct 2009 11:06:51 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 19 Oct 2009 11:06:51 GMT Message-Id: <200910191106.n9JB6pK2063426@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-fs@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-fs@FreeBSD.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Oct 2009 11:06:52 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/139725 fs [zfs] zdb(1) dumps core on i386 when examining zpool c o kern/139715 fs [zfs] vfs.numvnodes leak on busy zfs o bin/139651 fs [nfs] mount(8): read-only remount of NFS volume does n o kern/139597 fs [patch] [tmpfs] tmpfs initializes va_gen but doesn't u o kern/139564 fs [zfs] [panic] 8.0-RC1 - Fatal trap 12 at end of shutdo o kern/139407 fs [smbfs] [panic] smb mount causes system crash if remot o kern/139363 fs [nfs] diskless root nfs mount from non FreeBSD server o kern/138790 fs [zfs] ZFS ceases caching when mem demand is high o kern/138524 fs [msdosfs] disks and usb flashes/cards with Russian lab o kern/138421 fs [ufs] [patch] remove UFS label limitations o kern/138367 fs [tmpfs] [panic] 'panic: Assertion pages > 0 failed' wh o kern/138202 fs mount_msdosfs(1) see only 2Gb o kern/138109 fs [extfs] [patch] Minor cleanups to the sys/gnu/fs/ext2f f kern/137037 fs [zfs] [hang] zfs rollback on root causes FreeBSD to fr o kern/136968 fs [ufs] [lor] ufs/bufwait/ufs (open) o kern/136945 fs [ufs] [lor] filedesc structure/ufs (poll) o kern/136944 fs [ffs] [lor] bufwait/snaplk (fsync) o kern/136873 fs [ntfs] Missing directories/files on NTFS volume o kern/136865 fs [nfs] [patch] NFS exports atomic and on-the-fly atomic o kern/136470 fs [nfs] Cannot mount / in read-only, over NFS o kern/135594 fs [zfs] Single dataset unresponsive with Samba o kern/135546 fs [zfs] zfs.ko module doesn't ignore zpool.cache filenam o kern/135469 fs [ufs] [panic] kernel crash on md operation in ufs_dirb o kern/135050 fs [zfs] ZFS clears/hides disk errors on reboot o kern/134491 fs [zfs] Hot spares are rather cold... o kern/133980 fs [panic] [ffs] panic: ffs_valloc: dup alloc o kern/133676 fs [smbfs] [panic] umount -f'ing a vnode-based memory dis o kern/133614 fs [smbfs] [panic] panic: ffs_truncate: read-only filesys o kern/133174 fs [msdosfs] [patch] msdosfs must support utf-encoded int f kern/133150 fs [zfs] Page fault with ZFS on 7.1-RELEASE/amd64 while w o kern/132960 fs [ufs] [panic] panic:ffs_blkfree: freeing free frag o kern/132597 fs [tmpfs] [panic] tmpfs-related panic while interrupting o kern/132397 fs reboot causes filesystem corruption (failure to sync b o kern/132331 fs [ufs] [lor] LOR ufs and syncer o kern/132237 fs [msdosfs] msdosfs has problems to read MSDOS Floppy o kern/132145 fs [panic] File System Hard Crashes o kern/131995 fs [nfs] Failure to mount NFSv4 server o kern/131441 fs [unionfs] [nullfs] unionfs and/or nullfs not combineab o kern/131360 fs [nfs] poor scaling behavior of the NFS server under lo o kern/131342 fs [nfs] mounting/unmounting of disks causes NFS to fail o bin/131341 fs makefs: error "Bad file descriptor" on the mount poin o kern/131086 fs [ext2fs] [patch] mkfs.ext2 creates rotten partition o kern/130979 fs [smbfs] [panic] boot/kernel/smbfs.ko o kern/130920 fs [msdosfs] cp(1) takes 100% CPU time while copying file o kern/130229 fs [iconv] usermount fails on fs that need iconv o kern/130210 fs [nullfs] Error by check nullfs o kern/129760 fs [nfs] after 'umount -f' of a stale NFS share FreeBSD l o kern/129488 fs [smbfs] Kernel "bug" when using smbfs in smbfs_smb.c: o kern/129231 fs [ufs] [patch] New UFS mount (norandom) option - mostly o kern/129152 fs [panic] non-userfriendly panic when trying to mount(8) o kern/129059 fs [zfs] [patch] ZFS bootloader whitelistable via WITHOUT f kern/128829 fs smbd(8) causes periodic panic on 7-RELEASE f kern/128173 fs [ext2fs] ls gives "Input/output error" on mounted ext3 o kern/127659 fs [tmpfs] tmpfs memory leak o kern/127420 fs [gjournal] [panic] Journal overflow on gmirrored gjour o kern/127029 fs [panic] mount(8): trying to mount a write protected zi o kern/126287 fs [ufs] [panic] Kernel panics while mounting an UFS file s kern/125738 fs [zfs] [request] SHA256 acceleration in ZFS f kern/125536 fs [ext2fs] ext 2 mounts cleanly but fails on commands li f kern/124621 fs [ext3] [patch] Cannot mount ext2fs partition f bin/124424 fs [zfs] zfs(8): zfs list -r shows strange snapshots' siz o kern/123939 fs [msdosfs] corrupts new files o kern/122380 fs [ffs] ffs_valloc:dup alloc (Soekris 4801/7.0/USB Flash o bin/122172 fs [fs]: amd(8) automount daemon dies on 6.3-STABLE i386, o kern/122047 fs [ext2fs] [patch] incorrect handling of UF_IMMUTABLE / o bin/121898 fs [nullfs] pwd(1)/getcwd(2) fails with Permission denied o bin/121779 fs [ufs] snapinfo(8) (and related tools?) only work for t o bin/121366 fs [zfs] [patch] Automatic disk scrubbing from periodic(8 o bin/121072 fs [smbfs] mount_smbfs(8) cannot normally convert the cha f kern/120991 fs [panic] [fs] [snapshot] System crashes when manipulati o kern/120483 fs [ntfs] [patch] NTFS filesystem locking changes o kern/120482 fs [ntfs] [patch] Sync style changes between NetBSD and F f kern/119735 fs [zfs] geli + ZFS + samba starting on boot panics 7.0-B o kern/118912 fs [2tb] disk sizing/geometry problem with large array o kern/118713 fs [minidump] [patch] Display media size required for a k o bin/118249 fs mv(1): moving a directory changes its mtime o kern/118107 fs [ntfs] [panic] Kernel panic when accessing a file at N o bin/117315 fs [smbfs] mount_smbfs(8) and related options can't mount o kern/117314 fs [ntfs] Long-filename only NTFS fs'es cause kernel pani o kern/117158 fs [zfs] zpool scrub causes panic if geli vdevs detach on o bin/116980 fs [msdosfs] [patch] mount_msdosfs(8) resets some flags f o kern/116913 fs [ffs] [panic] ffs_blkfree: freeing free block p kern/116608 fs [msdosfs] [patch] msdosfs fails to check mount options o kern/116583 fs [ffs] [hang] System freezes for short time when using o kern/116170 fs [panic] Kernel panic when mounting /tmp o kern/115645 fs [snapshots] [panic] lockmgr: thread 0xc4c00d80, not ex o bin/115361 fs [zfs] mount(8) gets into a state where it won't set/un o kern/114955 fs [cd9660] [patch] [request] support for mask,dirmask,ui o kern/114847 fs [ntfs] [patch] [request] dirmask support for NTFS ala o kern/114676 fs [ufs] snapshot creation panics: snapacct_ufs2: bad blo o bin/114468 fs [patch] [request] add -d option to umount(8) to detach o kern/113852 fs [smbfs] smbfs does not properly implement DFS referral o bin/113838 fs [patch] [request] mount(8): add support for relative p o bin/113049 fs [patch] [request] make quot(8) use getopt(3) and show o kern/112658 fs [smbfs] [patch] smbfs and caching problems (resolves b f usb/112640 fs [ext2fs] [hang] Kernel freezes when writing a file to o kern/111843 fs [msdosfs] Long Names of files are incorrectly created o kern/111782 fs [ufs] dump(8) fails horribly for large filesystems s bin/111146 fs [2tb] fsck(8) fails on 6T filesystem o kern/109024 fs [msdosfs] mount_msdosfs: msdosfs_iconv: Operation not o kern/109010 fs [msdosfs] can't mv directory within fat32 file system o bin/107829 fs [2TB] fdisk(8): invalid boundary checking in fdisk / w o kern/106030 fs [ufs] [panic] panic in ufs from geom when a dead disk o kern/105093 fs [ext2fs] [patch] ext2fs on read-only media cannot be m o kern/104406 fs [ufs] Processes get stuck in "ufs" state under persist o kern/104133 fs [ext2fs] EXT2FS module corrupts EXT2/3 filesystems o kern/103035 fs [ntfs] Directories in NTFS mounted disc images appear o kern/101324 fs [smbfs] smbfs sometimes not case sensitive when it's s o kern/99290 fs [ntfs] mount_ntfs ignorant of cluster sizes o kern/97377 fs [ntfs] [patch] syntax cleanup for ntfs_ihash.c o kern/95222 fs [iso9660] File sections on ISO9660 level 3 CDs ignored o kern/94849 fs [ufs] rename on UFS filesystem is not atomic o kern/94769 fs [ufs] Multiple file deletions on multi-snapshotted fil o kern/94733 fs [smbfs] smbfs may cause double unlock o kern/93942 fs [vfs] [patch] panic: ufs_dirbad: bad dir (patch from D o kern/92272 fs [ffs] [hang] Filling a filesystem while creating a sna f kern/91568 fs [ufs] [panic] writing to UFS/softupdates DVD media in o kern/91134 fs [smbfs] [patch] Preserve access and modification time a kern/90815 fs [smbfs] [patch] SMBFS with character conversions somet o kern/89991 fs [ufs] softupdates with mount -ur causes fs UNREFS o kern/88657 fs [smbfs] windows client hang when browsing a samba shar o kern/88266 fs [smbfs] smbfs does not implement UIO_NOCOPY and sendfi o kern/87859 fs [smbfs] System reboot while umount smbfs. o kern/86587 fs [msdosfs] rm -r /PATH fails with lots of small files o kern/85326 fs [smbfs] [panic] saving a file via samba to an overquot o kern/84589 fs [2TB] 5.4-STABLE unresponsive during background fsck 2 o kern/80088 fs [smbfs] Incorrect file time setting on NTFS mounted vi o kern/77826 fs [ext2fs] ext2fs usb filesystem will not mount RW o kern/73484 fs [ntfs] Kernel panic when doing `ls` from the client si o bin/73019 fs [ufs] fsck_ufs(8) cannot alloc 607016868 bytes for ino o kern/71774 fs [ntfs] NTFS cannot "see" files on a WinXP filesystem o kern/68978 fs [panic] [ufs] crashes with failing hard disk, loose po o kern/65920 fs [nwfs] Mounted Netware filesystem behaves strange o kern/65901 fs [smbfs] [patch] smbfs fails fsx write/truncate-down/tr o kern/61503 fs [smbfs] mount_smbfs does not work as non-root o kern/55617 fs [smbfs] Accessing an nsmb-mounted drive via a smb expo o kern/51685 fs [hang] Unbounded inode allocation causes kernel to loc o kern/51583 fs [nullfs] [patch] allow to work with devices and socket o kern/36566 fs [smbfs] System reboot with dead smb mount and umount o kern/18874 fs [2TB] 32bit NFS servers export wrong negative values t 140 problems total. From owner-freebsd-fs@FreeBSD.ORG Tue Oct 20 15:20:04 2009 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0AD4F1065692 for ; Tue, 20 Oct 2009 15:20:04 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id D489B8FC1D for ; Tue, 20 Oct 2009 15:20:03 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n9KFK398091921 for ; Tue, 20 Oct 2009 15:20:03 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id n9KFK3ic091920; Tue, 20 Oct 2009 15:20:03 GMT (envelope-from gnats) Date: Tue, 20 Oct 2009 15:20:03 GMT Message-Id: <200910201520.n9KFK3ic091920@freefall.freebsd.org> To: freebsd-fs@FreeBSD.org From: Mark Linimon Cc: Subject: Re: kern/139715: [zfs] vfs.numvnodes leak on busy zfs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Mark Linimon List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Oct 2009 15:20:04 -0000 The following reply was made to PR kern/139715; it has been noted by GNATS. From: Mark Linimon To: bug-followup@FreeBSD.org Cc: Subject: Re: kern/139715: [zfs] vfs.numvnodes leak on busy zfs Date: Tue, 20 Oct 2009 10:13:05 -0500 ----- Forwarded message from kickbsd kickbsd ----- From: kickbsd kickbsd To: linimon@freebsd.org Subject: Re: kern/139715: [zfs] vfs.numvnodes leak on busy zfs There is newer graph to show tendency http://www.freeimagehosting.net/image.php?e5f0195542.jpg ----- End forwarded message ----- From owner-freebsd-fs@FreeBSD.ORG Wed Oct 21 17:38:41 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4BFBC106566B; Wed, 21 Oct 2009 17:38:41 +0000 (UTC) (envelope-from rnoland@FreeBSD.org) Received: from gizmo.2hip.net (gizmo.2hip.net [64.74.207.195]) by mx1.freebsd.org (Postfix) with ESMTP id 1E19D8FC18; Wed, 21 Oct 2009 17:38:40 +0000 (UTC) Received: from [192.168.1.4] (adsl-154-198-93.ard.bellsouth.net [72.154.198.93]) (authenticated bits=0) by gizmo.2hip.net (8.14.3/8.14.3) with ESMTP id n9LHcZc0082724 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Wed, 21 Oct 2009 13:38:35 -0400 (EDT) (envelope-from rnoland@FreeBSD.org) From: Robert Noland To: Radek =?iso-8859-2?Q?Val=E1=B9ek?= In-Reply-To: <4AD779FC.1070204@buchlovice.org> References: <4AD710D6.70404@buchlovice.org> <1255633430.2175.12.camel@balrog.2hip.net> <4AD779FC.1070204@buchlovice.org> Content-Type: text/plain; charset="iso-8859-2" Organization: FreeBSD Date: Wed, 21 Oct 2009 12:38:29 -0500 Message-Id: <1256146709.2310.9.camel@balrog.2hip.net> Mime-Version: 1.0 X-Mailer: Evolution 2.26.3 FreeBSD GNOME Team Port Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-1.3 required=5.0 tests=AWL,BAYES_00,RCVD_IN_PBL, RCVD_IN_SORBS_DUL, RDNS_DYNAMIC, SPF_SOFTFAIL autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on gizmo.2hip.net Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org Subject: Re: GPT boot with ZFS RAIDZ "ZFS: i/o error - all block copies unavailable" X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Oct 2009 17:38:41 -0000 On Thu, 2009-10-15 at 21:37 +0200, Radek Valá¹ek wrote: > Robert Noland napsal(a): > > On Thu, 2009-10-15 at 14:08 +0200, Radek Valá¹ek wrote: > > > >> Hi, > >> > >> I want to ask if there is something new in adding support to > >> gptzfsboot/zfsboot for reading gang-blocks? > >> I think that the gang block patch will work, though still haven't gotten it tested. However, I'm fairly confident that the issue is not gang block related. Right now, I have setup a disk like this: => 34 1953525101 ada1 GPT (932G) 34 128 1 freebsd-boot (64K) 162 8388608 2 freebsd-swap (4.0G) 8388770 648019968 3 freebsd-zfs (309G) 656408738 648019968 4 freebsd-zfs (309G) 1304428706 648019968 5 freebsd-zfs (309G) 1952448674 1076461 - free - (526M) Note that this is not a raidz pool right now. It is just 3 toplevel partitions setup as a single pool. I finally have this configuration working reliably. At least in this case, the issue is due to all of the partitions not being probed during early boot and so not being added to the list of vdevs for the pool. When zio_read finds a dva that points to a device it doesn't know about, it gives up and whines. Can you detail for me how you have everything configured, so that I can try to replicate it. gpart show, zpool status and zpool get all would be good. I'm not sure that I have enough spare disks lying around to do this properly, but maybe I can use virtual disks or something. robert. > > Ok, I can't figure out any way to test this... beyond the fact that it > > builds and doesn't break my currently working setup. Can you give this > > a try? It should still report if it finds gang blocks, but hopefully > > now will read them as well. > > > > robert. > > > > > Big thanks for the patches Robert, I will definitely test them as soon > as possible (tomorrow) and report the results immediately to list. I can > repeat this issue probably at any time (up to cca 30 times tested with > the same result), so don't bother about the broken booting, I'm prepared > for it... > > vaLin > >> From Sun's docs: > >> > >> Gang blocks > >> > >> When there is not enough contiguous space to write a complete block, the ZIO > >> pipeline will break the I/O up into smaller 'gang blocks' which can later be > >> assembled transparently to appear as complete blocks. > >> > >> Everything works fine for me, until I rewrite kernel/world after system > >> upgrade to latest one (releng_8). After this am I no longer able to boot > >> from zfs raidz1 pool with following messages: > >> > >> >/ ZFS: i/o error - all block copies unavailable > >> />/ ZFS: can't read MOS > >> />/ ZFS: unexpected object set type lld > >> />/ ZFS: unexpected object set type lld > >> />/ > >> />/ FreeBSD/i386 boot > >> />/ Default: z:/boot/kernel/kernel > >> />/ boot: > >> />/ ZFS: unexpected object set type lld > >> />/ > >> />/ FreeBSD/i386 boot > >> />/ Default: tank:/boot/kernel/kernel > >> />/ boot: > >> // > >> /I presume it's the same issue as talked in june-2009 current mailing > >> list > >> http://lists.freebsd.org/pipermail/freebsd-current/2009-June/008589.html > >> > >> Any success in that matter? > >> > >> Thnx for answer. > >> > >> vaLin > >> _______________________________________________ > >> freebsd-current@freebsd.org mailing list > >> http://lists.freebsd.org/mailman/listinfo/freebsd-current > >> To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" > >> > -- Robert Noland FreeBSD From owner-freebsd-fs@FreeBSD.ORG Wed Oct 21 20:15:31 2009 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CFD61106566B; Wed, 21 Oct 2009 20:15:31 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id A773C8FC0C; Wed, 21 Oct 2009 20:15:31 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n9LKFVm9024694; Wed, 21 Oct 2009 20:15:31 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id n9LKFVZx024690; Wed, 21 Oct 2009 20:15:31 GMT (envelope-from linimon) Date: Wed, 21 Oct 2009 20:15:31 GMT Message-Id: <200910212015.n9LKFVZx024690@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-amd64@FreeBSD.org, freebsd-fs@FreeBSD.org From: linimon@FreeBSD.org Cc: Subject: Re: kern/139806: [zfs] [panic] Write attempt to file in ZFS snapshot dir causes panic X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Oct 2009 20:15:31 -0000 Old Synopsis: Write attempt to file in ZFS snapshot dir causes panic New Synopsis: [zfs] [panic] Write attempt to file in ZFS snapshot dir causes panic Responsible-Changed-From-To: freebsd-amd64->freebsd-fs Responsible-Changed-By: linimon Responsible-Changed-When: Wed Oct 21 20:15:16 UTC 2009 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=139806 From owner-freebsd-fs@FreeBSD.ORG Wed Oct 21 23:22:10 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0BF5E106566C; Wed, 21 Oct 2009 23:22:10 +0000 (UTC) (envelope-from artemb@gmail.com) Received: from mail-yw0-f178.google.com (mail-yw0-f178.google.com [209.85.211.178]) by mx1.freebsd.org (Postfix) with ESMTP id 847A18FC13; Wed, 21 Oct 2009 23:22:09 +0000 (UTC) Received: by ywh8 with SMTP id 8so6421781ywh.3 for ; Wed, 21 Oct 2009 16:22:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:received:in-reply-to :references:date:x-google-sender-auth:message-id:subject:from:to:cc :content-type; bh=zErOiIBD3wTDQ80cMtXL3EIM40VLaLzGhX9KwriYoU8=; b=E965eZJsD2c35TF9LxDtHsuIJ0ozRVEjlmJUyWvBs925CaAClgmaph8F+0oZTEu9J8 tHPko7Of0ZVGB22pZ28pMEJdajFzm2ydberKp5TnStL268oSVIWQPI3z7KZBS7ett0X5 PB2p7lk8KEi3KvezxB1Ti4ZjN29ZpI/UmsHcI= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; b=f6kRnCQmGE7+Q+LvMcvc8uEG0mEEWTe6DLagiMf4CTDXnIwNoY1pIQWQa8LMacBJHv QpWrynjAin/R1gCigz7qK/v89Ymk6Ipp2Y9eXO10nahiuxId8f986ekS1qGNDIDEvQls YQa9dea3qICUsdSkylnzdNcUFRx5qeNylW+Sw= MIME-Version: 1.0 Sender: artemb@gmail.com Received: by 10.91.20.28 with SMTP id x28mr8940818agi.23.1256167328941; Wed, 21 Oct 2009 16:22:08 -0700 (PDT) In-Reply-To: References: <4AC1E540.9070001@fsn.hu> <4AC5B2C7.2000200@fsn.hu> <20091002184526.GA1660@garage.freebsd.pl> <4ACDA5EA.2010600@fsn.hu> <4ACDDED0.2070707@fsn.hu> <20091008160718.GB2134@garage.freebsd.pl> <4AD2E118.2050202@fsn.hu> <47C0A3F4-6431-49E5-B780-FA162946C288@freebsd.org> <4AD2E998.8090409@fsn.hu> Date: Wed, 21 Oct 2009 16:22:08 -0700 X-Google-Sender-Auth: 15a0875e3f7e54e8 Message-ID: From: Artem Belevich To: "K. Macy" Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-fs@freebsd.org, Pawel Jakub Dawidek Subject: Re: ARC size constantly shrinks, then ZFS slows down extremely X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Oct 2009 23:22:10 -0000 Hi, Will r197816 from head be MFC'd to stable-8 as commit comment suggested? I've been running my box with the r197816 applied to stable/8 and so far it's been working fine for me. #svn log -v -c 197816 ------------------------------------------------------------------------ r197816 | kmacy | 2009-10-06 14:40:50 -0700 (Tue, 06 Oct 2009) | 5 lines Changed paths: M /head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Prevent paging pressure from draining arc too much - always drain arc if above arc_c_max - never drain arc if arc is below arc_c_max MFC after: 3 days ------------------------------------------------------------------------ --Artem From owner-freebsd-fs@FreeBSD.ORG Thu Oct 22 20:10:04 2009 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 20BF01065672 for ; Thu, 22 Oct 2009 20:10:04 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 0FEED8FC12 for ; Thu, 22 Oct 2009 20:10:04 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n9MKA3Bm090956 for ; Thu, 22 Oct 2009 20:10:03 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id n9MKA3YG090955; Thu, 22 Oct 2009 20:10:03 GMT (envelope-from gnats) Date: Thu, 22 Oct 2009 20:10:03 GMT Message-Id: <200910222010.n9MKA3YG090955@freefall.freebsd.org> To: freebsd-fs@FreeBSD.org From: Jaakko Heinonen Cc: Subject: Re: kern/139806: [zfs] [panic] Write attempt to file in ZFS snapshot dir causes panic X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Jaakko Heinonen List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Oct 2009 20:10:04 -0000 The following reply was made to PR kern/139806; it has been noted by GNATS. From: Jaakko Heinonen To: Carl Chave Cc: bug-followup@FreeBSD.org Subject: Re: kern/139806: [zfs] [panic] Write attempt to file in ZFS snapshot dir causes panic Date: Thu, 22 Oct 2009 22:50:56 +0300 Hi, On 2009-10-21, Carl Chave wrote: > Fixit# echo hello >> test.txt > panic: dirtying snapshot! The problem seems to be that in certain conditions zfs_freebsd_access() uses only vaccess(9) for access check. However vaccess(9) doesn't handle the read-only file system case. Could you try this patch? --- patch begins here --- Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c (revision 198368) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c (working copy) @@ -3989,7 +3989,12 @@ struct thread *a_td; } */ *ap; { + int error; + error = zfs_access(ap->a_vp, ap->a_accmode, 0, ap->a_cred, NULL); + if (error != 0) + return (error); + /* * ZFS itself only knowns about VREAD, VWRITE and VEXEC, the rest * we have to handle by calling vaccess(). @@ -3999,11 +4004,11 @@ znode_t *zp = VTOZ(vp); znode_phys_t *zphys = zp->z_phys; - return (vaccess(vp->v_type, zphys->zp_mode, zphys->zp_uid, - zphys->zp_gid, ap->a_accmode, ap->a_cred, NULL)); + error = vaccess(vp->v_type, zphys->zp_mode, zphys->zp_uid, + zphys->zp_gid, ap->a_accmode, ap->a_cred, NULL); } - return (zfs_access(ap->a_vp, ap->a_accmode, 0, ap->a_cred, NULL)); + return (error); } static int --- patch ends here --- -- Jaakko From owner-freebsd-fs@FreeBSD.ORG Sat Oct 24 00:39:57 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8EAA3106566C for ; Sat, 24 Oct 2009 00:39:57 +0000 (UTC) (envelope-from randy@psg.com) Received: from ran.psg.com (ran.psg.com [IPv6:2001:418:1::36]) by mx1.freebsd.org (Postfix) with ESMTP id D2A158FC18 for ; Sat, 24 Oct 2009 00:39:56 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=rmac.psg.com) by ran.psg.com with esmtp (Exim 4.69 (FreeBSD)) (envelope-from ) id 1N1Ufi-00016q-G6 for freebsd-fs@freebsd.org; Sat, 24 Oct 2009 00:39:55 +0000 Received: from 173.0.128.10.in-addr.arpa.noptr.antlabs.com.psg.com (localhost [127.0.0.1]) by rmac.psg.com (Postfix) with ESMTP id DDFD12B8FA96 for ; Sat, 24 Oct 2009 09:39:53 +0900 (JST) Date: Sat, 24 Oct 2009 09:39:53 +0900 Message-ID: From: Randy Bush To: freebsd-fs User-Agent: Wanderlust/2.15.5 (Almost Unreal) Emacs/22.3 Mule/5.0 (SAKAKI) MIME-Version: 1.0 (generated by SEMI 1.14.6 - "Maruoka") Content-Type: text/plain; charset=US-ASCII Subject: to err or not to err X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Oct 2009 00:39:57 -0000 so, were there errors or not? do i worry/act or not? randy --- dfw1# zpool status -v pool: tank state: ONLINE status: One or more devices has experienced an error resulting in data corruption. Applications may be affected. action: Restore the file in question if possible. Otherwise restore the entire pool from backup. see: http://www.sun.com/msg/ZFS-8000-8A scrub: scrub in progress for 0h5m, 8.59% done, 1h3m to go config: NAME STATE READ WRITE CKSUM tank ONLINE 0 0 0 mirror ONLINE 0 0 0 label/m00-d01 ONLINE 0 0 0 label/m00-d00 ONLINE 0 0 0 mirror ONLINE 0 0 0 label/m01-d00 ONLINE 0 0 0 label/m01-d01 ONLINE 0 0 0 mirror ONLINE 0 0 0 label/m02-d00 ONLINE 0 0 0 label/m02-d01 ONLINE 0 0 0 mirror ONLINE 0 0 0 label/m03-d00 ONLINE 0 0 0 label/m03-d01 ONLINE 0 0 0 mirror ONLINE 0 0 0 label/m04-d00 ONLINE 0 0 0 label/m04-d01 ONLINE 0 0 0 mirror ONLINE 0 0 0 label/m05-d00 ONLINE 0 0 0 label/m05-d01 ONLINE 0 0 0 errors: Permanent errors have been detected in the following files: tank/dfw1:<0x8dffc> tank/dfw1:<0xcfd> tank/dfw1:<0x8dffd> < some hours later > dfw1# zpool status -v pool: tank state: ONLINE scrub: scrub completed after 0h53m with 0 errors on Fri Oct 23 15:20:43 2009 config: NAME STATE READ WRITE CKSUM tank ONLINE 0 0 0 mirror ONLINE 0 0 0 label/m00-d01 ONLINE 0 0 0 label/m00-d00 ONLINE 0 0 0 mirror ONLINE 0 0 0 label/m01-d00 ONLINE 0 0 0 label/m01-d01 ONLINE 0 0 0 mirror ONLINE 0 0 0 label/m02-d00 ONLINE 0 0 0 label/m02-d01 ONLINE 0 0 0 mirror ONLINE 0 0 0 label/m03-d00 ONLINE 0 0 0 label/m03-d01 ONLINE 0 0 0 mirror ONLINE 0 0 0 label/m04-d00 ONLINE 0 0 0 label/m04-d01 ONLINE 0 0 0 mirror ONLINE 0 0 0 label/m05-d00 ONLINE 0 0 0 label/m05-d01 ONLINE 0 0 0 errors: No known data errors -30- From owner-freebsd-fs@FreeBSD.ORG Sat Oct 24 17:44:28 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AA417106566B; Sat, 24 Oct 2009 17:44:28 +0000 (UTC) (envelope-from valin@buchlovice.org) Received: from smtp-sfn.sitkom.cz (smtp-sfn.sitkom.cz [88.146.175.4]) by mx1.freebsd.org (Postfix) with ESMTP id 071EE8FC0C; Sat, 24 Oct 2009 17:44:27 +0000 (UTC) Received: from osiris.buchlovice.sfn (osiris.buchlovice.sfn [10.6.193.10]) by smtp-sfn.sitkom.cz (Postfix) with ESMTP id D89811FA6F5; Sat, 24 Oct 2009 19:44:25 +0200 (CEST) Message-ID: <4AE33CF9.3050308@buchlovice.org> Date: Sat, 24 Oct 2009 19:44:25 +0200 From: =?ISO-8859-2?Q?Radek_Val=E1=B9ek?= User-Agent: Thunderbird 2.0.0.23 (X11/20091015) MIME-Version: 1.0 To: Robert Noland References: <4AD710D6.70404@buchlovice.org> <1255633430.2175.12.camel@balrog.2hip.net> <4AD779FC.1070204@buchlovice.org> <1256146709.2310.9.camel@balrog.2hip.net> In-Reply-To: <1256146709.2310.9.camel@balrog.2hip.net> Content-Type: text/plain; charset=ISO-8859-2; format=flowed Content-Transfer-Encoding: 8bit Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org Subject: Re: GPT boot with ZFS RAIDZ "ZFS: i/o error - all block copies unavailable" X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Oct 2009 17:44:28 -0000 Robert Noland napsal(a): > On Thu, 2009-10-15 at 21:37 +0200, Radek Valá¹ek wrote: > >> Robert Noland napsal(a): >> >>> On Thu, 2009-10-15 at 14:08 +0200, Radek Valá¹ek wrote: >>> >>> >>>> Hi, >>>> >>>> I want to ask if there is something new in adding support to >>>> gptzfsboot/zfsboot for reading gang-blocks? >>>> >>>> > > I think that the gang block patch will work, though still haven't gotten > it tested. However, I'm fairly confident that the issue is not gang > block related. Right now, I have setup a disk like this: > > => 34 1953525101 ada1 GPT (932G) > 34 128 1 freebsd-boot (64K) > 162 8388608 2 freebsd-swap (4.0G) > 8388770 648019968 3 freebsd-zfs (309G) > 656408738 648019968 4 freebsd-zfs (309G) > 1304428706 648019968 5 freebsd-zfs (309G) > 1952448674 1076461 - free - (526M) > > Note that this is not a raidz pool right now. It is just 3 toplevel > partitions setup as a single pool. I finally have this configuration > working reliably. At least in this case, the issue is due to all of the > partitions not being probed during early boot and so not being added to > the list of vdevs for the pool. When zio_read finds a dva that points > to a device it doesn't know about, it gives up and whines. > > Can you detail for me how you have everything configured, so that I can > try to replicate it. gpart show, zpool status and zpool get all > would be good. I'm not sure that I have enough spare disks lying around > to do this properly, but maybe I can use virtual disks or something. > > robert. > > Sorry for not responding so long. Here are details you want from me: # gpart show => 34 1953525101 ad6 GPT (932G) 34 128 1 freebsd-boot (64K) 162 1953524973 2 freebsd-zfs (932G) => 34 1953525101 ad8 GPT (932G) 34 128 1 freebsd-boot (64K) 162 1953524973 2 freebsd-zfs (932G) => 34 1953525101 ad10 GPT (932G) 34 128 1 freebsd-boot (64K) 162 1953524973 2 freebsd-zfs (932G) => 34 1953525101 ad12 GPT (932G) 34 128 1 freebsd-boot (64K) 162 1953524973 2 freebsd-zfs (932G) # zpool status pool: z state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM z ONLINE 0 0 0 raidz1 ONLINE 0 0 0 ad6p2 ONLINE 0 0 0 ad8p2 ONLINE 0 0 0 ad10p2 ONLINE 0 0 0 ad12p2 ONLINE 0 0 0 errors: No known data errors # zpool get all z NAME PROPERTY VALUE SOURCE z size 3.62T - z used 4.62G - z available 3.62T - z capacity 0% - z altroot - default z health ONLINE - z guid 17857007133862981114 - z version 13 default z bootfs z/system local z delegation on default z autoreplace off default z cachefile - default z failmode wait default z listsnapshots off default I've tested your patches but it seems that you're right and it's not gang related issue. I was able to discover these things on a fully functional zfs pool (system compiled with your patches): 1, If I overwrite the file /boot/loader.conf (with copy of itself, or when upgrading kernel/world), next reboot comes with these messages: BTX loader 1.00 BTX version is 1.02 Consoles: internal video/keyboard BIOS drive C: is disk0 BIOS drive D: is disk1 BIOS drive E: is disk2 BIOS drive F: is disk3 BIOS 627kB/3405248kB available memory FreeBSD/i386 bootstrap loader, Revision 1.1 (root@ztest, Thu Oct 22 22:27:22 CEST 2009) Loading /boot/defaults/loader.conf ZFS: i/o error - all block copies unavailable Warning: error reading file /boot/loader.conf Then I'm still able to boot the system, but I must set the boot variables included in loader.conf by hand 2, Next I overwrite the file /boot/loader (with copy of itself, or when upgrading kernel/world) and reboot comes with these messages: BTX loader 1.00 BTX version is 1.02 Consoles: internal video/keyboard BIOS drive C: is disk0 BIOS drive D: is disk1 BIOS drive E: is disk2 BIOS drive F: is disk3 BIOS 627kB/3405248kB available memory FreeBSD/i386 bootstrap loader, Revision 1.1 (root@ztest, Thu Oct 22 22:27:22 CEST 2009) Loading /boot/defaults/loader.conf ZFS: i/o error - all block copies unavailable Warning: error reading file /boot/loader.conf ZFS: i/o error - all block copies unavailable ZFS: i/o error - all block copies unavailable ZFS: i/o error - all block copies unavailable ZFS: i/o error - all block copies unavailable ZFS: i/o error - all block copies unavailable ZFS: i/o error - all block copies unavailable ZFS: i/o error - all block copies unavailable ZFS: i/o error - all block copies unavailable Unable to load a kernel! After that I'm no longer able to boot the system from zfs pool. Hope you have some ideas... vaLin >>> Ok, I can't figure out any way to test this... beyond the fact that it >>> builds and doesn't break my currently working setup. Can you give this >>> a try? It should still report if it finds gang blocks, but hopefully >>> now will read them as well. >>> >>> robert. >>> >>> >>> >> Big thanks for the patches Robert, I will definitely test them as soon >> as possible (tomorrow) and report the results immediately to list. I can >> repeat this issue probably at any time (up to cca 30 times tested with >> the same result), so don't bother about the broken booting, I'm prepared >> for it... >> >> vaLin >> >>>> From Sun's docs: >>>> >>>> Gang blocks >>>> >>>> When there is not enough contiguous space to write a complete block, the ZIO >>>> pipeline will break the I/O up into smaller 'gang blocks' which can later be >>>> assembled transparently to appear as complete blocks. >>>> >>>> Everything works fine for me, until I rewrite kernel/world after system >>>> upgrade to latest one (releng_8). After this am I no longer able to boot >>>> from zfs raidz1 pool with following messages: >>>> >>>> >/ ZFS: i/o error - all block copies unavailable >>>> />/ ZFS: can't read MOS >>>> />/ ZFS: unexpected object set type lld >>>> />/ ZFS: unexpected object set type lld >>>> />/ >>>> />/ FreeBSD/i386 boot >>>> />/ Default: z:/boot/kernel/kernel >>>> />/ boot: >>>> />/ ZFS: unexpected object set type lld >>>> />/ >>>> />/ FreeBSD/i386 boot >>>> />/ Default: tank:/boot/kernel/kernel >>>> />/ boot: >>>> // >>>> /I presume it's the same issue as talked in june-2009 current mailing >>>> list >>>> http://lists.freebsd.org/pipermail/freebsd-current/2009-June/008589.html >>>> >>>> Any success in that matter? >>>> >>>> Thnx for answer. >>>> >>>> vaLin >>>> _______________________________________________ >>>> freebsd-current@freebsd.org mailing list >>>> http://lists.freebsd.org/mailman/listinfo/freebsd-current >>>> To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" >>>> >>>> From owner-freebsd-fs@FreeBSD.ORG Sat Oct 24 17:46:59 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1535E106566C; Sat, 24 Oct 2009 17:46:59 +0000 (UTC) (envelope-from stef-list@memberwebs.com) Received: from memberwebs.com (memberwebs.com [94.75.203.95]) by mx1.freebsd.org (Postfix) with ESMTP id EF46D8FC19; Sat, 24 Oct 2009 17:46:57 +0000 (UTC) Received: from [172.27.5.159] (unknown [172.27.5.159]) by memberwebs.com (Postfix) with ESMTP id B41F283E4C6; Sat, 24 Oct 2009 17:46:52 +0000 (UTC) Message-ID: <4AE33D89.6050807@memberwebs.com> Date: Sat, 24 Oct 2009 12:46:49 -0500 From: Stef Walter User-Agent: Thunderbird 2.0.0.23 (X11/20090817) MIME-Version: 1.0 To: freebsd-fs@freebsd.org References: <4AD77230.3030803@memberwebs.com> In-Reply-To: <4AD77230.3030803@memberwebs.com> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: pjd@FreeBSD.org Subject: Re: Deadlock after canceled zfs recv X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: stef@memberwebs.com List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Oct 2009 17:46:59 -0000 Stef Walter wrote: > I'm running the latest RELENG_8, and been doing some pre-production > stress testing. > > I can deadlock (reproduceable) after a canceled ssh + zfs recv. Here's > how I reproduce the problem: Pawel, I hope it's okay if I CC you on this problem. I'd like to contribute and help get this problem sorted out before the 8.0 release. I've done a deadlock post mortem on this problem. It's below. There are several zfs processes waiting on locks. Is there anything I can do to further research the problem. Thanks in advance. Cheers, Stef ---------------------------------------------------------------------- DEADLOCK POSTMORTEM db> show pcpu cpuid = 0 dynamic pcpu = 0x68e280 curthread = 0xc596fb40: pid 11 "idle: cpu0" curpcb = 0xc54cdd90 fpcurthread = none idlethread = 0xc596fb40: pid 11 "idle: cpu0" APIC ID = 0 currentldt = 0x50 spin locks held: db> show allpcpu Current CPU: 0 cpuid = 0 dynamic pcpu = 0x68e280 curthread = 0xc596fb40: pid 11 "idle: cpu0" curpcb = 0xc54cdd90 fpcurthread = none idlethread = 0xc596fb40: pid 11 "idle: cpu0" APIC ID = 0 currentldt = 0x50 spin locks held: db> show locks db> show alllocks db> show lockedvnods Locked vnodes db> alltrace Tracing command zfs pid 1559 tid 100126 td 0xc673ab40 sched_switch(c673ab40,0,104,191,ecf50dd2,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c673ab40,0,c0c7a69a,24b,c673ab40,...) at sleepq_switch+0x15f sleepq_wait(c97710a8,0,e8188ab4,1,0,...) at sleepq_wait+0x63 _cv_wait(c97710a8,c9771078,208,206,c97710a8,...) at _cv_wait+0x240 dsl_dataset_hold_ref(c6b0f9c0,e8188b58,c6268de1,241,e8188b28,...) at dsl_dataset_hold_ref+0xa0 dsl_dataset_hold(d1be8000,c6b0f9c0,e8188b58,c086d87d,1,...) at dsl_dataset_hold+0x10c dmu_objset_open(d1be8000,5,9,e8188b84,e8188b94,...) at dmu_objset_open+0xc3 zfs_ioc_objset_stats(d1be8000,24,d1be800e,0,d1be8938,...) at zfs_ioc_objset_stats+0x38 zfs_ioc_dataset_list_next(d1be8000,0,0,0,0,...) at zfs_ioc_dataset_list_next+0x11f zfsdev_ioctl(c6136100,cc285a13,d1be8000,3,c673ab40,...) at zfsdev_ioctl+0x7d devfs_ioctl_f(c5ed3a10,cc285a13,d1be8000,c60ff680,c673ab40,...) at devfs_ioctl_f+0xf8 kern_ioctl(c673ab40,3,cc285a13,d1be8000,173ab40,...) at kern_ioctl+0x1fd ioctl(c673ab40,e8188cf8,c,e8188d38,c0d5c928,...) at ioctl+0x134 syscall(e8188d38) at syscall+0x2b4 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (54, FreeBSD ELF32, ioctl), eip = 0x282211f3, esp = 0xbfbfc9cc, ebp = 0xbfbfd638 --- Tracing command zfs pid 1558 tid 100124 td 0xc6122480 sched_switch(c6122480,0,104,191,fc4c78d2,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c6122480,0,c0c7a69a,26e,2,...) at sleepq_switch+0x15f sleepq_timedwait(c0dcac64,0,c62689d8,2,0,...) at sleepq_timedwait+0x6b _sleep(c0dcac64,0,0,c62689d8,1,...) at _sleep+0x339 pause(c62689d8,1,c9771200,e818250c,c61a1ec2,...) at pause+0x47 dnode_special_close(c9446000,c626870f,c61a0c60,c9771200,c9771200,...) at dnode_special_close+0x2c dmu_objset_evict(c9771000,c9771200,1cb,0,c6265be3,...) at dmu_objset_evict+0x82 dsl_dataset_destroy(c9771000,c6268520,16,e8182984,e8182944,...) at dsl_dataset_destroy+0x24c dmu_recv_abort_cleanup(e8182984,c626d723,1cb,c6265810,e818292c,...) at dmu_recv_abort_cleanup+0x2f dmu_recv_stream(e8182984,c5ed4dc8,e818299c,0,0,...) at dmu_recv_stream+0x9de zfs_ioc_recv(cfa8f000,0,0,0,0,...) at zfs_ioc_recv+0x2d4 zfsdev_ioctl(c6136100,cc285a1c,cfa8f000,3,c6122480,...) at zfsdev_ioctl+0x7d devfs_ioctl_f(c5ed4700,cc285a1c,cfa8f000,c8488d00,c6122480,...) at devfs_ioctl_f+0xf8 kern_ioctl(c6122480,3,cc285a1c,cfa8f000,1000000,...) at kern_ioctl+0x1fd ioctl(c6122480,e8182cf8,c,c0c8e3e7,c0d5c928,...) at ioctl+0x134 syscall(e8182d38) at syscall+0x2b4 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (54, FreeBSD ELF32, ioctl), eip = 0x282211f3, esp = 0xbfbfbcfc, ebp = 0xbfbfbd28 --- Tracing command zfskern pid 1521 tid 100117 td 0xc61226c0 sched_switch(c61226c0,0,104,191,a1331866,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c61226c0,0,c0c7a69a,26e,c61226c0,...) at sleepq_switch+0x15f sleepq_timedwait(c65a65dc,0,e8164c28,1,0,...) at sleepq_timedwait+0x6b _cv_timedwait(c65a65dc,c65a6584,7530,c65a65dc,c65a6580,...) at _cv_timedwait+0x250 txg_thread_wait(7530,0,3b9aca00,0,3e8,...) at txg_thread_wait+0x44 txg_sync_thread(c65a6400,e8164d38,c0c717a4,343,c611d550,...) at txg_sync_thread+0x122 fork_exit(c61d37e0,c65a6400,e8164d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe8164d70, ebp = 0 --- Tracing command zfskern pid 1521 tid 100116 td 0xc6122900 sched_switch(c6122900,0,104,191,a133372e,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c6122900,0,c0c7a69a,24b,c6122900,...) at sleepq_switch+0x15f sleepq_wait(c65a65ec,0,e8160c50,1,0,...) at sleepq_wait+0x63 _cv_wait(c65a65ec,c65a6584,c65a65f4,c65a65ec,c65a6580,...) at _cv_wait+0x240 txg_thread_wait(0,0,c626d723,18d,10,...) at txg_thread_wait+0x87 txg_quiesce_thread(c65a6400,e8160d38,c0c717a4,343,c611d550,...) at txg_quiesce_thread+0xb5 fork_exit(c61d3070,c65a6400,e8160d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe8160d70, ebp = 0 --- Tracing command zfskern pid 1521 tid 100114 td 0xc6122d80 sched_switch(c6122d80,0,104,191,a12ac4fe,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,4c,...) at mi_switch+0x200 sleepq_switch(c6122d80,0,c0c7a69a,24b,0,...) at sleepq_switch+0x15f sleepq_wait(c6541908,4c,c62713d8,0,0,...) at sleepq_wait+0x63 _sleep(c6541908,c654191c,24c,c62713d8,0,...) at _sleep+0x36b vdev_geom_worker(c6541900,e8154d38,c0c717a4,343,c611d550,...) at vdev_geom_worker+0x124 fork_exit(c621a170,c6541900,e8154d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe8154d70, ebp = 0 --- Tracing command zfskern pid 1521 tid 100087 td 0xc6120b40 sched_switch(c6120b40,0,104,191,7f209aca,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c6120b40,0,c0c7a69a,26e,c6120b40,...) at sleepq_switch+0x15f sleepq_timedwait(c627e534,0,e8109c24,1,0,...) at sleepq_timedwait+0x6b _cv_timedwait(c627e534,c627e520,3e8,1131,0,...) at _cv_timedwait+0x250 l2arc_feed_thread(0,e8109d38,c0c717a4,343,c611d550,...) at l2arc_feed_thread+0x131 fork_exit(c618f090,0,e8109d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe8109d70, ebp = 0 --- Tracing command zfskern pid 1521 tid 100082 td 0xc61216c0 sched_switch(c61216c0,0,104,191,7f20c4da,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c61216c0,0,c0c7a69a,26e,c61216c0,...) at sleepq_switch+0x15f sleepq_timedwait(c627614c,0,c57f6c80,1,0,...) at sleepq_timedwait+0x6b _cv_timedwait(c627614c,c6276138,3e8,0,3e8,...) at _cv_timedwait+0x250 arc_reclaim_thread(0,c57f6d38,c0c717a4,343,c611d550,...) at arc_reclaim_thread+0x39f fork_exit(c6191f00,0,c57f6d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc57f6d70, ebp = 0 --- Tracing command bash pid 1519 tid 100080 td 0xc6121b40 sched_switch(c6121b40,0,104,191,ebff50ce,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,5c,...) at mi_switch+0x200 sleepq_switch(c6121b40,0,c0c7a69a,18b,5c,...) at sleepq_switch+0x15f sleepq_catch_signals(c0c7a69a,15b,0,100,100,...) at sleepq_catch_signals+0xb7 sleepq_wait_sig(c611daa0,5c,c0c7cf0b,100,0,...) at sleepq_wait_sig+0x17 _sleep(c611daa0,c611db28,15c,c0c7cf0b,0,...) at _sleep+0x354 kern_wait(c6121b40,ffffffff,c57f0c74,6,0,...) at kern_wait+0xb76 wait4(c6121b40,c57f0cf8,10,c0c7cdfa,c0d5c404,...) at wait4+0x3b syscall(c57f0d38) at syscall+0x2b4 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (7, FreeBSD ELF32, wait4), eip = 0x80e1f4f, esp = 0xbfbfe9dc, ebp = 0xbfbfe9f8 --- Tracing command sshd pid 1516 tid 100075 td 0xc5ede480 sched_switch(c5ede480,0,104,191,ebe40d02,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c5ede480,0,c0c7a69a,18b,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c087111a,c5e7cd50,0,c0c74a49,c5ede480,...) at sleepq_catch_signals+0xb7 sleepq_wait_sig(c5e7cd64,0,c57cda7c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c5e7cd64,c5e7cd50,c0c7c95a,5d1,c5ed30e0,...) at _cv_wait_sig+0x240 seltdwait(c5ed30e0,58,c5f40200,c5ede480,58,...) at seltdwait+0xa2 kern_select(c5ede480,9,286030c4,286030cc,0,0,20,65,280fdd50) at kern_select+0x4f4 select(c5ede480,c57cdcf8,14,c0c5e8da,c0d5cd6c,...) at select+0x66 syscall(c57cdd38) at syscall+0x2b4 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (93, FreeBSD ELF32, select), eip = 0x283cc1d3, esp = 0xbfbfde5c, ebp = 0xbfbfdea8 --- Tracing command login pid 1514 tid 100070 td 0xc5ee0000 sched_switch(c5ee0000,0,104,191,2065f36,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c5ee0000,0,c0c7a69a,18b,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c087111a,c59ede04,0,c0c74a49,c5ee0000,...) at sleepq_catch_signals+0xb7 sleepq_wait_sig(c59ede70,0,c57bcb0c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c59ede70,c59ede04,c0c7e645,4d5,0,...) at _cv_wait_sig+0x240 tty_wait(c59ede00,c59ede70,1,c57bcb87,c57bcb94,...) at tty_wait+0x71 ttydisc_read(c59ede00,c57bcc58,0,9e,0,...) at ttydisc_read+0xef ttydev_read(c5afd400,c57bcc58,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c5ed39a0,c57bcc58,c596b100,0,c5ee0000,...) at devfs_read_f+0x7e dofileread(c57bcc58,ffffffff,ffffffff,0,c5ed39a0,...) at dofileread+0x96 kern_readv(c5ee0000,0,c57bcc58,c57bcc78,1,...) at kern_readv+0x58 read(c5ee0000,c57bccf8,c,c0c7cefa,c0d5c394,...) at read+0x4f syscall(c57bcd38) at syscall+0x2b4 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (3, FreeBSD ELF32, read), eip = 0x281af253, esp = 0xbfbfe6bc, ebp = 0xbfbfe958 --- Tracing command getty pid 1513 tid 100083 td 0xc6121480 sched_switch(c6121480,0,104,191,51bbee4e,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c6121480,0,c0c7a69a,18b,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c0c7a69a,15b,0,c6121480,c6121480,...) at sleepq_catch_signals+0xb7 sleepq_wait_sig(c5c7d470,0,c0c7ec9a,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c5c7d470,c0dca5b0,c0c7e645,4d5,0,...) at _cv_wait_sig+0x240 tty_wait(c5c7d400,c5c7d470,c57f9c58,1,0,...) at tty_wait+0x71 ttydisc_read(c5c7d400,c57f9c58,0,9e,0,...) at ttydisc_read+0x22c ttydev_read(c5c85900,c57f9c58,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c5ed33f0,c57f9c58,c596b100,0,c6121480,...) at devfs_read_f+0x7e dofileread(c57f9c58,ffffffff,ffffffff,0,c5ed33f0,...) at dofileread+0x96 kern_readv(c6121480,0,c57f9c58,c57f9c78,1,...) at kern_readv+0x58 read(c6121480,c57f9cf8,c,c0c8e3e7,c0d5c394,...) at read+0x4f syscall(c57f9d38) at syscall+0x2b4 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (3, FreeBSD ELF32, read), eip = 0x2818f253, esp = 0xbfbfedcc, ebp = 0xbfbfedf8 --- Tracing command getty pid 1512 tid 100077 td 0xc5ede000 sched_switch(c5ede000,0,104,191,519be396,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c5ede000,0,c0c7a69a,18b,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c0c7a69a,15b,0,c5ede000,c5ede000,...) at sleepq_catch_signals+0xb7 sleepq_wait_sig(c59eda70,0,c0c7ec9a,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c59eda70,c0dca5b0,c0c7e645,4d5,0,...) at _cv_wait_sig+0x240 tty_wait(c59eda00,c59eda70,c57e7c58,1,0,...) at tty_wait+0x71 ttydisc_read(c59eda00,c57e7c58,0,9e,0,...) at ttydisc_read+0x22c ttydev_read(c5c85700,c57e7c58,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c5ed38c0,c57e7c58,c596b100,0,c5ede000,...) at devfs_read_f+0x7e dofileread(c57e7c58,ffffffff,ffffffff,0,c5ed38c0,...) at dofileread+0x96 kern_readv(c5ede000,0,c57e7c58,c57e7c78,1,...) at kern_readv+0x58 read(c5ede000,c57e7cf8,c,c0c8e3e7,c0d5c394,...) at read+0x4f syscall(c57e7d38) at syscall+0x2b4 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (3, FreeBSD ELF32, read), eip = 0x2818f253, esp = 0xbfbfedcc, ebp = 0xbfbfedf8 --- Tracing command getty pid 1511 tid 100069 td 0xc5ee0240 sched_switch(c5ee0240,0,104,191,517bb2e2,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c5ee0240,0,c0c7a69a,18b,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c0c7a69a,15b,0,c5ee0240,c5ee0240,...) at sleepq_catch_signals+0xb7 sleepq_wait_sig(c59ed670,0,c0c7ec9a,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c59ed670,c0dca5b0,c0c7e645,4d5,0,...) at _cv_wait_sig+0x240 tty_wait(c59ed600,c59ed670,c57b8c58,1,0,...) at tty_wait+0x71 ttydisc_read(c59ed600,c57b8c58,0,9e,0,...) at ttydisc_read+0x22c ttydev_read(c5c85e00,c57b8c58,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c5ed3268,c57b8c58,c596b100,0,c5ee0240,...) at devfs_read_f+0x7e dofileread(c57b8c58,ffffffff,ffffffff,0,c5ed3268,...) at dofileread+0x96 kern_readv(c5ee0240,0,c57b8c58,c57b8c78,1,...) at kern_readv+0x58 read(c5ee0240,c57b8cf8,c,c0c8e3e7,c0d5c394,...) at read+0x4f syscall(c57b8d38) at syscall+0x2b4 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (3, FreeBSD ELF32, read), eip = 0x2818f253, esp = 0xbfbfedcc, ebp = 0xbfbfedf8 --- Tracing command getty pid 1510 tid 100081 td 0xc6121900 sched_switch(c6121900,0,104,191,515ba0da,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c6121900,0,c0c7a69a,18b,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c0c7a69a,15b,0,c6121900,c6121900,...) at sleepq_catch_signals+0xb7 sleepq_wait_sig(c59ed470,0,c0c7ec9a,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c59ed470,c0dca5b0,c0c7e645,4d5,0,...) at _cv_wait_sig+0x240 tty_wait(c59ed400,c59ed470,c57f3c58,1,0,...) at tty_wait+0x71 ttydisc_read(c59ed400,c57f3c58,0,9e,0,...) at ttydisc_read+0x22c ttydev_read(c5c86100,c57f3c58,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c5ed3ce8,c57f3c58,c596b100,0,c6121900,...) at devfs_read_f+0x7e dofileread(c57f3c58,ffffffff,ffffffff,0,c5ed3ce8,...) at dofileread+0x96 kern_readv(c6121900,0,c57f3c58,c57f3c78,1,...) at kern_readv+0x58 read(c6121900,c57f3cf8,c,c0c8e3e7,c0d5c394,...) at read+0x4f syscall(c57f3d38) at syscall+0x2b4 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (3, FreeBSD ELF32, read), eip = 0x2818f253, esp = 0xbfbfedcc, ebp = 0xbfbfedf8 --- Tracing command getty pid 1509 tid 100063 td 0xc5e56240 sched_switch(c5e56240,0,104,191,513b103e,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c5e56240,0,c0c7a69a,18b,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c0c7a69a,15b,0,c5e56240,c5e56240,...) at sleepq_catch_signals+0xb7 sleepq_wait_sig(c59eca70,0,c0c7ec9a,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c59eca70,c0dca5b0,c0c7e645,4d5,0,...) at _cv_wait_sig+0x240 tty_wait(c59eca00,c59eca70,c5799c58,1,0,...) at tty_wait+0x71 ttydisc_read(c59eca00,c5799c58,0,9e,0,...) at ttydisc_read+0x22c ttydev_read(c5c85b00,c5799c58,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c5ed3818,c5799c58,c596b100,0,c5e56240,...) at devfs_read_f+0x7e dofileread(c5799c58,ffffffff,ffffffff,0,c5ed3818,...) at dofileread+0x96 kern_readv(c5e56240,0,c5799c58,c5799c78,1,...) at kern_readv+0x58 read(c5e56240,c5799cf8,c,c0c8e3e7,c0d5c394,...) at read+0x4f syscall(c5799d38) at syscall+0x2b4 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (3, FreeBSD ELF32, read), eip = 0x2818f253, esp = 0xbfbfedcc, ebp = 0xbfbfedf8 --- Tracing command getty pid 1508 tid 100086 td 0xc6120d80 sched_switch(c6120d80,0,104,191,511a69a2,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c6120d80,0,c0c7a69a,18b,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c0c7a69a,15b,0,c6120d80,c6120d80,...) at sleepq_catch_signals+0xb7 sleepq_wait_sig(c59ed270,0,c0c7ec9a,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c59ed270,c0dca5b0,c0c7e645,4d5,0,...) at _cv_wait_sig+0x240 tty_wait(c59ed200,c59ed270,e80fcc58,1,0,...) at tty_wait+0x71 ttydisc_read(c59ed200,e80fcc58,0,9e,0,...) at ttydisc_read+0x22c ttydev_read(c5c85600,e80fcc58,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c5ed3310,e80fcc58,c596b100,0,c6120d80,...) at devfs_read_f+0x7e dofileread(e80fcc58,ffffffff,ffffffff,0,c5ed3310,...) at dofileread+0x96 kern_readv(c6120d80,0,e80fcc58,e80fcc78,1,...) at kern_readv+0x58 read(c6120d80,e80fccf8,c,c0c8e3e7,c0d5c394,...) at read+0x4f syscall(e80fcd38) at syscall+0x2b4 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (3, FreeBSD ELF32, read), eip = 0x2818f253, esp = 0xbfbfedcc, ebp = 0xbfbfedf8 --- Tracing command getty pid 1507 tid 100085 td 0xc6121000 sched_switch(c6121000,0,104,191,51fdf8aa,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c6121000,0,c0c7a69a,18b,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c0c7a69a,15b,0,c6121000,c6121000,...) at sleepq_catch_signals+0xb7 sleepq_wait_sig(c59edc70,0,c0c7ec9a,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c59edc70,c0dca5b0,c0c7e645,4d5,0,...) at _cv_wait_sig+0x240 tty_wait(c59edc00,c59edc70,c57ffc58,1,0,...) at tty_wait+0x71 ttydisc_read(c59edc00,c57ffc58,0,9e,0,...) at ttydisc_read+0x22c ttydev_read(c5c86000,c57ffc58,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c5ed3e70,c57ffc58,c596b100,0,c6121000,...) at devfs_read_f+0x7e dofileread(c57ffc58,ffffffff,ffffffff,0,c5ed3e70,...) at dofileread+0x96 kern_readv(c6121000,0,c57ffc58,c57ffc78,1,...) at kern_readv+0x58 read(c6121000,c57ffcf8,c,c0c8e3e7,c0d5c394,...) at read+0x4f syscall(c57ffd38) at syscall+0x2b4 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (3, FreeBSD ELF32, read), eip = 0x2818f253, esp = 0xbfbfedcc, ebp = 0xbfbfedf8 --- Tracing command getty pid 1506 tid 100084 td 0xc6121240 sched_switch(c6121240,0,104,191,50fa4d42,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c6121240,0,c0c7a69a,18b,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c0c7a69a,15b,0,c6121240,c6121240,...) at sleepq_catch_signals+0xb7 sleepq_wait_sig(c59ed870,0,c0c7ec9a,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c59ed870,c0dca5b0,c0c7e645,4d5,0,...) at _cv_wait_sig+0x240 tty_wait(c59ed800,c59ed870,c57fcc58,1,0,...) at tty_wait+0x71 ttydisc_read(c59ed800,c57fcc58,0,9e,0,...) at ttydisc_read+0x22c ttydev_read(c5c85a00,c57fcc58,0,0,1,...) at ttydev_read+0xaa devfs_read_f(c5ed3a80,c57fcc58,c596b100,0,c6121240,...) at devfs_read_f+0x7e dofileread(c57fcc58,ffffffff,ffffffff,0,c5ed3a80,...) at dofileread+0x96 kern_readv(c6121240,0,c57fcc58,c57fcc78,1,...) at kern_readv+0x58 read(c6121240,c57fccf8,c,c0c8e3e7,c0d5c394,...) at read+0x4f syscall(c57fcd38) at syscall+0x2b4 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (3, FreeBSD ELF32, read), eip = 0x2818f253, esp = 0xbfbfedcc, ebp = 0xbfbfedf8 --- Tracing command cron pid 1455 tid 100065 td 0xc5ee0b40 sched_switch(c5ee0b40,0,104,191,153fd976,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,5c,...) at mi_switch+0x200 sleepq_switch(c5ee0b40,0,c0c7a69a,18b,5c,...) at sleepq_switch+0x15f sleepq_catch_signals(ea61,c08b8560,c5ee0b40,0,100,...) at sleepq_catch_signals+0xb7 sleepq_timedwait_sig(c0dcae04,5c,c0c7759a,100,0,...) at sleepq_timedwait_sig+0x1a _sleep(c0dcae04,0,15c,c0c7759a,ea61,...) at _sleep+0x31e kern_nanosleep(c5ee0b40,c57a1c64,c57a1c6c,3c,0,...) at kern_nanosleep+0xc1 nanosleep(c5ee0b40,c57a1cf8,8,c0c7d048,c0d5dd80,...) at nanosleep+0x6f syscall(c57a1d38) at syscall+0x2b4 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (240, FreeBSD ELF32, nanosleep), eip = 0x28179c2f, esp = 0xbfbfec8c, ebp = 0xbfbfecb8 --- Tracing command sendmail pid 1449 tid 100064 td 0xc5ee0d80 sched_switch(c5ee0d80,0,104,191,22a85a16,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,68,...) at mi_switch+0x200 sleepq_switch(c5ee0d80,0,c0c7a69a,18b,68,...) at sleepq_switch+0x15f sleepq_catch_signals(c0c7a69a,15b,0,100,100,...) at sleepq_catch_signals+0xb7 sleepq_wait_sig(c59deaf8,68,c0c2a55d,100,0,...) at sleepq_wait_sig+0x17 _sleep(c59deaf8,c59deb28,168,c0c2a55d,0,...) at _sleep+0x354 kern_sigsuspend(c5ee0d80,0,0,0,0,...) at kern_sigsuspend+0xe4 sigsuspend(c5ee0d80,c579dcf8,4,c0c7cdfa,c0d5e88c,...) at sigsuspend+0x4d syscall(c579dd38) at syscall+0x2b4 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (341, FreeBSD ELF32, sigsuspend), eip = 0x28332a3b, esp = 0xbfbfcf9c, ebp = 0xbfbfcfc8 --- Tracing command sendmail pid 1443 tid 100067 td 0xc5ee06c0 sched_switch(c5ee06c0,0,104,191,ecb7afa2,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c5ee06c0,0,c0c7a69a,18b,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c57b0a4c,c087111a,c5e5cb50,0,c5ee06c0,...) at sleepq_catch_signals+0xb7 sleepq_timedwait_sig(c5e5cb64,0,c57b0a7c,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(c5e5cb64,c5e5cb50,1389,5d1,c5ed3bd0,...) at _cv_timedwait_sig+0x250 seltdwait(c57b0c28,c57b0c30,c60ff500,c5ee06c0,c188b014,...) at seltdwait+0x8a kern_select(c5ee06c0,5,bfbfc520,0,0,c57b0c70,20,5,0) at kern_select+0x4f4 select(c5ee06c0,c57b0cf8,14,c0c7d31e,c0d5cd6c,...) at select+0x66 syscall(c57b0d38) at syscall+0x2b4 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (93, FreeBSD ELF32, select), eip = 0x283d81d3, esp = 0xbfbfc48c, ebp = 0xbfbfcfb8 --- Tracing command sshd pid 1436 tid 100071 td 0xc5eded80 sched_switch(c5eded80,0,104,191,4d9d04fe,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c5eded80,0,c0c7a69a,18b,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c087111a,c5f2d850,0,c0c74a49,c5eded80,...) at sleepq_catch_signals+0xb7 sleepq_wait_sig(c5f2d864,0,c57c0a7c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c5f2d864,c5f2d850,c0c7c95a,5d1,c5ed3850,...) at _cv_wait_sig+0x240 seltdwait(c5ed3850,58,c6100b00,c5eded80,0,...) at seltdwait+0xa2 kern_select(c5eded80,5,286090ac,0,0,0,20,65,a) at kern_select+0x4f4 select(c5eded80,c57c0cf8,14,c0c7d770,c0d5cd6c,...) at select+0x66 syscall(c57c0d38) at syscall+0x2b4 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (93, FreeBSD ELF32, select), eip = 0x283cc1d3, esp = 0xbfbfdf1c, ebp = 0xbfbfee38 --- Tracing command syslogd pid 1206 tid 100074 td 0xc5ede6c0 sched_switch(c5ede6c0,0,104,191,b65fba92,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c5ede6c0,0,c0c7a69a,18b,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c087111a,c5e7bc10,0,c0c74a49,c5ede6c0,...) at sleepq_catch_signals+0xb7 sleepq_wait_sig(c5e7bc24,0,c57caa7c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c5e7bc24,c5e7bc10,c0c7c95a,5d1,c5ed32d8,...) at _cv_wait_sig+0x240 seltdwait(c5ed32d8,58,c596b100,c5ede6c0,0,...) at seltdwait+0xa2 kern_select(c5ede6c0,9,282290a8,0,0,0,20,65,281a4ad8) at kern_select+0x4f4 select(c5ede6c0,c57cacf8,14,c0c7cefa,c0d5cd6c,...) at select+0x66 syscall(c57cad38) at syscall+0x2b4 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (93, FreeBSD ELF32, select), eip = 0x281921d3, esp = 0xbfbfde7c, ebp = 0xbfbfee28 --- Tracing command dhclient pid 1157 tid 100068 td 0xc5ee0480 sched_switch(c5ee0480,0,104,191,125256de,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c5ee0480,0,c0c7a69a,18b,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c57b4aa8,c087111a,c5e7cdd0,0,c5ee0480,...) at sleepq_catch_signals+0xb7 sleepq_timedwait_sig(c5e7cde4,0,c57b4ad8,101,0,...) at sleepq_timedwait_sig+0x1a _cv_timedwait_sig(c5e7cde4,c5e7cdd0,5265431,5d1,c57b4b64,...) at _cv_timedwait_sig+0x250 seltdwait(c57b4c5c,c57b4c64,4df,c5ee0480,c57b4b5c,...) at seltdwait+0x8a poll(c5ee0480,c57b4cf8,c,c0c15d47,c0d5da1c,...) at poll+0x300 syscall(c57b4d38) at syscall+0x2b4 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (209, FreeBSD ELF32, poll), eip = 0x2813122f, esp = 0xbfbfedcc, ebp = 0xbfbfedf8 --- Tracing command dhclient pid 1120 tid 100073 td 0xc5ede900 sched_switch(c5ede900,0,104,191,8b5d5aba,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c5ede900,0,c0c7a69a,18b,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c087111a,c5f2e5d0,0,c0c74a49,c5ede900,...) at sleepq_catch_signals+0xb7 sleepq_wait_sig(c5f2e5e4,0,c57c7ad8,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c5f2e5e4,c5f2e5d0,c0c7c95a,5d1,c57c7b5c,...) at _cv_wait_sig+0x240 seltdwait(c5f6692c,c0c7c95a,4df,c5ede900,c57c7b5c,...) at seltdwait+0xa2 poll(c5ede900,c57c7cf8,c,c57c7cb8,c0d5da1c,...) at poll+0x300 syscall(c57c7d38) at syscall+0x2b4 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (209, FreeBSD ELF32, poll), eip = 0x2813122f, esp = 0xbfbfedcc, ebp = 0xbfbfedf8 --- Tracing command devd pid 971 tid 100072 td 0xc5edeb40 sched_switch(c5edeb40,0,104,191,7c9b5d12,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c5edeb40,0,c0c7a69a,18b,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c087111a,c5f2d650,0,c0c74a49,c5edeb40,...) at sleepq_catch_signals+0xb7 sleepq_wait_sig(c5f2d664,0,c57c4a7c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c5f2d664,c5f2d650,c0c7c95a,5d1,c5ed34d0,...) at _cv_wait_sig+0x240 seltdwait(c5ed34d0,58,c596b100,c5edeb40,c0c6e5c4,...) at seltdwait+0xa2 kern_select(c5edeb40,6,bfbfe9b0,0,0,0,20,65,10) at kern_select+0x4f4 select(c5edeb40,c57c4cf8,14,c0c15d47,c0d5cd6c,...) at select+0x66 syscall(c57c4d38) at syscall+0x2b4 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (93, FreeBSD ELF32, select), eip = 0x80880e3, esp = 0xbfbfe97c, ebp = 0xbfbfee58 --- Tracing command moused pid 954 tid 100066 td 0xc5ee0900 sched_switch(c5ee0900,0,104,191,5e2c030e,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c5ee0900,0,c0c7a69a,18b,0,...) at sleepq_switch+0x15f sleepq_catch_signals(c087111a,c5f2ed50,0,c0c74a49,c5ee0900,...) at sleepq_catch_signals+0xb7 sleepq_wait_sig(c5f2ed64,0,c57aca7c,101,0,...) at sleepq_wait_sig+0x17 _cv_wait_sig(c5f2ed64,c5f2ed50,c0c7c95a,5d1,c5ed3000,...) at _cv_wait_sig+0x240 seltdwait(c5ed3000,58,c596b100,c5ee0900,0,...) at seltdwait+0xa2 kern_select(c5ee0900,7,bfbfea14,0,0,0,20,65,0) at kern_select+0x4f4 select(c5ee0900,c57accf8,14,c0c8e3e7,c0d5cd6c,...) at select+0x66 syscall(c57acd38) at syscall+0x2b4 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (93, FreeBSD ELF32, select), eip = 0x281ac1d3, esp = 0xbfbfe99c, ebp = 0xbfbfeb38 --- Tracing command flowcleaner pid 21 tid 100062 td 0xc5e56480 sched_switch(c5e56480,0,104,191,8f896f2e,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c5e56480,0,c0c7a69a,26e,c5e56480,...) at sleepq_switch+0x15f sleepq_timedwait(c0f37888,0,c5791cc4,1,0,...) at sleepq_timedwait+0x6b _cv_timedwait(c0f37888,c0f37890,2710,3ec,0,...) at _cv_timedwait+0x250 flowtable_cleaner(0,c5791d38,c0c717a4,343,c5ea1000,...) at flowtable_cleaner+0x1af fork_exit(c0927ec0,0,c5791d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5791d70, ebp = 0 --- Tracing command softdepflush pid 20 tid 100061 td 0xc5e566c0 sched_switch(c5e566c0,0,104,191,a95525e6,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,44,...) at mi_switch+0x200 sleepq_switch(c5e566c0,0,c0c7a69a,26e,0,...) at sleepq_switch+0x15f sleepq_timedwait(c0f42f80,44,c0c9cf37,0,0,...) at sleepq_timedwait+0x6b _sleep(c0f42f80,c0f42f24,44,c0c9cf37,3e8,...) at _sleep+0x339 softdep_flush(0,c578ed38,c0c717a4,343,c5ea12a8,...) at softdep_flush+0x244 fork_exit(c0ab1150,0,c578ed38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc578ed70, ebp = 0 --- Tracing command vnlru pid 19 tid 100060 td 0xc5e56900 sched_switch(c5e56900,0,104,191,a954e79e,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,50,...) at mi_switch+0x200 sleepq_switch(c5e56900,0,c0c7a69a,26e,0,...) at sleepq_switch+0x15f sleepq_timedwait(c5ea1550,50,c0c847ac,0,0,...) at sleepq_timedwait+0x6b _sleep(c5ea1550,c0f37654,250,c0c847ac,3e8,...) at _sleep+0x339 vnlru_proc(0,c578bd38,c0c717a4,343,c5ea1550,...) at vnlru_proc+0xe7 fork_exit(c09138c0,0,c578bd38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc578bd70, ebp = 0 --- Tracing command syncer pid 18 tid 100059 td 0xc5e56b40 sched_switch(c5e56b40,0,104,191,a954bed6,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c5e56b40,0,c0c7a69a,26e,c5e56b40,...) at sleepq_switch+0x15f sleepq_timedwait(c0f37694,0,c5788c88,1,0,...) at sleepq_timedwait+0x6b _cv_timedwait(c0f37694,c0f37680,3e8,6cc,4e20,...) at _cv_timedwait+0x250 sched_sync(0,c5788d38,c0c717a4,343,c5ea17f8,...) at sched_sync+0x502 fork_exit(c0912cf0,0,c5788d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5788d70, ebp = 0 --- Tracing command bufdaemon pid 17 tid 100058 td 0xc5e56d80 sched_switch(c5e56d80,0,104,191,a954977e,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,44,...) at mi_switch+0x200 sleepq_switch(c5e56d80,0,c0c7a69a,26e,0,...) at sleepq_switch+0x15f sleepq_timedwait(c0f373c8,44,c0c81d22,0,0,...) at sleepq_timedwait+0x6b _sleep(c0f373c8,c0f373cc,44,c0c81d22,3e8,...) at _sleep+0x339 buf_daemon(0,c5785d38,c0c717a4,343,c5ea1aa0,...) at buf_daemon+0x138 fork_exit(c08fb060,0,c5785d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5785d70, ebp = 0 --- Tracing command pagezero pid 16 tid 100057 td 0xc5e5b000 sched_switch(c5e5b000,0,104,191,8e2607de,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c5e5b000,0,c0c7a69a,26e,0,...) at sleepq_switch+0x15f sleepq_timedwait(c0f43c54,0,c0ca2840,0,0,...) at sleepq_timedwait+0x6b _sleep(c0f43c54,c0f43810,0,c0ca2840,493e0,...) at _sleep+0x339 vm_pagezero(0,c5782d38,c0c717a4,343,c5ea1d48,...) at vm_pagezero+0xdc fork_exit(c0aee490,0,c5782d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5782d70, ebp = 0 --- Tracing command vmdaemon pid 15 tid 100056 td 0xc5e5b240 sched_switch(c5e5b240,0,104,191,2a76132,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,68,...) at mi_switch+0x200 sleepq_switch(c5e5b240,0,c0c7a69a,24b,0,...) at sleepq_switch+0x15f sleepq_wait(c0f4387c,68,c0c81d22,0,0,...) at sleepq_wait+0x63 _sleep(c0f4387c,c0f43880,68,c0c81d22,0,...) at _sleep+0x36b vm_daemon(0,c577fd38,c0c717a4,343,c596e2a8,...) at vm_daemon+0x59 fork_exit(c0ae88e0,0,c577fd38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc577fd70, ebp = 0 --- Tracing command pagedaemon pid 9 tid 100055 td 0xc5e5b480 sched_switch(c5e5b480,0,104,191,5b168876,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,44,...) at mi_switch+0x200 sleepq_switch(c5e5b480,0,c0c7a69a,26e,0,...) at sleepq_switch+0x15f sleepq_timedwait(c0f43844,44,c0c81d22,0,0,...) at sleepq_timedwait+0x6b _sleep(c0f43844,c0f43810,44,c0c81d22,1388,...) at _sleep+0x339 vm_pageout(0,c577cd38,c0c717a4,343,c596e550,...) at vm_pageout+0x2bb fork_exit(c0ae9780,0,c577cd38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc577cd70, ebp = 0 --- Tracing command usb pid 14 tid 100054 td 0xc5e5b6c0 sched_switch(c5e5b6c0,0,104,191,5ff31610,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c5e5b6c0,0,c0c7a69a,24b,c5e5b6c0,...) at sleepq_switch+0x15f sleepq_wait(c5aead0c,0,c5779cb8,1,0,...) at sleepq_wait+0x63 _cv_wait(c5aead0c,c5aeadac,c0c650ce,6a,c0dceb40,...) at _cv_wait+0x240 usb_process(c5aead04,c5779d38,c0c717a4,343,c596e7f8,...) at usb_process+0x193 fork_exit(c07b7150,c5aead04,c5779d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5779d70, ebp = 0 --- Tracing command usb pid 14 tid 100053 td 0xc5e5b900 sched_switch(c5e5b900,0,104,191,730c5dd2,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c5e5b900,0,c0c7a69a,24b,c5e5b900,...) at sleepq_switch+0x15f sleepq_wait(c5aeacdc,0,c5776cb8,1,0,...) at sleepq_wait+0x63 _cv_wait(c5aeacdc,c5aeadac,c0c650ce,6a,c0dceb40,...) at _cv_wait+0x240 usb_process(c5aeacd4,c5776d38,c0c717a4,343,c596e7f8,...) at usb_process+0x193 fork_exit(c07b7150,c5aeacd4,c5776d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5776d70, ebp = 0 --- Tracing command usb pid 14 tid 100052 td 0xc5e5bb40 sched_switch(c5e5bb40,0,104,191,a1274526,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c5e5bb40,0,c0c7a69a,24b,c5e5bb40,...) at sleepq_switch+0x15f sleepq_wait(c5aeacac,0,c5773cb8,1,0,...) at sleepq_wait+0x63 _cv_wait(c5aeacac,c5aeadac,c0c650ce,6a,c0dceb40,...) at _cv_wait+0x240 usb_process(c5aeaca4,c5773d38,c0c717a4,343,c596e7f8,...) at usb_process+0x193 fork_exit(c07b7150,c5aeaca4,c5773d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5773d70, ebp = 0 --- Tracing command usb pid 14 tid 100051 td 0xc5e5bd80 sched_switch(c5e5bd80,0,104,191,5a5f3bf8,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c5e5bd80,0,c0c7a69a,24b,c5e5bd80,...) at sleepq_switch+0x15f sleepq_wait(c5aeac7c,0,c5770cb8,1,0,...) at sleepq_wait+0x63 _cv_wait(c5aeac7c,c5aeadac,c0c650ce,6a,c0dceb40,...) at _cv_wait+0x240 usb_process(c5aeac74,c5770d38,c0c717a4,343,c596e7f8,...) at usb_process+0x193 fork_exit(c07b7150,c5aeac74,c5770d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5770d70, ebp = 0 --- Tracing command usb pid 14 tid 100050 td 0xc5b5f6c0 sched_switch(c5b5f6c0,0,104,191,5a5f14f4,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c5b5f6c0,0,c0c7a69a,24b,c5b5f6c0,...) at sleepq_switch+0x15f sleepq_wait(c5ad7dac,0,c576dcb8,1,0,...) at sleepq_wait+0x63 _cv_wait(c5ad7dac,c5ad7e4c,c0c650ce,6a,c0dceb40,...) at _cv_wait+0x240 usb_process(c5ad7da4,c576dd38,c0c717a4,343,c596e7f8,...) at usb_process+0x193 fork_exit(c07b7150,c5ad7da4,c576dd38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc576dd70, ebp = 0 --- Tracing command usb pid 14 tid 100049 td 0xc5b5f900 sched_switch(c5b5f900,0,104,191,6ef6c8da,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c5b5f900,0,c0c7a69a,24b,c5b5f900,...) at sleepq_switch+0x15f sleepq_wait(c5ad7d7c,0,c576acb8,1,0,...) at sleepq_wait+0x63 _cv_wait(c5ad7d7c,c5ad7e4c,c0c650ce,6a,c0dceb40,...) at _cv_wait+0x240 usb_process(c5ad7d74,c576ad38,c0c717a4,343,c596e7f8,...) at usb_process+0x193 fork_exit(c07b7150,c5ad7d74,c576ad38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc576ad70, ebp = 0 --- Tracing command usb pid 14 tid 100048 td 0xc5b5fb40 sched_switch(c5b5fb40,0,104,191,544dcdcc,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c5b5fb40,0,c0c7a69a,24b,c5b5fb40,...) at sleepq_switch+0x15f sleepq_wait(c5ad7d4c,0,c5767cb8,1,0,...) at sleepq_wait+0x63 _cv_wait(c5ad7d4c,c5ad7e4c,c0c650ce,6a,c0dceb40,...) at _cv_wait+0x240 usb_process(c5ad7d44,c5767d38,c0c717a4,343,c596e7f8,...) at usb_process+0x193 fork_exit(c07b7150,c5ad7d44,c5767d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5767d70, ebp = 0 --- Tracing command usb pid 14 tid 100047 td 0xc5b5fd80 sched_switch(c5b5fd80,0,104,191,544da8c4,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c5b5fd80,0,c0c7a69a,24b,c5b5fd80,...) at sleepq_switch+0x15f sleepq_wait(c5ad7d1c,0,c5764cb8,1,0,...) at sleepq_wait+0x63 _cv_wait(c5ad7d1c,c5ad7e4c,c0c650ce,6a,c0dceb40,...) at _cv_wait+0x240 usb_process(c5ad7d14,c5764d38,c0c717a4,343,c596e7f8,...) at usb_process+0x193 fork_exit(c07b7150,c5ad7d14,c5764d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5764d70, ebp = 0 --- Tracing command usb pid 14 tid 100046 td 0xc5e55000 sched_switch(c5e55000,0,104,191,544d83f0,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c5e55000,0,c0c7a69a,24b,c5e55000,...) at sleepq_switch+0x15f sleepq_wait(c5ac5dac,0,c5761cb8,1,0,...) at sleepq_wait+0x63 _cv_wait(c5ac5dac,c5ac5e4c,c0c650ce,6a,c0dceb40,...) at _cv_wait+0x240 usb_process(c5ac5da4,c5761d38,c0c717a4,343,c596e7f8,...) at usb_process+0x193 fork_exit(c07b7150,c5ac5da4,c5761d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5761d70, ebp = 0 --- Tracing command usb pid 14 tid 100045 td 0xc5e55240 sched_switch(c5e55240,0,104,191,6e88eef6,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c5e55240,0,c0c7a69a,24b,c5e55240,...) at sleepq_switch+0x15f sleepq_wait(c5ac5d7c,0,c575ecb8,1,0,...) at sleepq_wait+0x63 _cv_wait(c5ac5d7c,c5ac5e4c,c0c650ce,6a,c0dceb40,...) at _cv_wait+0x240 usb_process(c5ac5d74,c575ed38,c0c717a4,343,c596e7f8,...) at usb_process+0x193 fork_exit(c07b7150,c5ac5d74,c575ed38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc575ed70, ebp = 0 --- Tracing command usb pid 14 tid 100044 td 0xc5e55480 sched_switch(c5e55480,0,104,191,4e39af0c,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c5e55480,0,c0c7a69a,24b,c5e55480,...) at sleepq_switch+0x15f sleepq_wait(c5ac5d4c,0,c575bcb8,1,0,...) at sleepq_wait+0x63 _cv_wait(c5ac5d4c,c5ac5e4c,c0c650ce,6a,c0dceb40,...) at _cv_wait+0x240 usb_process(c5ac5d44,c575bd38,c0c717a4,343,c596e7f8,...) at usb_process+0x193 fork_exit(c07b7150,c5ac5d44,c575bd38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc575bd70, ebp = 0 --- Tracing command usb pid 14 tid 100043 td 0xc5e556c0 sched_switch(c5e556c0,0,104,191,4e398d00,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c5e556c0,0,c0c7a69a,24b,c5e556c0,...) at sleepq_switch+0x15f sleepq_wait(c5ac5d1c,0,c5758cb8,1,0,...) at sleepq_wait+0x63 _cv_wait(c5ac5d1c,c5ac5e4c,c0c650ce,6a,c0dceb40,...) at _cv_wait+0x240 usb_process(c5ac5d14,c5758d38,c0c717a4,343,c596e7f8,...) at usb_process+0x193 fork_exit(c07b7150,c5ac5d14,c5758d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5758d70, ebp = 0 --- Tracing command usb pid 14 tid 100042 td 0xc5e55900 sched_switch(c5e55900,0,104,191,6f1cda86,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c5e55900,0,c0c7a69a,24b,c5e55900,...) at sleepq_switch+0x15f sleepq_wait(c5ab0dac,0,c5755cb8,1,0,...) at sleepq_wait+0x63 _cv_wait(c5ab0dac,c5ab0e4c,c0c650ce,6a,c0dceb40,...) at _cv_wait+0x240 usb_process(c5ab0da4,c5755d38,c0c717a4,343,c596e7f8,...) at usb_process+0x193 fork_exit(c07b7150,c5ab0da4,c5755d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5755d70, ebp = 0 --- Tracing command usb pid 14 tid 100041 td 0xc5e55b40 sched_switch(c5e55b40,0,104,191,6f1d1da2,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c5e55b40,0,c0c7a69a,24b,c5e55b40,...) at sleepq_switch+0x15f sleepq_wait(c5ab0d7c,0,c5752cb8,1,0,...) at sleepq_wait+0x63 _cv_wait(c5ab0d7c,c5ab0e4c,c0c650ce,6a,c0dceb40,...) at _cv_wait+0x240 usb_process(c5ab0d74,c5752d38,c0c717a4,343,c596e7f8,...) at usb_process+0x193 fork_exit(c07b7150,c5ab0d74,c5752d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5752d70, ebp = 0 --- Tracing command usb pid 14 tid 100040 td 0xc5e55d80 sched_switch(c5e55d80,0,104,191,5e2bc3d2,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c5e55d80,0,c0c7a69a,24b,c5e55d80,...) at sleepq_switch+0x15f sleepq_wait(c5ab0d4c,0,c574fcb8,1,0,...) at sleepq_wait+0x63 _cv_wait(c5ab0d4c,c5ab0e4c,c0c650ce,6a,c0dceb40,...) at _cv_wait+0x240 usb_process(c5ab0d44,c574fd38,c0c717a4,343,c596e7f8,...) at usb_process+0x193 fork_exit(c07b7150,c5ab0d44,c574fd38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc574fd70, ebp = 0 --- Tracing command usb pid 14 tid 100039 td 0xc5e56000 sched_switch(c5e56000,0,104,191,3a0de216,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c5e56000,0,c0c7a69a,24b,c5e56000,...) at sleepq_switch+0x15f sleepq_wait(c5ab0d1c,0,c574ccb8,1,0,...) at sleepq_wait+0x63 _cv_wait(c5ab0d1c,c5ab0e4c,c0c650ce,6a,c0dceb40,...) at _cv_wait+0x240 usb_process(c5ab0d14,c574cd38,c0c717a4,343,c596e7f8,...) at usb_process+0x193 fork_exit(c07b7150,c5ab0d14,c574cd38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc574cd70, ebp = 0 --- Tracing command sctp_iterator pid 8 tid 100038 td 0xc5a85d80 sched_switch(c5a85d80,0,104,191,804aa3ce,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c5a85d80,0,c0c7a69a,24b,0,...) at sleepq_switch+0x15f sleepq_wait(c0f3917c,0,c0c8fb50,0,0,...) at sleepq_wait+0x63 _sleep(c0f3917c,c0f39090,0,c0c8fb50,0,...) at _sleep+0x36b sctp_iterator_thread(0,c5749d38,c0c717a4,343,c596eaa0,...) at sctp_iterator_thread+0x60 fork_exit(c099c420,0,c5749d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5749d70, ebp = 0 --- Tracing command fdc0 pid 7 tid 100034 td 0xc5b5d6c0 sched_switch(c5b5d6c0,0,104,191,b1e97d3e,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,4c,...) at mi_switch+0x200 sleepq_switch(c5b5d6c0,0,c0c7a69a,26e,0,...) at sleepq_switch+0x15f sleepq_timedwait(c59ec23c,4c,c0c6c015,0,0,...) at sleepq_timedwait+0x6b _sleep(c59ec23c,c59ec2f0,4c,c0c6c015,3e8,...) at _sleep+0x339 fdc_thread(c59ec200,c573cd38,c0c717a4,343,c596ed48,...) at fdc_thread+0x2be fork_exit(c0b6ba00,c59ec200,c573cd38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc573cd70, ebp = 0 --- Tracing command fw0_probe pid 6 tid 100031 td 0xc5b5dd80 sched_switch(c5b5dd80,0,104,191,7c72a4a,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,5c,...) at mi_switch+0x200 sleepq_switch(c5b5dd80,0,c0c7a69a,18b,5c,...) at sleepq_switch+0x15f sleepq_catch_signals(c0c7a69a,15b,0,100,100,...) at sleepq_catch_signals+0xb7 sleepq_wait_sig(c5b62000,5c,c0c6c015,100,0,...) at sleepq_wait_sig+0x17 _sleep(c5b62000,c5b66488,15c,c0c6c015,0,...) at _sleep+0x354 fw_bus_probe_thread(c5b62000,c5558d38,c0c717a4,343,c59de000,...) at fw_bus_probe_thread+0xa08 fork_exit(c0654500,c5b62000,c5558d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5558d70, ebp = 0 --- Tracing command xpt_thrd pid 5 tid 100015 td 0xc59716c0 sched_switch(c59716c0,0,104,191,481a07e0,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,4c,...) at mi_switch+0x200 sleepq_switch(c59716c0,0,c0c7a69a,24b,0,...) at sleepq_switch+0x15f sleepq_wait(c0d97a54,4c,c0c117a4,0,0,...) at sleepq_wait+0x63 _sleep(c0d97a54,c0d97a6c,4c,c0c117a4,0,...) at _sleep+0x36b xpt_scanner_thread(0,c54f2d38,c0c717a4,343,c59de2a8,...) at xpt_scanner_thread+0x4a fork_exit(c0482f30,0,c54f2d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc54f2d70, ebp = 0 --- Tracing command yarrow pid 13 tid 100011 td 0xc59b5000 sched_switch(c59b5000,0,104,191,f525b556,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c59b5000,0,c0c7a69a,26e,2,...) at sleepq_switch+0x15f sleepq_timedwait(c0dcac64,0,c0c6c015,2,0,...) at sleepq_timedwait+0x6b _sleep(c0dcac64,0,0,c0c6c015,64,...) at _sleep+0x339 pause(c0c6c015,64,c0c58f7a,111,0,...) at pause+0x47 random_kthread(0,c54e6d38,c0c717a4,343,c59de550,...) at random_kthread+0x1ef fork_exit(c0730080,0,c54e6d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc54e6d70, ebp = 0 --- Tracing command g_down pid 4 tid 100009 td 0xc59b5480 sched_switch(c59b5480,0,104,191,f124675e,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,4c,...) at mi_switch+0x200 sleepq_switch(c59b5480,0,c0c7a69a,26e,0,...) at sleepq_switch+0x15f sleepq_timedwait(c0dc8a24,4c,c0c6c015,0,0,...) at sleepq_timedwait+0x6b _sleep(c0dc8a24,c0dc8988,24c,c0c6c015,64,...) at _sleep+0x339 g_io_schedule_down(c59b5480,0,c0c6d64b,74,0,...) at g_io_schedule_down+0x6b g_down_procbody(0,c54e0d38,c0c717a4,343,c596d000,...) at g_down_procbody+0x8d fork_exit(c081e2f0,0,c54e0d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc54e0d70, ebp = 0 --- Tracing command g_up pid 3 tid 100008 td 0xc596f000 sched_switch(c596f000,0,104,191,f1248b8e,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,4c,...) at mi_switch+0x200 sleepq_switch(c596f000,0,c0c7a69a,26e,0,...) at sleepq_switch+0x15f sleepq_timedwait(c0dc8a20,4c,c0c6c015,0,0,...) at sleepq_timedwait+0x6b _sleep(c0dc8a20,c0dc89a8,24c,c0c6c015,64,...) at _sleep+0x339 g_io_schedule_up(c596f000,0,c0c6d64b,5d,0,...) at g_io_schedule_up+0x133 g_up_procbody(0,c54ddd38,c0c717a4,343,c596d2a8,...) at g_up_procbody+0x8d fork_exit(c081e380,0,c54ddd38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc54ddd70, ebp = 0 --- Tracing command g_event pid 2 tid 100007 td 0xc596f240 sched_switch(c596f240,0,104,191,f525e4ee,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,4c,...) at mi_switch+0x200 sleepq_switch(c596f240,0,c0c7a69a,26e,0,...) at sleepq_switch+0x15f sleepq_timedwait(c0dc8a18,4c,c0c6c015,0,0,...) at sleepq_timedwait+0x6b _sleep(c0dc8a18,0,4c,c0c6c015,64,...) at _sleep+0x339 g_event_procbody(0,c54dad38,c0c717a4,343,c596d550,...) at g_event_procbody+0xcb fork_exit(c081e410,0,c54dad38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc54dad70, ebp = 0 --- Tracing command intr pid 12 tid 100037 td 0xc5b5d000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100036 td 0xc5b5d240 sched_switch(c5b5d240,0,109,191,f631f0a6,...) at sched_switch+0x406 mi_switch(109,0,c0c71a2b,4f6,c5c77170,...) at mi_switch+0x200 ithread_loop(c5c802f0,c5743d38,c0c717a4,343,c596d7f8,...) at ithread_loop+0x1f6 fork_exit(c0859f70,c5c802f0,c5743d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5743d70, ebp = 0 --- Tracing command intr pid 12 tid 100035 td 0xc5b5d480 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100033 td 0xc5b5d900 sched_switch(c5b5d900,0,109,191,59d95806,...) at sched_switch+0x406 mi_switch(109,0,c0c71a2b,4f6,c59b42f0,...) at mi_switch+0x200 ithread_loop(c5c6a630,c572fd38,c0c717a4,343,c596d7f8,...) at ithread_loop+0x1f6 fork_exit(c0859f70,c5c6a630,c572fd38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc572fd70, ebp = 0 --- Tracing command intr pid 12 tid 100032 td 0xc5b5db40 sched_switch(c5b5db40,0,109,191,479c27be,...) at sched_switch+0x406 mi_switch(109,0,c0c71a2b,4f6,c59b4370,...) at mi_switch+0x200 ithread_loop(c5c6a4a0,c56c4d38,c0c717a4,343,c596d7f8,...) at ithread_loop+0x1f6 fork_exit(c0859f70,c5c6a4a0,c56c4d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc56c4d70, ebp = 0 --- Tracing command intr pid 12 tid 100029 td 0xc5b5f240 sched_switch(c5b5f240,0,109,191,47dd9b8c,...) at sched_switch+0x406 mi_switch(109,0,c0c71a2b,4f6,c59b41f0,...) at mi_switch+0x200 ithread_loop(c5b028e0,c554fd38,c0c717a4,343,c596d7f8,...) at ithread_loop+0x1f6 fork_exit(c0859f70,c5b028e0,c554fd38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc554fd70, ebp = 0 --- Tracing command intr pid 12 tid 100027 td 0xc59b56c0 sched_switch(c59b56c0,0,109,191,e51b0c76,...) at sched_switch+0x406 mi_switch(109,0,c0c71a2b,4f6,c59b3ef0,...) at mi_switch+0x200 ithread_loop(c5b02980,c5546d38,c0c717a4,343,c596d7f8,...) at ithread_loop+0x1f6 fork_exit(c0859f70,c5b02980,c5546d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5546d70, ebp = 0 --- Tracing command intr pid 12 tid 100026 td 0xc59b5900 sched_switch(c59b5900,0,109,191,a1270286,...) at sched_switch+0x406 mi_switch(109,0,c0c71a2b,4f6,c59b3df0,...) at mi_switch+0x200 ithread_loop(c5aa95c0,c552ed38,c0c717a4,343,c596d7f8,...) at ithread_loop+0x1f6 fork_exit(c0859f70,c5aa95c0,c552ed38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc552ed70, ebp = 0 --- Tracing command intr pid 12 tid 100025 td 0xc59b5b40 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100024 td 0xc59b5d80 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100023 td 0xc5a85000 sched_switch(c5a85000,0,109,191,6f1c989a,...) at sched_switch+0x406 mi_switch(109,0,c0c71a2b,4f6,c59b4270,...) at mi_switch+0x200 ithread_loop(c5aac820,c5521d38,c0c717a4,343,c596d7f8,...) at ithread_loop+0x1f6 fork_exit(c0859f70,c5aac820,c5521d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5521d70, ebp = 0 --- Tracing command intr pid 12 tid 100022 td 0xc5a85240 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100020 td 0xc5a856c0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100016 td 0xc5971480 sched_switch(c5971480,0,109,191,a12a276a,...) at sched_switch+0x406 mi_switch(109,0,c0c71a2b,4f6,c5a83970,...) at mi_switch+0x200 ithread_loop(c590a980,c54f5d38,c0c717a4,343,c596d7f8,...) at ithread_loop+0x1f6 fork_exit(c0859f70,c590a980,c54f5d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc54f5d70, ebp = 0 --- Tracing command intr pid 12 tid 100013 td 0xc5971b40 sched_switch(c5971b40,0,109,191,8a3a33da,...) at sched_switch+0x406 mi_switch(109,0,c0c71a2b,4f6,c59b3070,...) at mi_switch+0x200 ithread_loop(c590a9b0,c54ecd38,c0c717a4,343,c596d7f8,...) at ithread_loop+0x1f6 fork_exit(c0859f70,c590a9b0,c54ecd38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc54ecd70, ebp = 0 --- Tracing command intr pid 12 tid 100012 td 0xc5971d80 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100006 td 0xc596f480 sched_switch(c596f480,0,109,191,fc4c5766,...) at sched_switch+0x406 mi_switch(109,0,c0c71a2b,4f6,c59b3c70,...) at mi_switch+0x200 ithread_loop(c596c0c0,c54d7d38,c0c717a4,343,c596d7f8,...) at ithread_loop+0x1f6 fork_exit(c0859f70,c596c0c0,c54d7d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc54d7d70, ebp = 0 --- Tracing command intr pid 12 tid 100005 td 0xc596f6c0 sched_switch(c596f6c0,0,109,191,4b847aea,...) at sched_switch+0x406 mi_switch(109,0,c0c71a2b,4f6,c59b3cf0,...) at mi_switch+0x200 ithread_loop(c596c0d0,c54d4d38,c0c717a4,343,c596d7f8,...) at ithread_loop+0x1f6 fork_exit(c0859f70,c596c0d0,c54d4d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc54d4d70, ebp = 0 --- Tracing command intr pid 12 tid 100004 td 0xc596f900 fork_trampoline() at fork_trampoline Tracing command idle pid 11 tid 100003 td 0xc596fb40 kdb_enter(c0c1f62f,c0c5fa9d,80000000,c5afd380,0,...) at kdb_enter+0x3a uart_intr(c5afd300,c596fb40,c5953cd0,c596ad80,4,...) at uart_intr+0x126 intr_event_handle(c596ad80,c54cdc34,0,1f4,c5c7d000,...) at intr_event_handle+0x5c intr_execute_handlers(c5953cd0,c54cdc34,0,c54cdc74,c0b96044,...) at intr_execute_handlers+0x49 lapic_handle_intr(3b,c54cdc34) at lapic_handle_intr+0x4c Xapic_isr1() at Xapic_isr1+0x34 --- interrupt, eip = 0xc0b8a525, esp = 0xc54cdc74, ebp = 0xc54cdc74 --- acpi_cpu_c1(c54c0008,c54c0028,0,c596fb40,c0dceb40,...) at acpi_cpu_c1+0x5 acpi_cpu_idle(c54cdcb4,c0ba194b,1,c54cdcf8,c08a31be,...) at acpi_cpu_idle+0x11c cpu_idle_acpi(1,c54cdcf8,c08a31be,1,0,...) at cpu_idle_acpi+0x1b cpu_idle(1,0,c0c77e8a,9e9,c596fb40,...) at cpu_idle+0x1b sched_idletd(0,c54cdd38,c0c717a4,343,c596daa0,...) at sched_idletd+0x23e fork_exit(c08a2f80,0,c54cdd38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc54cdd70, ebp = 0 --- Tracing command init pid 1 tid 100002 td 0xc596fd80 sched_switch(c596fd80,0,104,191,6b6f0d6a,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,5c,...) at mi_switch+0x200 sleepq_switch(c596fd80,0,c0c7a69a,18b,5c,...) at sleepq_switch+0x15f sleepq_catch_signals(c0c7a69a,15b,0,100,100,...) at sleepq_catch_signals+0xb7 sleepq_wait_sig(c596dd48,5c,c0c7cf0b,100,0,...) at sleepq_wait_sig+0x17 _sleep(c596dd48,c596ddd0,15c,c0c7cf0b,0,...) at _sleep+0x354 kern_wait(c596fd80,ffffffff,c54c9c74,0,0,...) at kern_wait+0xb76 wait4(c596fd80,c54c9cf8,10,c0c7cd3b,c0d5c404,...) at wait4+0x3b syscall(c54c9d38) at syscall+0x2b4 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (7, FreeBSD ELF32, wait4), eip = 0x8054b8f, esp = 0xbfbfe90c, ebp = 0xbfbfe928 --- Tracing command audit pid 10 tid 100001 td 0xc5971000 sched_switch(c5971000,0,104,191,48159710,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c5971000,0,c0c7a69a,24b,c5971000,...) at sleepq_switch+0x15f sleepq_wait(c0f428a0,0,c54c6c9c,1,0,...) at sleepq_wait+0x63 _cv_wait(c0f428a0,c0f42884,c0c9a679,194,0,...) at _cv_wait+0x240 audit_worker(0,c54c6d38,c0c717a4,343,c596e000,...) at audit_worker+0x84 fork_exit(c0a84b10,0,c54c6d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc54c6d70, ebp = 0 --- Tracing command kernel pid 0 tid 100130 td 0xc673a000 sched_switch(c673a000,0,104,191,fea55740,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c673a000,0,c0c7a69a,24b,0,...) at sleepq_switch+0x15f sleepq_wait(d4a446c0,0,c0c6c015,0,0,...) at sleepq_wait+0x63 _sleep(d4a446c0,d4a446d8,0,c0c6c015,0,...) at _sleep+0x36b taskqueue_thread_loop(ce9352f0,e8194d38,c0c717a4,343,c0dc8ae0,...) at taskqueue_thread_loop+0xba fork_exit(c08baf50,ce9352f0,e8194d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe8194d70, ebp = 0 --- Tracing command kernel pid 0 tid 100115 td 0xc6122b40 sched_switch(c6122b40,0,104,191,9e7765e8,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c6122b40,0,c0c7a69a,24b,0,...) at sleepq_switch+0x15f sleepq_wait(c6542280,0,c0c6c015,0,0,...) at sleepq_wait+0x63 _sleep(c6542280,c6542298,0,c0c6c015,0,...) at _sleep+0x36b taskqueue_thread_loop(c5b07c70,e8157d38,c0c717a4,343,c0dc8ae0,...) at taskqueue_thread_loop+0xba fork_exit(c08baf50,c5b07c70,e8157d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe8157d70, ebp = 0 --- Tracing command kernel pid 0 tid 100113 td 0xc6545000 sched_switch(c6545000,0,104,191,a12b1c96,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c6545000,0,c0c7a69a,24b,0,...) at sleepq_switch+0x15f sleepq_wait(c6541700,0,c0c6c015,0,0,...) at sleepq_wait+0x63 _sleep(c6541700,c6541718,0,c0c6c015,0,...) at _sleep+0x36b taskqueue_thread_loop(c5f3f3b0,e8151d38,c0c717a4,343,c0dc8ae0,...) at taskqueue_thread_loop+0xba fork_exit(c08baf50,c5f3f3b0,e8151d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe8151d70, ebp = 0 --- Tracing command kernel pid 0 tid 100112 td 0xc6545240 sched_switch(c6545240,0,104,191,96bcc54e,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c6545240,0,c0c7a69a,24b,0,...) at sleepq_switch+0x15f sleepq_wait(c65416c0,0,c0c6c015,0,0,...) at sleepq_wait+0x63 _sleep(c65416c0,c65416d8,0,c0c6c015,0,...) at _sleep+0x36b taskqueue_thread_loop(c5f3f4e0,e814ed38,c0c717a4,343,c0dc8ae0,...) at taskqueue_thread_loop+0xba fork_exit(c08baf50,c5f3f4e0,e814ed38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe814ed70, ebp = 0 --- Tracing command kernel pid 0 tid 100111 td 0xc6545480 sched_switch(c6545480,0,104,191,96bca382,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c6545480,0,c0c7a69a,24b,0,...) at sleepq_switch+0x15f sleepq_wait(c6541680,0,c0c6c015,0,0,...) at sleepq_wait+0x63 _sleep(c6541680,c6541698,0,c0c6c015,0,...) at _sleep+0x36b taskqueue_thread_loop(c5e9a1d0,e814bd38,c0c717a4,343,c0dc8ae0,...) at taskqueue_thread_loop+0xba fork_exit(c08baf50,c5e9a1d0,e814bd38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe814bd70, ebp = 0 --- Tracing command kernel pid 0 tid 100110 td 0xc65456c0 sched_switch(c65456c0,0,104,191,96bc803e,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c65456c0,0,c0c7a69a,24b,0,...) at sleepq_switch+0x15f sleepq_wait(c65412c0,0,c0c6c015,0,0,...) at sleepq_wait+0x63 _sleep(c65412c0,c65412d8,0,c0c6c015,0,...) at _sleep+0x36b taskqueue_thread_loop(c5f3f470,e8148d38,c0c717a4,343,c0dc8ae0,...) at taskqueue_thread_loop+0xba fork_exit(c08baf50,c5f3f470,e8148d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe8148d70, ebp = 0 --- Tracing command kernel pid 0 tid 100109 td 0xc6546480 sched_switch(c6546480,0,104,191,96bc5daa,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c6546480,0,c0c7a69a,24b,0,...) at sleepq_switch+0x15f sleepq_wait(c6541280,0,c0c6c015,0,0,...) at sleepq_wait+0x63 _sleep(c6541280,c6541298,0,c0c6c015,0,...) at _sleep+0x36b taskqueue_thread_loop(c5e9a700,e8145d38,c0c717a4,343,c0dc8ae0,...) at taskqueue_thread_loop+0xba fork_exit(c08baf50,c5e9a700,e8145d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe8145d70, ebp = 0 --- Tracing command kernel pid 0 tid 100108 td 0xc6120000 sched_switch(c6120000,0,104,191,96bc39da,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c6120000,0,c0c7a69a,24b,0,...) at sleepq_switch+0x15f sleepq_wait(c6541240,0,c0c6c015,0,0,...) at sleepq_wait+0x63 _sleep(c6541240,c6541258,0,c0c6c015,0,...) at _sleep+0x36b taskqueue_thread_loop(c5f3f520,e8142d38,c0c717a4,343,c0dc8ae0,...) at taskqueue_thread_loop+0xba fork_exit(c08baf50,c5f3f520,e8142d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe8142d70, ebp = 0 --- Tracing command kernel pid 0 tid 100107 td 0xc6120240 sched_switch(c6120240,0,104,191,a0dc8922,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c6120240,0,c0c7a69a,24b,0,...) at sleepq_switch+0x15f sleepq_wait(c6541800,0,c0c6c015,0,0,...) at sleepq_wait+0x63 _sleep(c6541800,c6541818,0,c0c6c015,0,...) at _sleep+0x36b taskqueue_thread_loop(c5f3f500,e813fd38,c0c717a4,343,c0dc8ae0,...) at taskqueue_thread_loop+0xba fork_exit(c08baf50,c5f3f500,e813fd38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe813fd70, ebp = 0 --- Tracing command kernel pid 0 tid 100106 td 0xc6545900 sched_switch(c6545900,0,104,191,9fc3328a,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c6545900,0,c0c7a69a,24b,0,...) at sleepq_switch+0x15f sleepq_wait(c6541840,0,c0c6c015,0,0,...) at sleepq_wait+0x63 _sleep(c6541840,c6541858,0,c0c6c015,0,...) at _sleep+0x36b taskqueue_thread_loop(c5f3f4f0,e813cd38,c0c717a4,343,c0dc8ae0,...) at taskqueue_thread_loop+0xba fork_exit(c08baf50,c5f3f4f0,e813cd38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe813cd70, ebp = 0 --- Tracing command kernel pid 0 tid 100105 td 0xc6545b40 sched_switch(c6545b40,0,104,191,9fc30fca,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c6545b40,0,c0c7a69a,24b,0,...) at sleepq_switch+0x15f sleepq_wait(c6541840,0,c0c6c015,0,0,...) at sleepq_wait+0x63 _sleep(c6541840,c6541858,0,c0c6c015,0,...) at _sleep+0x36b taskqueue_thread_loop(c5f3f4f0,e8139d38,c0c717a4,343,c0dc8ae0,...) at taskqueue_thread_loop+0xba fork_exit(c08baf50,c5f3f4f0,e8139d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe8139d70, ebp = 0 --- Tracing command kernel pid 0 tid 100104 td 0xc6545d80 sched_switch(c6545d80,0,104,191,9fc38dc2,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c6545d80,0,c0c7a69a,24b,0,...) at sleepq_switch+0x15f sleepq_wait(c6541840,0,c0c6c015,0,0,...) at sleepq_wait+0x63 _sleep(c6541840,c6541858,0,c0c6c015,0,...) at _sleep+0x36b taskqueue_thread_loop(c5f3f4f0,e8136d38,c0c717a4,343,c0dc8ae0,...) at taskqueue_thread_loop+0xba fork_exit(c08baf50,c5f3f4f0,e8136d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe8136d70, ebp = 0 --- Tracing command kernel pid 0 tid 100103 td 0xc6546000 sched_switch(c6546000,0,104,191,9fc3aa6e,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c6546000,0,c0c7a69a,24b,0,...) at sleepq_switch+0x15f sleepq_wait(c6541840,0,c0c6c015,0,0,...) at sleepq_wait+0x63 _sleep(c6541840,c6541858,0,c0c6c015,0,...) at _sleep+0x36b taskqueue_thread_loop(c5f3f4f0,e8133d38,c0c717a4,343,c0dc8ae0,...) at taskqueue_thread_loop+0xba fork_exit(c08baf50,c5f3f4f0,e8133d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe8133d70, ebp = 0 --- Tracing command kernel pid 0 tid 100102 td 0xc6546240 sched_switch(c6546240,0,104,191,9fc37132,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c6546240,0,c0c7a69a,24b,0,...) at sleepq_switch+0x15f sleepq_wait(c6541840,0,c0c6c015,0,0,...) at sleepq_wait+0x63 _sleep(c6541840,c6541858,0,c0c6c015,0,...) at _sleep+0x36b taskqueue_thread_loop(c5f3f4f0,e8130d38,c0c717a4,343,c0dc8ae0,...) at taskqueue_thread_loop+0xba fork_exit(c08baf50,c5f3f4f0,e8130d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe8130d70, ebp = 0 --- Tracing command kernel pid 0 tid 100101 td 0xc6120480 sched_switch(c6120480,0,104,191,9fc2dec2,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c6120480,0,c0c7a69a,24b,0,...) at sleepq_switch+0x15f sleepq_wait(c6541840,0,c0c6c015,0,0,...) at sleepq_wait+0x63 _sleep(c6541840,c6541858,0,c0c6c015,0,...) at _sleep+0x36b taskqueue_thread_loop(c5f3f4f0,e812dd38,c0c717a4,343,c0dc8ae0,...) at taskqueue_thread_loop+0xba fork_exit(c08baf50,c5f3f4f0,e812dd38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe812dd70, ebp = 0 --- Tracing command kernel pid 0 tid 100100 td 0xc61206c0 sched_switch(c61206c0,0,104,191,9fc3cc86,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c61206c0,0,c0c7a69a,24b,0,...) at sleepq_switch+0x15f sleepq_wait(c6541840,0,c0c6c015,0,0,...) at sleepq_wait+0x63 _sleep(c6541840,c6541858,0,c0c6c015,0,...) at _sleep+0x36b taskqueue_thread_loop(c5f3f4f0,e811bd38,c0c717a4,343,c0dc8ae0,...) at taskqueue_thread_loop+0xba fork_exit(c08baf50,c5f3f4f0,e811bd38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe811bd70, ebp = 0 --- Tracing command kernel pid 0 tid 100099 td 0xc6120900 sched_switch(c6120900,0,104,191,9fc35486,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c6120900,0,c0c7a69a,24b,0,...) at sleepq_switch+0x15f sleepq_wait(c6541840,0,c0c6c015,0,0,...) at sleepq_wait+0x63 _sleep(c6541840,c6541858,0,c0c6c015,0,...) at _sleep+0x36b taskqueue_thread_loop(c5f3f4f0,e8118d38,c0c717a4,343,c0dc8ae0,...) at taskqueue_thread_loop+0xba fork_exit(c08baf50,c5f3f4f0,e8118d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe8118d70, ebp = 0 --- Tracing command kernel pid 0 tid 100098 td 0xc6547d80 sched_switch(c6547d80,0,104,191,4b17e030,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c6547d80,0,c0c7a69a,24b,0,...) at sleepq_switch+0x15f sleepq_wait(c6541880,0,c0c6c015,0,0,...) at sleepq_wait+0x63 _sleep(c6541880,c6541898,0,c0c6c015,0,...) at _sleep+0x36b taskqueue_thread_loop(c5f3f4d0,e8115d38,c0c717a4,343,c0dc8ae0,...) at taskqueue_thread_loop+0xba fork_exit(c08baf50,c5f3f4d0,e8115d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe8115d70, ebp = 0 --- Tracing command kernel pid 0 tid 100097 td 0xc6547b40 sched_switch(c6547b40,0,104,191,4af7fcd8,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c6547b40,0,c0c7a69a,24b,0,...) at sleepq_switch+0x15f sleepq_wait(c6541880,0,c0c6c015,0,0,...) at sleepq_wait+0x63 _sleep(c6541880,c6541898,0,c0c6c015,0,...) at _sleep+0x36b taskqueue_thread_loop(c5f3f4d0,e812ad38,c0c717a4,343,c0dc8ae0,...) at taskqueue_thread_loop+0xba fork_exit(c08baf50,c5f3f4d0,e812ad38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe812ad70, ebp = 0 --- Tracing command kernel pid 0 tid 100096 td 0xc65466c0 sched_switch(c65466c0,0,104,191,4adc90e8,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c65466c0,0,c0c7a69a,24b,0,...) at sleepq_switch+0x15f sleepq_wait(c6541880,0,c0c6c015,0,0,...) at sleepq_wait+0x63 _sleep(c6541880,c6541898,0,c0c6c015,0,...) at _sleep+0x36b taskqueue_thread_loop(c5f3f4d0,e8127d38,c0c717a4,343,c0dc8ae0,...) at taskqueue_thread_loop+0xba fork_exit(c08baf50,c5f3f4d0,e8127d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe8127d70, ebp = 0 --- Tracing command kernel pid 0 tid 100095 td 0xc6546900 sched_switch(c6546900,0,104,191,47f2818e,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c6546900,0,c0c7a69a,24b,0,...) at sleepq_switch+0x15f sleepq_wait(c6541880,0,c0c6c015,0,0,...) at sleepq_wait+0x63 _sleep(c6541880,c6541898,0,c0c6c015,0,...) at _sleep+0x36b taskqueue_thread_loop(c5f3f4d0,e8124d38,c0c717a4,343,c0dc8ae0,...) at taskqueue_thread_loop+0xba fork_exit(c08baf50,c5f3f4d0,e8124d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe8124d70, ebp = 0 --- Tracing command kernel pid 0 tid 100094 td 0xc6546b40 sched_switch(c6546b40,0,104,191,47d25caa,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c6546b40,0,c0c7a69a,24b,0,...) at sleepq_switch+0x15f sleepq_wait(c6541880,0,c0c6c015,0,0,...) at sleepq_wait+0x63 _sleep(c6541880,c6541898,0,c0c6c015,0,...) at _sleep+0x36b taskqueue_thread_loop(c5f3f4d0,e8121d38,c0c717a4,343,c0dc8ae0,...) at taskqueue_thread_loop+0xba fork_exit(c08baf50,c5f3f4d0,e8121d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe8121d70, ebp = 0 --- Tracing command kernel pid 0 tid 100093 td 0xc6546d80 sched_switch(c6546d80,0,104,191,4bdfd712,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c6546d80,0,c0c7a69a,24b,0,...) at sleepq_switch+0x15f sleepq_wait(c6541880,0,c0c6c015,0,0,...) at sleepq_wait+0x63 _sleep(c6541880,c6541898,0,c0c6c015,0,...) at _sleep+0x36b taskqueue_thread_loop(c5f3f4d0,e811ed38,c0c717a4,343,c0dc8ae0,...) at taskqueue_thread_loop+0xba fork_exit(c08baf50,c5f3f4d0,e811ed38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe811ed70, ebp = 0 --- Tracing command kernel pid 0 tid 100092 td 0xc6547000 sched_switch(c6547000,0,104,191,4ba7157e,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c6547000,0,c0c7a69a,24b,0,...) at sleepq_switch+0x15f sleepq_wait(c6541880,0,c0c6c015,0,0,...) at sleepq_wait+0x63 _sleep(c6541880,c6541898,0,c0c6c015,0,...) at _sleep+0x36b taskqueue_thread_loop(c5f3f4d0,e8112d38,c0c717a4,343,c0dc8ae0,...) at taskqueue_thread_loop+0xba fork_exit(c08baf50,c5f3f4d0,e8112d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe8112d70, ebp = 0 --- Tracing command kernel pid 0 tid 100091 td 0xc6547240 sched_switch(c6547240,0,104,191,4b7c372e,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c6547240,0,c0c7a69a,24b,0,...) at sleepq_switch+0x15f sleepq_wait(c6541880,0,c0c6c015,0,0,...) at sleepq_wait+0x63 _sleep(c6541880,c6541898,0,c0c6c015,0,...) at _sleep+0x36b taskqueue_thread_loop(c5f3f4d0,e810fd38,c0c717a4,343,c0dc8ae0,...) at taskqueue_thread_loop+0xba fork_exit(c08baf50,c5f3f4d0,e810fd38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe810fd70, ebp = 0 --- Tracing command kernel pid 0 tid 100090 td 0xc6547480 sched_switch(c6547480,0,104,191,a4c861da,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c6547480,0,c0c7a69a,24b,0,...) at sleepq_switch+0x15f sleepq_wait(c65418c0,0,c0c6c015,0,0,...) at sleepq_wait+0x63 _sleep(c65418c0,c65418d8,0,c0c6c015,0,...) at _sleep+0x36b taskqueue_thread_loop(c5f3f490,e810cd38,c0c717a4,343,c0dc8ae0,...) at taskqueue_thread_loop+0xba fork_exit(c08baf50,c5f3f490,e810cd38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe810cd70, ebp = 0 --- Tracing command kernel pid 0 tid 100089 td 0xc65476c0 sched_switch(c65476c0,0,104,191,a128ad06,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c65476c0,0,c0c7a69a,24b,0,...) at sleepq_switch+0x15f sleepq_wait(c5e5d680,0,c0c6c015,0,0,...) at sleepq_wait+0x63 _sleep(c5e5d680,c5e5d698,0,c0c6c015,0,...) at _sleep+0x36b taskqueue_thread_loop(c5f3f4c0,e815ad38,c0c717a4,343,c0dc8ae0,...) at taskqueue_thread_loop+0xba fork_exit(c08baf50,c5f3f4c0,e815ad38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe815ad70, ebp = 0 --- Tracing command kernel pid 0 tid 100088 td 0xc6547900 sched_switch(c6547900,0,104,191,a0e7711e,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c6547900,0,c0c7a69a,24b,0,...) at sleepq_switch+0x15f sleepq_wait(c5e7c100,0,c0c6c015,0,0,...) at sleepq_wait+0x63 _sleep(c5e7c100,c5e7c118,0,c0c6c015,0,...) at _sleep+0x36b taskqueue_thread_loop(c5c4d360,e815dd38,c0c717a4,343,c0dc8ae0,...) at taskqueue_thread_loop+0xba fork_exit(c08baf50,c5c4d360,e815dd38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe815dd70, ebp = 0 --- Tracing command kernel pid 0 tid 100030 td 0xc5b5f000 sched_switch(c5b5f000,0,104,191,f94b86a6,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c5b5f000,0,c0c7a69a,24b,c5b5f000,...) at sleepq_switch+0x15f sleepq_wait(c5abeb80,0,c0c76ece,c0c6c015,0,...) at sleepq_wait+0x63 msleep_spin(c5abeb80,c5abeb98,c0c6c015,0,c0c74a49,...) at msleep_spin+0x21d taskqueue_thread_loop(c5b6649c,c5555d38,c0c717a4,343,c0dc8ae0,...) at taskqueue_thread_loop+0x94 fork_exit(c08baf50,c5b6649c,c5555d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5555d70, ebp = 0 --- Tracing command kernel pid 0 tid 100028 td 0xc5b5f480 sched_switch(c5b5f480,0,104,191,39b65fdc,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c5b5f480,0,c0c7a69a,24b,0,...) at sleepq_switch+0x15f sleepq_wait(c5b05500,0,c0c6c015,0,0,...) at sleepq_wait+0x63 _sleep(c5b05500,c5b05518,0,c0c6c015,0,...) at _sleep+0x36b taskqueue_thread_loop(c5b5b074,c554bd38,c0c717a4,343,c0dc8ae0,...) at taskqueue_thread_loop+0xba fork_exit(c08baf50,c5b5b074,c554bd38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc554bd70, ebp = 0 --- Tracing command kernel pid 0 tid 100021 td 0xc5a85480 sched_switch(c5a85480,0,104,191,737dd5ea,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c5a85480,0,c0c7a69a,24b,0,...) at sleepq_switch+0x15f sleepq_wait(c5a66e80,0,c0c6c015,0,0,...) at sleepq_wait+0x63 _sleep(c5a66e80,c5a66e98,0,c0c6c015,0,...) at _sleep+0x36b taskqueue_thread_loop(c0ddd5c8,c5504d38,c0c717a4,343,c0dc8ae0,...) at taskqueue_thread_loop+0xba fork_exit(c08baf50,c0ddd5c8,c5504d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc5504d70, ebp = 0 --- Tracing command kernel pid 0 tid 100019 td 0xc5a85900 sched_switch(c5a85900,0,104,191,609f5440,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c5a85900,0,c0c7a69a,24b,c5a85900,...) at sleepq_switch+0x15f sleepq_wait(c5a67240,0,c0c76ece,c0c6c015,0,...) at sleepq_wait+0x63 msleep_spin(c5a67240,c5a67258,c0c6c015,0,c0c74a49,...) at msleep_spin+0x21d taskqueue_thread_loop(c0d9a860,c54fed38,c0c717a4,343,c0dc8ae0,...) at taskqueue_thread_loop+0x94 fork_exit(c08baf50,c0d9a860,c54fed38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc54fed70, ebp = 0 --- Tracing command kernel pid 0 tid 100018 td 0xc5a85b40 sched_switch(c5a85b40,0,104,191,609f3204,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c5a85b40,0,c0c7a69a,24b,c5a85b40,...) at sleepq_switch+0x15f sleepq_wait(c5a67240,0,c0c76ece,c0c6c015,0,...) at sleepq_wait+0x63 msleep_spin(c5a67240,c5a67258,c0c6c015,0,c0c74a49,...) at msleep_spin+0x21d taskqueue_thread_loop(c0d9a860,c54fbd38,c0c717a4,343,c0dc8ae0,...) at taskqueue_thread_loop+0x94 fork_exit(c08baf50,c0d9a860,c54fbd38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc54fbd70, ebp = 0 --- Tracing command kernel pid 0 tid 100017 td 0xc5971240 sched_switch(c5971240,0,104,191,609f07ac,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c5971240,0,c0c7a69a,24b,c5971240,...) at sleepq_switch+0x15f sleepq_wait(c5a67240,0,c0c76ece,c0c6c015,0,...) at sleepq_wait+0x63 msleep_spin(c5a67240,c5a67258,c0c6c015,0,c0c74a49,...) at msleep_spin+0x21d taskqueue_thread_loop(c0d9a860,c54f8d38,c0c717a4,343,c0dc8ae0,...) at taskqueue_thread_loop+0x94 fork_exit(c08baf50,c0d9a860,c54f8d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc54f8d70, ebp = 0 --- Tracing command kernel pid 0 tid 100014 td 0xc5971900 sched_switch(c5971900,0,104,191,609dbb5c,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c5971900,0,c0c7a69a,24b,0,...) at sleepq_switch+0x15f sleepq_wait(c5a673c0,0,c0c6c015,0,0,...) at sleepq_wait+0x63 _sleep(c5a673c0,c5a673d8,0,c0c6c015,0,...) at _sleep+0x36b taskqueue_thread_loop(c0dc9384,c54efd38,c0c717a4,343,c0dc8ae0,...) at taskqueue_thread_loop+0xba fork_exit(c08baf50,c0dc9384,c54efd38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc54efd70, ebp = 0 --- Tracing command kernel pid 0 tid 100010 td 0xc59b5240 sched_switch(c59b5240,0,104,191,73b9f616,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,0,...) at mi_switch+0x200 sleepq_switch(c59b5240,0,c0c7a69a,24b,0,...) at sleepq_switch+0x15f sleepq_wait(c5955c40,0,c0c6c015,0,0,...) at sleepq_wait+0x63 _sleep(c5955c40,c5955c58,0,c0c6c015,0,...) at _sleep+0x36b taskqueue_thread_loop(c0ddc060,c54e3d38,c0c717a4,343,c0dc8ae0,...) at taskqueue_thread_loop+0xba fork_exit(c08baf50,c0ddc060,c54e3d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc54e3d70, ebp = 0 --- Tracing command kernel pid 0 tid 100000 td 0xc0dc8d90 sched_switch(c0dc8d90,0,104,191,8f890d82,...) at sched_switch+0x406 mi_switch(104,0,c0c7a69a,1d6,44,...) at mi_switch+0x200 sleepq_switch(c0dc8d90,0,c0c7a69a,26e,0,...) at sleepq_switch+0x15f sleepq_timedwait(c0dc8ae0,44,c0c78703,0,0,...) at sleepq_timedwait+0x6b _sleep(c0dc8ae0,0,44,c0c78703,2710,...) at _sleep+0x339 scheduler(0,141ec00,141ec00,141e000,1425000,...) at scheduler+0x23e mi_startup() at mi_startup+0x96 begin() at begin+0x2c ---------------------------------------------------------------------- TOP OF DMESG KDB: debugger backends: ddb KDB: current backend: ddb Copyright (c) 1992-2009 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 8.0-RC1 #1: Sat Oct 24 18:12:41 UTC 2009 root@box8.ws.local:/usr/obj/usr/src/sys/WITHDDB WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Intel(R) Pentium(R) 4 CPU 2.40GHz (2400.10-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0xf27 Stepping = 7 Features=0xbfebfbff Features2=0x4400 real memory = 2147483648 (2048 MB) avail memory = 2086248448 (1989 MB) ---------------------------------------------------------------------- KERNEL CONFIG include GENERIC ident WITHDDB options KDB options DDB options BREAK_TO_DEBUGGER makeoptions DEBUG=-g options INVARIANTS options INVARIANT_SUPPORT options WITNESS options DEBUG_LOCKS options DEBUG_VFS_LOCKS options DIAGNOSTIC